-----============= acceptance-small: conf-sanity ============----- Thu Oct 23 13:05:30 EDT 2025 oleg601-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball 110 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg601-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg601-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg601-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg601-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg601-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg601-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg601-server oleg601-server: oleg601-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg601-server' oleg601-server: oleg601-server.virtnet: executing load_modules_local oleg601-server: Loading modules from /home/green/git/lustre-release/lustre oleg601-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg601-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg601-server: detected 4 online CPUs by sysfs oleg601-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg601-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg601-server: oleg601-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg601-client: oleg601-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 start mds service on oleg601-server Starting mds2: -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg601-server: oleg601-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg601-client: oleg601-server: ssh exited with exit code 1 Commit the device label on lustre-mdt2/mdt2 Started lustre-MDT0001 oleg601-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg601-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg601-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg601-server: oleg601-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg601-client: oleg601-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg601-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg601-server: oleg601-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg601-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg601-server: oleg601-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 40 oleg601-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg601-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg601-server stop mds service on oleg601-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg601-server stop mds service on oleg601-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg601-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 13:08:22 (1761239302) Stopping clients: oleg601-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg601-client.virtnet /mnt/lustre2 (opts:-f) oleg601-server: oleg601-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg601-server' oleg601-server: oleg601-server.virtnet: executing load_modules_local oleg601-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg601-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg601-server: Loading modules from /home/green/git/lustre-release/lustre oleg601-server: detected 4 online CPUs by sysfs oleg601-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg601-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg601-server: oleg601-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg601-client: oleg601-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 start mds service on oleg601-server Starting mds2: -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg601-server: oleg601-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg601-client: oleg601-server: ssh exited with exit code 1 Commit the device label on lustre-mdt2/mdt2 Started lustre-MDT0001 oleg601-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg601-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg601-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg601-server: oleg601-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg601-client: oleg601-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg601-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg601-client.virtnet: -o user_xattr,flock oleg601-server@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86890 331 86559 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 87086 312 86774 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 16232 424 15808 3% /mnt/lustre[OST:0] filesystem_summary: 16451 643 15808 4% /mnt/lustre fail_loc=0x1312 fail_val=5 - open/close 734 (time 1761239415.67 total 10.01 last 73.34) - open/close 1492 (time 1761239425.67 total 20.01 last 75.79) - open/close 2302 (time 1761239435.67 total 30.01 last 80.95) - open/close 3075 (time 1761239445.68 total 40.02 last 77.25) - open/close 3879 (time 1761239455.69 total 50.03 last 80.30) - open/close 4688 (time 1761239465.70 total 60.04 last 80.84) - open/close 5387 (time 1761239475.71 total 70.05 last 69.84) - open/close 6152 (time 1761239485.71 total 80.05 last 76.49) - open/close 6900 (time 1761239495.71 total 90.05 last 74.80) - open/close 7677 (time 1761239505.71 total 100.06 last 77.65) - open/close 8494 (time 1761239515.72 total 110.06 last 81.65) - open/close 9366 (time 1761239525.73 total 120.07 last 87.09) - open/close 10000 (time 1761239533.03 total 127.37 last 86.93) - open/close 10896 (time 1761239543.04 total 137.38 last 89.50) - open/close 11849 (time 1761239553.04 total 147.38 last 95.25) - open/close 12743 (time 1761239563.04 total 157.39 last 89.39) - open/close 13771 (time 1761239573.05 total 167.40 last 102.71) - open/close 14753 (time 1761239583.06 total 177.40 last 98.18) - open/close 15681 (time 1761239593.06 total 187.40 last 92.77) - open/close 16550 (time 1761239603.07 total 197.41 last 86.78) - open/close 17431 (time 1761239613.08 total 207.42 last 88.08) - open/close 18273 (time 1761239623.08 total 217.43 last 84.13) - open/close 19131 (time 1761239633.10 total 227.44 last 85.69) - open/close 19975 (time 1761239643.11 total 237.45 last 84.33) - open/close 20805 (time 1761239653.12 total 247.46 last 82.87) - open/close 21669 (time 1761239663.13 total 257.47 last 86.34) - open/close 22529 (time 1761239673.13 total 267.47 last 86.00) - open/close 23297 (time 1761239683.14 total 277.48 last 76.73) - open/close 24081 (time 1761239693.15 total 287.49 last 78.32) - open/close 24929 (time 1761239703.16 total 297.50 last 84.71) - open/close 25722 (time 1761239713.17 total 307.51 last 79.19) - open/close 26447 (time 1761239723.18 total 317.52 last 72.44) - open/close 27277 (time 1761239733.19 total 327.53 last 82.95) - open/close 28154 (time 1761239743.20 total 337.54 last 87.60) - open/close 28952 (time 1761239753.21 total 347.55 last 79.75) - open/close 29745 (time 1761239763.22 total 357.56 last 79.22) - open/close 30000 (time 1761239766.26 total 360.60 last 83.88) - open/close 30824 (time 1761239776.26 total 370.60 last 82.35) - open/close 31714 (time 1761239786.27 total 380.62 last 88.90) - open/close 32572 (time 1761239796.28 total 390.62 last 85.76) - open/close 33332 (time 1761239806.29 total 400.63 last 75.90) - open/close 34220 (time 1761239816.30 total 410.64 last 88.75) - open/close 35099 (time 1761239826.30 total 420.64 last 87.88) - open/close 35939 (time 1761239836.30 total 430.64 last 83.99) - open/close 36725 (time 1761239846.32 total 440.66 last 78.48) - open/close 37567 (time 1761239856.32 total 450.66 last 84.17) - open/close 38366 (time 1761239866.33 total 460.67 last 79.85) - open/close 39308 (time 1761239876.33 total 470.67 last 94.17) - open/close 40000 (time 1761239885.51 total 479.85 last 75.40) - open/close 40838 (time 1761239895.51 total 489.85 last 83.79) - open/close 41733 (time 1761239905.51 total 499.85 last 89.48) - open/close 42601 (time 1761239915.52 total 509.87 last 86.69) - open/close 43379 (time 1761239925.54 total 519.88 last 77.66) - open/close 44252 (time 1761239935.54 total 529.89 last 87.28) - open/close 45022 (time 1761239945.55 total 539.89 last 76.98) - open/close 45857 (time 1761239955.55 total 549.90 last 83.45) - open/close 46790 (time 1761239965.56 total 559.90 last 93.29) - open/close 47662 (time 1761239975.56 total 569.90 last 87.19) - open/close 48663 (time 1761239985.56 total 579.90 last 100.09) - open/close 49606 (time 1761239995.56 total 589.90 last 94.29) - open/close 50000 (time 1761239999.40 total 593.75 last 102.50) - open/close 50847 (time 1761240009.41 total 603.75 last 84.62) - open/close 51831 (time 1761240019.42 total 613.76 last 98.30) - open/close 52641 (time 1761240029.43 total 623.77 last 80.97) - open/close 53467 (time 1761240039.44 total 633.78 last 82.51) - open/close 54308 (time 1761240049.45 total 643.79 last 84.02) - open/close 55215 (time 1761240059.45 total 653.79 last 90.66) - open/close 56286 (time 1761240069.45 total 663.79 last 107.09) - open/close 57147 (time 1761240079.45 total 673.79 last 86.10) - open/close 57993 (time 1761240089.46 total 683.80 last 84.53) - open/close 58793 (time 1761240099.47 total 693.81 last 79.97) - open/close 59723 (time 1761240109.48 total 703.82 last 92.90) - open/close 60000 (time 1761240112.75 total 707.09 last 84.72) - open/close 60847 (time 1761240122.75 total 717.09 last 84.66) - open/close 61699 (time 1761240132.76 total 727.10 last 85.15) - open/close 62508 (time 1761240142.76 total 737.10 last 80.86) - open/close 63399 (time 1761240152.77 total 747.11 last 89.06) - open/close 64176 (time 1761240162.78 total 757.12 last 77.59) total: 64768 open/close in 764.08 seconds: 84.77 ops/second - /unlink 1560 (time 1761240182.61 total 10.00 last 155.93) - /unlink 2909 (time 1761240192.62 total 20.01 last 134.82) - /unlink 4351 (time 1761240202.63 total 30.02 last 144.04) - /unlink 5702 (time 1761240212.63 total 40.02 last 135.09) - /unlink 7353 (time 1761240222.63 total 50.03 last 164.99) - /unlink 9136 (time 1761240232.64 total 60.03 last 178.27) - /unlink 10000 (time 1761240238.25 total 65.64 last 154.00) - /unlink 11779 (time 1761240248.26 total 75.65 last 177.73) - /unlink 13439 (time 1761240258.27 total 85.66 last 165.83) - /unlink 14980 (time 1761240268.27 total 95.66 last 154.07) - /unlink 16383 (time 1761240278.27 total 105.67 last 140.25) - /unlink 17851 (time 1761240288.28 total 115.67 last 146.70) - /unlink 19255 (time 1761240298.28 total 125.68 last 140.36) - /unlink 20000 (time 1761240303.91 total 131.30 last 132.42) - /unlink 21299 (time 1761240313.92 total 141.31 last 129.79) - /unlink 22609 (time 1761240323.92 total 151.31 last 131.00) - /unlink 23983 (time 1761240333.92 total 161.31 last 137.36) - /unlink 25336 (time 1761240343.92 total 171.31 last 135.29) - /unlink 26665 (time 1761240353.92 total 181.32 last 132.84) - /unlink 27993 (time 1761240363.93 total 191.32 last 132.75) - /unlink 29277 (time 1761240373.93 total 201.33 last 128.36) - /unlink 30000 (time 1761240379.04 total 206.43 last 141.56) - /unlink 31457 (time 1761240389.04 total 216.43 last 145.69) - /unlink 32795 (time 1761240399.04 total 226.44 last 133.74) - /unlink 34166 (time 1761240409.04 total 236.44 last 137.09) - /unlink 35499 (time 1761240419.05 total 246.44 last 133.23) - /unlink 36712 (time 1761240429.05 total 256.45 last 121.29) - /unlink 37978 (time 1761240439.05 total 266.45 last 126.58) - /unlink 39305 (time 1761240449.06 total 276.45 last 132.61) - /unlink 40000 (time 1761240454.55 total 281.94 last 126.60) - /unlink 41433 (time 1761240464.55 total 291.95 last 143.27) - /unlink 42705 (time 1761240474.56 total 301.95 last 127.11) - /unlink 44073 (time 1761240484.56 total 311.95 last 136.78) - /unlink 45490 (time 1761240494.56 total 321.96 last 141.62) - /unlink 46913 (time 1761240504.57 total 331.96 last 142.25) - /unlink 48218 (time 1761240514.57 total 341.97 last 130.44) - /unlink 49490 (time 1761240524.58 total 351.97 last 127.12) - /unlink 50000 (time 1761240528.42 total 355.82 last 132.71) - /unlink 51422 (time 1761240538.43 total 365.82 last 142.16) - /unlink 52805 (time 1761240548.43 total 375.82 last 138.28) - /unlink 54122 (time 1761240558.44 total 385.83 last 131.56) - /unlink 55382 (time 1761240568.44 total 395.84 last 125.92) - /unlink 56665 (time 1761240578.44 total 405.84 last 128.29) - /unlink 58009 (time 1761240588.45 total 415.85 last 134.28) - /unlink 59301 (time 1761240598.46 total 425.86 last 129.10) - /unlink 60000 (time 1761240604.65 total 432.05 last 112.91) - /unlink 61347 (time 1761240614.65 total 442.05 last 134.66) - /unlink 62608 (time 1761240624.66 total 452.05 last 126.06) - /unlink 63937 (time 1761240634.66 total 462.06 last 132.83) total: 64768 /unlink in 469.15 seconds: 138.06 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 820 (time 1761240678.74 total 10.00 last 81.99) - open/close 1576 (time 1761240688.74 total 20.01 last 75.53) - open/close 2416 (time 1761240698.75 total 30.02 last 83.93) - open/close 3201 (time 1761240708.76 total 40.02 last 78.46) - open/close 4014 (time 1761240718.76 total 50.03 last 81.28) - open/close 4829 (time 1761240728.76 total 60.03 last 81.49) - open/close 5648 (time 1761240738.77 total 70.04 last 81.84) - open/close 6436 (time 1761240748.79 total 80.05 last 78.67) - open/close 7194 (time 1761240758.79 total 90.06 last 75.76) - open/close 8004 (time 1761240768.80 total 100.06 last 80.94) - open/close 8809 (time 1761240778.81 total 110.08 last 80.38) - open/close 9601 (time 1761240788.82 total 120.09 last 79.13) - open/close 10000 (time 1761240793.73 total 124.99 last 81.36) - open/close 10805 (time 1761240803.73 total 135.00 last 80.48) - open/close 11628 (time 1761240813.74 total 145.00 last 82.25) - open/close 12343 (time 1761240823.74 total 155.01 last 71.44) - open/close 13096 (time 1761240833.75 total 165.01 last 75.29) - open/close 13845 (time 1761240843.76 total 175.03 last 74.80) - open/close 14602 (time 1761240853.76 total 185.03 last 75.68) - open/close 15425 (time 1761240863.78 total 195.05 last 82.14) - open/close 16195 (time 1761240873.79 total 205.06 last 76.93) - open/close 16993 (time 1761240883.79 total 215.06 last 79.79) - open/close 17793 (time 1761240893.79 total 225.06 last 79.99) - open/close 18642 (time 1761240903.79 total 235.06 last 84.90) - open/close 19434 (time 1761240913.80 total 245.06 last 79.18) - open/close 20000 (time 1761240922.02 total 253.29 last 68.81) - open/close 20750 (time 1761240932.02 total 263.29 last 74.97) - open/close 21473 (time 1761240942.03 total 273.30 last 72.24) - open/close 22219 (time 1761240952.04 total 283.30 last 74.57) - open/close 22924 (time 1761240962.05 total 293.32 last 70.39) - open/close 23648 (time 1761240972.06 total 303.32 last 72.36) - open/close 24407 (time 1761240982.07 total 313.33 last 75.85) - open/close 25230 (time 1761240992.07 total 323.33 last 82.29) - open/close 25982 (time 1761241002.07 total 333.34 last 75.16) - open/close 26785 (time 1761241012.08 total 343.35 last 80.23) - open/close 27626 (time 1761241022.09 total 353.35 last 84.05) - open/close 28426 (time 1761241032.10 total 363.37 last 79.86) - open/close 29245 (time 1761241042.12 total 373.38 last 81.79) - open/close 30000 (time 1761241051.69 total 382.96 last 78.86) - open/close 30740 (time 1761241061.70 total 392.96 last 73.94) - open/close 31591 (time 1761241071.70 total 402.97 last 85.05) - open/close 32424 (time 1761241081.71 total 412.98 last 83.23) - open/close 33217 (time 1761241091.73 total 422.99 last 79.19) - open/close 34003 (time 1761241101.74 total 433.00 last 78.54) - open/close 34797 (time 1761241111.74 total 443.01 last 79.34) - open/close 35515 (time 1761241121.76 total 453.02 last 71.70) - open/close 36286 (time 1761241131.76 total 463.02 last 77.08) - open/close 37112 (time 1761241141.76 total 473.02 last 82.60) - open/close 37914 (time 1761241151.77 total 483.03 last 80.15) - open/close 38708 (time 1761241161.77 total 493.03 last 79.39) - open/close 39569 (time 1761241171.77 total 503.04 last 86.06) - open/close 40000 (time 1761241176.50 total 507.77 last 91.09) - open/close 40938 (time 1761241186.51 total 517.78 last 93.73) - open/close 41791 (time 1761241196.52 total 527.78 last 85.26) - open/close 42636 (time 1761241206.53 total 537.80 last 84.34) - open/close 43570 (time 1761241216.54 total 547.80 last 93.40) - open/close 44467 (time 1761241226.54 total 557.81 last 89.64) - open/close 45336 (time 1761241236.55 total 567.82 last 86.80) - open/close 46224 (time 1761241246.56 total 577.83 last 88.74) - open/close 47070 (time 1761241256.56 total 587.83 last 84.55) - open/close 47920 (time 1761241266.58 total 597.85 last 84.87) - open/close 48734 (time 1761241276.59 total 607.85 last 81.34) - open/close 49611 (time 1761241286.59 total 617.86 last 87.64) - open/close 50000 (time 1761241291.21 total 622.48 last 84.21) - open/close 50816 (time 1761241301.21 total 632.48 last 81.59) - open/close 51579 (time 1761241311.23 total 642.49 last 76.22) - open/close 52357 (time 1761241321.23 total 652.50 last 77.76) - open/close 53120 (time 1761241331.25 total 662.51 last 76.18) - open/close 53905 (time 1761241341.25 total 672.52 last 78.47) - open/close 54710 (time 1761241351.26 total 682.52 last 80.44) - open/close 55530 (time 1761241361.26 total 692.53 last 81.98) - open/close 56430 (time 1761241371.26 total 702.53 last 89.97) - open/close 57308 (time 1761241381.27 total 712.53 last 87.75) - open/close 58183 (time 1761241391.27 total 722.54 last 87.48) - open/close 59025 (time 1761241401.28 total 732.54 last 84.16) - open/close 59976 (time 1761241411.29 total 742.56 last 94.94) - open/close 60854 (time 1761241421.29 total 752.56 last 87.80) - open/close 61773 (time 1761241431.30 total 762.57 last 91.82) - open/close 62655 (time 1761241441.31 total 772.57 last 88.16) - open/close 63585 (time 1761241451.32 total 782.58 last 92.90) - open/close 64427 (time 1761241461.33 total 792.59 last 84.12) total: 64768 open/close in 796.32 seconds: 81.33 ops/second - /unlink 1527 (time 1761241478.21 total 10.00 last 152.64) - /unlink 2869 (time 1761241488.22 total 20.01 last 134.10) - /unlink 4288 (time 1761241498.23 total 30.02 last 141.72) - /unlink 5626 (time 1761241508.23 total 40.02 last 133.79) - /unlink 7100 (time 1761241518.23 total 50.03 last 147.39) - /unlink 8669 (time 1761241528.24 total 60.03 last 156.80) - /unlink 10000 (time 1761241535.92 total 67.71 last 173.27) - /unlink 11884 (time 1761241545.93 total 77.72 last 188.31) - /unlink 13305 (time 1761241555.93 total 87.72 last 142.09) - /unlink 14655 (time 1761241565.93 total 97.72 last 134.96) - /unlink 16011 (time 1761241575.93 total 107.73 last 135.54) - /unlink 17345 (time 1761241585.94 total 117.73 last 133.32) - /unlink 18591 (time 1761241595.94 total 127.73 last 124.60) - /unlink 19938 (time 1761241605.97 total 137.76 last 134.36) - /unlink 21412 (time 1761241615.97 total 147.76 last 147.37) - /unlink 22763 (time 1761241625.97 total 157.76 last 135.03) - /unlink 24227 (time 1761241635.97 total 167.76 last 146.39) - /unlink 25400 (time 1761241645.98 total 177.77 last 117.28) - /unlink 26684 (time 1761241655.98 total 187.77 last 128.35) - /unlink 27964 (time 1761241665.98 total 197.77 last 128.00) - /unlink 29422 (time 1761241675.98 total 207.78 last 145.73) - /unlink 30000 (time 1761241679.92 total 211.71 last 146.72) - /unlink 31521 (time 1761241689.93 total 221.73 last 151.95) - /unlink 33023 (time 1761241699.94 total 231.73 last 150.13) - /unlink 34342 (time 1761241709.94 total 241.73 last 131.89) - /unlink 35621 (time 1761241719.94 total 251.73 last 127.90) - /unlink 37066 (time 1761241729.95 total 261.74 last 144.36) - /unlink 38511 (time 1761241739.95 total 271.74 last 144.47) - /unlink 39780 (time 1761241749.95 total 281.75 last 126.86) - /unlink 41131 (time 1761241759.96 total 291.75 last 135.05) - /unlink 42342 (time 1761241769.96 total 301.75 last 121.05) - /unlink 43716 (time 1761241779.96 total 311.75 last 137.37) - /unlink 45063 (time 1761241789.97 total 321.76 last 134.63) - /unlink 46464 (time 1761241799.97 total 331.76 last 140.07) - /unlink 47915 (time 1761241809.98 total 341.77 last 145.05) - /unlink 49402 (time 1761241819.98 total 351.77 last 148.63) - /unlink 50000 (time 1761241823.96 total 355.75 last 150.31) - /unlink 51437 (time 1761241833.96 total 365.75 last 143.66) - /unlink 52777 (time 1761241843.97 total 375.76 last 133.91) - /unlink 53983 (time 1761241853.97 total 385.76 last 120.54) - /unlink 55214 (time 1761241863.98 total 395.77 last 122.99) - /unlink 56504 (time 1761241873.99 total 405.78 last 128.94) - /unlink 57820 (time 1761241883.99 total 415.78 last 131.55) - /unlink 59125 (time 1761241894.00 total 425.79 last 130.36) - /unlink 60000 (time 1761241901.14 total 432.93 last 122.52) - /unlink 61304 (time 1761241911.15 total 442.94 last 130.30) - /unlink 62625 (time 1761241921.15 total 452.95 last 132.06) - /unlink 63935 (time 1761241931.17 total 462.96 last 130.83) total: 64768 /unlink in 470.32 seconds: 137.71 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 726 (time 1761241976.33 total 10.00 last 72.58) - open/close 1496 (time 1761241986.38 total 20.04 last 76.68) - open/close 2327 (time 1761241996.39 total 30.06 last 82.97) - open/close 3171 (time 1761242006.40 total 40.07 last 84.35) - open/close 3967 (time 1761242016.41 total 50.08 last 79.53) - open/close 4830 (time 1761242026.41 total 60.08 last 86.28) - open/close 5623 (time 1761242036.42 total 70.08 last 79.25) - open/close 6425 (time 1761242046.42 total 80.09 last 80.18) - open/close 7254 (time 1761242056.43 total 90.10 last 82.83) - open/close 8077 (time 1761242066.43 total 100.10 last 82.28) - open/close 8817 (time 1761242076.44 total 110.11 last 73.88) - open/close 9580 (time 1761242086.44 total 120.11 last 76.29) - open/close 10000 (time 1761242091.39 total 125.06 last 84.99) - open/close 10867 (time 1761242101.41 total 135.08 last 86.51) - open/close 11684 (time 1761242111.41 total 145.08 last 81.67) - open/close 12501 (time 1761242121.42 total 155.09 last 81.63) - open/close 13321 (time 1761242131.43 total 165.10 last 81.95) - open/close 14092 (time 1761242141.43 total 175.10 last 77.07) - open/close 14867 (time 1761242151.44 total 185.11 last 77.45) - open/close 15671 (time 1761242161.44 total 195.11 last 80.40) - open/close 16533 (time 1761242171.45 total 205.12 last 86.13) - open/close 17332 (time 1761242181.45 total 215.12 last 79.88) - open/close 18259 (time 1761242191.46 total 225.13 last 92.59) - open/close 19101 (time 1761242201.47 total 235.14 last 84.16) - open/close 19914 (time 1761242211.48 total 245.15 last 81.19) - open/close 20669 (time 1761242221.48 total 255.15 last 75.49) - open/close 21493 (time 1761242231.48 total 265.15 last 82.40) - open/close 22318 (time 1761242241.48 total 275.15 last 82.47) - open/close 23199 (time 1761242251.49 total 285.16 last 88.05) - open/close 23987 (time 1761242261.49 total 295.16 last 78.77) - open/close 24847 (time 1761242271.50 total 305.17 last 85.90) - open/close 25568 (time 1761242281.51 total 315.18 last 72.08) - open/close 26256 (time 1761242291.52 total 325.19 last 68.72) - open/close 26922 (time 1761242301.54 total 335.21 last 66.48) - open/close 27600 (time 1761242311.54 total 345.21 last 67.80) - open/close 28380 (time 1761242321.54 total 355.21 last 77.96) - open/close 29121 (time 1761242331.56 total 365.22 last 74.01) - open/close 29884 (time 1761242341.56 total 375.23 last 76.23) - open/close 30660 (time 1761242351.57 total 385.24 last 77.59) - open/close 31439 (time 1761242361.57 total 395.24 last 77.87) - open/close 32238 (time 1761242371.57 total 405.24 last 79.88) - open/close 33022 (time 1761242381.58 total 415.25 last 78.37) - open/close 33778 (time 1761242391.58 total 425.25 last 75.58) - open/close 34528 (time 1761242401.59 total 435.26 last 74.95) - open/close 35293 (time 1761242411.59 total 445.26 last 76.46) - open/close 36067 (time 1761242421.59 total 455.26 last 77.40) - open/close 36785 (time 1761242431.60 total 465.27 last 71.77) - open/close 37526 (time 1761242441.60 total 475.27 last 74.07) - open/close 38231 (time 1761242451.61 total 485.28 last 70.45) - open/close 38956 (time 1761242461.62 total 495.29 last 72.40) - open/close 39718 (time 1761242471.63 total 505.30 last 76.16) - open/close 40000 (time 1761242474.98 total 508.65 last 83.97) - open/close 40731 (time 1761242484.99 total 518.66 last 73.03) - open/close 41525 (time 1761242495.00 total 528.67 last 79.39) - open/close 42257 (time 1761242505.01 total 538.68 last 73.11) - open/close 43064 (time 1761242515.02 total 548.69 last 80.61) - open/close 43871 (time 1761242525.03 total 558.69 last 80.65) - open/close 44798 (time 1761242535.03 total 568.70 last 92.66) - open/close 45634 (time 1761242545.04 total 578.71 last 83.55) - open/close 46493 (time 1761242555.04 total 588.71 last 85.86) - open/close 47347 (time 1761242565.04 total 598.71 last 85.37) - open/close 48132 (time 1761242575.05 total 608.72 last 78.43) - open/close 48946 (time 1761242585.06 total 618.73 last 81.33) - open/close 49682 (time 1761242595.07 total 628.73 last 73.57) - open/close 50000 (time 1761242599.26 total 632.93 last 75.85) - open/close 50561 (time 1761242609.27 total 642.94 last 56.05) - open/close 51198 (time 1761242619.27 total 652.94 last 63.70) - open/close 52015 (time 1761242629.27 total 662.94 last 81.68) - open/close 52881 (time 1761242639.28 total 672.95 last 86.52) - open/close 53798 (time 1761242649.29 total 682.96 last 91.59) - open/close 54770 (time 1761242659.30 total 692.97 last 97.12) - open/close 55554 (time 1761242669.31 total 702.98 last 78.34) - open/close 56322 (time 1761242679.32 total 712.99 last 76.68) - open/close 57237 (time 1761242689.33 total 723.00 last 91.46) - open/close 58085 (time 1761242699.33 total 733.00 last 84.79) - open/close 58917 (time 1761242709.33 total 743.00 last 83.19) - open/close 59780 (time 1761242719.34 total 753.01 last 86.18) - open/close 60000 (time 1761242721.98 total 755.65 last 83.43) - open/close 60878 (time 1761242731.99 total 765.66 last 87.70) - open/close 61719 (time 1761242741.99 total 775.66 last 84.07) - open/close 62506 (time 1761242752.01 total 785.68 last 78.56) - open/close 63304 (time 1761242762.03 total 795.70 last 79.67) - open/close 64140 (time 1761242772.03 total 805.70 last 83.59) total: 64768 open/close in 813.01 seconds: 79.66 ops/second - /unlink 1549 (time 1761242793.25 total 10.01 last 154.74) - /unlink 3033 (time 1761242803.26 total 20.02 last 148.31) - /unlink 4399 (time 1761242813.26 total 30.02 last 136.56) - /unlink 5737 (time 1761242823.27 total 40.02 last 133.76) - /unlink 7157 (time 1761242833.27 total 50.02 last 141.97) - /unlink 8628 (time 1761242843.27 total 60.02 last 147.10) - /unlink 10000 (time 1761242852.32 total 69.08 last 151.52) - /unlink 11501 (time 1761242862.33 total 79.08 last 150.02) - /unlink 13235 (time 1761242872.33 total 89.09 last 173.30) - /unlink 14874 (time 1761242882.34 total 99.10 last 163.81) - /unlink 16237 (time 1761242892.35 total 109.11 last 136.14) - /unlink 17507 (time 1761242902.36 total 119.11 last 126.95) - /unlink 18809 (time 1761242912.36 total 129.12 last 130.14) - /unlink 20000 (time 1761242920.64 total 137.39 last 143.93) - /unlink 21484 (time 1761242930.64 total 147.40 last 148.33) - /unlink 22758 (time 1761242940.64 total 157.40 last 127.36) - /unlink 24207 (time 1761242950.64 total 167.40 last 144.89) - /unlink 25729 (time 1761242960.65 total 177.40 last 152.18) - /unlink 27282 (time 1761242970.65 total 187.41 last 155.22) - /unlink 28659 (time 1761242980.65 total 197.41 last 137.70) - /unlink 30000 (time 1761242990.53 total 207.29 last 135.69) - /unlink 31363 (time 1761243000.54 total 217.29 last 136.26) - /unlink 32770 (time 1761243010.54 total 227.30 last 140.64) - /unlink 34188 (time 1761243020.54 total 237.30 last 141.75) - /unlink 35601 (time 1761243030.55 total 247.30 last 141.24) - /unlink 37012 (time 1761243040.56 total 257.31 last 140.96) - /unlink 38430 (time 1761243050.56 total 267.32 last 141.72) - /unlink 39980 (time 1761243060.56 total 277.32 last 155.00) - /unlink 41431 (time 1761243070.57 total 287.32 last 145.05) - /unlink 42892 (time 1761243080.57 total 297.33 last 146.05) - /unlink 44391 (time 1761243090.57 total 307.33 last 149.89) - /unlink 45728 (time 1761243100.57 total 317.33 last 133.69) - /unlink 46966 (time 1761243110.59 total 327.35 last 123.52) - /unlink 48277 (time 1761243120.61 total 337.36 last 130.94) - /unlink 49646 (time 1761243130.62 total 347.38 last 136.73) - /unlink 50000 (time 1761243133.06 total 349.82 last 145.12) - /unlink 51457 (time 1761243143.06 total 359.82 last 145.63) - /unlink 53098 (time 1761243153.06 total 369.82 last 164.08) - /unlink 54802 (time 1761243163.07 total 379.83 last 170.33) - /unlink 56492 (time 1761243173.07 total 389.83 last 168.98) - /unlink 58388 (time 1761243183.07 total 399.83 last 189.53) - /unlink 60000 (time 1761243191.82 total 408.58 last 184.21) - /unlink 61449 (time 1761243201.84 total 418.59 last 144.73) - /unlink 62954 (time 1761243211.84 total 428.60 last 150.45) - /unlink 64335 (time 1761243221.85 total 438.60 last 138.00) total: 64768 /unlink in 442.28 seconds: 146.44 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 860 (time 1761243263.01 total 10.01 last 85.96) - open/close 1865 (time 1761243273.01 total 20.01 last 100.45) - open/close 2825 (time 1761243283.02 total 30.02 last 95.93) - open/close 3603 (time 1761243293.03 total 40.02 last 77.75) - open/close 4356 (time 1761243303.03 total 50.03 last 75.28) - open/close 5236 (time 1761243313.03 total 60.03 last 87.98) - open/close 6109 (time 1761243323.04 total 70.03 last 87.29) - open/close 6955 (time 1761243333.05 total 80.05 last 84.47) - open/close 7735 (time 1761243343.06 total 90.05 last 77.95) - open/close 8590 (time 1761243353.07 total 100.06 last 85.41) - open/close 9495 (time 1761243363.07 total 110.07 last 90.45) - open/close 10000 (time 1761243368.93 total 115.93 last 86.17) - open/close 10771 (time 1761243378.94 total 125.93 last 77.06) - open/close 11677 (time 1761243388.95 total 135.94 last 90.50) - open/close 12582 (time 1761243398.95 total 145.95 last 90.45) - open/close 13485 (time 1761243408.96 total 155.95 last 90.26) - open/close 14298 (time 1761243418.96 total 165.96 last 81.29) - open/close 15136 (time 1761243428.97 total 175.96 last 83.74) - open/close 16012 (time 1761243438.97 total 185.97 last 87.55) - open/close 16948 (time 1761243448.98 total 195.98 last 93.51) - open/close 17872 (time 1761243458.98 total 205.98 last 92.39) - open/close 18811 (time 1761243468.99 total 215.98 last 93.86) - open/close 19792 (time 1761243478.99 total 225.99 last 98.07) - open/close 20000 (time 1761243481.71 total 228.71 last 76.38) - open/close 20847 (time 1761243491.72 total 238.72 last 84.64) - open/close 21694 (time 1761243501.73 total 248.72 last 84.64) - open/close 22542 (time 1761243511.73 total 258.73 last 84.78) - open/close 23349 (time 1761243521.74 total 268.73 last 80.66) - open/close 24209 (time 1761243531.74 total 278.73 last 85.98) - open/close 25043 (time 1761243541.76 total 288.76 last 83.20) - open/close 25897 (time 1761243551.77 total 298.76 last 85.36) - open/close 26736 (time 1761243561.79 total 308.79 last 83.71) - open/close 27562 (time 1761243571.80 total 318.80 last 82.52) - open/close 28357 (time 1761243581.81 total 328.80 last 79.45) - open/close 29167 (time 1761243591.82 total 338.81 last 80.94) - open/close 29906 (time 1761243601.82 total 348.81 last 73.89) - open/close 30706 (time 1761243611.82 total 358.82 last 79.96) - open/close 31461 (time 1761243621.83 total 368.82 last 75.45) - open/close 32187 (time 1761243631.85 total 378.84 last 72.47) - open/close 32952 (time 1761243641.85 total 388.85 last 76.46) - open/close 33722 (time 1761243651.87 total 398.87 last 76.82) - open/close 34346 (time 1761243661.89 total 408.88 last 62.31) - open/close 34979 (time 1761243671.90 total 418.89 last 63.25) - open/close 35653 (time 1761243681.90 total 428.90 last 67.35) - open/close 36355 (time 1761243691.93 total 438.93 last 70.01) - open/close 37132 (time 1761243701.94 total 448.93 last 77.65) - open/close 37916 (time 1761243711.94 total 458.94 last 78.36) - open/close 38669 (time 1761243721.96 total 468.95 last 75.19) - open/close 39381 (time 1761243731.96 total 478.95 last 71.19) - open/close 40000 (time 1761243740.62 total 487.61 last 71.46) - open/close 40715 (time 1761243750.62 total 497.62 last 71.49) - open/close 41462 (time 1761243760.63 total 507.62 last 74.65) - open/close 42228 (time 1761243770.64 total 517.63 last 76.51) - open/close 42928 (time 1761243780.65 total 527.64 last 69.92) - open/close 43618 (time 1761243790.66 total 537.65 last 68.94) - open/close 44345 (time 1761243800.67 total 547.66 last 72.63) - open/close 45037 (time 1761243810.68 total 557.68 last 69.11) - open/close 45777 (time 1761243820.69 total 567.68 last 73.95) - open/close 46513 (time 1761243830.69 total 577.69 last 73.58) - open/close 47172 (time 1761243840.70 total 587.69 last 65.85) - open/close 47883 (time 1761243850.70 total 597.69 last 71.09) - open/close 48601 (time 1761243860.70 total 607.70 last 71.78) - open/close 49315 (time 1761243870.72 total 617.71 last 71.29) - open/close 50000 (time 1761243879.50 total 626.49 last 78.00) - open/close 50717 (time 1761243889.50 total 636.50 last 71.66) - open/close 51412 (time 1761243899.52 total 646.51 last 69.41) - open/close 52101 (time 1761243909.52 total 656.52 last 68.88) - open/close 52745 (time 1761243919.52 total 666.52 last 64.40) - open/close 53479 (time 1761243929.52 total 676.52 last 73.39) - open/close 54178 (time 1761243939.53 total 686.53 last 69.85) - open/close 54970 (time 1761243949.53 total 696.53 last 79.19) - open/close 55646 (time 1761243959.53 total 706.53 last 67.59) - open/close 56359 (time 1761243969.54 total 716.53 last 71.26) - open/close 57060 (time 1761243979.55 total 726.54 last 70.04) - open/close 57755 (time 1761243989.55 total 736.55 last 69.48) - open/close 58501 (time 1761243999.56 total 746.56 last 74.51) - open/close 59278 (time 1761244009.57 total 756.56 last 77.67) - open/close 60000 (time 1761244019.34 total 766.34 last 73.85) - open/close 60629 (time 1761244029.35 total 776.35 last 62.84) - open/close 61307 (time 1761244039.37 total 786.36 last 67.68) - open/close 62021 (time 1761244049.39 total 796.39 last 71.23) - open/close 62698 (time 1761244059.40 total 806.40 last 67.65) - open/close 63446 (time 1761244069.40 total 816.40 last 74.77) - open/close 64171 (time 1761244079.41 total 826.41 last 72.46) total: 64768 open/close in 835.17 seconds: 77.55 ops/second - /unlink 1109 (time 1761244102.13 total 10.00 last 110.89) - /unlink 2199 (time 1761244112.14 total 20.01 last 108.87) - /unlink 3340 (time 1761244122.14 total 30.02 last 114.08) - /unlink 4434 (time 1761244132.15 total 40.02 last 109.34) - /unlink 5561 (time 1761244142.16 total 50.04 last 112.53) - /unlink 6716 (time 1761244152.17 total 60.04 last 115.45) - /unlink 7877 (time 1761244162.17 total 70.04 last 116.04) - /unlink 9036 (time 1761244172.18 total 80.05 last 115.87) - /unlink 10000 (time 1761244180.34 total 88.22 last 118.02) - /unlink 11130 (time 1761244190.35 total 98.22 last 112.90) - /unlink 12375 (time 1761244200.36 total 108.23 last 124.44) - /unlink 13669 (time 1761244210.37 total 118.24 last 129.29) - /unlink 14783 (time 1761244220.37 total 128.24 last 111.35) - /unlink 15862 (time 1761244230.37 total 138.24 last 107.89) - /unlink 16971 (time 1761244240.38 total 148.25 last 110.83) - /unlink 18267 (time 1761244250.38 total 158.25 last 129.56) - /unlink 19367 (time 1761244260.39 total 168.26 last 109.92) - /unlink 20000 (time 1761244265.40 total 173.28 last 126.18) - /unlink 21290 (time 1761244275.41 total 183.28 last 128.95) - /unlink 22650 (time 1761244285.41 total 193.28 last 135.92) - /unlink 23992 (time 1761244295.41 total 203.29 last 134.18) - /unlink 25487 (time 1761244305.42 total 213.29 last 149.47) - /unlink 26902 (time 1761244315.42 total 223.29 last 141.46) - /unlink 28267 (time 1761244325.43 total 233.30 last 136.40) - /unlink 29654 (time 1761244335.43 total 243.30 last 138.62) - /unlink 30000 (time 1761244337.81 total 245.68 last 145.32) - /unlink 31556 (time 1761244347.82 total 255.69 last 155.53) - /unlink 33132 (time 1761244357.82 total 265.69 last 157.52) - /unlink 34589 (time 1761244367.82 total 275.69 last 145.69) - /unlink 35806 (time 1761244377.83 total 285.70 last 121.64) - /unlink 37105 (time 1761244387.83 total 295.70 last 129.87) - /unlink 38448 (time 1761244397.83 total 305.70 last 134.28) - /unlink 39805 (time 1761244407.84 total 315.71 last 135.56) - /unlink 41143 (time 1761244417.85 total 325.72 last 133.75) - /unlink 42686 (time 1761244427.85 total 335.72 last 154.19) - /unlink 44158 (time 1761244437.87 total 345.74 last 146.94) - /unlink 45709 (time 1761244447.87 total 355.74 last 155.09) - /unlink 47076 (time 1761244457.87 total 365.75 last 136.65) - /unlink 48379 (time 1761244467.88 total 375.75 last 130.27) - /unlink 49783 (time 1761244477.88 total 385.75 last 140.36) - /unlink 51212 (time 1761244487.88 total 395.75 last 142.87) - /unlink 52536 (time 1761244497.88 total 405.76 last 132.38) - /unlink 53983 (time 1761244507.88 total 415.76 last 144.68) - /unlink 55607 (time 1761244517.89 total 425.76 last 162.37) - /unlink 57133 (time 1761244527.89 total 435.76 last 152.51) - /unlink 58518 (time 1761244537.89 total 445.77 last 138.46) - /unlink 60000 (time 1761244546.91 total 454.78 last 164.47) - /unlink 61673 (time 1761244556.91 total 464.78 last 167.28) - /unlink 63215 (time 1761244566.91 total 474.78 last 154.20) - /unlink 64676 (time 1761244576.91 total 484.78 last 146.08) total: 64768 /unlink in 485.40 seconds: 133.43 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 931 (time 1761244611.33 total 10.05 last 92.66) - open/close 1936 (time 1761244621.35 total 20.06 last 100.35) - open/close 2964 (time 1761244631.35 total 30.07 last 102.72) - open/close 3871 (time 1761244641.35 total 40.07 last 90.70) - open/close 4801 (time 1761244651.36 total 50.08 last 92.95) - open/close 5597 (time 1761244661.37 total 60.09 last 79.49) - open/close 6269 (time 1761244671.38 total 70.09 last 67.18) - open/close 7032 (time 1761244681.38 total 80.10 last 76.26) - open/close 7839 (time 1761244691.40 total 90.11 last 80.59) - open/close 8655 (time 1761244701.40 total 100.12 last 81.53) - open/close 9494 (time 1761244711.41 total 110.13 last 83.81) - open/close 10000 (time 1761244717.88 total 116.60 last 78.24) - open/close 10911 (time 1761244727.89 total 126.61 last 91.04) - open/close 11758 (time 1761244737.89 total 136.61 last 84.70) - open/close 12616 (time 1761244747.90 total 146.61 last 85.75) - open/close 13478 (time 1761244757.90 total 156.62 last 86.13) - open/close 14231 (time 1761244767.92 total 166.63 last 75.21) - open/close 15052 (time 1761244777.92 total 176.64 last 82.07) - open/close 15915 (time 1761244787.92 total 186.64 last 86.27) - open/close 16804 (time 1761244797.93 total 196.64 last 88.86) - open/close 17552 (time 1761244807.93 total 206.65 last 74.77) - open/close 18309 (time 1761244817.93 total 216.65 last 75.69) - open/close 19267 (time 1761244827.93 total 226.65 last 95.76) - open/close 20000 (time 1761244836.28 total 234.99 last 87.88) - open/close 20859 (time 1761244846.28 total 245.00 last 85.84) - open/close 21625 (time 1761244856.29 total 255.01 last 76.55) - open/close 22468 (time 1761244866.30 total 265.01 last 84.24) - open/close 23285 (time 1761244876.31 total 275.03 last 81.59) - open/close 24166 (time 1761244886.32 total 285.04 last 88.00) - open/close 24989 (time 1761244896.33 total 295.05 last 82.21) - open/close 25939 (time 1761244906.34 total 305.06 last 94.94) - open/close 26819 (time 1761244916.35 total 315.06 last 87.93) - open/close 27622 (time 1761244926.35 total 325.07 last 80.25) - open/close 28493 (time 1761244936.36 total 335.08 last 87.04) - open/close 29398 (time 1761244946.36 total 345.08 last 90.46) - open/close 30000 (time 1761244953.94 total 352.66 last 79.43) - open/close 30849 (time 1761244963.95 total 362.67 last 84.83) - open/close 31652 (time 1761244973.95 total 372.67 last 80.28) - open/close 32634 (time 1761244983.95 total 382.67 last 98.19) - open/close 33546 (time 1761244993.96 total 392.68 last 91.15) - open/close 34496 (time 1761245003.97 total 402.68 last 94.95) - open/close 35469 (time 1761245013.97 total 412.68 last 97.27) - open/close 36234 (time 1761245023.98 total 422.70 last 76.42) - open/close 37123 (time 1761245033.99 total 432.70 last 88.83) - open/close 38073 (time 1761245043.99 total 442.71 last 94.94) - open/close 39052 (time 1761245053.99 total 452.71 last 97.88) - open/close 39910 (time 1761245064.00 total 462.71 last 85.78) - open/close 40776 (time 1761245074.00 total 472.72 last 86.57) - open/close 41546 (time 1761245084.01 total 482.72 last 76.95) - open/close 42359 (time 1761245094.01 total 492.73 last 81.24) - open/close 43220 (time 1761245104.02 total 502.74 last 86.04) - open/close 44076 (time 1761245114.02 total 512.74 last 85.59) - open/close 44994 (time 1761245124.03 total 522.75 last 91.71) - open/close 45851 (time 1761245134.03 total 532.75 last 85.69) - open/close 46757 (time 1761245144.04 total 542.75 last 90.58) - open/close 47669 (time 1761245154.05 total 552.77 last 91.05) - open/close 48505 (time 1761245164.06 total 562.78 last 83.53) - open/close 49266 (time 1761245174.08 total 572.79 last 75.99) - open/close 50000 (time 1761245183.65 total 582.37 last 76.69) - open/close 50776 (time 1761245193.66 total 592.38 last 77.50) - open/close 51514 (time 1761245203.68 total 602.39 last 73.69) - open/close 52321 (time 1761245213.68 total 612.40 last 80.64) - open/close 53033 (time 1761245223.69 total 622.40 last 71.19) - open/close 53771 (time 1761245233.69 total 632.41 last 73.76) - open/close 54534 (time 1761245243.69 total 642.41 last 76.29) - open/close 55286 (time 1761245253.69 total 652.41 last 75.20) - open/close 56089 (time 1761245263.70 total 662.42 last 80.26) - open/close 56857 (time 1761245273.71 total 672.43 last 76.71) - open/close 57624 (time 1761245283.72 total 682.44 last 76.63) - open/close 58404 (time 1761245293.72 total 692.44 last 77.98) - open/close 59239 (time 1761245303.73 total 702.45 last 83.44) - open/close 60000 (time 1761245313.74 total 712.45 last 76.05) - open/close 60774 (time 1761245323.74 total 722.46 last 77.37) - open/close 61622 (time 1761245333.74 total 732.46 last 84.80) - open/close 62424 (time 1761245343.75 total 742.46 last 80.15) - open/close 63208 (time 1761245353.80 total 752.51 last 78.01) - open/close 64158 (time 1761245363.81 total 762.52 last 94.91) total: 64768 open/close in 769.72 seconds: 84.14 ops/second - /unlink 1517 (time 1761245384.05 total 10.01 last 151.52) - /unlink 2909 (time 1761245394.06 total 20.02 last 139.12) - /unlink 4402 (time 1761245404.06 total 30.02 last 149.26) - /unlink 5677 (time 1761245414.06 total 40.02 last 127.47) - /unlink 6922 (time 1761245424.07 total 50.02 last 124.47) - /unlink 8317 (time 1761245434.07 total 60.03 last 139.48) - /unlink 9714 (time 1761245444.07 total 70.03 last 139.62) - /unlink 10000 (time 1761245446.40 total 72.36 last 123.10) - /unlink 11388 (time 1761245456.43 total 82.39 last 138.36) - /unlink 12747 (time 1761245466.45 total 92.41 last 135.62) - /unlink 14157 (time 1761245476.45 total 102.41 last 140.92) - /unlink 15517 (time 1761245486.46 total 112.42 last 135.95) - /unlink 16900 (time 1761245496.46 total 122.42 last 138.29) - /unlink 18234 (time 1761245506.46 total 132.42 last 133.30) - /unlink 19468 (time 1761245516.47 total 142.43 last 123.32) - /unlink 20000 (time 1761245520.72 total 146.68 last 125.18) - /unlink 21290 (time 1761245530.73 total 156.69 last 128.84) - /unlink 22587 (time 1761245540.73 total 166.69 last 129.68) - /unlink 23923 (time 1761245550.73 total 176.69 last 133.60) - /unlink 25307 (time 1761245560.74 total 186.70 last 138.28) - /unlink 26593 (time 1761245570.75 total 196.71 last 128.54) - /unlink 27809 (time 1761245580.75 total 206.71 last 121.53) - /unlink 29031 (time 1761245590.76 total 216.72 last 122.16) - /unlink 30000 (time 1761245598.17 total 224.13 last 130.67) - /unlink 31394 (time 1761245608.18 total 234.14 last 139.33) - /unlink 32757 (time 1761245618.19 total 244.15 last 136.17) - /unlink 34162 (time 1761245628.19 total 254.15 last 140.43) - /unlink 35530 (time 1761245638.20 total 264.16 last 136.76) - /unlink 36971 (time 1761245648.20 total 274.16 last 144.09) - /unlink 38360 (time 1761245658.20 total 284.16 last 138.87) - /unlink 39652 (time 1761245668.20 total 294.16 last 129.14) - /unlink 40000 (time 1761245670.41 total 296.37 last 157.65) - /unlink 41484 (time 1761245680.41 total 306.37 last 148.35) - /unlink 42910 (time 1761245690.41 total 316.37 last 142.58) - /unlink 44359 (time 1761245700.42 total 326.38 last 144.81) - /unlink 45716 (time 1761245710.42 total 336.38 last 135.66) - /unlink 47188 (time 1761245720.43 total 346.39 last 147.14) - /unlink 48479 (time 1761245730.43 total 356.39 last 129.02) - /unlink 49829 (time 1761245740.44 total 366.40 last 134.94) - /unlink 51229 (time 1761245750.44 total 376.40 last 139.99) - /unlink 52642 (time 1761245760.44 total 386.40 last 141.24) - /unlink 53944 (time 1761245770.44 total 396.40 last 130.18) - /unlink 55203 (time 1761245780.45 total 406.41 last 125.80) - /unlink 56560 (time 1761245790.46 total 416.42 last 135.64) - /unlink 58429 (time 1761245800.46 total 426.42 last 186.79) - /unlink 59962 (time 1761245810.46 total 436.42 last 153.30) - /unlink 61434 (time 1761245820.47 total 446.43 last 147.16) - /unlink 62904 (time 1761245830.47 total 456.43 last 146.96) - /unlink 64245 (time 1761245840.47 total 466.43 last 134.03) total: 64768 /unlink in 470.10 seconds: 137.77 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg601-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg601-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg601-server stop mds service on oleg601-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg601-server stop mds service on oleg601-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg601-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg601-server' oleg601-server: oleg601-server.virtnet: executing unload_modules_local oleg601-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg601-server: LNET ready to unload oleg601-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. pdsh@oleg601-client: oleg601-client: ssh exited with exit code 2 pdsh@oleg601-client: oleg601-server: ssh exited with exit code 2 pdsh@oleg601-client: oleg601-client: ssh exited with exit code 2 pdsh@oleg601-client: oleg601-server: ssh exited with exit code 2 PASS 106 (6606s) Stopping clients: oleg601-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg601-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg601-client: oleg601-server: ssh exited with exit code 2 oleg601-server: oleg601-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg601-server' oleg601-server: oleg601-server.virtnet: executing load_modules_local oleg601-server: Loading modules from /home/green/git/lustre-release/lustre oleg601-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg601-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg601-server: detected 4 online CPUs by sysfs oleg601-server: Force libcfs to create 2 CPU partitions oleg601-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg601-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 6839 sec ========== 14:59:30 (1761245970)