-----============= acceptance-small: conf-sanity ============----- Fri Oct 24 10:05:36 EDT 2025 oleg429-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball 110 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg429-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg429-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg429-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg429-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg429-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg429-server oleg429-server: oleg429-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg429-server' oleg429-server: oleg429-server.virtnet: executing load_modules_local oleg429-server: Loading modules from /home/green/git/lustre-release/lustre oleg429-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg429-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg429-server: detected 4 online CPUs by sysfs oleg429-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg429-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg429-server: oleg429-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg429-client: oleg429-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg429-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg429-server: oleg429-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg429-client: oleg429-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg429-server: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg429-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg429-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg429-server stop mds service on oleg429-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg429-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 10:07:51 (1761314871) Stopping clients: oleg429-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg429-client.virtnet /mnt/lustre2 (opts:-f) oleg429-server: oleg429-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg429-server' oleg429-server: oleg429-server.virtnet: executing load_modules_local oleg429-server: Loading modules from /home/green/git/lustre-release/lustre oleg429-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg429-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg429-server: detected 4 online CPUs by sysfs oleg429-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg429-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg429-server: oleg429-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg429-client: oleg429-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg429-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg429-server: oleg429-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg429-client: oleg429-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg429-client.virtnet: -o user_xattr,flock oleg429-server@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 16166 358 15808 3% /mnt/lustre[OST:0] filesystem_summary: 16137 329 15808 3% /mnt/lustre fail_loc=0x1312 fail_val=5 - open/close 789 (time 1761314963.55 total 10.00 last 78.86) - open/close 1565 (time 1761314973.57 total 20.02 last 77.46) - open/close 2289 (time 1761314983.57 total 30.03 last 72.36) - open/close 2999 (time 1761314993.58 total 40.03 last 70.96) - open/close 3602 (time 1761315003.58 total 50.04 last 60.28) - open/close 4295 (time 1761315013.59 total 60.04 last 69.26) - open/close 5010 (time 1761315023.59 total 70.05 last 71.47) - open/close 5771 (time 1761315033.60 total 80.06 last 76.03) - open/close 6510 (time 1761315043.61 total 90.07 last 73.82) - open/close 7248 (time 1761315053.61 total 100.07 last 73.80) - open/close 7916 (time 1761315063.62 total 110.08 last 66.70) - open/close 8661 (time 1761315073.63 total 120.08 last 74.49) - open/close 9387 (time 1761315083.64 total 130.09 last 72.53) - open/close 10000 (time 1761315092.21 total 138.66 last 71.52) - open/close 10694 (time 1761315102.21 total 148.67 last 69.36) - open/close 11477 (time 1761315112.23 total 158.68 last 78.18) - open/close 12245 (time 1761315122.24 total 168.69 last 76.74) - open/close 13018 (time 1761315132.25 total 178.71 last 77.20) - open/close 13654 (time 1761315142.26 total 188.72 last 63.53) - open/close 14344 (time 1761315152.27 total 198.72 last 68.96) - open/close 14965 (time 1761315162.27 total 208.73 last 62.04) - open/close 15634 (time 1761315172.28 total 218.74 last 66.84) - open/close 16388 (time 1761315182.29 total 228.75 last 75.34) - open/close 17093 (time 1761315192.30 total 238.75 last 70.45) - open/close 17849 (time 1761315202.31 total 248.76 last 75.55) - open/close 18572 (time 1761315212.32 total 258.78 last 72.20) - open/close 19226 (time 1761315222.33 total 268.78 last 65.36) - open/close 19931 (time 1761315232.34 total 278.79 last 70.44) - open/close 20647 (time 1761315242.34 total 288.80 last 71.56) - open/close 21379 (time 1761315252.34 total 298.80 last 73.18) - open/close 22116 (time 1761315262.36 total 308.81 last 73.61) - open/close 22797 (time 1761315272.36 total 318.82 last 68.06) - open/close 23518 (time 1761315282.37 total 328.83 last 72.04) - open/close 24214 (time 1761315292.38 total 338.83 last 69.57) - open/close 24940 (time 1761315302.38 total 348.84 last 72.57) - open/close 25661 (time 1761315312.39 total 358.85 last 72.02) - open/close 26380 (time 1761315322.40 total 368.86 last 71.82) - open/close 27080 (time 1761315332.42 total 378.87 last 69.91) - open/close 27745 (time 1761315342.42 total 388.88 last 66.47) - open/close 28472 (time 1761315352.42 total 398.88 last 72.68) - open/close 29162 (time 1761315362.43 total 408.89 last 68.94) - open/close 29880 (time 1761315372.44 total 418.90 last 71.74) - open/close 30575 (time 1761315382.45 total 428.91 last 69.43) - open/close 31270 (time 1761315392.46 total 438.92 last 69.43) - open/close 31928 (time 1761315402.47 total 448.92 last 65.77) - open/close 32598 (time 1761315412.48 total 458.94 last 66.89) - open/close 33289 (time 1761315422.48 total 468.94 last 69.09) - open/close 33941 (time 1761315432.50 total 478.95 last 65.12) - open/close 34619 (time 1761315442.51 total 488.97 last 67.71) - open/close 35294 (time 1761315452.52 total 498.97 last 67.44) - open/close 35899 (time 1761315462.55 total 509.00 last 60.33) - open/close 36462 (time 1761315472.58 total 519.04 last 56.11) - open/close 37009 (time 1761315482.58 total 529.04 last 54.68) - open/close 37578 (time 1761315492.59 total 539.05 last 56.85) - open/close 38261 (time 1761315502.59 total 549.05 last 68.29) - open/close 38938 (time 1761315512.60 total 559.06 last 67.65) - open/close 39640 (time 1761315522.61 total 569.06 last 70.16) - open/close 40000 (time 1761315528.21 total 574.67 last 64.25) - open/close 40627 (time 1761315538.21 total 584.67 last 62.67) - open/close 41209 (time 1761315548.23 total 594.69 last 58.10) - open/close 41788 (time 1761315558.24 total 604.69 last 57.87) - open/close 42448 (time 1761315568.24 total 614.70 last 65.95) - open/close 43100 (time 1761315578.26 total 624.72 last 65.08) - open/close 43814 (time 1761315588.27 total 634.73 last 71.33) - open/close 44485 (time 1761315598.28 total 644.74 last 67.02) - open/close 45113 (time 1761315608.29 total 654.74 last 62.76) - open/close 45774 (time 1761315618.31 total 664.77 last 65.93) - open/close 46470 (time 1761315628.31 total 674.77 last 69.59) - open/close 47186 (time 1761315638.32 total 684.78 last 71.53) - open/close 47865 (time 1761315648.33 total 694.78 last 67.89) - open/close 48565 (time 1761315658.33 total 704.79 last 69.95) - open/close 49223 (time 1761315668.34 total 714.79 last 65.78) - open/close 49815 (time 1761315678.34 total 724.79 last 59.20) - open/close 50000 (time 1761315681.32 total 727.78 last 61.98) - open/close 50616 (time 1761315691.32 total 737.78 last 61.59) - open/close 51270 (time 1761315701.33 total 747.79 last 65.36) - open/close 51894 (time 1761315711.34 total 757.80 last 62.33) - open/close 52578 (time 1761315721.35 total 767.80 last 68.36) - open/close 53234 (time 1761315731.36 total 777.82 last 65.51) - open/close 53883 (time 1761315741.37 total 787.82 last 64.84) - open/close 54507 (time 1761315751.38 total 797.83 last 62.36) - open/close 55157 (time 1761315761.38 total 807.83 last 65.00) - open/close 55799 (time 1761315771.38 total 817.84 last 64.18) - open/close 56415 (time 1761315781.38 total 827.84 last 61.58) - open/close 57087 (time 1761315791.38 total 837.84 last 67.19) - open/close 57792 (time 1761315801.39 total 847.84 last 70.49) - open/close 58441 (time 1761315811.39 total 857.84 last 64.89) - open/close 59107 (time 1761315821.40 total 867.86 last 66.52) - open/close 59775 (time 1761315831.41 total 877.86 last 66.74) - open/close 60000 (time 1761315834.93 total 881.39 last 63.84) - open/close 60642 (time 1761315844.94 total 891.40 last 64.15) - open/close 61344 (time 1761315854.95 total 901.41 last 70.12) - open/close 62073 (time 1761315864.96 total 911.42 last 72.82) - open/close 62785 (time 1761315874.97 total 921.43 last 71.14) - open/close 63454 (time 1761315884.99 total 931.45 last 66.77) - open/close 64020 (time 1761315895.00 total 941.46 last 56.54) - open/close 64531 (time 1761315905.02 total 951.48 last 50.98) total: 64768 open/close in 956.00 seconds: 67.75 ops/second - /unlink 1145 (time 1761315924.07 total 10.00 last 114.50) - /unlink 2279 (time 1761315934.11 total 20.04 last 113.00) - /unlink 3409 (time 1761315944.11 total 30.04 last 112.97) - /unlink 4503 (time 1761315954.14 total 40.06 last 109.13) - /unlink 5578 (time 1761315964.14 total 50.07 last 107.46) - /unlink 6738 (time 1761315974.14 total 60.07 last 115.98) - /unlink 7841 (time 1761315984.15 total 70.08 last 110.21) - /unlink 9155 (time 1761315994.15 total 80.08 last 131.38) - /unlink 10000 (time 1761316002.16 total 88.09 last 105.53) - /unlink 11143 (time 1761316012.16 total 98.09 last 114.28) - /unlink 12225 (time 1761316022.17 total 108.10 last 108.09) - /unlink 13330 (time 1761316032.18 total 118.11 last 110.40) - /unlink 14501 (time 1761316042.19 total 128.11 last 117.03) - /unlink 15709 (time 1761316052.19 total 138.11 last 120.79) - /unlink 16875 (time 1761316062.19 total 148.12 last 116.56) - /unlink 18110 (time 1761316072.19 total 158.12 last 123.43) - /unlink 19384 (time 1761316082.19 total 168.12 last 127.40) - /unlink 20000 (time 1761316086.66 total 172.59 last 137.90) - /unlink 21190 (time 1761316096.67 total 182.59 last 118.95) - /unlink 22354 (time 1761316106.67 total 192.59 last 116.40) - /unlink 23518 (time 1761316116.67 total 202.60 last 116.33) - /unlink 24656 (time 1761316126.68 total 212.61 last 113.74) - /unlink 25760 (time 1761316136.68 total 222.61 last 110.35) - /unlink 26947 (time 1761316146.69 total 232.62 last 118.58) - /unlink 28083 (time 1761316156.70 total 242.62 last 113.55) - /unlink 29228 (time 1761316166.70 total 252.63 last 114.44) - /unlink 30000 (time 1761316173.70 total 259.62 last 110.38) - /unlink 31190 (time 1761316183.70 total 269.63 last 118.95) - /unlink 32526 (time 1761316193.70 total 279.63 last 133.54) - /unlink 33687 (time 1761316203.71 total 289.63 last 116.08) - /unlink 34692 (time 1761316213.71 total 299.64 last 100.44) - /unlink 35844 (time 1761316223.71 total 309.64 last 115.19) - /unlink 37026 (time 1761316233.71 total 319.64 last 118.20) - /unlink 38083 (time 1761316243.72 total 329.65 last 105.63) - /unlink 39245 (time 1761316253.72 total 339.65 last 116.19) - /unlink 40000 (time 1761316260.35 total 346.28 last 113.87) - /unlink 41235 (time 1761316270.36 total 356.29 last 123.38) - /unlink 42457 (time 1761316280.36 total 366.29 last 122.15) - /unlink 43724 (time 1761316290.37 total 376.30 last 126.61) - /unlink 44914 (time 1761316300.38 total 386.31 last 118.87) - /unlink 46228 (time 1761316310.39 total 396.32 last 131.28) - /unlink 47511 (time 1761316320.39 total 406.32 last 128.26) - /unlink 48660 (time 1761316330.40 total 416.32 last 114.88) - /unlink 49774 (time 1761316340.40 total 426.33 last 111.38) - /unlink 50000 (time 1761316342.57 total 428.49 last 104.24) - /unlink 51096 (time 1761316352.57 total 438.50 last 109.53) - /unlink 52354 (time 1761316362.57 total 448.50 last 125.79) - /unlink 53677 (time 1761316372.58 total 458.51 last 132.22) - /unlink 54922 (time 1761316382.58 total 468.51 last 124.48) - /unlink 56151 (time 1761316392.59 total 478.52 last 122.80) - /unlink 57485 (time 1761316402.59 total 488.52 last 133.39) - /unlink 58781 (time 1761316412.59 total 498.52 last 129.59) - /unlink 60000 (time 1761316421.99 total 507.92 last 129.69) - /unlink 61291 (time 1761316432.00 total 517.92 last 129.02) - /unlink 62683 (time 1761316442.00 total 527.93 last 139.17) - /unlink 63966 (time 1761316452.01 total 537.94 last 128.19) total: 64768 /unlink in 544.16 seconds: 119.02 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 671 (time 1761316491.89 total 10.00 last 67.10) - open/close 1277 (time 1761316501.90 total 20.01 last 60.54) - open/close 1935 (time 1761316511.91 total 30.02 last 65.74) - open/close 2606 (time 1761316521.92 total 40.03 last 67.04) - open/close 3306 (time 1761316531.92 total 50.03 last 69.97) - open/close 4047 (time 1761316541.92 total 60.03 last 74.08) - open/close 4756 (time 1761316551.93 total 70.04 last 70.84) - open/close 5477 (time 1761316561.94 total 80.05 last 72.04) - open/close 6191 (time 1761316571.94 total 90.06 last 71.36) - open/close 6883 (time 1761316581.95 total 100.06 last 69.19) - open/close 7579 (time 1761316591.96 total 110.07 last 69.52) - open/close 8329 (time 1761316601.96 total 120.08 last 74.95) - open/close 9091 (time 1761316611.97 total 130.08 last 76.19) - open/close 9792 (time 1761316621.98 total 140.10 last 69.98) - open/close 10000 (time 1761316625.04 total 143.15 last 68.07) - open/close 10721 (time 1761316635.05 total 153.16 last 72.03) - open/close 11436 (time 1761316645.06 total 163.18 last 71.39) - open/close 12143 (time 1761316655.07 total 173.19 last 70.63) - open/close 12839 (time 1761316665.08 total 183.19 last 69.55) - open/close 13448 (time 1761316675.09 total 193.20 last 60.84) - open/close 13908 (time 1761316685.13 total 203.24 last 45.80) - open/close 14311 (time 1761316695.14 total 213.25 last 40.28) - open/close 14960 (time 1761316705.14 total 223.25 last 64.87) - open/close 15612 (time 1761316715.15 total 233.27 last 65.10) - open/close 16305 (time 1761316725.16 total 243.27 last 69.27) - open/close 17041 (time 1761316735.17 total 253.28 last 73.52) - open/close 17765 (time 1761316745.18 total 263.29 last 72.32) - open/close 18407 (time 1761316755.18 total 273.29 last 64.20) - open/close 19109 (time 1761316765.19 total 283.30 last 70.17) - open/close 19773 (time 1761316775.19 total 293.31 last 66.34) - open/close 20000 (time 1761316778.73 total 296.84 last 64.25) - open/close 20646 (time 1761316788.74 total 306.85 last 64.54) - open/close 21273 (time 1761316798.74 total 316.86 last 62.66) - open/close 21968 (time 1761316808.76 total 326.87 last 69.39) - open/close 22642 (time 1761316818.77 total 336.88 last 67.36) - open/close 23343 (time 1761316828.77 total 346.89 last 70.04) - open/close 24066 (time 1761316838.78 total 356.89 last 72.24) - open/close 24796 (time 1761316848.80 total 366.91 last 72.90) - open/close 25519 (time 1761316858.81 total 376.92 last 72.23) - open/close 26219 (time 1761316868.81 total 386.92 last 69.99) - open/close 26901 (time 1761316878.81 total 396.92 last 68.19) - open/close 27549 (time 1761316888.82 total 406.94 last 64.69) - open/close 28198 (time 1761316898.83 total 416.94 last 64.86) - open/close 28883 (time 1761316908.84 total 426.95 last 68.46) - open/close 29551 (time 1761316918.85 total 436.96 last 66.69) - open/close 30000 (time 1761316925.17 total 443.28 last 71.08) - open/close 30670 (time 1761316935.18 total 453.29 last 66.94) - open/close 31314 (time 1761316945.18 total 463.29 last 64.39) - open/close 32024 (time 1761316955.19 total 473.30 last 70.94) - open/close 32683 (time 1761316965.20 total 483.31 last 65.86) - open/close 33359 (time 1761316975.20 total 493.31 last 67.60) - open/close 34036 (time 1761316985.21 total 503.32 last 67.62) - open/close 34762 (time 1761316995.21 total 513.32 last 72.57) - open/close 35468 (time 1761317005.23 total 523.34 last 70.51) - open/close 36135 (time 1761317015.24 total 533.35 last 66.61) - open/close 36817 (time 1761317025.24 total 543.35 last 68.20) - open/close 37522 (time 1761317035.24 total 553.35 last 70.48) - open/close 38228 (time 1761317045.24 total 563.36 last 70.59) - open/close 38940 (time 1761317055.26 total 573.37 last 71.11) - open/close 39655 (time 1761317065.26 total 583.38 last 71.44) - open/close 40000 (time 1761317070.17 total 588.28 last 70.37) - open/close 40690 (time 1761317080.18 total 598.29 last 68.90) - open/close 41395 (time 1761317090.19 total 608.30 last 70.46) - open/close 42131 (time 1761317100.19 total 618.30 last 73.56) - open/close 42894 (time 1761317110.19 total 628.31 last 76.28) - open/close 43635 (time 1761317120.20 total 638.31 last 74.08) - open/close 44376 (time 1761317130.21 total 648.32 last 74.02) - open/close 45162 (time 1761317140.22 total 658.33 last 78.49) - open/close 45888 (time 1761317150.23 total 668.34 last 72.53) - open/close 46654 (time 1761317160.25 total 678.36 last 76.44) - open/close 47408 (time 1761317170.26 total 688.37 last 75.34) - open/close 48149 (time 1761317180.27 total 698.38 last 74.04) - open/close 48883 (time 1761317190.28 total 708.39 last 73.33) - open/close 49656 (time 1761317200.29 total 718.40 last 77.19) - open/close 50000 (time 1761317204.81 total 722.92 last 76.16) - open/close 50701 (time 1761317214.82 total 732.93 last 70.05) - open/close 51401 (time 1761317224.82 total 742.93 last 69.95) - open/close 52067 (time 1761317234.84 total 752.95 last 66.50) - open/close 52754 (time 1761317244.85 total 762.96 last 68.64) - open/close 53492 (time 1761317254.86 total 772.97 last 73.74) - open/close 54216 (time 1761317264.87 total 782.98 last 72.29) - open/close 54947 (time 1761317274.87 total 792.99 last 73.09) - open/close 55743 (time 1761317284.88 total 803.00 last 79.52) - open/close 56559 (time 1761317294.89 total 813.01 last 81.52) - open/close 57341 (time 1761317304.91 total 823.02 last 78.12) - open/close 58050 (time 1761317314.91 total 833.02 last 70.85) - open/close 58720 (time 1761317324.91 total 843.03 last 66.98) - open/close 59369 (time 1761317334.92 total 853.03 last 64.87) - open/close 59997 (time 1761317344.95 total 863.06 last 62.63) - open/close 60601 (time 1761317354.95 total 873.07 last 60.35) - open/close 61190 (time 1761317364.96 total 883.07 last 58.89) - open/close 61814 (time 1761317374.96 total 893.07 last 62.40) - open/close 62488 (time 1761317384.96 total 903.08 last 67.35) - open/close 63114 (time 1761317394.97 total 913.08 last 62.56) - open/close 63817 (time 1761317404.97 total 923.09 last 70.26) - open/close 64467 (time 1761317414.99 total 933.10 last 64.93) total: 64768 open/close in 937.45 seconds: 69.09 ops/second - /unlink 1097 (time 1761317432.93 total 10.01 last 109.63) - /unlink 2272 (time 1761317442.93 total 20.01 last 117.47) - /unlink 3440 (time 1761317452.93 total 30.01 last 116.74) - /unlink 4515 (time 1761317462.93 total 40.01 last 107.49) - /unlink 5576 (time 1761317472.94 total 50.02 last 106.07) - /unlink 6631 (time 1761317482.96 total 60.04 last 105.24) - /unlink 7678 (time 1761317492.97 total 70.05 last 104.64) - /unlink 8794 (time 1761317502.97 total 80.05 last 111.56) - /unlink 9829 (time 1761317512.99 total 90.07 last 103.32) - /unlink 10690 (time 1761317522.99 total 100.07 last 86.08) - /unlink 11593 (time 1761317533.00 total 110.08 last 90.25) - /unlink 12634 (time 1761317543.00 total 120.08 last 104.05) - /unlink 13728 (time 1761317553.00 total 130.08 last 109.37) - /unlink 14828 (time 1761317563.01 total 140.09 last 109.98) - /unlink 15925 (time 1761317573.01 total 150.09 last 109.65) - /unlink 17092 (time 1761317583.01 total 160.09 last 116.66) - /unlink 18459 (time 1761317593.02 total 170.10 last 136.60) - /unlink 19609 (time 1761317603.03 total 180.11 last 114.93) - /unlink 20000 (time 1761317606.19 total 183.27 last 123.48) - /unlink 21255 (time 1761317616.20 total 193.28 last 125.44) - /unlink 22477 (time 1761317626.20 total 203.28 last 122.18) - /unlink 23590 (time 1761317636.20 total 213.28 last 111.30) - /unlink 25010 (time 1761317646.20 total 223.28 last 141.99) - /unlink 26262 (time 1761317656.20 total 233.28 last 125.18) - /unlink 27347 (time 1761317666.21 total 243.29 last 108.42) - /unlink 28546 (time 1761317676.22 total 253.30 last 119.77) - /unlink 29795 (time 1761317686.23 total 263.31 last 124.84) - /unlink 30000 (time 1761317688.28 total 265.36 last 99.88) - /unlink 31301 (time 1761317698.28 total 275.36 last 130.08) - /unlink 32497 (time 1761317708.28 total 285.36 last 119.55) - /unlink 33629 (time 1761317718.29 total 295.37 last 113.15) - /unlink 34702 (time 1761317728.29 total 305.37 last 107.24) - /unlink 35872 (time 1761317738.30 total 315.38 last 116.98) - /unlink 37045 (time 1761317748.30 total 325.38 last 117.28) - /unlink 38074 (time 1761317758.34 total 335.42 last 102.48) - /unlink 39253 (time 1761317768.34 total 345.42 last 117.84) - /unlink 40000 (time 1761317775.35 total 352.43 last 106.68) - /unlink 41097 (time 1761317785.35 total 362.43 last 109.65) - /unlink 42217 (time 1761317795.35 total 372.43 last 111.99) - /unlink 43363 (time 1761317805.38 total 382.46 last 114.29) - /unlink 44478 (time 1761317815.38 total 392.46 last 111.47) - /unlink 45547 (time 1761317825.39 total 402.47 last 106.79) - /unlink 46717 (time 1761317835.39 total 412.48 last 116.96) - /unlink 47879 (time 1761317845.40 total 422.48 last 116.18) - /unlink 49040 (time 1761317855.40 total 432.48 last 116.08) - /unlink 50000 (time 1761317863.57 total 440.65 last 117.44) - /unlink 51156 (time 1761317873.58 total 450.66 last 115.54) - /unlink 52301 (time 1761317883.59 total 460.67 last 114.40) - /unlink 53449 (time 1761317893.59 total 470.67 last 114.77) - /unlink 54670 (time 1761317903.60 total 480.69 last 121.92) - /unlink 55779 (time 1761317913.62 total 490.70 last 110.78) - /unlink 56955 (time 1761317923.62 total 500.70 last 117.56) - /unlink 58216 (time 1761317933.63 total 510.71 last 125.98) - /unlink 59541 (time 1761317943.63 total 520.71 last 132.49) - /unlink 60000 (time 1761317946.76 total 523.84 last 146.42) - /unlink 61436 (time 1761317956.77 total 533.85 last 143.50) - /unlink 62669 (time 1761317966.78 total 543.86 last 123.24) - /unlink 63868 (time 1761317976.78 total 553.86 last 119.88) total: 64768 /unlink in 561.41 seconds: 115.37 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 702 (time 1761318017.90 total 10.01 last 70.16) - open/close 1371 (time 1761318027.91 total 20.02 last 66.84) - open/close 2078 (time 1761318037.91 total 30.02 last 70.64) - open/close 2800 (time 1761318047.93 total 40.04 last 72.07) - open/close 3504 (time 1761318057.94 total 50.05 last 70.32) - open/close 4218 (time 1761318067.95 total 60.06 last 71.38) - open/close 4888 (time 1761318077.95 total 70.06 last 66.96) - open/close 5587 (time 1761318087.96 total 80.07 last 69.87) - open/close 6287 (time 1761318097.97 total 90.08 last 69.88) - open/close 6995 (time 1761318107.99 total 100.10 last 70.72) - open/close 7718 (time 1761318118.00 total 110.11 last 72.22) - open/close 8433 (time 1761318128.01 total 120.12 last 71.41) - open/close 9107 (time 1761318138.03 total 130.14 last 67.27) - open/close 9832 (time 1761318148.03 total 140.14 last 72.45) - open/close 10000 (time 1761318150.29 total 142.40 last 74.41) - open/close 10692 (time 1761318160.30 total 152.41 last 69.13) - open/close 11445 (time 1761318170.31 total 162.42 last 75.25) - open/close 12139 (time 1761318180.32 total 172.43 last 69.32) - open/close 12805 (time 1761318190.33 total 182.44 last 66.51) - open/close 13477 (time 1761318200.34 total 192.45 last 67.16) - open/close 14091 (time 1761318210.35 total 202.46 last 61.31) - open/close 14720 (time 1761318220.36 total 212.47 last 62.84) - open/close 15481 (time 1761318230.37 total 222.48 last 76.07) - open/close 16193 (time 1761318240.39 total 232.50 last 71.07) - open/close 16845 (time 1761318250.39 total 242.50 last 65.18) - open/close 17487 (time 1761318260.39 total 252.50 last 64.17) - open/close 18129 (time 1761318270.40 total 262.51 last 64.17) - open/close 18799 (time 1761318280.41 total 272.52 last 66.94) - open/close 19453 (time 1761318290.41 total 282.52 last 65.36) - open/close 20000 (time 1761318298.67 total 290.78 last 66.24) - open/close 20599 (time 1761318308.68 total 300.79 last 59.83) - open/close 21279 (time 1761318318.69 total 310.80 last 67.94) - open/close 21872 (time 1761318328.71 total 320.82 last 59.21) - open/close 22546 (time 1761318338.71 total 330.82 last 67.37) - open/close 23252 (time 1761318348.71 total 340.82 last 70.58) - open/close 23979 (time 1761318358.72 total 350.83 last 72.67) - open/close 24678 (time 1761318368.73 total 360.83 last 69.86) - open/close 25333 (time 1761318378.73 total 370.84 last 65.48) - open/close 25992 (time 1761318388.73 total 380.84 last 65.87) - open/close 26664 (time 1761318398.74 total 390.85 last 67.14) - open/close 27340 (time 1761318408.76 total 400.87 last 67.50) - open/close 28013 (time 1761318418.76 total 410.87 last 67.27) - open/close 28708 (time 1761318428.76 total 420.87 last 69.48) - open/close 29427 (time 1761318438.78 total 430.89 last 71.76) - open/close 30000 (time 1761318447.45 total 439.56 last 66.14) - open/close 30683 (time 1761318457.46 total 449.57 last 68.23) - open/close 31344 (time 1761318467.47 total 459.58 last 66.03) - open/close 32049 (time 1761318477.48 total 469.59 last 70.40) - open/close 32735 (time 1761318487.49 total 479.60 last 68.53) - open/close 33444 (time 1761318497.49 total 489.60 last 70.90) - open/close 34157 (time 1761318507.49 total 499.60 last 71.30) - open/close 34860 (time 1761318517.50 total 509.61 last 70.28) - open/close 35542 (time 1761318527.50 total 519.61 last 68.19) - open/close 36192 (time 1761318537.50 total 529.61 last 64.98) - open/close 36803 (time 1761318547.51 total 539.62 last 61.07) - open/close 37469 (time 1761318557.51 total 549.62 last 66.55) - open/close 38135 (time 1761318567.52 total 559.63 last 66.55) - open/close 38859 (time 1761318577.53 total 569.64 last 72.36) - open/close 39569 (time 1761318587.53 total 579.64 last 70.99) - open/close 40000 (time 1761318594.31 total 586.42 last 63.53) - open/close 40642 (time 1761318604.31 total 596.42 last 64.18) - open/close 41292 (time 1761318614.33 total 606.44 last 64.90) - open/close 41936 (time 1761318624.34 total 616.45 last 64.32) - open/close 42592 (time 1761318634.35 total 626.46 last 65.54) - open/close 43305 (time 1761318644.35 total 636.46 last 71.28) - open/close 43954 (time 1761318654.37 total 646.48 last 64.81) - open/close 44598 (time 1761318664.37 total 656.48 last 64.39) - open/close 45328 (time 1761318674.37 total 666.48 last 72.99) - open/close 45997 (time 1761318684.39 total 676.50 last 66.75) - open/close 46646 (time 1761318694.39 total 686.50 last 64.90) - open/close 47280 (time 1761318704.40 total 696.51 last 63.35) - open/close 47894 (time 1761318714.43 total 706.54 last 61.20) - open/close 48482 (time 1761318724.44 total 716.55 last 58.74) - open/close 49125 (time 1761318734.45 total 726.56 last 64.29) - open/close 49772 (time 1761318744.46 total 736.57 last 64.61) - open/close 50000 (time 1761318747.93 total 740.04 last 65.77) - open/close 50677 (time 1761318757.95 total 750.06 last 67.55) - open/close 51350 (time 1761318767.95 total 760.06 last 67.29) - open/close 52016 (time 1761318777.96 total 770.07 last 66.53) - open/close 52692 (time 1761318787.97 total 780.08 last 67.53) - open/close 53348 (time 1761318797.98 total 790.09 last 65.54) - open/close 54035 (time 1761318808.00 total 800.11 last 68.55) - open/close 54700 (time 1761318818.02 total 810.13 last 66.40) - open/close 55342 (time 1761318828.02 total 820.13 last 64.17) - open/close 56026 (time 1761318838.03 total 830.14 last 68.33) - open/close 56671 (time 1761318848.04 total 840.15 last 64.43) - open/close 57364 (time 1761318858.05 total 850.16 last 69.23) - open/close 58096 (time 1761318868.06 total 860.17 last 73.10) - open/close 58791 (time 1761318878.07 total 870.18 last 69.43) - open/close 59505 (time 1761318888.08 total 880.19 last 71.34) - open/close 60000 (time 1761318895.06 total 887.17 last 70.88) - open/close 60665 (time 1761318905.07 total 897.18 last 66.49) - open/close 61387 (time 1761318915.07 total 907.18 last 72.14) - open/close 62071 (time 1761318925.07 total 917.18 last 68.40) - open/close 62807 (time 1761318935.09 total 927.20 last 73.50) - open/close 63495 (time 1761318945.10 total 937.21 last 68.73) - open/close 64160 (time 1761318955.10 total 947.21 last 66.50) total: 64768 open/close in 956.72 seconds: 67.70 ops/second - /unlink 1173 (time 1761318979.07 total 10.00 last 117.25) - /unlink 2367 (time 1761318989.08 total 20.02 last 119.23) - /unlink 3515 (time 1761318999.08 total 30.02 last 114.76) - /unlink 4860 (time 1761319009.09 total 40.03 last 134.41) - /unlink 6060 (time 1761319019.09 total 50.03 last 119.97) - /unlink 7278 (time 1761319029.10 total 60.03 last 121.78) - /unlink 8478 (time 1761319039.10 total 70.04 last 119.94) - /unlink 9721 (time 1761319049.11 total 80.04 last 124.23) - /unlink 10000 (time 1761319051.34 total 82.27 last 125.12) - /unlink 11210 (time 1761319061.34 total 92.28 last 120.93) - /unlink 12472 (time 1761319071.35 total 102.29 last 126.07) - /unlink 13739 (time 1761319081.36 total 112.30 last 126.61) - /unlink 14938 (time 1761319091.36 total 122.30 last 119.86) - /unlink 16135 (time 1761319101.37 total 132.31 last 119.57) - /unlink 17316 (time 1761319111.37 total 142.31 last 118.09) - /unlink 18474 (time 1761319121.39 total 152.33 last 115.65) - /unlink 19743 (time 1761319131.39 total 162.33 last 126.85) - /unlink 21009 (time 1761319141.40 total 172.33 last 126.54) - /unlink 22236 (time 1761319151.40 total 182.34 last 122.64) - /unlink 23472 (time 1761319161.41 total 192.34 last 123.54) - /unlink 24688 (time 1761319171.41 total 202.35 last 121.59) - /unlink 25919 (time 1761319181.41 total 212.35 last 123.04) - /unlink 27130 (time 1761319191.42 total 222.36 last 121.01) - /unlink 28334 (time 1761319201.43 total 232.36 last 120.33) - /unlink 29704 (time 1761319211.44 total 242.37 last 136.89) - /unlink 30000 (time 1761319213.92 total 244.86 last 118.97) - /unlink 31372 (time 1761319223.93 total 254.86 last 137.14) - /unlink 32894 (time 1761319233.93 total 264.87 last 152.19) - /unlink 34111 (time 1761319243.93 total 274.87 last 121.69) - /unlink 35622 (time 1761319253.93 total 284.87 last 151.05) - /unlink 37144 (time 1761319263.93 total 294.87 last 152.18) - /unlink 38824 (time 1761319273.93 total 304.87 last 167.98) - /unlink 40000 (time 1761319279.94 total 310.88 last 195.80) - /unlink 41517 (time 1761319289.95 total 320.89 last 151.52) - /unlink 43017 (time 1761319299.96 total 330.89 last 149.94) - /unlink 44612 (time 1761319309.96 total 340.90 last 159.44) - /unlink 46191 (time 1761319319.96 total 350.90 last 157.86) - /unlink 47707 (time 1761319329.97 total 360.90 last 151.56) - /unlink 49320 (time 1761319339.97 total 370.91 last 161.28) - /unlink 50000 (time 1761319344.58 total 375.52 last 147.29) - /unlink 51465 (time 1761319354.59 total 385.53 last 146.45) - /unlink 53122 (time 1761319364.59 total 395.53 last 165.65) - /unlink 54618 (time 1761319374.59 total 405.53 last 149.59) - /unlink 56223 (time 1761319384.59 total 415.53 last 160.48) - /unlink 57888 (time 1761319394.60 total 425.54 last 166.38) - /unlink 59654 (time 1761319404.61 total 435.54 last 176.50) - /unlink 61485 (time 1761319414.61 total 445.54 last 183.09) - /unlink 63667 (time 1761319424.61 total 455.54 last 218.19) total: 64768 /unlink in 461.56 seconds: 140.32 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1020 (time 1761319459.18 total 10.00 last 101.96) - open/close 2157 (time 1761319469.18 total 20.00 last 113.69) - open/close 3536 (time 1761319479.18 total 30.01 last 137.82) - open/close 4851 (time 1761319489.18 total 40.01 last 131.49) - open/close 6139 (time 1761319499.19 total 50.02 last 128.75) - open/close 7319 (time 1761319509.19 total 60.02 last 117.99) - open/close 8480 (time 1761319519.19 total 70.02 last 116.05) - open/close 9820 (time 1761319529.20 total 80.02 last 133.95) - open/close 11149 (time 1761319539.20 total 90.03 last 132.85) - open/close 12537 (time 1761319549.20 total 100.03 last 138.78) - open/close 13820 (time 1761319559.20 total 110.03 last 128.25) - open/close 15049 (time 1761319569.21 total 120.03 last 122.89) - open/close 16249 (time 1761319579.21 total 130.03 last 119.98) - open/close 17474 (time 1761319589.21 total 140.04 last 122.47) - open/close 18117 (time 1761319599.22 total 150.05 last 64.22) - open/close 18720 (time 1761319609.22 total 160.05 last 60.29) - open/close 19323 (time 1761319619.24 total 170.06 last 60.23) - open/close 19847 (time 1761319629.25 total 180.08 last 52.33) - open/close 20000 (time 1761319632.24 total 183.07 last 51.09) - open/close 20533 (time 1761319642.26 total 193.08 last 53.24) - open/close 21237 (time 1761319652.26 total 203.09 last 70.38) - open/close 21998 (time 1761319662.27 total 213.09 last 76.05) - open/close 22768 (time 1761319672.27 total 223.10 last 76.95) - open/close 23495 (time 1761319682.28 total 233.10 last 72.67) - open/close 24176 (time 1761319692.30 total 243.12 last 67.97) - open/close 24832 (time 1761319702.31 total 253.14 last 65.52) - open/close 25470 (time 1761319712.31 total 263.14 last 63.80) - open/close 26157 (time 1761319722.32 total 273.15 last 68.63) - open/close 26919 (time 1761319732.33 total 283.15 last 76.14) - open/close 27624 (time 1761319742.34 total 293.17 last 70.39) - open/close 28373 (time 1761319752.35 total 303.17 last 74.87) - open/close 29076 (time 1761319762.35 total 313.18 last 70.27) - open/close 29765 (time 1761319772.36 total 323.19 last 68.82) - open/close 30000 (time 1761319775.89 total 326.71 last 66.64) - open/close 30733 (time 1761319785.90 total 336.72 last 73.23) - open/close 31502 (time 1761319795.91 total 346.74 last 76.79) - open/close 32284 (time 1761319805.94 total 356.76 last 78.02) - open/close 32977 (time 1761319815.94 total 366.77 last 69.25) - open/close 33680 (time 1761319825.94 total 376.77 last 70.30) - open/close 34468 (time 1761319835.95 total 386.78 last 78.74) - open/close 35182 (time 1761319845.96 total 396.78 last 71.35) - open/close 35863 (time 1761319855.98 total 406.80 last 67.97) - open/close 36607 (time 1761319865.98 total 416.81 last 74.39) - open/close 37418 (time 1761319875.98 total 426.81 last 81.08) - open/close 38305 (time 1761319886.00 total 436.83 last 88.53) - open/close 39170 (time 1761319896.01 total 446.84 last 86.43) - open/close 39922 (time 1761319906.01 total 456.84 last 75.15) - open/close 40688 (time 1761319916.03 total 466.85 last 76.50) - open/close 41504 (time 1761319926.03 total 476.86 last 81.57) - open/close 42206 (time 1761319936.04 total 486.87 last 70.14) - open/close 42966 (time 1761319946.05 total 496.87 last 75.93) - open/close 43719 (time 1761319956.06 total 506.89 last 75.21) - open/close 44528 (time 1761319966.06 total 516.89 last 80.89) - open/close 45371 (time 1761319976.06 total 526.89 last 84.29) - open/close 46158 (time 1761319986.07 total 536.89 last 78.67) - open/close 47044 (time 1761319996.08 total 546.91 last 88.49) - open/close 47860 (time 1761320006.08 total 556.91 last 81.59) - open/close 48595 (time 1761320016.09 total 566.92 last 73.41) - open/close 49328 (time 1761320026.10 total 576.93 last 73.23) - open/close 50000 (time 1761320034.70 total 585.53 last 78.12) - open/close 50887 (time 1761320044.71 total 595.54 last 88.63) - open/close 51831 (time 1761320054.72 total 605.55 last 94.31) - open/close 52732 (time 1761320064.73 total 615.56 last 89.99) - open/close 53616 (time 1761320074.73 total 625.56 last 88.39) - open/close 54453 (time 1761320084.73 total 635.56 last 83.69) - open/close 55227 (time 1761320094.74 total 645.57 last 77.34) - open/close 56073 (time 1761320104.75 total 655.57 last 84.57) - open/close 56922 (time 1761320114.75 total 665.57 last 84.88) - open/close 57693 (time 1761320124.76 total 675.58 last 77.02) - open/close 58538 (time 1761320134.76 total 685.59 last 84.48) - open/close 59358 (time 1761320144.77 total 695.60 last 81.93) - open/close 60000 (time 1761320152.21 total 703.04 last 86.23) - open/close 60922 (time 1761320162.22 total 713.04 last 92.18) - open/close 61790 (time 1761320172.22 total 723.05 last 86.75) - open/close 62731 (time 1761320182.22 total 733.05 last 94.10) - open/close 63626 (time 1761320192.23 total 743.06 last 89.41) - open/close 64598 (time 1761320202.24 total 753.06 last 97.16) total: 64768 open/close in 754.57 seconds: 85.83 ops/second - /unlink 2082 (time 1761320215.97 total 10.00 last 208.16) - /unlink 4108 (time 1761320225.97 total 20.00 last 202.57) - /unlink 6014 (time 1761320235.98 total 30.01 last 190.51) - /unlink 7911 (time 1761320245.98 total 40.01 last 189.65) - /unlink 9620 (time 1761320255.98 total 50.02 last 170.80) - /unlink 10000 (time 1761320258.35 total 52.38 last 160.78) - /unlink 11459 (time 1761320268.35 total 62.38 last 145.90) - /unlink 12922 (time 1761320278.35 total 72.38 last 146.25) - /unlink 14357 (time 1761320288.36 total 82.39 last 143.38) - /unlink 15878 (time 1761320298.36 total 92.40 last 152.05) - /unlink 17735 (time 1761320308.37 total 102.40 last 185.59) - /unlink 19759 (time 1761320318.37 total 112.40 last 202.39) - /unlink 21285 (time 1761320328.37 total 122.41 last 152.54) - /unlink 22839 (time 1761320338.37 total 132.41 last 155.39) - /unlink 24583 (time 1761320348.38 total 142.41 last 174.34) - /unlink 26200 (time 1761320358.38 total 152.41 last 161.64) - /unlink 28000 (time 1761320368.38 total 162.42 last 179.95) - /unlink 30000 (time 1761320378.02 total 172.05 last 207.53) - /unlink 31769 (time 1761320388.02 total 182.05 last 176.87) - /unlink 34081 (time 1761320398.02 total 192.06 last 231.20) - /unlink 36335 (time 1761320408.02 total 202.06 last 225.36) - /unlink 38430 (time 1761320418.03 total 212.06 last 209.48) - /unlink 40000 (time 1761320425.03 total 219.06 last 224.20) - /unlink 42395 (time 1761320435.03 total 229.07 last 239.38) - /unlink 44533 (time 1761320445.04 total 239.07 last 213.71) - /unlink 46528 (time 1761320455.04 total 249.07 last 199.39) - /unlink 48479 (time 1761320465.05 total 259.08 last 195.05) - /unlink 50000 (time 1761320471.67 total 265.71 last 229.47) - /unlink 52086 (time 1761320481.68 total 275.71 last 208.55) - /unlink 54673 (time 1761320491.68 total 285.71 last 258.59) - /unlink 57251 (time 1761320501.68 total 295.72 last 257.69) - /unlink 59556 (time 1761320511.68 total 305.72 last 230.49) - /unlink 61793 (time 1761320521.69 total 315.72 last 223.66) - /unlink 63708 (time 1761320531.69 total 325.72 last 191.42) total: 64768 /unlink in 331.71 seconds: 195.26 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1258 (time 1761320564.42 total 10.01 last 125.69) - open/close 2525 (time 1761320574.43 total 20.01 last 126.64) - open/close 3859 (time 1761320584.43 total 30.02 last 133.37) - open/close 5388 (time 1761320594.43 total 40.02 last 152.85) - open/close 6866 (time 1761320604.44 total 50.02 last 147.74) - open/close 8134 (time 1761320614.44 total 60.03 last 126.76) - open/close 9352 (time 1761320624.44 total 70.03 last 121.77) - open/close 10000 (time 1761320629.27 total 74.85 last 134.30) - open/close 11072 (time 1761320639.27 total 84.86 last 107.16) - open/close 12445 (time 1761320649.28 total 94.86 last 137.20) - open/close 13665 (time 1761320659.28 total 104.86 last 121.99) - open/close 14984 (time 1761320669.29 total 114.88 last 131.73) - open/close 16161 (time 1761320679.30 total 124.88 last 117.66) - open/close 17287 (time 1761320689.30 total 134.88 last 112.59) - open/close 18588 (time 1761320699.33 total 144.91 last 129.74) - open/close 19957 (time 1761320709.33 total 154.92 last 136.79) - open/close 21207 (time 1761320719.34 total 164.93 last 124.87) - open/close 22454 (time 1761320729.35 total 174.93 last 124.64) - open/close 23827 (time 1761320739.36 total 184.94 last 137.17) - open/close 25035 (time 1761320749.36 total 194.95 last 120.73) - open/close 26705 (time 1761320759.37 total 204.95 last 166.90) - open/close 28270 (time 1761320769.37 total 214.95 last 156.50) - open/close 29947 (time 1761320779.37 total 224.96 last 167.63) - open/close 31583 (time 1761320789.38 total 234.96 last 163.55) - open/close 33656 (time 1761320799.38 total 244.96 last 207.30) - open/close 35234 (time 1761320809.38 total 254.97 last 157.75) - open/close 36954 (time 1761320819.38 total 264.97 last 171.97) - open/close 38556 (time 1761320829.39 total 274.97 last 160.15) - open/close 40000 (time 1761320838.98 total 284.56 last 150.51) - open/close 41512 (time 1761320848.98 total 294.57 last 151.16) - open/close 42988 (time 1761320858.98 total 304.57 last 147.57) - open/close 44517 (time 1761320868.99 total 314.57 last 152.84) - open/close 46047 (time 1761320878.99 total 324.58 last 152.95) - open/close 47446 (time 1761320888.99 total 334.58 last 139.88) - open/close 48995 (time 1761320899.00 total 344.59 last 154.76) - open/close 50000 (time 1761320905.34 total 350.92 last 158.62) - open/close 51860 (time 1761320915.34 total 360.93 last 185.92) - open/close 53422 (time 1761320925.34 total 370.93 last 156.17) - open/close 55076 (time 1761320935.34 total 380.93 last 165.39) - open/close 56581 (time 1761320945.35 total 390.94 last 150.41) - open/close 58004 (time 1761320955.36 total 400.94 last 142.18) - open/close 59661 (time 1761320965.36 total 410.95 last 165.62) - open/close 60000 (time 1761320967.47 total 413.06 last 160.76) - open/close 61724 (time 1761320977.48 total 423.06 last 172.33) - open/close 63293 (time 1761320987.48 total 433.07 last 156.80) total: 64768 open/close in 441.70 seconds: 146.63 ops/second - /unlink 3912 (time 1761321007.83 total 10.00 last 391.08) - /unlink 7035 (time 1761321017.83 total 20.00 last 312.29) - /unlink 10000 (time 1761321026.15 total 28.33 last 356.12) - /unlink 13375 (time 1761321036.15 total 38.33 last 337.47) - /unlink 16968 (time 1761321046.16 total 48.33 last 359.23) - /unlink 20000 (time 1761321055.03 total 57.21 last 341.70) - /unlink 23244 (time 1761321065.03 total 67.21 last 324.33) - /unlink 26739 (time 1761321075.03 total 77.21 last 349.50) - /unlink 30000 (time 1761321083.27 total 85.44 last 395.93) - /unlink 33707 (time 1761321093.27 total 95.44 last 370.67) - /unlink 37025 (time 1761321103.27 total 105.45 last 331.74) - /unlink 40000 (time 1761321112.79 total 114.96 last 312.57) - /unlink 43224 (time 1761321122.79 total 124.97 last 322.38) - /unlink 46283 (time 1761321132.79 total 134.97 last 305.80) - /unlink 49898 (time 1761321142.79 total 144.97 last 361.45) - /unlink 53511 (time 1761321152.79 total 154.97 last 361.28) - /unlink 57040 (time 1761321162.80 total 164.97 last 352.82) - /unlink 60000 (time 1761321168.74 total 170.91 last 498.32) - /unlink 64706 (time 1761321178.74 total 180.91 last 470.59) total: 64768 /unlink in 181.05 seconds: 357.74 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg429-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg429-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg429-server stop mds service on oleg429-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg429-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg429-server' oleg429-server: oleg429-server.virtnet: executing unload_modules_local oleg429-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg429-server: LNET ready to unload oleg429-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. pdsh@oleg429-client: oleg429-client: ssh exited with exit code 2 pdsh@oleg429-client: oleg429-server: ssh exited with exit code 2 pdsh@oleg429-client: oleg429-client: ssh exited with exit code 2 pdsh@oleg429-client: oleg429-server: ssh exited with exit code 2 PASS 106 (6354s) Stopping clients: oleg429-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg429-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg429-client: oleg429-server: ssh exited with exit code 2 oleg429-server: oleg429-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg429-server' oleg429-server: oleg429-server.virtnet: executing load_modules_local oleg429-server: Loading modules from /home/green/git/lustre-release/lustre oleg429-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg429-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg429-server: detected 4 online CPUs by sysfs oleg429-server: Force libcfs to create 2 CPU partitions oleg429-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg429-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 6518 sec ========== 11:54:16 (1761321256)