-----============= acceptance-small: conf-sanity ============----- Mon Dec 22 09:02:43 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg433-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg433-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg433-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg433-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg433-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg433-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg433-server oleg433-server: oleg433-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg433-server' oleg433-server: oleg433-server.virtnet: executing load_modules_local oleg433-server: Loading modules from /home/green/git/lustre-release/lustre oleg433-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg433-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg433-server: detected 4 online CPUs by sysfs oleg433-server: MODOPTS_LIBCFS= oleg433-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg433-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg433-server: oleg433-server.virtnet: executing set_default_debug -1 all pdsh@oleg433-client: oleg433-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg433-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg433-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg433-server: oleg433-server.virtnet: executing set_default_debug -1 all pdsh@oleg433-client: oleg433-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg433-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg433-server: oleg433-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg433-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg433-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg433-server stop mds service on oleg433-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg433-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 09:05:05 (1766412305) Stopping clients: oleg433-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg433-client.virtnet /mnt/lustre2 (opts:-f) oleg433-server: oleg433-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg433-server' oleg433-server: oleg433-server.virtnet: executing load_modules_local oleg433-server: Loading modules from /home/green/git/lustre-release/lustre oleg433-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg433-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg433-server: detected 4 online CPUs by sysfs oleg433-server: MODOPTS_LIBCFS= oleg433-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg433-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg433-server: oleg433-server.virtnet: executing set_default_debug -1 all pdsh@oleg433-client: oleg433-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg433-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg433-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg433-server: oleg433-server.virtnet: executing set_default_debug -1 all pdsh@oleg433-client: oleg433-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg433-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg433-client.virtnet: -o user_xattr,flock 192.168.204.133@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32648 392 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 589 (time 1766412409.93 total 10.02 last 58.80) - open/close 1251 (time 1766412419.94 total 20.03 last 66.13) - open/close 1831 (time 1766412429.95 total 30.03 last 57.98) - open/close 2499 (time 1766412439.95 total 40.03 last 66.78) - open/close 3211 (time 1766412449.95 total 50.03 last 71.20) - open/close 3950 (time 1766412459.96 total 60.04 last 73.84) - open/close 4738 (time 1766412469.96 total 70.04 last 78.79) - open/close 5581 (time 1766412479.96 total 80.05 last 84.28) - open/close 6328 (time 1766412489.96 total 90.05 last 74.69) - open/close 7100 (time 1766412499.97 total 100.06 last 77.13) - open/close 7788 (time 1766412509.98 total 110.07 last 68.73) - open/close 8529 (time 1766412519.99 total 120.07 last 74.05) - open/close 9230 (time 1766412529.99 total 130.08 last 70.06) - open/close 9877 (time 1766412540.00 total 140.09 last 64.65) - open/close 10543 (time 1766412550.01 total 150.10 last 66.53) - open/close 11328 (time 1766412560.01 total 160.10 last 78.48) - open/close 12064 (time 1766412570.02 total 170.11 last 73.53) - open/close 12821 (time 1766412580.03 total 180.12 last 75.65) - open/close 13627 (time 1766412590.05 total 190.13 last 80.44) - open/close 14437 (time 1766412600.06 total 200.14 last 80.95) - open/close 15240 (time 1766412610.07 total 210.15 last 80.21) - open/close 15984 (time 1766412620.07 total 220.16 last 74.35) - open/close 16762 (time 1766412630.08 total 230.17 last 77.75) - open/close 17465 (time 1766412640.09 total 240.18 last 70.23) - open/close 18150 (time 1766412650.10 total 250.19 last 68.43) - open/close 18719 (time 1766412660.10 total 260.19 last 56.90) - open/close 19406 (time 1766412670.11 total 270.20 last 68.64) - open/close 20000 (time 1766412678.29 total 278.37 last 72.66) - open/close 20739 (time 1766412688.29 total 288.37 last 73.88) - open/close 21495 (time 1766412698.30 total 298.38 last 75.54) - open/close 22231 (time 1766412708.30 total 308.38 last 73.57) - open/close 22942 (time 1766412718.30 total 318.39 last 71.09) - open/close 23700 (time 1766412728.30 total 328.39 last 75.77) - open/close 24437 (time 1766412738.31 total 338.40 last 73.65) - open/close 25152 (time 1766412748.32 total 348.40 last 71.45) - open/close 25884 (time 1766412758.33 total 358.41 last 73.14) - open/close 26551 (time 1766412768.34 total 368.42 last 66.62) - open/close 27159 (time 1766412778.35 total 378.44 last 60.71) - open/close 27865 (time 1766412788.37 total 388.45 last 70.51) - open/close 28558 (time 1766412798.38 total 398.46 last 69.20) - open/close 29267 (time 1766412808.38 total 408.47 last 70.88) - open/close 30000 (time 1766412817.95 total 418.04 last 76.58) - open/close 30706 (time 1766412827.96 total 428.04 last 70.56) - open/close 31360 (time 1766412837.97 total 438.06 last 65.33) - open/close 32018 (time 1766412847.98 total 448.07 last 65.73) - open/close 32753 (time 1766412857.98 total 458.07 last 73.49) - open/close 33433 (time 1766412867.99 total 468.08 last 67.93) - open/close 34177 (time 1766412878.00 total 478.08 last 74.38) - open/close 34811 (time 1766412888.02 total 488.10 last 63.29) - open/close 35417 (time 1766412898.02 total 498.11 last 60.54) - open/close 36041 (time 1766412908.03 total 508.11 last 62.39) - open/close 36699 (time 1766412918.03 total 518.11 last 65.77) - open/close 37398 (time 1766412928.03 total 528.12 last 69.88) - open/close 38044 (time 1766412938.04 total 538.13 last 64.53) - open/close 38725 (time 1766412948.05 total 548.14 last 68.04) - open/close 39344 (time 1766412958.07 total 558.15 last 61.82) - open/close 39971 (time 1766412968.07 total 568.15 last 62.69) - open/close 40645 (time 1766412978.09 total 578.18 last 67.24) - open/close 41278 (time 1766412988.10 total 588.19 last 63.22) - open/close 41987 (time 1766412998.11 total 598.20 last 70.83) - open/close 42650 (time 1766413008.12 total 608.21 last 66.25) - open/close 43296 (time 1766413018.14 total 618.22 last 64.49) - open/close 43974 (time 1766413028.14 total 628.23 last 67.76) - open/close 44656 (time 1766413038.17 total 638.25 last 68.05) - open/close 45367 (time 1766413048.18 total 648.26 last 71.03) - open/close 46051 (time 1766413058.18 total 658.27 last 68.34) - open/close 46672 (time 1766413068.19 total 668.27 last 62.08) - open/close 47217 (time 1766413078.25 total 678.34 last 54.15) - open/close 47715 (time 1766413088.26 total 688.35 last 49.75) - open/close 48312 (time 1766413098.27 total 698.35 last 59.68) - open/close 48990 (time 1766413108.27 total 708.35 last 67.78) - open/close 49628 (time 1766413118.27 total 718.36 last 63.78) - open/close 50000 (time 1766413123.38 total 723.46 last 72.90) - open/close 50599 (time 1766413133.38 total 733.47 last 59.87) - open/close 51200 (time 1766413143.38 total 743.47 last 60.08) - open/close 51792 (time 1766413153.40 total 753.48 last 59.12) - open/close 52470 (time 1766413163.40 total 763.49 last 67.75) - open/close 53160 (time 1766413173.42 total 773.50 last 68.91) - open/close 53838 (time 1766413183.43 total 783.51 last 67.74) - open/close 54512 (time 1766413193.44 total 793.52 last 67.33) - open/close 55209 (time 1766413203.44 total 803.52 last 69.68) - open/close 55857 (time 1766413213.44 total 813.53 last 64.77) - open/close 56467 (time 1766413223.46 total 823.55 last 60.89) - open/close 57096 (time 1766413233.46 total 833.55 last 62.89) - open/close 57723 (time 1766413243.47 total 843.56 last 62.64) - open/close 58313 (time 1766413253.47 total 853.56 last 58.99) - open/close 58922 (time 1766413263.48 total 863.56 last 60.90) - open/close 59611 (time 1766413273.49 total 873.57 last 68.83) - open/close 60000 (time 1766413279.46 total 879.54 last 65.16) - open/close 60680 (time 1766413289.48 total 889.56 last 67.86) - open/close 61353 (time 1766413299.48 total 899.57 last 67.27) - open/close 62057 (time 1766413309.51 total 909.60 last 70.17) - open/close 62738 (time 1766413319.52 total 919.60 last 68.08) - open/close 63417 (time 1766413329.53 total 929.61 last 67.84) - open/close 64101 (time 1766413339.54 total 939.62 last 68.30) total: 64768 open/close in 948.92 seconds: 68.25 ops/second - /unlink 689 (time 1766413362.89 total 10.00 last 68.87) - /unlink 1398 (time 1766413372.91 total 20.02 last 70.77) - /unlink 2072 (time 1766413382.91 total 30.02 last 67.38) - /unlink 2799 (time 1766413392.92 total 40.03 last 72.64) - /unlink 3528 (time 1766413402.92 total 50.04 last 72.85) - /unlink 4265 (time 1766413412.94 total 60.05 last 73.59) - /unlink 4990 (time 1766413422.94 total 70.06 last 72.46) - /unlink 5685 (time 1766413432.95 total 80.07 last 69.42) - /unlink 6391 (time 1766413442.96 total 90.08 last 70.55) - /unlink 7080 (time 1766413452.96 total 100.08 last 68.89) - /unlink 7779 (time 1766413462.97 total 110.09 last 69.82) - /unlink 8514 (time 1766413472.98 total 120.10 last 73.43) - /unlink 9159 (time 1766413482.99 total 130.10 last 64.49) - /unlink 9782 (time 1766413492.99 total 140.11 last 62.27) - /unlink 10000 (time 1766413495.76 total 142.87 last 78.78) - /unlink 10748 (time 1766413505.77 total 152.88 last 74.74) - /unlink 11440 (time 1766413515.77 total 162.89 last 69.15) - /unlink 12067 (time 1766413525.78 total 172.90 last 62.66) - /unlink 12734 (time 1766413535.81 total 182.93 last 66.49) - /unlink 13381 (time 1766413545.82 total 192.94 last 64.61) - /unlink 14072 (time 1766413555.83 total 202.95 last 69.07) - /unlink 14785 (time 1766413565.83 total 212.95 last 71.27) - /unlink 15482 (time 1766413575.83 total 222.95 last 69.69) - /unlink 16199 (time 1766413585.84 total 232.96 last 71.66) - /unlink 16895 (time 1766413595.84 total 242.96 last 69.60) - /unlink 17593 (time 1766413605.85 total 252.97 last 69.74) - /unlink 18329 (time 1766413615.85 total 262.97 last 73.58) - /unlink 19033 (time 1766413625.85 total 272.97 last 70.39) - /unlink 19729 (time 1766413635.86 total 282.97 last 69.57) - /unlink 20000 (time 1766413639.21 total 286.33 last 80.78) - /unlink 20732 (time 1766413649.22 total 296.34 last 73.14) - /unlink 21438 (time 1766413659.22 total 306.34 last 70.59) - /unlink 22146 (time 1766413669.23 total 316.35 last 70.71) - /unlink 22825 (time 1766413679.25 total 326.37 last 67.78) - /unlink 23519 (time 1766413689.26 total 336.37 last 69.36) - /unlink 24258 (time 1766413699.27 total 346.39 last 73.81) - /unlink 24984 (time 1766413709.28 total 356.39 last 72.54) - /unlink 25701 (time 1766413719.28 total 366.40 last 71.65) - /unlink 26433 (time 1766413729.29 total 376.40 last 73.17) - /unlink 27076 (time 1766413739.29 total 386.41 last 64.26) - /unlink 27784 (time 1766413749.30 total 396.42 last 70.73) - /unlink 28483 (time 1766413759.31 total 406.42 last 69.87) - /unlink 29202 (time 1766413769.32 total 416.43 last 71.83) - /unlink 29933 (time 1766413779.32 total 426.44 last 73.09) - /unlink 30619 (time 1766413789.33 total 436.44 last 68.55) - /unlink 31268 (time 1766413799.34 total 446.46 last 64.81) - /unlink 31898 (time 1766413809.34 total 456.46 last 62.99) - /unlink 32611 (time 1766413819.36 total 466.47 last 71.19) - /unlink 33319 (time 1766413829.36 total 476.48 last 70.76) - /unlink 33972 (time 1766413839.36 total 486.48 last 65.30) - /unlink 34604 (time 1766413849.37 total 496.49 last 63.15) - /unlink 35257 (time 1766413859.37 total 506.49 last 65.27) - /unlink 35910 (time 1766413869.38 total 516.49 last 65.28) - /unlink 36514 (time 1766413879.38 total 526.50 last 60.36) - /unlink 37129 (time 1766413889.39 total 536.51 last 61.45) - /unlink 37796 (time 1766413899.40 total 546.51 last 66.67) - /unlink 38448 (time 1766413909.40 total 556.52 last 65.15) - /unlink 39142 (time 1766413919.41 total 566.53 last 69.37) - /unlink 39857 (time 1766413929.41 total 576.53 last 71.50) - /unlink 40000 (time 1766413931.78 total 578.90 last 60.23) - /unlink 40653 (time 1766413941.86 total 588.97 last 64.83) - /unlink 41390 (time 1766413951.88 total 599.00 last 73.54) - /unlink 42009 (time 1766413961.88 total 609.00 last 61.89) - /unlink 42654 (time 1766413971.88 total 619.00 last 64.48) - /unlink 43266 (time 1766413981.89 total 629.01 last 61.14) - /unlink 43967 (time 1766413991.91 total 639.03 last 69.98) - /unlink 44668 (time 1766414001.91 total 649.03 last 70.07) - /unlink 45356 (time 1766414011.92 total 659.03 last 68.77) - /unlink 45991 (time 1766414021.92 total 669.04 last 63.49) - /unlink 46651 (time 1766414031.92 total 679.04 last 65.99) - /unlink 47296 (time 1766414041.93 total 689.05 last 64.46) - /unlink 47941 (time 1766414051.94 total 699.06 last 64.43) - /unlink 48566 (time 1766414061.94 total 709.06 last 62.47) - /unlink 49215 (time 1766414071.95 total 719.06 last 64.89) - /unlink 49831 (time 1766414081.95 total 729.07 last 61.57) - /unlink 50000 (time 1766414084.49 total 731.61 last 66.53) - /unlink 50617 (time 1766414094.51 total 741.62 last 61.61) - /unlink 51281 (time 1766414104.51 total 751.63 last 66.36) - /unlink 51964 (time 1766414114.52 total 761.64 last 68.24) - /unlink 52562 (time 1766414124.53 total 771.65 last 59.72) - /unlink 53237 (time 1766414134.54 total 781.66 last 67.45) - /unlink 53919 (time 1766414144.55 total 791.67 last 68.13) - /unlink 54643 (time 1766414154.56 total 801.68 last 72.32) - /unlink 55307 (time 1766414164.57 total 811.68 last 66.37) - /unlink 55895 (time 1766414174.57 total 821.69 last 58.79) - /unlink 56516 (time 1766414184.57 total 831.69 last 62.07) - /unlink 57143 (time 1766414194.58 total 841.69 last 62.68) - /unlink 57737 (time 1766414204.59 total 851.71 last 59.32) - /unlink 58311 (time 1766414214.60 total 861.72 last 57.36) - /unlink 59023 (time 1766414224.61 total 871.73 last 71.13) - /unlink 59676 (time 1766414234.62 total 881.74 last 65.23) - /unlink 60000 (time 1766414239.49 total 886.61 last 66.50) - /unlink 60632 (time 1766414249.49 total 896.61 last 63.18) - /unlink 61328 (time 1766414259.50 total 906.61 last 69.57) - /unlink 62005 (time 1766414269.51 total 916.62 last 67.64) - /unlink 62698 (time 1766414279.54 total 926.66 last 69.06) - /unlink 63397 (time 1766414289.55 total 936.67 last 69.84) - /unlink 64092 (time 1766414299.56 total 946.68 last 69.42) - /unlink 64743 (time 1766414309.57 total 956.69 last 65.06) total: 64768 /unlink in 957.02 seconds: 67.68 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 736 (time 1766414343.05 total 10.01 last 73.54) - open/close 1437 (time 1766414353.06 total 20.02 last 70.04) - open/close 2160 (time 1766414363.06 total 30.02 last 72.29) - open/close 2888 (time 1766414373.07 total 40.03 last 72.73) - open/close 3589 (time 1766414383.08 total 50.03 last 70.06) - open/close 4281 (time 1766414393.08 total 60.04 last 69.16) - open/close 4951 (time 1766414403.08 total 70.04 last 66.98) - open/close 5619 (time 1766414413.09 total 80.05 last 66.77) - open/close 6323 (time 1766414423.09 total 90.05 last 70.40) - open/close 7072 (time 1766414433.10 total 100.06 last 74.82) - open/close 7858 (time 1766414443.11 total 110.07 last 78.51) - open/close 8600 (time 1766414453.12 total 120.07 last 74.17) - open/close 9401 (time 1766414463.12 total 130.08 last 80.03) - open/close 10000 (time 1766414471.02 total 137.97 last 75.91) - open/close 10761 (time 1766414481.02 total 147.97 last 76.10) - open/close 11462 (time 1766414491.03 total 157.99 last 69.97) - open/close 12172 (time 1766414501.05 total 168.01 last 70.90) - open/close 12880 (time 1766414511.06 total 178.01 last 70.74) - open/close 13610 (time 1766414521.08 total 188.03 last 72.86) - open/close 14300 (time 1766414531.08 total 198.04 last 68.98) - open/close 15021 (time 1766414541.10 total 208.05 last 71.96) - open/close 15687 (time 1766414551.11 total 218.06 last 66.54) - open/close 16354 (time 1766414561.11 total 228.07 last 66.66) - open/close 17024 (time 1766414571.12 total 238.08 last 66.96) - open/close 17742 (time 1766414581.13 total 248.09 last 71.72) - open/close 18425 (time 1766414591.13 total 258.09 last 68.27) - open/close 19174 (time 1766414601.14 total 268.10 last 74.82) - open/close 19812 (time 1766414611.15 total 278.11 last 63.75) - open/close 20000 (time 1766414614.18 total 281.13 last 62.14) - open/close 20640 (time 1766414624.19 total 291.14 last 63.94) - open/close 21237 (time 1766414634.19 total 301.15 last 59.68) - open/close 21784 (time 1766414644.20 total 311.15 last 54.67) - open/close 22330 (time 1766414654.21 total 321.16 last 54.56) - open/close 22992 (time 1766414664.21 total 331.16 last 66.20) - open/close 23710 (time 1766414674.21 total 341.17 last 71.78) - open/close 24417 (time 1766414684.21 total 351.17 last 70.66) - open/close 25132 (time 1766414694.22 total 361.18 last 71.44) - open/close 25812 (time 1766414704.24 total 371.20 last 67.88) - open/close 26533 (time 1766414714.25 total 381.20 last 72.05) - open/close 27282 (time 1766414724.26 total 391.21 last 74.81) - open/close 28023 (time 1766414734.26 total 401.21 last 74.10) - open/close 28700 (time 1766414744.27 total 411.23 last 67.60) - open/close 29433 (time 1766414754.28 total 421.23 last 73.26) - open/close 30000 (time 1766414762.03 total 428.99 last 73.09) - open/close 30654 (time 1766414772.04 total 439.00 last 65.34) - open/close 31354 (time 1766414782.05 total 449.01 last 69.94) - open/close 32031 (time 1766414792.07 total 459.02 last 67.61) - open/close 32729 (time 1766414802.07 total 469.03 last 69.75) - open/close 33471 (time 1766414812.08 total 479.04 last 74.15) - open/close 34186 (time 1766414822.09 total 489.05 last 71.44) - open/close 34879 (time 1766414832.10 total 499.05 last 69.25) - open/close 35569 (time 1766414842.10 total 509.06 last 68.99) - open/close 36267 (time 1766414852.10 total 519.06 last 69.79) - open/close 36902 (time 1766414862.11 total 529.07 last 63.44) - open/close 37623 (time 1766414872.12 total 539.07 last 72.05) - open/close 38345 (time 1766414882.17 total 549.13 last 71.80) - open/close 39009 (time 1766414892.19 total 559.15 last 66.27) - open/close 39727 (time 1766414902.20 total 569.16 last 71.74) - open/close 40000 (time 1766414906.24 total 573.20 last 67.58) - open/close 40712 (time 1766414916.25 total 583.20 last 71.14) - open/close 41409 (time 1766414926.25 total 593.21 last 69.66) - open/close 42021 (time 1766414936.26 total 603.21 last 61.17) - open/close 42752 (time 1766414946.26 total 613.22 last 73.07) - open/close 43508 (time 1766414956.27 total 623.22 last 75.56) - open/close 44234 (time 1766414966.28 total 633.24 last 72.47) - open/close 44919 (time 1766414976.29 total 643.25 last 68.48) - open/close 45612 (time 1766414986.30 total 653.26 last 69.21) - open/close 46305 (time 1766414996.31 total 663.27 last 69.25) - open/close 46993 (time 1766415006.32 total 673.28 last 68.70) - open/close 47696 (time 1766415016.33 total 683.29 last 70.24) - open/close 48456 (time 1766415026.35 total 693.31 last 75.82) - open/close 49175 (time 1766415036.36 total 703.32 last 71.86) - open/close 49879 (time 1766415046.36 total 713.32 last 70.37) - open/close 50602 (time 1766415056.37 total 723.33 last 72.22) - open/close 51363 (time 1766415066.38 total 733.34 last 76.04) - open/close 52158 (time 1766415076.39 total 743.35 last 79.40) - open/close 52857 (time 1766415086.40 total 753.36 last 69.84) - open/close 53567 (time 1766415096.41 total 763.37 last 70.93) - open/close 54263 (time 1766415106.42 total 773.38 last 69.52) - open/close 54950 (time 1766415116.43 total 783.39 last 68.66) - open/close 55636 (time 1766415126.44 total 793.40 last 68.54) - open/close 56296 (time 1766415136.45 total 803.41 last 65.92) - open/close 56974 (time 1766415146.46 total 813.42 last 67.72) - open/close 57674 (time 1766415156.47 total 823.42 last 70.00) - open/close 58349 (time 1766415166.48 total 833.44 last 67.41) - open/close 59072 (time 1766415176.48 total 843.44 last 72.27) - open/close 59817 (time 1766415186.49 total 853.44 last 74.48) - open/close 60000 (time 1766415188.92 total 855.88 last 75.14) - open/close 60724 (time 1766415198.94 total 865.90 last 72.27) - open/close 61437 (time 1766415208.95 total 875.91 last 71.23) - open/close 62128 (time 1766415218.96 total 885.91 last 69.04) - open/close 62861 (time 1766415228.96 total 895.92 last 73.27) - open/close 63572 (time 1766415238.97 total 905.92 last 71.05) - open/close 64303 (time 1766415248.97 total 915.93 last 73.08) total: 64768 open/close in 922.97 seconds: 70.17 ops/second - /unlink 693 (time 1766415269.67 total 10.00 last 69.30) - /unlink 1398 (time 1766415279.68 total 20.01 last 70.41) - /unlink 2108 (time 1766415289.69 total 30.02 last 70.96) - /unlink 2838 (time 1766415299.70 total 40.03 last 72.94) - /unlink 3580 (time 1766415309.70 total 50.03 last 74.15) - /unlink 4305 (time 1766415319.72 total 60.05 last 72.38) - /unlink 4984 (time 1766415329.73 total 70.06 last 67.85) - /unlink 5672 (time 1766415339.73 total 80.06 last 68.79) - /unlink 6359 (time 1766415349.74 total 90.07 last 68.63) - /unlink 7073 (time 1766415359.74 total 100.07 last 71.40) - /unlink 7779 (time 1766415369.74 total 110.07 last 70.59) - /unlink 8485 (time 1766415379.74 total 120.07 last 70.59) - /unlink 9199 (time 1766415389.75 total 130.08 last 71.32) - /unlink 9934 (time 1766415399.76 total 140.08 last 73.48) - /unlink 10665 (time 1766415409.76 total 150.09 last 73.08) - /unlink 11343 (time 1766415419.77 total 160.09 last 67.75) - /unlink 12024 (time 1766415429.78 total 170.10 last 68.03) - /unlink 12724 (time 1766415439.79 total 180.12 last 69.92) - /unlink 13441 (time 1766415449.79 total 190.12 last 71.64) - /unlink 14135 (time 1766415459.80 total 200.13 last 69.39) - /unlink 14810 (time 1766415469.80 total 210.13 last 67.45) - /unlink 15408 (time 1766415479.81 total 220.14 last 59.76) - /unlink 16023 (time 1766415489.82 total 230.14 last 61.47) - /unlink 16689 (time 1766415499.82 total 240.15 last 66.56) - /unlink 17389 (time 1766415509.83 total 250.16 last 69.96) - /unlink 18128 (time 1766415519.83 total 260.16 last 73.89) - /unlink 18854 (time 1766415529.84 total 270.17 last 72.53) - /unlink 19552 (time 1766415539.84 total 280.17 last 69.79) - /unlink 20000 (time 1766415545.13 total 285.46 last 84.60) - /unlink 20720 (time 1766415555.13 total 295.46 last 72.00) - /unlink 21437 (time 1766415565.13 total 305.46 last 71.70) - /unlink 22206 (time 1766415575.14 total 315.47 last 76.84) - /unlink 22969 (time 1766415585.15 total 325.48 last 76.24) - /unlink 23755 (time 1766415595.15 total 335.48 last 78.59) - /unlink 24568 (time 1766415605.16 total 345.49 last 81.23) - /unlink 25284 (time 1766415615.16 total 355.49 last 71.58) - /unlink 25975 (time 1766415625.16 total 365.49 last 69.09) - /unlink 26648 (time 1766415635.17 total 375.50 last 67.25) - /unlink 27353 (time 1766415645.18 total 385.51 last 70.44) - /unlink 28075 (time 1766415655.18 total 395.51 last 72.17) - /unlink 28823 (time 1766415665.19 total 405.52 last 74.76) - /unlink 29593 (time 1766415675.22 total 415.55 last 76.76) - /unlink 30000 (time 1766415680.38 total 420.71 last 78.85) - /unlink 30662 (time 1766415690.40 total 430.73 last 66.07) - /unlink 31392 (time 1766415700.41 total 440.74 last 72.94) - /unlink 32173 (time 1766415710.41 total 450.74 last 78.07) - /unlink 32912 (time 1766415720.41 total 460.74 last 73.89) - /unlink 33742 (time 1766415730.42 total 470.75 last 82.96) - /unlink 34572 (time 1766415740.43 total 480.76 last 82.87) - /unlink 35260 (time 1766415750.45 total 490.77 last 68.72) - /unlink 36038 (time 1766415760.45 total 500.78 last 77.75) - /unlink 36634 (time 1766415770.45 total 510.78 last 59.59) - /unlink 37361 (time 1766415780.46 total 520.79 last 72.66) - /unlink 38057 (time 1766415790.46 total 530.79 last 69.58) - /unlink 38760 (time 1766415800.48 total 540.81 last 70.16) - /unlink 39466 (time 1766415810.49 total 550.82 last 70.53) - /unlink 40000 (time 1766415817.73 total 558.06 last 73.82) - /unlink 40687 (time 1766415827.74 total 568.07 last 68.63) - /unlink 41382 (time 1766415837.75 total 578.08 last 69.42) - /unlink 42121 (time 1766415847.76 total 588.09 last 73.84) - /unlink 42839 (time 1766415857.76 total 598.09 last 71.78) - /unlink 43565 (time 1766415867.77 total 608.10 last 72.55) - /unlink 44329 (time 1766415877.78 total 618.10 last 76.33) - /unlink 45057 (time 1766415887.79 total 628.12 last 72.71) - /unlink 45735 (time 1766415897.80 total 638.13 last 67.72) - /unlink 46516 (time 1766415907.81 total 648.13 last 78.05) - /unlink 47274 (time 1766415917.81 total 658.14 last 75.74) - /unlink 48004 (time 1766415927.83 total 668.16 last 72.90) - /unlink 48712 (time 1766415937.84 total 678.17 last 70.70) - /unlink 49401 (time 1766415947.84 total 688.17 last 68.87) - /unlink 50000 (time 1766415956.50 total 696.83 last 69.23) - /unlink 50695 (time 1766415966.50 total 706.83 last 69.47) - /unlink 51440 (time 1766415976.51 total 716.84 last 74.43) - /unlink 52178 (time 1766415986.52 total 726.85 last 73.72) - /unlink 52878 (time 1766415996.53 total 736.86 last 69.90) - /unlink 53526 (time 1766416006.54 total 746.87 last 64.75) - /unlink 54276 (time 1766416016.55 total 756.87 last 74.97) - /unlink 55128 (time 1766416026.56 total 766.89 last 85.09) - /unlink 55901 (time 1766416036.56 total 776.89 last 77.28) - /unlink 56696 (time 1766416046.58 total 786.91 last 79.38) - /unlink 57480 (time 1766416056.58 total 796.91 last 78.36) - /unlink 58314 (time 1766416066.58 total 806.91 last 83.38) - /unlink 59139 (time 1766416076.60 total 816.92 last 82.41) - /unlink 59844 (time 1766416086.62 total 826.95 last 70.35) - /unlink 60000 (time 1766416088.68 total 829.01 last 75.65) - /unlink 60762 (time 1766416098.68 total 839.01 last 76.18) - /unlink 61579 (time 1766416108.69 total 849.02 last 81.64) - /unlink 62530 (time 1766416118.70 total 859.03 last 94.99) - /unlink 63392 (time 1766416128.71 total 869.03 last 86.15) - /unlink 64291 (time 1766416138.71 total 879.04 last 89.85) total: 64768 /unlink in 884.96 seconds: 73.19 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 748 (time 1766416174.43 total 10.01 last 74.75) - open/close 1590 (time 1766416184.45 total 20.02 last 84.08) - open/close 2380 (time 1766416194.45 total 30.02 last 78.97) - open/close 3110 (time 1766416204.46 total 40.03 last 72.96) - open/close 3945 (time 1766416214.47 total 50.04 last 83.39) - open/close 4765 (time 1766416224.47 total 60.04 last 82.00) - open/close 5658 (time 1766416234.48 total 70.05 last 89.25) - open/close 6421 (time 1766416244.49 total 80.06 last 76.22) - open/close 7151 (time 1766416254.50 total 90.07 last 72.92) - open/close 7889 (time 1766416264.50 total 100.08 last 73.76) - open/close 8626 (time 1766416274.50 total 110.08 last 73.70) - open/close 9366 (time 1766416284.51 total 120.09 last 73.92) - open/close 10000 (time 1766416292.70 total 128.27 last 77.48) - open/close 10854 (time 1766416302.70 total 138.28 last 85.33) - open/close 11705 (time 1766416312.71 total 148.28 last 85.04) - open/close 12469 (time 1766416322.72 total 158.29 last 76.37) - open/close 13265 (time 1766416332.73 total 168.30 last 79.52) - open/close 14090 (time 1766416342.73 total 178.30 last 82.48) - open/close 14837 (time 1766416352.74 total 188.31 last 74.61) - open/close 15580 (time 1766416362.76 total 198.33 last 74.17) - open/close 16296 (time 1766416372.76 total 208.33 last 71.60) - open/close 17053 (time 1766416382.78 total 218.35 last 75.56) - open/close 17836 (time 1766416392.78 total 228.35 last 78.29) - open/close 18677 (time 1766416402.78 total 238.35 last 84.09) - open/close 19485 (time 1766416412.78 total 248.35 last 80.79) - open/close 20000 (time 1766416419.66 total 255.23 last 74.90) - open/close 20856 (time 1766416429.66 total 265.23 last 85.57) - open/close 21668 (time 1766416439.66 total 275.24 last 81.17) - open/close 22380 (time 1766416449.67 total 285.24 last 71.13) - open/close 23200 (time 1766416459.68 total 295.25 last 81.95) - open/close 24043 (time 1766416469.69 total 305.26 last 84.23) - open/close 24963 (time 1766416479.69 total 315.26 last 91.96) - open/close 25779 (time 1766416489.69 total 325.27 last 81.57) - open/close 26596 (time 1766416499.70 total 335.27 last 81.67) - open/close 27448 (time 1766416509.70 total 345.27 last 85.16) - open/close 28263 (time 1766416519.70 total 355.27 last 81.50) - open/close 29050 (time 1766416529.71 total 365.28 last 78.65) - open/close 29840 (time 1766416539.72 total 375.29 last 78.93) - open/close 30679 (time 1766416549.72 total 385.30 last 83.85) - open/close 31525 (time 1766416559.73 total 395.30 last 84.55) - open/close 32324 (time 1766416569.73 total 405.30 last 79.90) - open/close 33137 (time 1766416579.74 total 415.31 last 81.25) - open/close 33897 (time 1766416589.74 total 425.31 last 75.97) - open/close 34712 (time 1766416599.75 total 435.32 last 81.43) - open/close 35446 (time 1766416609.75 total 445.32 last 73.39) - open/close 36276 (time 1766416619.75 total 455.32 last 82.99) - open/close 37093 (time 1766416629.76 total 465.33 last 81.61) - open/close 37867 (time 1766416639.76 total 475.34 last 77.39) - open/close 38648 (time 1766416649.77 total 485.35 last 78.02) - open/close 39431 (time 1766416659.78 total 495.35 last 78.23) - open/close 40000 (time 1766416667.24 total 502.81 last 76.29) - open/close 40894 (time 1766416677.24 total 512.81 last 89.40) - open/close 41783 (time 1766416687.25 total 522.82 last 88.83) - open/close 42597 (time 1766416697.26 total 532.83 last 81.29) - open/close 43347 (time 1766416707.27 total 542.84 last 74.96) - open/close 44117 (time 1766416717.27 total 552.85 last 76.96) - open/close 44888 (time 1766416727.28 total 562.85 last 77.09) - open/close 45618 (time 1766416737.29 total 572.86 last 72.88) - open/close 46413 (time 1766416747.29 total 582.87 last 79.49) - open/close 47193 (time 1766416757.30 total 592.87 last 77.95) - open/close 47912 (time 1766416767.31 total 602.88 last 71.86) - open/close 48676 (time 1766416777.31 total 612.88 last 76.39) - open/close 49423 (time 1766416787.32 total 622.89 last 74.59) - open/close 50000 (time 1766416794.80 total 630.37 last 77.15) - open/close 50716 (time 1766416804.80 total 640.37 last 71.59) - open/close 51413 (time 1766416814.81 total 650.38 last 69.63) - open/close 52139 (time 1766416824.83 total 660.40 last 72.47) - open/close 52995 (time 1766416834.84 total 670.41 last 85.54) - open/close 53728 (time 1766416844.84 total 680.42 last 73.26) - open/close 54484 (time 1766416854.86 total 690.43 last 75.50) - open/close 55244 (time 1766416864.87 total 700.44 last 75.91) - open/close 56072 (time 1766416874.88 total 710.45 last 82.72) - open/close 56850 (time 1766416884.89 total 720.46 last 77.75) - open/close 57668 (time 1766416894.89 total 730.46 last 81.77) - open/close 58480 (time 1766416904.90 total 740.47 last 81.14) - open/close 59405 (time 1766416914.90 total 750.47 last 92.46) - open/close 60000 (time 1766416921.89 total 757.46 last 85.17) - open/close 61074 (time 1766416931.89 total 767.46 last 107.36) - open/close 62121 (time 1766416941.90 total 777.47 last 104.64) - open/close 63178 (time 1766416951.91 total 787.48 last 105.57) - open/close 64382 (time 1766416961.91 total 797.49 last 120.32) total: 64768 open/close in 801.06 seconds: 80.85 ops/second - /unlink 1161 (time 1766416977.57 total 10.01 last 116.02) - /unlink 2147 (time 1766416987.58 total 20.01 last 98.58) - /unlink 3171 (time 1766416997.58 total 30.01 last 102.37) - /unlink 4267 (time 1766417007.59 total 40.02 last 109.53) - /unlink 5409 (time 1766417017.59 total 50.02 last 114.17) - /unlink 6535 (time 1766417027.59 total 60.02 last 112.59) - /unlink 7557 (time 1766417037.59 total 70.02 last 102.19) - /unlink 8654 (time 1766417047.60 total 80.03 last 109.64) - /unlink 9865 (time 1766417057.60 total 90.03 last 121.09) - /unlink 10943 (time 1766417067.61 total 100.04 last 107.71) - /unlink 12130 (time 1766417077.61 total 110.04 last 118.66) - /unlink 13362 (time 1766417087.61 total 120.05 last 123.14) - /unlink 14953 (time 1766417097.62 total 130.05 last 159.01) - /unlink 16138 (time 1766417107.62 total 140.05 last 118.46) - /unlink 17104 (time 1766417117.62 total 150.06 last 96.59) - /unlink 18244 (time 1766417127.63 total 160.06 last 113.95) - /unlink 19466 (time 1766417137.63 total 170.06 last 122.20) - /unlink 20000 (time 1766417142.18 total 174.61 last 117.35) - /unlink 21186 (time 1766417152.18 total 184.61 last 118.60) - /unlink 22584 (time 1766417162.18 total 194.61 last 139.77) - /unlink 23958 (time 1766417172.18 total 204.61 last 137.40) - /unlink 25095 (time 1766417182.18 total 214.62 last 113.66) - /unlink 26534 (time 1766417192.19 total 224.62 last 143.84) - /unlink 28067 (time 1766417202.19 total 234.62 last 153.25) - /unlink 29320 (time 1766417212.19 total 244.63 last 125.27) - /unlink 30000 (time 1766417217.02 total 249.46 last 140.78) - /unlink 31545 (time 1766417227.03 total 259.46 last 154.48) - /unlink 32896 (time 1766417237.03 total 269.46 last 135.05) - /unlink 34223 (time 1766417247.03 total 279.47 last 132.65) - /unlink 35618 (time 1766417257.04 total 289.47 last 139.41) - /unlink 36939 (time 1766417267.04 total 299.47 last 132.09) - /unlink 38304 (time 1766417277.05 total 309.48 last 136.43) - /unlink 39539 (time 1766417287.05 total 319.48 last 123.43) - /unlink 40000 (time 1766417290.49 total 322.92 last 133.95) - /unlink 41255 (time 1766417300.49 total 332.92 last 125.50) - /unlink 42584 (time 1766417310.49 total 342.93 last 132.90) - /unlink 44221 (time 1766417320.50 total 352.93 last 163.65) - /unlink 46638 (time 1766417330.50 total 362.93 last 241.66) - /unlink 49359 (time 1766417340.50 total 372.93 last 272.08) - /unlink 50000 (time 1766417343.16 total 375.59 last 241.03) - /unlink 52282 (time 1766417353.16 total 385.59 last 228.13) - /unlink 54709 (time 1766417363.16 total 395.60 last 242.65) - /unlink 57402 (time 1766417373.17 total 405.60 last 269.18) - /unlink 60000 (time 1766417382.25 total 414.68 last 285.98) - /unlink 62732 (time 1766417392.25 total 424.69 last 273.15) total: 64768 /unlink in 432.45 seconds: 149.77 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1544 (time 1766417431.66 total 10.00 last 154.35) - open/close 3088 (time 1766417441.66 total 20.01 last 154.34) - open/close 4648 (time 1766417451.66 total 30.01 last 155.97) - open/close 6223 (time 1766417461.67 total 40.01 last 157.49) - open/close 8124 (time 1766417471.67 total 50.01 last 190.06) - open/close 10000 (time 1766417481.59 total 59.94 last 189.03) - open/close 11913 (time 1766417491.60 total 69.94 last 191.21) - open/close 13411 (time 1766417501.60 total 79.95 last 149.69) - open/close 14779 (time 1766417511.60 total 89.95 last 136.79) - open/close 16533 (time 1766417521.61 total 99.95 last 175.37) - open/close 18126 (time 1766417531.61 total 109.95 last 159.29) - open/close 19873 (time 1766417541.61 total 119.95 last 174.63) - open/close 21416 (time 1766417551.62 total 129.96 last 154.23) - open/close 23078 (time 1766417561.62 total 139.96 last 166.18) - open/close 24762 (time 1766417571.62 total 149.96 last 168.40) - open/close 26257 (time 1766417581.62 total 159.97 last 149.40) - open/close 27751 (time 1766417591.63 total 169.97 last 149.37) - open/close 29263 (time 1766417601.63 total 179.97 last 151.12) - open/close 30000 (time 1766417606.34 total 184.68 last 156.56) - open/close 31843 (time 1766417616.34 total 194.68 last 184.25) - open/close 33571 (time 1766417626.34 total 204.69 last 172.75) - open/close 35144 (time 1766417636.35 total 214.69 last 157.25) - open/close 36968 (time 1766417646.35 total 224.69 last 182.35) - open/close 38636 (time 1766417656.35 total 234.70 last 166.74) - open/close 40000 (time 1766417665.35 total 243.69 last 151.62) - open/close 41623 (time 1766417675.35 total 253.69 last 162.29) - open/close 43189 (time 1766417685.36 total 263.71 last 156.42) - open/close 44594 (time 1766417695.37 total 273.72 last 140.34) - open/close 45851 (time 1766417705.38 total 283.72 last 125.64) - open/close 47026 (time 1766417715.38 total 293.72 last 117.48) - open/close 48370 (time 1766417725.39 total 303.73 last 134.26) - open/close 49757 (time 1766417735.39 total 313.74 last 138.64) - open/close 51199 (time 1766417745.39 total 323.74 last 144.19) - open/close 52895 (time 1766417755.40 total 333.74 last 169.55) - open/close 54354 (time 1766417765.40 total 343.74 last 145.85) - open/close 56353 (time 1766417775.41 total 353.75 last 199.73) - open/close 57981 (time 1766417785.41 total 363.76 last 162.76) - open/close 59734 (time 1766417795.42 total 373.76 last 175.25) - open/close 61113 (time 1766417805.42 total 383.76 last 137.82) - open/close 62828 (time 1766417815.42 total 393.77 last 171.44) total: 64768 open/close in 402.58 seconds: 160.88 ops/second - /unlink 3239 (time 1766417836.16 total 10.00 last 323.75) - /unlink 6019 (time 1766417846.16 total 20.00 last 277.99) - /unlink 9369 (time 1766417856.17 total 30.01 last 334.95) - /unlink 10000 (time 1766417858.58 total 32.42 last 261.80) - /unlink 12244 (time 1766417868.58 total 42.42 last 224.30) - /unlink 15317 (time 1766417878.58 total 52.42 last 307.26) - /unlink 18244 (time 1766417888.58 total 62.42 last 292.64) - /unlink 20000 (time 1766417895.12 total 68.96 last 268.78) - /unlink 22647 (time 1766417905.12 total 78.96 last 264.64) - /unlink 25462 (time 1766417915.12 total 88.96 last 281.41) - /unlink 28572 (time 1766417925.13 total 98.97 last 310.91) - /unlink 30000 (time 1766417928.91 total 102.76 last 376.81) - /unlink 33381 (time 1766417938.92 total 112.76 last 338.08) - /unlink 36188 (time 1766417948.92 total 122.76 last 280.67) - /unlink 38736 (time 1766417958.92 total 132.76 last 254.73) - /unlink 40000 (time 1766417964.56 total 138.40 last 224.02) - /unlink 42206 (time 1766417974.56 total 148.40 last 220.58) - /unlink 44869 (time 1766417984.56 total 158.41 last 266.25) - /unlink 48098 (time 1766417994.57 total 168.41 last 322.88) - /unlink 50000 (time 1766417999.27 total 173.11 last 404.38) - /unlink 54157 (time 1766418009.27 total 183.11 last 415.62) - /unlink 56686 (time 1766418019.27 total 193.11 last 252.90) - /unlink 59645 (time 1766418029.27 total 203.11 last 295.86) - /unlink 62966 (time 1766418039.27 total 213.11 last 332.05) total: 64768 /unlink in 218.81 seconds: 296.00 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1851 (time 1766418077.35 total 10.01 last 185.00) - open/close 4000 (time 1766418087.35 total 20.01 last 214.82) - open/close 5871 (time 1766418097.36 total 30.02 last 186.95) - open/close 7458 (time 1766418107.36 total 40.02 last 158.63) - open/close 9261 (time 1766418117.37 total 50.03 last 180.23) - open/close 10000 (time 1766418121.66 total 54.31 last 172.35) - open/close 12066 (time 1766418131.66 total 64.31 last 206.56) - open/close 14383 (time 1766418141.66 total 74.32 last 231.66) - open/close 16417 (time 1766418151.66 total 84.32 last 203.28) - open/close 18259 (time 1766418161.67 total 94.33 last 184.13) - open/close 20000 (time 1766418169.85 total 102.50 last 212.93) - open/close 21751 (time 1766418179.85 total 112.50 last 175.08) - open/close 23428 (time 1766418189.85 total 122.50 last 167.68) - open/close 25250 (time 1766418199.85 total 132.51 last 182.08) - open/close 26894 (time 1766418209.86 total 142.51 last 164.35) - open/close 28401 (time 1766418219.86 total 152.52 last 150.67) - open/close 30000 (time 1766418228.10 total 160.76 last 194.03) - open/close 32319 (time 1766418238.10 total 170.76 last 231.89) - open/close 34121 (time 1766418248.10 total 180.76 last 180.16) - open/close 35617 (time 1766418258.11 total 190.76 last 149.54) - open/close 37484 (time 1766418268.11 total 200.77 last 186.67) - open/close 39538 (time 1766418278.11 total 210.77 last 205.40) - open/close 40000 (time 1766418280.67 total 213.33 last 180.16) - open/close 42499 (time 1766418290.67 total 223.33 last 249.84) - open/close 44742 (time 1766418300.67 total 233.33 last 224.30) - open/close 47221 (time 1766418310.68 total 243.33 last 247.84) - open/close 49489 (time 1766418320.68 total 253.34 last 226.78) - open/close 51757 (time 1766418330.68 total 263.34 last 226.78) - open/close 53921 (time 1766418340.68 total 273.34 last 216.33) - open/close 56018 (time 1766418350.68 total 283.34 last 209.68) - open/close 58467 (time 1766418360.68 total 293.34 last 244.88) - open/close 60000 (time 1766418366.29 total 298.95 last 273.47) - open/close 62817 (time 1766418376.29 total 308.95 last 281.67) total: 64768 open/close in 317.20 seconds: 204.18 ops/second - /unlink 4575 (time 1766418396.18 total 10.00 last 457.48) - /unlink 8786 (time 1766418406.18 total 20.00 last 421.03) - /unlink 10000 (time 1766418409.07 total 22.89 last 420.19) - /unlink 14760 (time 1766418419.07 total 32.89 last 475.94) - /unlink 18946 (time 1766418429.07 total 42.89 last 418.57) - /unlink 20000 (time 1766418431.29 total 45.11 last 474.91) - /unlink 24786 (time 1766418441.29 total 55.11 last 478.59) - /unlink 29622 (time 1766418451.29 total 65.11 last 483.58) - /unlink 34832 (time 1766418461.29 total 75.12 last 520.84) - /unlink 39769 (time 1766418471.33 total 85.15 last 492.12) - /unlink 43468 (time 1766418481.33 total 95.15 last 369.89) - /unlink 47186 (time 1766418491.33 total 105.15 last 371.80) - /unlink 50000 (time 1766418498.32 total 112.15 last 402.17) - /unlink 54494 (time 1766418508.32 total 122.15 last 449.39) - /unlink 58708 (time 1766418518.33 total 132.15 last 421.32) - /unlink 60000 (time 1766418521.35 total 135.17 last 426.92) - /unlink 64106 (time 1766418531.35 total 145.18 last 410.55) total: 64768 /unlink in 146.44 seconds: 442.30 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg433-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg433-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg433-server stop mds service on oleg433-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg433-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg433-server' oleg433-server: oleg433-server.virtnet: executing unload_modules_local oleg433-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg433-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg433-client: oleg433-client: ssh exited with exit code 2 pdsh@oleg433-client: oleg433-server: ssh exited with exit code 2 pdsh@oleg433-client: oleg433-client: ssh exited with exit code 2 pdsh@oleg433-client: oleg433-server: ssh exited with exit code 2 PASS 106 (6269s) Stopping clients: oleg433-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg433-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg433-client: oleg433-server: ssh exited with exit code 2 oleg433-server: oleg433-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg433-server' oleg433-server: oleg433-server.virtnet: executing load_modules_local oleg433-server: Loading modules from /home/green/git/lustre-release/lustre oleg433-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg433-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg433-server: detected 4 online CPUs by sysfs oleg433-server: MODOPTS_LIBCFS= oleg433-server: Force libcfs to create 2 CPU partitions oleg433-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg433-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 6440 sec ========== 10:50:05 (1766418605) === conf-sanity: start cleanup 10:50:05 (1766418605) === === conf-sanity: finish cleanup 10:50:07 (1766418607) ===