-----============= acceptance-small: conf-sanity ============----- Tue Nov 25 16:26:47 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg302-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg302-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg302-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg302-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg302-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg302-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg302-server oleg302-server: oleg302-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg302-server' oleg302-server: oleg302-server.virtnet: executing load_modules_local oleg302-server: Loading modules from /home/green/git/lustre-release/lustre oleg302-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg302-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg302-server: detected 4 online CPUs by sysfs oleg302-server: MODOPTS_LIBCFS= oleg302-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg302-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg302-server: oleg302-server.virtnet: executing set_default_debug -1 all pdsh@oleg302-client: oleg302-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg302-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg302-server: oleg302-server.virtnet: executing set_default_debug -1 all pdsh@oleg302-client: oleg302-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg302-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg302-server: oleg302-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg302-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg302-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg302-server stop mds service on oleg302-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg302-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 16:28:22 (1764106102) Stopping clients: oleg302-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg302-client.virtnet /mnt/lustre2 (opts:-f) oleg302-server: oleg302-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg302-server' oleg302-server: oleg302-server.virtnet: executing load_modules_local oleg302-server: Loading modules from /home/green/git/lustre-release/lustre oleg302-server: detected 4 online CPUs by sysfs oleg302-server: MODOPTS_LIBCFS= oleg302-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg302-server: Force libcfs to create 2 CPU partitions oleg302-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg302-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg302-server: oleg302-server.virtnet: executing set_default_debug -1 all pdsh@oleg302-client: oleg302-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg302-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg302-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg302-server: oleg302-server.virtnet: executing set_default_debug -1 all pdsh@oleg302-client: oleg302-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg302-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg302-client.virtnet: -o user_xattr,flock 192.168.203.102@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32648 392 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1211 (time 1764106170.10 total 10.00 last 121.06) - open/close 2384 (time 1764106180.10 total 20.01 last 117.21) - open/close 3785 (time 1764106190.11 total 30.01 last 140.05) - open/close 5045 (time 1764106200.11 total 40.02 last 125.98) - open/close 6240 (time 1764106210.11 total 50.02 last 119.44) - open/close 7557 (time 1764106220.12 total 60.03 last 131.65) - open/close 9045 (time 1764106230.12 total 70.03 last 148.73) - open/close 10000 (time 1764106236.47 total 76.38 last 150.41) - open/close 11782 (time 1764106246.48 total 86.39 last 178.07) - open/close 13147 (time 1764106256.48 total 96.39 last 136.44) - open/close 14383 (time 1764106266.49 total 106.39 last 123.56) - open/close 15836 (time 1764106276.50 total 116.40 last 145.17) - open/close 17555 (time 1764106286.50 total 126.40 last 171.88) - open/close 19068 (time 1764106296.50 total 136.41 last 151.22) - open/close 20000 (time 1764106302.89 total 142.80 last 145.86) - open/close 21628 (time 1764106312.90 total 152.81 last 162.68) - open/close 22971 (time 1764106322.90 total 162.81 last 134.28) - open/close 24671 (time 1764106332.90 total 172.81 last 169.96) - open/close 26778 (time 1764106342.91 total 182.82 last 210.54) - open/close 28712 (time 1764106352.91 total 192.82 last 193.34) - open/close 30000 (time 1764106359.33 total 199.24 last 200.69) - open/close 32615 (time 1764106369.33 total 209.24 last 261.43) - open/close 34500 (time 1764106379.34 total 219.25 last 188.39) - open/close 36668 (time 1764106389.34 total 229.25 last 216.77) - open/close 38742 (time 1764106399.34 total 239.25 last 207.37) - open/close 40000 (time 1764106406.28 total 246.19 last 181.33) - open/close 42074 (time 1764106416.28 total 256.19 last 207.32) - open/close 43538 (time 1764106426.29 total 266.19 last 146.38) - open/close 44909 (time 1764106436.29 total 276.20 last 137.01) - open/close 46721 (time 1764106446.29 total 286.20 last 181.16) - open/close 48647 (time 1764106456.30 total 296.21 last 192.54) - open/close 50000 (time 1764106462.69 total 302.60 last 211.50) - open/close 52476 (time 1764106472.70 total 312.61 last 247.50) - open/close 54653 (time 1764106482.70 total 322.61 last 217.68) - open/close 56653 (time 1764106492.70 total 332.61 last 199.97) - open/close 58901 (time 1764106502.70 total 342.61 last 224.75) - open/close 60000 (time 1764106507.90 total 347.81 last 211.36) - open/close 62105 (time 1764106517.90 total 357.81 last 210.50) - open/close 64452 (time 1764106527.90 total 367.81 last 234.68) total: 64768 open/close in 369.00 seconds: 175.52 ops/second - /unlink 3649 (time 1764106540.92 total 10.00 last 364.82) - /unlink 6282 (time 1764106550.93 total 20.01 last 263.12) - /unlink 9272 (time 1764106560.93 total 30.01 last 298.84) - /unlink 10000 (time 1764106563.47 total 32.55 last 286.94) - /unlink 12649 (time 1764106573.47 total 42.55 last 264.85) - /unlink 15618 (time 1764106583.47 total 52.55 last 296.85) - /unlink 18805 (time 1764106593.48 total 62.56 last 318.62) - /unlink 20000 (time 1764106596.81 total 65.89 last 358.42) - /unlink 22701 (time 1764106606.86 total 75.94 last 268.67) - /unlink 26484 (time 1764106616.87 total 85.95 last 378.26) - /unlink 30000 (time 1764106626.05 total 95.13 last 382.67) - /unlink 34730 (time 1764106636.06 total 105.14 last 472.92) - /unlink 39313 (time 1764106646.06 total 115.14 last 458.27) - /unlink 44601 (time 1764106656.06 total 125.14 last 528.74) - /unlink 50000 (time 1764106665.72 total 134.80 last 558.85) - /unlink 55200 (time 1764106675.72 total 144.80 last 519.95) - /unlink 60000 (time 1764106684.67 total 153.75 last 536.04) total: 64768 /unlink in 161.49 seconds: 401.05 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 3391 (time 1764106724.31 total 10.00 last 339.02) - open/close 6813 (time 1764106734.31 total 20.00 last 342.14) - open/close 10000 (time 1764106742.48 total 28.18 last 389.82) - open/close 13958 (time 1764106752.48 total 38.18 last 395.78) - open/close 16688 (time 1764106762.48 total 48.18 last 273.00) - open/close 19843 (time 1764106772.49 total 58.18 last 315.45) - open/close 23260 (time 1764106782.49 total 68.19 last 341.50) - open/close 26550 (time 1764106792.49 total 78.19 last 328.92) - open/close 29674 (time 1764106802.50 total 88.19 last 312.34) - open/close 32234 (time 1764106812.50 total 98.19 last 255.92) - open/close 33970 (time 1764106822.50 total 108.20 last 173.53) - open/close 36195 (time 1764106832.50 total 118.20 last 222.47) - open/close 38555 (time 1764106842.51 total 128.20 last 235.90) - open/close 40000 (time 1764106848.82 total 134.52 last 228.76) - open/close 42403 (time 1764106858.83 total 144.53 last 240.20) - open/close 44871 (time 1764106868.83 total 154.53 last 246.70) - open/close 46979 (time 1764106878.84 total 164.53 last 210.72) - open/close 49212 (time 1764106888.84 total 174.53 last 223.29) - open/close 50000 (time 1764106891.59 total 177.29 last 286.33) - open/close 52754 (time 1764106901.59 total 187.29 last 275.34) - open/close 55677 (time 1764106911.59 total 197.29 last 292.22) - open/close 58389 (time 1764106921.63 total 207.32 last 270.33) - open/close 59336 (time 1764106931.64 total 217.33 last 94.60) - open/close 60000 (time 1764106940.12 total 225.82 last 78.27) - open/close 60630 (time 1764106950.13 total 235.83 last 62.94) - open/close 61324 (time 1764106960.13 total 245.83 last 69.36) - open/close 61988 (time 1764106970.14 total 255.84 last 66.35) - open/close 62698 (time 1764106980.15 total 265.85 last 70.91) - open/close 63239 (time 1764106990.16 total 275.85 last 54.08) - open/close 63673 (time 1764107000.16 total 285.86 last 43.39) - open/close 64107 (time 1764107010.19 total 295.89 last 43.28) - open/close 64674 (time 1764107020.20 total 305.89 last 56.65) total: 64768 open/close in 307.67 seconds: 210.51 ops/second - /unlink 1251 (time 1764107036.48 total 10.00 last 125.05) - /unlink 2642 (time 1764107046.48 total 20.00 last 139.09) - /unlink 4050 (time 1764107056.49 total 30.01 last 140.73) - /unlink 5379 (time 1764107066.50 total 40.02 last 132.79) - /unlink 6712 (time 1764107076.50 total 50.03 last 133.20) - /unlink 8178 (time 1764107086.50 total 60.03 last 146.58) - /unlink 9564 (time 1764107096.51 total 70.03 last 138.57) - /unlink 10000 (time 1764107099.48 total 73.00 last 146.65) - /unlink 11449 (time 1764107109.48 total 83.01 last 144.85) - /unlink 12834 (time 1764107119.49 total 93.01 last 138.44) - /unlink 14270 (time 1764107129.49 total 103.01 last 143.53) - /unlink 15843 (time 1764107139.49 total 113.02 last 157.26) - /unlink 17556 (time 1764107149.50 total 123.02 last 171.20) - /unlink 18983 (time 1764107159.50 total 133.02 last 142.68) - /unlink 20000 (time 1764107165.13 total 138.65 last 180.82) - /unlink 21781 (time 1764107175.13 total 148.65 last 178.03) - /unlink 23155 (time 1764107185.14 total 158.66 last 137.30) - /unlink 24394 (time 1764107195.14 total 168.66 last 123.89) - /unlink 25679 (time 1764107205.14 total 178.66 last 128.47) - /unlink 27206 (time 1764107215.14 total 188.67 last 152.66) - /unlink 28700 (time 1764107225.14 total 198.67 last 149.40) - /unlink 30000 (time 1764107233.23 total 206.75 last 160.70) - /unlink 32014 (time 1764107243.23 total 216.76 last 201.37) - /unlink 33734 (time 1764107253.24 total 226.76 last 171.96) - /unlink 35298 (time 1764107263.24 total 236.76 last 156.37) - /unlink 38058 (time 1764107273.24 total 246.76 last 275.95) - /unlink 40000 (time 1764107279.73 total 253.26 last 299.01) - /unlink 43185 (time 1764107289.74 total 263.26 last 318.45) - /unlink 48136 (time 1764107299.74 total 273.26 last 494.99) - /unlink 50000 (time 1764107303.44 total 276.96 last 503.29) - /unlink 54903 (time 1764107313.44 total 286.97 last 490.29) - /unlink 59612 (time 1764107323.44 total 296.97 last 470.85) - /unlink 64563 (time 1764107333.44 total 306.97 last 495.09) total: 64768 /unlink in 307.38 seconds: 210.71 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 3839 (time 1764107361.68 total 10.00 last 383.87) - open/close 7993 (time 1764107371.68 total 20.00 last 415.36) - open/close 10000 (time 1764107375.77 total 24.09 last 490.84) - open/close 14383 (time 1764107385.77 total 34.09 last 438.18) - open/close 18587 (time 1764107395.77 total 44.09 last 420.36) - open/close 20000 (time 1764107399.11 total 47.43 last 423.23) - open/close 24413 (time 1764107409.11 total 57.43 last 441.29) - open/close 28983 (time 1764107419.11 total 67.43 last 456.96) - open/close 30000 (time 1764107421.43 total 69.75 last 438.83) - open/close 34224 (time 1764107431.43 total 79.75 last 422.38) - open/close 38711 (time 1764107441.43 total 89.75 last 448.66) - open/close 40000 (time 1764107444.12 total 92.44 last 479.93) - open/close 43727 (time 1764107454.12 total 102.44 last 372.65) - open/close 47278 (time 1764107464.12 total 112.44 last 355.04) - open/close 50000 (time 1764107472.03 total 120.35 last 344.08) - open/close 53774 (time 1764107482.03 total 130.35 last 377.37) - open/close 58116 (time 1764107492.06 total 140.38 last 433.24) - open/close 60000 (time 1764107496.69 total 145.01 last 406.24) - open/close 64516 (time 1764107506.69 total 155.01 last 451.58) total: 64768 open/close in 155.84 seconds: 415.60 ops/second - /unlink 5904 (time 1764107519.03 total 10.00 last 590.35) - /unlink 10000 (time 1764107525.26 total 16.24 last 656.90) - /unlink 15897 (time 1764107535.26 total 26.24 last 589.66) - /unlink 20000 (time 1764107542.07 total 33.04 last 603.02) - /unlink 27069 (time 1764107552.07 total 43.04 last 706.85) - /unlink 30000 (time 1764107556.57 total 47.55 last 650.48) - /unlink 37027 (time 1764107566.57 total 57.55 last 702.64) - /unlink 40000 (time 1764107570.93 total 61.90 last 682.75) - /unlink 46666 (time 1764107580.93 total 71.90 last 666.55) - /unlink 50000 (time 1764107585.36 total 76.34 last 751.65) - /unlink 58894 (time 1764107595.36 total 86.34 last 889.35) total: 64768 /unlink in 93.56 seconds: 692.26 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 4290 (time 1764107639.78 total 10.00 last 428.95) - open/close 8880 (time 1764107649.78 total 20.00 last 458.94) - open/close 10000 (time 1764107652.26 total 22.48 last 452.15) - open/close 14415 (time 1764107662.26 total 32.48 last 441.48) - open/close 18517 (time 1764107672.26 total 42.48 last 410.20) - open/close 20000 (time 1764107676.02 total 46.24 last 394.66) - open/close 24340 (time 1764107686.02 total 56.24 last 433.92) - open/close 28833 (time 1764107696.02 total 66.24 last 449.20) - open/close 30000 (time 1764107698.54 total 68.76 last 464.00) - open/close 34481 (time 1764107708.54 total 78.76 last 447.97) - open/close 38969 (time 1764107718.54 total 88.76 last 448.75) - open/close 43769 (time 1764107728.54 total 98.76 last 479.98) - open/close 48124 (time 1764107738.54 total 108.76 last 435.43) - open/close 50000 (time 1764107742.19 total 112.41 last 514.60) - open/close 54307 (time 1764107752.19 total 122.41 last 430.64) - open/close 55440 (time 1764107762.20 total 132.41 last 113.25) - open/close 56342 (time 1764107772.21 total 142.43 last 90.10) - open/close 57198 (time 1764107782.22 total 152.44 last 85.49) - open/close 57691 (time 1764107792.23 total 162.45 last 49.24) - open/close 58473 (time 1764107802.23 total 172.45 last 78.18) - open/close 59304 (time 1764107812.24 total 182.46 last 83.01) - open/close 60000 (time 1764107817.11 total 187.32 last 143.16) - open/close 61907 (time 1764107827.11 total 197.33 last 190.66) - open/close 64403 (time 1764107837.11 total 207.33 last 249.54) total: 64768 open/close in 208.65 seconds: 310.41 ops/second - /unlink 4461 (time 1764107850.09 total 10.00 last 446.03) - /unlink 7278 (time 1764107860.09 total 20.00 last 281.66) - /unlink 10000 (time 1764107869.24 total 29.16 last 297.28) - /unlink 13599 (time 1764107879.25 total 39.16 last 359.83) - /unlink 15634 (time 1764107889.25 total 49.16 last 203.44) - /unlink 16878 (time 1764107899.25 total 59.17 last 124.36) - /unlink 18168 (time 1764107909.25 total 69.17 last 128.97) - /unlink 19405 (time 1764107919.26 total 79.17 last 123.66) - /unlink 20000 (time 1764107923.48 total 83.40 last 140.81) - /unlink 21340 (time 1764107933.49 total 93.41 last 133.89) - /unlink 23914 (time 1764107943.49 total 103.41 last 257.35) - /unlink 26376 (time 1764107953.50 total 113.41 last 246.08) - /unlink 28997 (time 1764107963.50 total 123.41 last 262.07) - /unlink 30000 (time 1764107967.49 total 127.40 last 251.67) - /unlink 32860 (time 1764107977.49 total 137.40 last 285.94) - /unlink 35829 (time 1764107987.49 total 147.40 last 296.90) - /unlink 38805 (time 1764107997.49 total 157.40 last 297.55) - /unlink 40000 (time 1764108001.94 total 161.86 last 268.29) - /unlink 43477 (time 1764108011.94 total 171.86 last 347.64) - /unlink 46411 (time 1764108021.95 total 181.86 last 293.37) - /unlink 49406 (time 1764108031.95 total 191.86 last 299.46) - /unlink 52930 (time 1764108041.95 total 201.86 last 352.33) - /unlink 57252 (time 1764108051.95 total 211.87 last 432.09) - /unlink 59395 (time 1764108061.96 total 221.87 last 214.22) - /unlink 60000 (time 1764108066.12 total 226.03 last 145.36) - /unlink 61857 (time 1764108076.13 total 236.04 last 185.54) - /unlink 64038 (time 1764108086.13 total 246.05 last 218.00) total: 64768 /unlink in 249.02 seconds: 260.09 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1222 (time 1764108120.46 total 10.00 last 122.14) - open/close 2395 (time 1764108130.46 total 20.00 last 117.30) - open/close 3596 (time 1764108140.46 total 30.01 last 120.05) - open/close 5194 (time 1764108150.47 total 40.01 last 159.75) - open/close 7034 (time 1764108160.47 total 50.02 last 183.93) - open/close 8637 (time 1764108170.47 total 60.02 last 160.29) - open/close 10000 (time 1764108177.91 total 67.46 last 183.22) - open/close 11821 (time 1764108187.92 total 77.46 last 181.97) - open/close 13001 (time 1764108197.92 total 87.47 last 117.97) - open/close 14152 (time 1764108207.92 total 97.47 last 115.07) - open/close 15513 (time 1764108217.93 total 107.47 last 136.01) - open/close 16639 (time 1764108227.94 total 117.48 last 112.54) - open/close 17794 (time 1764108237.95 total 127.49 last 115.39) - open/close 18941 (time 1764108247.95 total 137.49 last 114.66) - open/close 19947 (time 1764108257.95 total 147.50 last 100.55) - open/close 20945 (time 1764108267.96 total 157.50 last 99.75) - open/close 22262 (time 1764108277.97 total 167.51 last 131.60) - open/close 23899 (time 1764108287.97 total 177.51 last 163.69) - open/close 25348 (time 1764108297.97 total 187.51 last 144.89) - open/close 26752 (time 1764108307.97 total 197.51 last 140.39) - open/close 28047 (time 1764108317.97 total 207.51 last 129.49) - open/close 29488 (time 1764108327.97 total 217.51 last 144.10) - open/close 30000 (time 1764108331.93 total 221.48 last 129.20) - open/close 31455 (time 1764108341.96 total 231.50 last 145.09) - open/close 32716 (time 1764108351.96 total 241.51 last 126.06) - open/close 34369 (time 1764108361.96 total 251.51 last 165.29) - open/close 35533 (time 1764108371.96 total 261.51 last 116.40) - open/close 36738 (time 1764108381.97 total 271.51 last 120.43) - open/close 37794 (time 1764108391.97 total 281.52 last 105.59) - open/close 38893 (time 1764108401.98 total 291.52 last 109.83) - open/close 40000 (time 1764108410.44 total 299.99 last 130.79) - open/close 41589 (time 1764108420.44 total 309.99 last 158.87) - open/close 42703 (time 1764108430.45 total 319.99 last 111.36) - open/close 44001 (time 1764108440.45 total 330.00 last 129.73) - open/close 45516 (time 1764108450.45 total 340.00 last 151.49) - open/close 46917 (time 1764108460.46 total 350.00 last 140.06) - open/close 48280 (time 1764108470.48 total 360.02 last 135.99) - open/close 49886 (time 1764108480.48 total 370.02 last 160.59) - open/close 51382 (time 1764108490.48 total 380.03 last 149.52) - open/close 52503 (time 1764108500.49 total 390.04 last 112.00) - open/close 53668 (time 1764108510.50 total 400.04 last 116.46) - open/close 54855 (time 1764108520.51 total 410.05 last 118.57) - open/close 56289 (time 1764108530.51 total 420.05 last 143.39) - open/close 57630 (time 1764108540.52 total 430.06 last 133.99) - open/close 58633 (time 1764108550.53 total 440.07 last 100.19) - open/close 59345 (time 1764108560.54 total 450.09 last 71.10) - open/close 59823 (time 1764108570.54 total 460.09 last 47.79) - open/close 60000 (time 1764108574.24 total 463.78 last 47.91) - open/close 60558 (time 1764108584.25 total 473.79 last 55.74) - open/close 61419 (time 1764108594.25 total 483.80 last 86.08) - open/close 62419 (time 1764108604.25 total 493.80 last 99.98) - open/close 63735 (time 1764108614.26 total 503.80 last 131.58) total: 64768 open/close in 510.92 seconds: 126.77 ops/second - /unlink 2689 (time 1764108633.70 total 10.00 last 268.84) - /unlink 5148 (time 1764108643.70 total 20.00 last 245.88) - /unlink 7545 (time 1764108653.71 total 30.01 last 239.58) - /unlink 9972 (time 1764108663.71 total 40.01 last 242.68) - /unlink 12016 (time 1764108673.71 total 50.01 last 204.33) - /unlink 14609 (time 1764108683.71 total 60.01 last 259.29) - /unlink 16721 (time 1764108693.71 total 70.01 last 211.16) - /unlink 18932 (time 1764108703.72 total 80.02 last 221.01) - /unlink 20000 (time 1764108708.82 total 85.12 last 209.27) - /unlink 22062 (time 1764108718.82 total 95.12 last 206.17) - /unlink 24601 (time 1764108728.82 total 105.12 last 253.89) - /unlink 27194 (time 1764108738.82 total 115.13 last 259.24) - /unlink 29149 (time 1764108748.83 total 125.13 last 195.40) - /unlink 30000 (time 1764108753.37 total 129.67 last 187.55) - /unlink 31940 (time 1764108763.37 total 139.67 last 193.95) - /unlink 34166 (time 1764108773.37 total 149.67 last 222.58) - /unlink 36437 (time 1764108783.37 total 159.68 last 227.02) - /unlink 38641 (time 1764108793.38 total 169.68 last 220.33) - /unlink 40000 (time 1764108799.14 total 175.44 last 235.99) - /unlink 41864 (time 1764108809.14 total 185.44 last 186.31) - /unlink 43816 (time 1764108819.16 total 195.46 last 194.93) - /unlink 46298 (time 1764108829.16 total 205.46 last 248.17) - /unlink 48343 (time 1764108839.16 total 215.46 last 204.36) - /unlink 50000 (time 1764108846.97 total 223.27 last 212.29) - /unlink 51602 (time 1764108856.97 total 233.27 last 160.15) - /unlink 53045 (time 1764108866.98 total 243.28 last 144.15) - /unlink 54501 (time 1764108876.98 total 253.28 last 145.59) - /unlink 55879 (time 1764108886.99 total 263.29 last 137.77) - /unlink 57284 (time 1764108896.99 total 273.29 last 140.42) - /unlink 58412 (time 1764108907.00 total 283.30 last 112.75) - /unlink 59735 (time 1764108917.00 total 293.30 last 132.25) - /unlink 61250 (time 1764108927.00 total 303.30 last 151.46) - /unlink 62848 (time 1764108937.01 total 313.31 last 159.73) total: 64768 /unlink in 323.22 seconds: 200.38 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg302-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg302-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg302-server stop mds service on oleg302-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg302-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg302-server' oleg302-server: oleg302-server.virtnet: executing unload_modules_local oleg302-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg302-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg302-client: oleg302-client: ssh exited with exit code 2 pdsh@oleg302-client: oleg302-server: ssh exited with exit code 2 pdsh@oleg302-client: oleg302-client: ssh exited with exit code 2 pdsh@oleg302-client: oleg302-server: ssh exited with exit code 2 PASS 106 (2881s) Stopping clients: oleg302-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg302-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg302-client: oleg302-server: ssh exited with exit code 2 oleg302-server: oleg302-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg302-server' oleg302-server: oleg302-server.virtnet: executing load_modules_local oleg302-server: Loading modules from /home/green/git/lustre-release/lustre oleg302-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg302-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg302-server: detected 4 online CPUs by sysfs oleg302-server: MODOPTS_LIBCFS= oleg302-server: Force libcfs to create 2 CPU partitions oleg302-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg302-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3009 sec ========== 17:16:57 (1764109017) === conf-sanity: start cleanup 17:16:58 (1764109018) === === conf-sanity: finish cleanup 17:17:00 (1764109020) ===