-----============= acceptance-small: conf-sanity ============----- Mon Jul 14 12:17:21 EDT 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg335-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg335-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg335-server oleg335-server: oleg335-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg335-server' oleg335-server: oleg335-server.virtnet: executing load_modules_local oleg335-server: Loading modules from /home/green/git/lustre-release/lustre oleg335-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy oleg335-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy oleg335-server: detected 4 online CPUs by sysfs oleg335-server: MODOPTS_LIBCFS= oleg335-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg335-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg335-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg335-server: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg335-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg335-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server stop mds service on oleg335-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 12:18:46 (1752509926) Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:-f) oleg335-server: oleg335-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg335-server' oleg335-server: oleg335-server.virtnet: executing load_modules_local oleg335-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy oleg335-server: Loading modules from /home/green/git/lustre-release/lustre oleg335-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy oleg335-server: detected 4 online CPUs by sysfs oleg335-server: MODOPTS_LIBCFS= oleg335-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg335-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg335-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg335-client.virtnet: -o user_xattr,flock 192.168.203.135@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1300 (time 1752509984.65 total 10.00 last 129.95) - open/close 2674 (time 1752509994.65 total 20.01 last 137.36) - open/close 4577 (time 1752510004.66 total 30.01 last 190.18) - open/close 6700 (time 1752510014.66 total 40.02 last 212.22) - open/close 8554 (time 1752510024.66 total 50.02 last 185.35) - open/close 9913 (time 1752510034.67 total 60.03 last 135.79) - open/close 11526 (time 1752510044.67 total 70.03 last 161.28) - open/close 13041 (time 1752510054.67 total 80.03 last 151.50) - open/close 14343 (time 1752510064.67 total 90.03 last 130.18) - open/close 15777 (time 1752510074.68 total 100.04 last 143.29) - open/close 17438 (time 1752510084.69 total 110.04 last 166.00) - open/close 19132 (time 1752510094.69 total 120.05 last 169.36) - open/close 20000 (time 1752510098.93 total 124.28 last 204.87) - open/close 21778 (time 1752510108.93 total 134.29 last 177.73) - open/close 23673 (time 1752510118.93 total 144.29 last 189.48) - open/close 25578 (time 1752510128.93 total 154.29 last 190.47) - open/close 27674 (time 1752510138.94 total 164.29 last 209.55) - open/close 29736 (time 1752510148.94 total 174.29 last 206.15) - open/close 31530 (time 1752510158.94 total 184.30 last 179.38) - open/close 33472 (time 1752510168.94 total 194.30 last 194.16) - open/close 35616 (time 1752510178.95 total 204.30 last 214.34) - open/close 37622 (time 1752510188.95 total 214.30 last 200.57) - open/close 39614 (time 1752510198.95 total 224.30 last 199.18) - open/close 41460 (time 1752510208.95 total 234.30 last 184.58) - open/close 43245 (time 1752510218.95 total 244.31 last 178.43) - open/close 45090 (time 1752510228.96 total 254.31 last 184.45) - open/close 47161 (time 1752510238.96 total 264.32 last 206.99) - open/close 49052 (time 1752510248.96 total 274.32 last 189.06) - open/close 50000 (time 1752510254.10 total 279.45 last 184.72) - open/close 52003 (time 1752510264.10 total 289.46 last 200.18) - open/close 53875 (time 1752510274.11 total 299.46 last 187.12) - open/close 55607 (time 1752510284.11 total 309.47 last 173.06) - open/close 57630 (time 1752510294.12 total 319.47 last 202.26) - open/close 59588 (time 1752510304.12 total 329.47 last 195.80) - open/close 60000 (time 1752510306.23 total 331.59 last 194.64) - open/close 62227 (time 1752510316.24 total 341.59 last 222.64) - open/close 64378 (time 1752510326.24 total 351.59 last 215.07) total: 64768 open/close in 353.48 seconds: 183.23 ops/second - /unlink 3955 (time 1752510339.82 total 10.00 last 395.42) - /unlink 7766 (time 1752510349.82 total 20.00 last 381.09) - /unlink 10000 (time 1752510356.95 total 27.13 last 313.55) - /unlink 13389 (time 1752510366.95 total 37.13 last 338.72) - /unlink 16709 (time 1752510376.95 total 47.13 last 331.94) - /unlink 19557 (time 1752510386.96 total 57.14 last 284.65) - /unlink 23016 (time 1752510396.96 total 67.14 last 345.89) - /unlink 26417 (time 1752510406.96 total 77.14 last 340.01) - /unlink 29846 (time 1752510416.97 total 87.14 last 342.81) - /unlink 33673 (time 1752510426.97 total 97.15 last 382.68) - /unlink 37465 (time 1752510436.97 total 107.15 last 379.15) - /unlink 40000 (time 1752510443.42 total 113.60 last 392.96) - /unlink 44046 (time 1752510453.43 total 123.61 last 404.09) - /unlink 47534 (time 1752510463.43 total 133.61 last 348.73) - /unlink 50000 (time 1752510470.52 total 140.69 last 348.21) - /unlink 53867 (time 1752510480.52 total 150.69 last 386.69) - /unlink 57848 (time 1752510490.52 total 160.70 last 398.03) - /unlink 60000 (time 1752510495.99 total 166.16 last 393.55) - /unlink 63812 (time 1752510505.99 total 176.17 last 381.15) total: 64768 /unlink in 179.04 seconds: 361.75 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 2173 (time 1752510539.30 total 10.00 last 217.24) - open/close 4572 (time 1752510549.30 total 20.00 last 239.85) - open/close 7082 (time 1752510559.30 total 30.01 last 250.99) - open/close 10000 (time 1752510567.96 total 38.67 last 336.87) - open/close 12747 (time 1752510577.96 total 48.67 last 274.68) - open/close 15410 (time 1752510587.96 total 58.67 last 266.28) - open/close 18271 (time 1752510597.97 total 68.67 last 286.02) - open/close 20000 (time 1752510605.09 total 75.80 last 242.56) - open/close 23188 (time 1752510615.10 total 85.80 last 318.73) - open/close 26487 (time 1752510625.10 total 95.80 last 329.85) - open/close 29427 (time 1752510635.10 total 105.80 last 293.94) - open/close 32557 (time 1752510645.10 total 115.81 last 312.99) - open/close 35666 (time 1752510655.11 total 125.82 last 310.47) - open/close 38251 (time 1752510665.12 total 135.82 last 258.43) - open/close 40000 (time 1752510671.14 total 141.84 last 290.56) - open/close 42681 (time 1752510681.14 total 151.84 last 268.04) - open/close 45750 (time 1752510691.14 total 161.85 last 306.84) - open/close 48417 (time 1752510701.14 total 171.85 last 266.60) - open/close 50000 (time 1752510707.16 total 177.86 last 263.19) - open/close 52238 (time 1752510717.16 total 187.87 last 223.70) - open/close 54825 (time 1752510727.16 total 197.87 last 258.67) - open/close 57199 (time 1752510737.17 total 207.87 last 237.32) - open/close 59485 (time 1752510747.17 total 217.88 last 228.51) - open/close 60000 (time 1752510749.52 total 220.22 last 219.78) - open/close 62037 (time 1752510759.52 total 230.22 last 203.61) - open/close 64142 (time 1752510769.52 total 240.23 last 210.42) total: 64768 open/close in 242.92 seconds: 266.62 ops/second - /unlink 4478 (time 1752510783.89 total 10.00 last 447.74) - /unlink 8633 (time 1752510793.89 total 20.00 last 415.39) - /unlink 10000 (time 1752510796.69 total 22.80 last 488.36) - /unlink 13790 (time 1752510806.74 total 32.85 last 377.09) - /unlink 17523 (time 1752510816.74 total 42.86 last 373.25) - /unlink 20000 (time 1752510823.78 total 49.89 last 352.19) - /unlink 23843 (time 1752510833.78 total 59.89 last 384.25) - /unlink 27817 (time 1752510843.78 total 69.89 last 397.34) - /unlink 30000 (time 1752510848.45 total 74.57 last 467.07) - /unlink 34484 (time 1752510858.45 total 84.57 last 448.32) - /unlink 39388 (time 1752510868.46 total 94.57 last 490.35) - /unlink 44470 (time 1752510878.46 total 104.57 last 508.18) - /unlink 49935 (time 1752510888.46 total 114.57 last 546.44) - /unlink 55256 (time 1752510898.46 total 124.57 last 532.07) - /unlink 59476 (time 1752510908.46 total 134.57 last 421.99) - /unlink 62967 (time 1752510918.46 total 144.57 last 349.05) total: 64768 /unlink in 150.13 seconds: 431.41 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 2658 (time 1752510957.07 total 10.00 last 265.74) - open/close 5716 (time 1752510967.07 total 20.00 last 305.80) - open/close 8795 (time 1752510977.07 total 30.01 last 307.81) - open/close 10000 (time 1752510980.80 total 33.73 last 323.18) - open/close 12622 (time 1752510990.80 total 43.73 last 262.18) - open/close 15343 (time 1752511000.80 total 53.74 last 271.91) - open/close 17887 (time 1752511010.81 total 63.74 last 254.39) - open/close 20000 (time 1752511020.25 total 73.19 last 223.69) - open/close 23334 (time 1752511030.25 total 83.19 last 333.31) - open/close 25742 (time 1752511040.26 total 93.19 last 240.74) - open/close 28509 (time 1752511050.26 total 103.19 last 276.69) - open/close 30000 (time 1752511055.32 total 108.26 last 294.55) - open/close 32880 (time 1752511065.32 total 118.26 last 287.92) - open/close 35327 (time 1752511075.32 total 128.26 last 244.66) - open/close 38815 (time 1752511085.32 total 138.26 last 348.79) - open/close 40000 (time 1752511088.38 total 141.32 last 387.14) - open/close 43524 (time 1752511098.38 total 151.32 last 352.38) - open/close 46632 (time 1752511108.39 total 161.32 last 310.77) - open/close 49717 (time 1752511118.39 total 171.32 last 308.48) - open/close 52231 (time 1752511128.39 total 181.32 last 251.39) - open/close 54934 (time 1752511138.39 total 191.32 last 270.29) - open/close 58197 (time 1752511148.39 total 201.33 last 326.24) - open/close 60000 (time 1752511154.18 total 207.11 last 311.52) - open/close 63464 (time 1752511164.18 total 217.11 last 346.36) total: 64768 open/close in 220.75 seconds: 293.40 ops/second - /unlink 6957 (time 1752511179.17 total 10.00 last 695.64) - /unlink 10000 (time 1752511184.46 total 15.29 last 575.26) - /unlink 15695 (time 1752511194.47 total 25.29 last 569.43) - /unlink 20000 (time 1752511200.78 total 31.61 last 681.67) - /unlink 26974 (time 1752511210.78 total 41.61 last 697.36) - /unlink 30000 (time 1752511214.67 total 45.49 last 778.66) - /unlink 37014 (time 1752511224.67 total 55.49 last 701.34) - /unlink 40000 (time 1752511228.26 total 59.09 last 831.48) - /unlink 47299 (time 1752511238.26 total 69.09 last 729.85) - /unlink 50000 (time 1752511242.90 total 73.73 last 581.55) - /unlink 56363 (time 1752511252.91 total 83.73 last 636.20) - /unlink 60000 (time 1752511258.65 total 89.47 last 633.52) total: 64768 /unlink in 97.15 seconds: 666.65 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 5161 (time 1752511296.30 total 10.00 last 515.98) - open/close 9642 (time 1752511306.30 total 20.00 last 448.10) - open/close 14433 (time 1752511316.30 total 30.00 last 479.02) - open/close 18615 (time 1752511326.30 total 40.00 last 418.17) - open/close 20000 (time 1752511329.91 total 43.61 last 384.47) - open/close 23869 (time 1752511339.91 total 53.61 last 386.80) - open/close 27475 (time 1752511349.91 total 63.61 last 360.47) - open/close 30000 (time 1752511357.97 total 71.67 last 313.56) - open/close 34105 (time 1752511367.97 total 81.67 last 410.41) - open/close 37703 (time 1752511377.97 total 91.67 last 359.72) - open/close 40000 (time 1752511384.32 total 98.02 last 361.89) - open/close 43254 (time 1752511394.32 total 108.02 last 325.32) - open/close 44411 (time 1752511404.33 total 118.03 last 115.63) - open/close 45306 (time 1752511414.34 total 128.04 last 89.35) - open/close 46144 (time 1752511424.35 total 138.05 last 83.72) - open/close 47089 (time 1752511434.35 total 148.05 last 94.48) - open/close 48052 (time 1752511444.36 total 158.06 last 96.28) - open/close 49246 (time 1752511454.36 total 168.06 last 119.38) - open/close 50000 (time 1752511460.64 total 174.35 last 119.92) - open/close 51472 (time 1752511470.65 total 184.35 last 147.15) - open/close 52783 (time 1752511480.66 total 194.36 last 130.92) - open/close 54171 (time 1752511490.66 total 204.36 last 138.79) - open/close 55677 (time 1752511500.66 total 214.36 last 150.59) - open/close 57814 (time 1752511510.66 total 224.36 last 213.69) - open/close 59654 (time 1752511520.67 total 234.37 last 183.94) - open/close 60000 (time 1752511523.17 total 236.87 last 138.43) - open/close 62099 (time 1752511533.17 total 246.87 last 209.82) - open/close 63630 (time 1752511543.17 total 256.87 last 153.07) total: 64768 open/close in 264.04 seconds: 245.30 ops/second - /unlink 2697 (time 1752511562.24 total 10.00 last 269.63) - /unlink 6221 (time 1752511572.24 total 20.00 last 352.34) - /unlink 9722 (time 1752511582.24 total 30.01 last 350.04) - /unlink 11942 (time 1752511592.25 total 40.01 last 221.93) - /unlink 14234 (time 1752511602.25 total 50.01 last 229.15) - /unlink 16857 (time 1752511612.25 total 60.01 last 262.29) - /unlink 19718 (time 1752511622.25 total 70.01 last 286.08) - /unlink 21952 (time 1752511632.25 total 80.02 last 223.31) - /unlink 24974 (time 1752511642.25 total 90.02 last 302.16) - /unlink 27864 (time 1752511652.26 total 100.02 last 288.86) - /unlink 30000 (time 1752511660.70 total 108.46 last 253.15) - /unlink 32965 (time 1752511670.70 total 118.46 last 296.44) - /unlink 35587 (time 1752511680.70 total 128.46 last 262.19) - /unlink 39357 (time 1752511690.70 total 138.47 last 376.89) - /unlink 43290 (time 1752511700.70 total 148.47 last 393.29) - /unlink 47217 (time 1752511710.70 total 158.47 last 392.67) - /unlink 50000 (time 1752511718.94 total 166.70 last 337.90) - /unlink 53523 (time 1752511728.94 total 176.70 last 352.23) - /unlink 57615 (time 1752511738.94 total 186.71 last 409.12) - /unlink 60000 (time 1752511745.92 total 193.68 last 341.86) - /unlink 63175 (time 1752511755.92 total 203.69 last 317.42) total: 64768 /unlink in 209.29 seconds: 309.46 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1249 (time 1752511792.17 total 10.00 last 124.86) - open/close 2435 (time 1752511802.18 total 20.01 last 118.51) - open/close 3730 (time 1752511812.19 total 30.01 last 129.46) - open/close 4959 (time 1752511822.19 total 40.02 last 122.88) - open/close 6881 (time 1752511832.19 total 50.02 last 192.16) - open/close 8586 (time 1752511842.19 total 60.02 last 170.47) - open/close 10000 (time 1752511849.85 total 67.67 last 184.72) - open/close 11662 (time 1752511859.85 total 77.68 last 166.15) - open/close 13715 (time 1752511869.85 total 87.68 last 205.20) - open/close 15927 (time 1752511879.86 total 97.69 last 221.08) - open/close 18128 (time 1752511889.86 total 107.69 last 220.09) - open/close 20000 (time 1752511897.76 total 115.59 last 236.91) - open/close 22155 (time 1752511907.76 total 125.59 last 215.49) - open/close 25305 (time 1752511917.78 total 135.61 last 314.43) - open/close 27854 (time 1752511927.78 total 145.61 last 254.88) - open/close 30000 (time 1752511935.25 total 153.08 last 287.36) - open/close 32413 (time 1752511945.25 total 163.08 last 241.22) - open/close 34504 (time 1752511955.25 total 173.08 last 209.04) - open/close 37107 (time 1752511965.26 total 183.09 last 260.26) - open/close 39491 (time 1752511975.26 total 193.09 last 238.39) - open/close 40000 (time 1752511978.22 total 196.05 last 171.70) - open/close 41718 (time 1752511988.22 total 206.05 last 171.76) - open/close 43368 (time 1752511998.23 total 216.06 last 164.94) - open/close 44957 (time 1752512008.23 total 226.06 last 158.82) - open/close 46459 (time 1752512018.24 total 236.07 last 150.12) - open/close 48232 (time 1752512028.24 total 246.07 last 177.18) - open/close 50000 (time 1752512037.74 total 255.57 last 186.21) - open/close 51872 (time 1752512047.75 total 265.57 last 187.07) - open/close 53590 (time 1752512057.75 total 275.58 last 171.79) - open/close 54985 (time 1752512067.75 total 285.58 last 139.48) - open/close 56396 (time 1752512077.75 total 295.58 last 141.04) - open/close 57788 (time 1752512087.76 total 305.59 last 139.08) - open/close 59401 (time 1752512097.76 total 315.59 last 161.29) - open/close 60000 (time 1752512101.60 total 319.42 last 156.26) - open/close 61471 (time 1752512111.60 total 329.43 last 147.06) - open/close 63030 (time 1752512121.60 total 339.43 last 155.83) total: 64768 open/close in 348.88 seconds: 185.64 ops/second - /unlink 3214 (time 1752512142.86 total 10.00 last 321.30) - /unlink 6115 (time 1752512152.86 total 20.01 last 290.02) - /unlink 9408 (time 1752512162.87 total 30.01 last 329.17) - /unlink 10000 (time 1752512165.16 total 32.31 last 257.82) - /unlink 12476 (time 1752512175.16 total 42.31 last 247.55) - /unlink 15717 (time 1752512185.17 total 52.31 last 324.02) - /unlink 19004 (time 1752512195.17 total 62.31 last 328.66) - /unlink 20000 (time 1752512197.93 total 65.07 last 360.51) - /unlink 22972 (time 1752512207.93 total 75.08 last 297.16) - /unlink 26515 (time 1752512217.93 total 85.08 last 354.26) - /unlink 30000 (time 1752512227.65 total 94.79 last 358.66) - /unlink 32950 (time 1752512237.65 total 104.80 last 294.93) - /unlink 35559 (time 1752512247.66 total 114.80 last 260.79) - /unlink 38325 (time 1752512257.66 total 124.80 last 276.53) - /unlink 40000 (time 1752512263.46 total 130.61 last 288.65) - /unlink 43280 (time 1752512273.47 total 140.61 last 327.89) - /unlink 46977 (time 1752512283.47 total 150.61 last 369.64) - /unlink 50000 (time 1752512289.27 total 156.41 last 521.13) - /unlink 55247 (time 1752512299.27 total 166.41 last 524.56) - /unlink 59475 (time 1752512309.27 total 176.41 last 422.78) - /unlink 64196 (time 1752512319.27 total 186.42 last 472.04) total: 64768 /unlink in 187.41 seconds: 345.60 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg335-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg335-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server stop mds service on oleg335-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg335-server' oleg335-server: oleg335-server.virtnet: executing unload_modules_local oleg335-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg335-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg335-client: oleg335-client: ssh exited with exit code 2 pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2 pdsh@oleg335-client: oleg335-client: ssh exited with exit code 2 pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2 PASS 106 (2431s) Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2 oleg335-server: oleg335-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg335-server' oleg335-server: oleg335-server.virtnet: executing load_modules_local oleg335-server: Loading modules from /home/green/git/lustre-release/lustre oleg335-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy oleg335-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy oleg335-server: detected 4 online CPUs by sysfs oleg335-server: MODOPTS_LIBCFS= oleg335-server: Force libcfs to create 2 CPU partitions oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 2546 sec ========== 12:59:47 (1752512387) === conf-sanity: start cleanup 12:59:48 (1752512388) === === conf-sanity: finish cleanup 12:59:49 (1752512389) ===