-----============= acceptance-small: conf-sanity ============----- Tue Aug 5 10:50:50 EDT 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg430-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg430-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg430-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg430-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg430-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg430-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg430-server oleg430-server: oleg430-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg430-server' oleg430-server: oleg430-server.virtnet: executing load_modules_local oleg430-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy oleg430-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy oleg430-server: Loading modules from /home/green/git/lustre-release/lustre oleg430-server: detected 4 online CPUs by sysfs oleg430-server: MODOPTS_LIBCFS= oleg430-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg430-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg430-server: oleg430-server.virtnet: executing set_default_debug -1 all pdsh@oleg430-client: oleg430-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg430-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg430-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg430-server: oleg430-server.virtnet: executing set_default_debug -1 all pdsh@oleg430-client: oleg430-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg430-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg430-server: oleg430-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg430-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg430-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg430-server stop mds service on oleg430-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg430-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 10:52:09 (1754405529) Stopping clients: oleg430-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg430-client.virtnet /mnt/lustre2 (opts:-f) oleg430-server: oleg430-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg430-server' oleg430-server: oleg430-server.virtnet: executing load_modules_local oleg430-server: Loading modules from /home/green/git/lustre-release/lustre oleg430-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy oleg430-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy oleg430-server: detected 4 online CPUs by sysfs oleg430-server: MODOPTS_LIBCFS= oleg430-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg430-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg430-server: oleg430-server.virtnet: executing set_default_debug -1 all pdsh@oleg430-client: oleg430-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg430-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg430-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg430-server: oleg430-server.virtnet: executing set_default_debug -1 all pdsh@oleg430-client: oleg430-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg430-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg430-client.virtnet: -o user_xattr,flock 192.168.204.130@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 940 (time 1754405580.17 total 10.01 last 93.94) - open/close 1683 (time 1754405590.19 total 20.02 last 74.19) - open/close 2391 (time 1754405600.19 total 30.03 last 70.77) - open/close 3149 (time 1754405610.20 total 40.03 last 75.76) - open/close 3883 (time 1754405620.20 total 50.03 last 73.38) - open/close 4629 (time 1754405630.20 total 60.04 last 74.56) - open/close 5493 (time 1754405640.21 total 70.04 last 86.35) - open/close 6317 (time 1754405650.21 total 80.05 last 82.36) - open/close 7155 (time 1754405660.22 total 90.05 last 83.79) - open/close 7994 (time 1754405670.22 total 100.05 last 83.88) - open/close 8813 (time 1754405680.22 total 110.05 last 81.89) - open/close 9688 (time 1754405690.23 total 120.06 last 87.44) - open/close 10000 (time 1754405693.83 total 123.67 last 86.48) - open/close 10852 (time 1754405703.83 total 133.67 last 85.19) - open/close 11686 (time 1754405713.84 total 143.68 last 83.34) - open/close 12528 (time 1754405723.85 total 153.68 last 84.17) - open/close 13360 (time 1754405733.85 total 163.69 last 83.15) - open/close 14190 (time 1754405743.86 total 173.69 last 82.96) - open/close 15063 (time 1754405753.86 total 183.70 last 87.22) - open/close 15935 (time 1754405763.87 total 193.71 last 87.14) - open/close 17005 (time 1754405773.87 total 203.71 last 106.98) - open/close 17963 (time 1754405783.88 total 213.72 last 95.72) - open/close 18868 (time 1754405793.89 total 223.72 last 90.46) - open/close 19815 (time 1754405803.89 total 233.72 last 94.68) - open/close 20775 (time 1754405813.89 total 243.73 last 95.95) - open/close 21635 (time 1754405823.89 total 253.73 last 85.99) - open/close 22452 (time 1754405833.93 total 263.76 last 81.45) - open/close 23409 (time 1754405843.93 total 273.76 last 95.67) - open/close 24376 (time 1754405853.94 total 283.77 last 96.63) - open/close 25358 (time 1754405863.94 total 293.77 last 98.17) - open/close 26309 (time 1754405873.95 total 303.78 last 95.03) - open/close 27249 (time 1754405883.95 total 313.78 last 93.99) - open/close 28217 (time 1754405893.95 total 323.79 last 96.74) - open/close 29221 (time 1754405903.96 total 333.80 last 100.32) - open/close 30000 (time 1754405912.03 total 341.86 last 96.54) - open/close 30899 (time 1754405922.04 total 351.87 last 89.85) - open/close 31821 (time 1754405932.04 total 361.87 last 92.17) - open/close 32725 (time 1754405942.04 total 371.88 last 90.37) - open/close 33687 (time 1754405952.05 total 381.88 last 96.15) - open/close 34723 (time 1754405962.05 total 391.88 last 103.59) - open/close 35690 (time 1754405972.05 total 401.89 last 96.65) - open/close 36576 (time 1754405982.07 total 411.90 last 88.46) - open/close 37482 (time 1754405992.07 total 421.91 last 90.57) - open/close 38340 (time 1754406002.08 total 431.92 last 85.73) - open/close 39381 (time 1754406012.09 total 441.92 last 104.04) - open/close 40000 (time 1754406018.39 total 448.23 last 98.16) - open/close 40991 (time 1754406028.39 total 458.23 last 99.09) - open/close 42057 (time 1754406038.40 total 468.23 last 106.58) - open/close 43024 (time 1754406048.40 total 478.24 last 96.65) - open/close 43949 (time 1754406058.41 total 488.25 last 92.41) - open/close 44865 (time 1754406068.41 total 498.25 last 91.58) - open/close 45819 (time 1754406078.41 total 508.25 last 95.39) - open/close 46727 (time 1754406088.42 total 518.26 last 90.72) - open/close 47787 (time 1754406098.43 total 528.27 last 105.89) - open/close 48797 (time 1754406108.44 total 538.28 last 100.93) - open/close 49877 (time 1754406118.45 total 548.28 last 107.93) - open/close 50954 (time 1754406128.45 total 558.28 last 107.70) - open/close 52069 (time 1754406138.46 total 568.29 last 111.42) - open/close 53202 (time 1754406148.46 total 578.29 last 113.26) - open/close 54435 (time 1754406158.46 total 588.30 last 123.24) - open/close 55591 (time 1754406168.47 total 598.30 last 115.56) - open/close 56758 (time 1754406178.47 total 608.31 last 116.63) - open/close 57723 (time 1754406188.47 total 618.31 last 96.49) - open/close 58850 (time 1754406198.48 total 628.31 last 112.64) - open/close 60000 (time 1754406207.13 total 636.97 last 132.91) - open/close 61099 (time 1754406217.13 total 646.97 last 109.89) - open/close 62192 (time 1754406227.14 total 656.97 last 109.27) - open/close 63293 (time 1754406237.14 total 666.98 last 110.02) - open/close 64431 (time 1754406247.15 total 676.99 last 113.70) total: 64768 open/close in 679.53 seconds: 95.31 ops/second - /unlink 1335 (time 1754406261.68 total 10.00 last 133.45) - /unlink 2680 (time 1754406271.68 total 20.01 last 134.44) - /unlink 4238 (time 1754406281.68 total 30.01 last 155.79) - /unlink 5679 (time 1754406291.69 total 40.01 last 144.07) - /unlink 7067 (time 1754406301.69 total 50.02 last 138.68) - /unlink 8145 (time 1754406311.69 total 60.02 last 107.80) - /unlink 9623 (time 1754406321.70 total 70.03 last 147.71) - /unlink 10000 (time 1754406324.45 total 72.77 last 137.26) - /unlink 11442 (time 1754406334.45 total 82.77 last 144.19) - /unlink 13061 (time 1754406344.45 total 92.78 last 161.84) - /unlink 14514 (time 1754406354.46 total 102.78 last 145.19) - /unlink 16215 (time 1754406364.46 total 112.79 last 170.09) - /unlink 18754 (time 1754406374.46 total 122.79 last 253.86) - /unlink 20000 (time 1754406380.36 total 128.69 last 211.22) - /unlink 22347 (time 1754406390.36 total 138.69 last 234.68) - /unlink 24794 (time 1754406400.36 total 148.69 last 244.61) - /unlink 27274 (time 1754406410.37 total 158.70 last 247.84) - /unlink 29992 (time 1754406420.37 total 168.70 last 271.77) - /unlink 33129 (time 1754406430.37 total 178.70 last 313.64) - /unlink 36870 (time 1754406440.37 total 188.70 last 374.08) - /unlink 40000 (time 1754406450.18 total 198.51 last 319.07) - /unlink 43588 (time 1754406460.18 total 208.51 last 358.80) - /unlink 46458 (time 1754406470.19 total 218.51 last 286.96) - /unlink 49317 (time 1754406480.19 total 228.51 last 285.89) - /unlink 50000 (time 1754406482.49 total 230.81 last 296.67) - /unlink 52568 (time 1754406492.49 total 240.81 last 256.80) - /unlink 54972 (time 1754406502.49 total 250.82 last 240.34) - /unlink 57215 (time 1754406512.49 total 260.82 last 224.27) - /unlink 59846 (time 1754406522.49 total 270.82 last 263.06) - /unlink 62566 (time 1754406532.50 total 280.82 last 271.96) total: 64768 /unlink in 288.31 seconds: 224.65 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1386 (time 1754406573.81 total 10.00 last 138.56) - open/close 2993 (time 1754406583.81 total 20.01 last 160.64) - open/close 4413 (time 1754406593.81 total 30.01 last 141.99) - open/close 6138 (time 1754406603.81 total 40.01 last 172.49) - open/close 7843 (time 1754406613.82 total 50.01 last 170.43) - open/close 10000 (time 1754406623.60 total 59.79 last 220.60) - open/close 12519 (time 1754406633.60 total 69.79 last 251.85) - open/close 14461 (time 1754406643.60 total 79.79 last 194.18) - open/close 16267 (time 1754406653.60 total 89.79 last 180.55) - open/close 18440 (time 1754406663.60 total 99.80 last 217.25) - open/close 20000 (time 1754406670.86 total 107.05 last 215.00) - open/close 21702 (time 1754406680.86 total 117.05 last 170.19) - open/close 23514 (time 1754406690.86 total 127.05 last 181.18) - open/close 25625 (time 1754406700.86 total 137.06 last 211.04) - open/close 27742 (time 1754406710.87 total 147.06 last 211.68) - open/close 29914 (time 1754406720.87 total 157.06 last 217.12) - open/close 32101 (time 1754406730.87 total 167.07 last 218.57) - open/close 33821 (time 1754406740.88 total 177.07 last 171.98) - open/close 35504 (time 1754406750.88 total 187.07 last 168.24) - open/close 37454 (time 1754406760.88 total 197.07 last 194.97) - open/close 39212 (time 1754406770.89 total 207.08 last 175.71) - open/close 40000 (time 1754406775.38 total 211.58 last 175.20) - open/close 41755 (time 1754406785.39 total 221.58 last 175.44) - open/close 43777 (time 1754406795.39 total 231.58 last 202.14) - open/close 45701 (time 1754406805.40 total 241.59 last 192.27) - open/close 47835 (time 1754406815.40 total 251.59 last 213.35) - open/close 50000 (time 1754406824.36 total 260.56 last 241.50) - open/close 52531 (time 1754406834.36 total 270.56 last 253.09) - open/close 55251 (time 1754406844.37 total 280.56 last 271.91) - open/close 57639 (time 1754406854.37 total 290.57 last 238.69) - open/close 59449 (time 1754406864.37 total 300.57 last 180.95) - open/close 60000 (time 1754406866.92 total 303.11 last 216.39) - open/close 62562 (time 1754406876.92 total 313.12 last 256.17) - open/close 64505 (time 1754406886.93 total 323.12 last 194.21) total: 64768 open/close in 324.54 seconds: 199.57 ops/second - /unlink 3255 (time 1754406899.99 total 10.00 last 325.40) - /unlink 6425 (time 1754406910.00 total 20.00 last 316.95) - /unlink 10000 (time 1754406919.69 total 29.70 last 368.70) - /unlink 13781 (time 1754406929.69 total 39.70 last 378.08) - /unlink 18064 (time 1754406939.70 total 49.70 last 428.17) - /unlink 20000 (time 1754406944.20 total 54.21 last 429.68) - /unlink 23698 (time 1754406954.20 total 64.21 last 369.75) - /unlink 27625 (time 1754406964.20 total 74.21 last 392.61) - /unlink 30000 (time 1754406970.40 total 80.41 last 383.36) - /unlink 33154 (time 1754406980.40 total 90.41 last 315.32) - /unlink 36825 (time 1754406990.40 total 100.41 last 367.05) - /unlink 40000 (time 1754406999.08 total 109.09 last 365.99) - /unlink 43203 (time 1754407009.08 total 119.09 last 320.26) - /unlink 47375 (time 1754407019.08 total 129.09 last 417.19) - /unlink 50000 (time 1754407024.60 total 134.60 last 475.91) - /unlink 53862 (time 1754407034.60 total 144.61 last 385.96) - /unlink 57327 (time 1754407044.60 total 154.61 last 346.49) - /unlink 60000 (time 1754407051.05 total 161.06 last 414.68) - /unlink 63889 (time 1754407061.05 total 171.06 last 388.81) total: 64768 /unlink in 173.75 seconds: 372.77 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 2079 (time 1754407092.33 total 10.00 last 207.84) - open/close 4245 (time 1754407102.33 total 20.00 last 216.59) - open/close 6218 (time 1754407112.33 total 30.00 last 197.30) - open/close 8557 (time 1754407122.33 total 40.01 last 233.84) - open/close 10000 (time 1754407128.92 total 46.59 last 219.02) - open/close 12174 (time 1754407138.92 total 56.60 last 217.39) - open/close 14490 (time 1754407148.92 total 66.60 last 231.58) - open/close 16464 (time 1754407158.93 total 76.60 last 197.26) - open/close 18646 (time 1754407168.93 total 86.60 last 218.18) - open/close 20000 (time 1754407175.90 total 93.58 last 194.12) - open/close 21986 (time 1754407185.91 total 103.58 last 198.56) - open/close 23875 (time 1754407195.91 total 113.58 last 188.85) - open/close 25855 (time 1754407205.91 total 123.59 last 197.92) - open/close 27778 (time 1754407215.92 total 133.60 last 192.17) - open/close 30000 (time 1754407225.60 total 143.28 last 229.52) - open/close 32554 (time 1754407235.60 total 153.28 last 255.32) - open/close 35007 (time 1754407245.61 total 163.28 last 245.24) - open/close 37515 (time 1754407255.61 total 173.28 last 250.73) - open/close 39846 (time 1754407265.61 total 183.29 last 233.03) - open/close 42248 (time 1754407275.62 total 193.29 last 240.09) - open/close 44527 (time 1754407285.62 total 203.29 last 227.85) - open/close 46804 (time 1754407295.62 total 213.29 last 227.69) - open/close 49131 (time 1754407305.62 total 223.30 last 232.69) - open/close 50000 (time 1754407308.80 total 226.47 last 273.56) - open/close 52466 (time 1754407318.80 total 236.47 last 246.58) - open/close 55018 (time 1754407328.80 total 246.48 last 255.10) - open/close 57110 (time 1754407338.80 total 256.48 last 209.15) - open/close 60000 (time 1754407348.24 total 265.91 last 306.35) - open/close 62744 (time 1754407358.24 total 275.92 last 274.32) total: 64768 open/close in 282.38 seconds: 229.36 ops/second - /unlink 5725 (time 1754407376.18 total 10.00 last 572.50) - /unlink 10000 (time 1754407383.68 total 17.50 last 569.75) - /unlink 15304 (time 1754407393.68 total 27.50 last 530.38) - /unlink 20000 (time 1754407403.37 total 37.19 last 484.76) - /unlink 24736 (time 1754407413.37 total 47.19 last 473.53) - /unlink 30000 (time 1754407423.31 total 57.14 last 529.42) - /unlink 35713 (time 1754407433.31 total 67.14 last 571.28) - /unlink 40000 (time 1754407440.73 total 74.55 last 578.15) - /unlink 45173 (time 1754407450.73 total 84.55 last 517.29) - /unlink 50000 (time 1754407459.71 total 93.53 last 537.47) - /unlink 56078 (time 1754407469.71 total 103.53 last 607.78) - /unlink 60000 (time 1754407477.16 total 110.98 last 526.52) total: 64768 /unlink in 119.25 seconds: 543.14 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 2516 (time 1754407519.71 total 10.00 last 251.52) - open/close 5443 (time 1754407529.71 total 20.01 last 292.63) - open/close 8236 (time 1754407539.72 total 30.01 last 279.26) - open/close 10000 (time 1754407545.26 total 35.56 last 317.92) - open/close 12785 (time 1754407555.26 total 45.56 last 278.48) - open/close 15938 (time 1754407565.27 total 55.56 last 315.24) - open/close 19533 (time 1754407575.27 total 65.56 last 359.47) - open/close 23254 (time 1754407585.27 total 75.56 last 372.03) - open/close 26745 (time 1754407595.27 total 85.56 last 349.08) - open/close 29812 (time 1754407605.27 total 95.56 last 306.67) - open/close 32677 (time 1754407615.27 total 105.56 last 286.48) - open/close 35910 (time 1754407625.27 total 115.56 last 323.27) - open/close 38788 (time 1754407635.28 total 125.57 last 287.70) - open/close 40000 (time 1754407640.27 total 130.56 last 242.67) - open/close 42788 (time 1754407650.27 total 140.56 last 278.74) - open/close 45687 (time 1754407660.27 total 150.56 last 289.88) - open/close 48526 (time 1754407670.28 total 160.57 last 283.84) - open/close 50000 (time 1754407675.43 total 165.72 last 285.79) - open/close 53114 (time 1754407685.43 total 175.72 last 311.37) - open/close 56453 (time 1754407695.43 total 185.73 last 333.87) - open/close 59696 (time 1754407705.44 total 195.73 last 324.21) - open/close 62622 (time 1754407715.44 total 205.73 last 292.57) total: 64768 open/close in 212.51 seconds: 304.77 ops/second - /unlink 5925 (time 1754407733.66 total 10.00 last 592.49) - /unlink 10000 (time 1754407740.31 total 16.65 last 612.55) - /unlink 16508 (time 1754407750.31 total 26.65 last 650.75) - /unlink 20000 (time 1754407755.93 total 32.27 last 621.54) - /unlink 26116 (time 1754407765.93 total 42.27 last 611.60) - /unlink 30000 (time 1754407772.30 total 48.64 last 609.79) - /unlink 36993 (time 1754407782.30 total 58.64 last 699.27) - /unlink 40000 (time 1754407785.99 total 62.34 last 813.82) - /unlink 46930 (time 1754407795.99 total 72.34 last 692.97) - /unlink 50000 (time 1754407800.55 total 76.89 last 674.56) - /unlink 57562 (time 1754407810.55 total 86.89 last 756.13) - /unlink 60000 (time 1754407813.93 total 90.28 last 719.95) total: 64768 /unlink in 96.18 seconds: 673.39 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 4385 (time 1754407850.32 total 10.00 last 438.48) - open/close 8634 (time 1754407860.33 total 20.00 last 424.84) - open/close 10000 (time 1754407863.74 total 23.42 last 399.74) - open/close 13822 (time 1754407873.74 total 33.42 last 382.12) - open/close 17399 (time 1754407883.75 total 43.43 last 357.55) - open/close 20000 (time 1754407890.09 total 49.77 last 409.99) - open/close 24735 (time 1754407900.09 total 59.77 last 473.50) - open/close 29363 (time 1754407910.09 total 69.77 last 462.72) - open/close 33644 (time 1754407920.10 total 79.77 last 428.03) - open/close 38038 (time 1754407930.10 total 89.77 last 439.32) - open/close 40000 (time 1754407934.72 total 94.39 last 424.87) - open/close 44272 (time 1754407944.72 total 104.39 last 427.11) - open/close 48756 (time 1754407954.72 total 114.40 last 448.32) - open/close 50000 (time 1754407957.62 total 117.30 last 428.58) - open/close 54295 (time 1754407967.62 total 127.30 last 429.48) - open/close 58490 (time 1754407977.62 total 137.30 last 419.42) - open/close 60000 (time 1754407981.39 total 141.07 last 401.15) - open/close 64280 (time 1754407991.39 total 151.07 last 427.96) total: 64768 open/close in 152.28 seconds: 425.31 ops/second - /unlink 7574 (time 1754408003.98 total 10.00 last 757.39) - /unlink 10000 (time 1754408006.98 total 13.00 last 809.48) - /unlink 18234 (time 1754408016.98 total 23.00 last 823.34) - /unlink 20000 (time 1754408019.01 total 25.03 last 868.50) - /unlink 27889 (time 1754408029.01 total 35.03 last 788.82) - /unlink 30000 (time 1754408032.43 total 38.45 last 618.31) - /unlink 36970 (time 1754408042.43 total 48.45 last 697.00) - /unlink 40000 (time 1754408046.30 total 52.31 last 783.29) - /unlink 46558 (time 1754408056.30 total 62.32 last 655.61) - /unlink 48525 (time 1754408066.31 total 72.33 last 196.46) - /unlink 50000 (time 1754408075.69 total 81.71 last 157.30) - /unlink 51360 (time 1754408085.69 total 91.71 last 135.98) - /unlink 52996 (time 1754408095.70 total 101.71 last 163.51) - /unlink 56807 (time 1754408105.70 total 111.72 last 380.99) - /unlink 60000 (time 1754408113.07 total 119.09 last 432.94) - /unlink 64267 (time 1754408123.08 total 129.09 last 426.65) total: 64768 /unlink in 130.29 seconds: 497.11 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg430-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg430-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg430-server stop mds service on oleg430-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg430-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg430-server' oleg430-server: oleg430-server.virtnet: executing unload_modules_local oleg430-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg430-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg430-client: oleg430-client: ssh exited with exit code 2 pdsh@oleg430-client: oleg430-server: ssh exited with exit code 2 pdsh@oleg430-client: oleg430-client: ssh exited with exit code 2 pdsh@oleg430-client: oleg430-server: ssh exited with exit code 2 PASS 106 (2633s) Stopping clients: oleg430-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg430-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg430-client: oleg430-server: ssh exited with exit code 2 oleg430-server: oleg430-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg430-server' oleg430-server: oleg430-server.virtnet: executing load_modules_local oleg430-server: Loading modules from /home/green/git/lustre-release/lustre oleg430-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1024: echo: write error: Device or resource busy oleg430-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1025: echo: write error: Device or resource busy oleg430-server: detected 4 online CPUs by sysfs oleg430-server: MODOPTS_LIBCFS= oleg430-server: Force libcfs to create 2 CPU partitions oleg430-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg430-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 2740 sec ========== 11:36:31 (1754408191) === conf-sanity: start cleanup 11:36:31 (1754408191) === === conf-sanity: finish cleanup 11:36:32 (1754408192) ===