-----============= acceptance-small: conf-sanity ============----- Wed Apr 30 23:40:01 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg217-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg217-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg217-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg217-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg217-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg217-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg217-server oleg217-server: oleg217-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg217-server' oleg217-server: oleg217-server.virtnet: executing load_modules_local oleg217-server: Loading modules from /home/green/git/lustre-release/lustre oleg217-server: detected 4 online CPUs by sysfs oleg217-server: MODOPTS_LIBCFS= oleg217-server: Force libcfs to create 2 CPU partitions oleg217-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg217-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg217-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg217-server: oleg217-server.virtnet: executing set_default_debug -1 all pdsh@oleg217-client: oleg217-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg217-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg217-server: oleg217-server.virtnet: executing set_default_debug -1 all pdsh@oleg217-client: oleg217-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg217-server: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg217-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg217-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg217-server stop mds service on oleg217-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg217-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 23:41:18 (1746070878) Stopping clients: oleg217-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg217-client.virtnet /mnt/lustre2 (opts:-f) oleg217-server: oleg217-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg217-server' oleg217-server: oleg217-server.virtnet: executing load_modules_local oleg217-server: Loading modules from /home/green/git/lustre-release/lustre oleg217-server: detected 4 online CPUs by sysfs oleg217-server: MODOPTS_LIBCFS= oleg217-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg217-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg217-server: oleg217-server.virtnet: executing set_default_debug -1 all pdsh@oleg217-client: oleg217-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg217-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg217-server: oleg217-server.virtnet: executing set_default_debug -1 all pdsh@oleg217-client: oleg217-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg217-client.virtnet: -o user_xattr,flock 192.168.202.117@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1836 (time 1746070925.18 total 10.01 last 183.45) - open/close 3857 (time 1746070935.19 total 20.01 last 202.00) - open/close 5451 (time 1746070945.19 total 30.02 last 159.34) - open/close 6905 (time 1746070955.19 total 40.02 last 145.36) - open/close 8458 (time 1746070965.20 total 50.02 last 155.26) - open/close 9969 (time 1746070975.20 total 60.03 last 151.05) - open/close 11537 (time 1746070985.20 total 70.03 last 156.80) - open/close 13118 (time 1746070995.20 total 80.03 last 158.05) - open/close 14763 (time 1746071005.21 total 90.03 last 164.46) - open/close 16270 (time 1746071015.21 total 100.04 last 150.64) - open/close 17669 (time 1746071025.21 total 110.04 last 139.88) - open/close 19021 (time 1746071035.21 total 120.04 last 135.16) - open/close 20000 (time 1746071041.89 total 126.71 last 146.75) - open/close 21457 (time 1746071051.89 total 136.71 last 145.68) - open/close 23140 (time 1746071061.89 total 146.71 last 168.27) - open/close 24891 (time 1746071071.89 total 156.71 last 175.07) - open/close 26450 (time 1746071081.89 total 166.72 last 155.86) - open/close 27861 (time 1746071091.90 total 176.73 last 140.97) - open/close 29273 (time 1746071101.90 total 186.73 last 141.16) - open/close 30000 (time 1746071106.40 total 191.22 last 161.80) - open/close 31578 (time 1746071116.40 total 201.23 last 157.72) - open/close 33141 (time 1746071126.40 total 211.23 last 156.27) - open/close 35280 (time 1746071136.41 total 221.23 last 213.86) - open/close 37692 (time 1746071146.41 total 231.23 last 241.16) - open/close 39489 (time 1746071156.41 total 241.24 last 179.64) - open/close 40000 (time 1746071158.55 total 243.37 last 239.24) - open/close 42184 (time 1746071168.55 total 253.37 last 218.36) - open/close 44589 (time 1746071178.55 total 263.38 last 240.41) - open/close 47261 (time 1746071188.55 total 273.38 last 267.17) - open/close 49650 (time 1746071198.56 total 283.38 last 238.78) - open/close 51468 (time 1746071208.56 total 293.39 last 181.74) - open/close 53653 (time 1746071218.56 total 303.39 last 218.47) - open/close 55707 (time 1746071228.56 total 313.39 last 205.40) - open/close 57281 (time 1746071238.56 total 323.39 last 157.40) - open/close 59423 (time 1746071248.57 total 333.39 last 214.16) - open/close 60000 (time 1746071251.27 total 336.10 last 213.25) - open/close 61648 (time 1746071261.27 total 346.10 last 164.80) - open/close 64702 (time 1746071271.27 total 356.10 last 305.35) total: 64768 open/close in 356.41 seconds: 181.73 ops/second - /unlink 3750 (time 1746071282.72 total 10.00 last 374.97) - /unlink 6359 (time 1746071292.73 total 20.00 last 260.81) - /unlink 8707 (time 1746071302.73 total 30.01 last 234.76) - /unlink 10000 (time 1746071307.44 total 34.71 last 274.77) - /unlink 12618 (time 1746071317.44 total 44.71 last 261.75) - /unlink 15531 (time 1746071327.44 total 54.72 last 291.22) - /unlink 18046 (time 1746071337.44 total 64.72 last 251.49) - /unlink 20000 (time 1746071344.12 total 71.40 last 292.48) - /unlink 23648 (time 1746071354.12 total 81.40 last 364.75) - /unlink 26658 (time 1746071364.13 total 91.40 last 300.88) - /unlink 29588 (time 1746071374.13 total 101.41 last 292.91) - /unlink 32568 (time 1746071384.13 total 111.41 last 297.99) - /unlink 35250 (time 1746071394.13 total 121.41 last 268.11) - /unlink 37742 (time 1746071404.13 total 131.41 last 249.19) - /unlink 40000 (time 1746071410.92 total 138.19 last 332.92) - /unlink 42890 (time 1746071420.92 total 148.20 last 288.92) - /unlink 45770 (time 1746071430.92 total 158.20 last 287.97) - /unlink 47702 (time 1746071440.92 total 168.20 last 193.11) - /unlink 49602 (time 1746071450.93 total 178.20 last 189.94) - /unlink 51622 (time 1746071460.93 total 188.21 last 201.94) - /unlink 53351 (time 1746071470.93 total 198.21 last 172.83) - /unlink 55333 (time 1746071480.94 total 208.21 last 198.14) - /unlink 57360 (time 1746071490.94 total 218.21 last 202.70) - /unlink 59293 (time 1746071500.94 total 228.21 last 193.29) - /unlink 60000 (time 1746071503.88 total 231.16 last 239.99) - /unlink 62212 (time 1746071513.88 total 241.16 last 221.20) - /unlink 64424 (time 1746071523.89 total 251.16 last 221.17) total: 64768 /unlink in 252.71 seconds: 256.29 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1406 (time 1746071552.48 total 10.01 last 140.46) - open/close 2952 (time 1746071562.49 total 20.01 last 154.54) - open/close 4291 (time 1746071572.50 total 30.02 last 133.76) - open/close 5564 (time 1746071582.50 total 40.03 last 127.23) - open/close 7116 (time 1746071592.50 total 50.03 last 155.20) - open/close 8464 (time 1746071602.51 total 60.03 last 134.77) - open/close 10000 (time 1746071611.76 total 69.29 last 165.93) - open/close 11448 (time 1746071621.77 total 79.30 last 144.65) - open/close 12788 (time 1746071631.78 total 89.31 last 133.92) - open/close 14132 (time 1746071641.78 total 99.31 last 134.35) - open/close 15456 (time 1746071651.79 total 109.32 last 132.29) - open/close 16991 (time 1746071661.79 total 119.32 last 153.47) - open/close 19356 (time 1746071671.80 total 129.32 last 236.41) - open/close 20000 (time 1746071674.12 total 131.65 last 277.32) - open/close 21677 (time 1746071684.12 total 141.65 last 167.65) - open/close 23238 (time 1746071694.13 total 151.65 last 156.03) - open/close 24642 (time 1746071704.13 total 161.65 last 140.39) - open/close 25805 (time 1746071714.13 total 171.66 last 116.23) - open/close 27503 (time 1746071724.14 total 181.66 last 169.76) - open/close 29102 (time 1746071734.14 total 191.67 last 159.85) - open/close 30000 (time 1746071741.39 total 198.91 last 123.91) - open/close 31152 (time 1746071751.39 total 208.92 last 115.12) - open/close 32752 (time 1746071761.40 total 218.92 last 159.96) - open/close 34064 (time 1746071771.41 total 228.94 last 130.98) - open/close 35336 (time 1746071781.41 total 238.94 last 127.19) - open/close 36740 (time 1746071791.42 total 248.95 last 140.31) - open/close 38105 (time 1746071801.43 total 258.95 last 136.43) - open/close 39262 (time 1746071811.43 total 268.95 last 115.65) - open/close 40000 (time 1746071816.00 total 273.52 last 161.50) - open/close 41766 (time 1746071826.00 total 283.53 last 176.56) - open/close 43291 (time 1746071836.01 total 293.53 last 152.43) - open/close 44683 (time 1746071846.01 total 303.54 last 139.10) - open/close 46102 (time 1746071856.02 total 313.54 last 141.86) - open/close 47394 (time 1746071866.02 total 323.54 last 129.20) - open/close 48588 (time 1746071876.02 total 333.54 last 119.37) - open/close 49786 (time 1746071886.02 total 343.55 last 119.75) - open/close 51569 (time 1746071896.02 total 353.55 last 178.29) - open/close 52952 (time 1746071906.05 total 363.57 last 137.96) - open/close 54453 (time 1746071916.05 total 373.57 last 150.09) - open/close 56257 (time 1746071926.05 total 383.57 last 180.38) - open/close 57826 (time 1746071936.05 total 393.58 last 156.88) - open/close 59635 (time 1746071946.06 total 403.58 last 180.79) - open/close 60000 (time 1746071948.67 total 406.20 last 139.59) - open/close 62242 (time 1746071958.67 total 416.20 last 224.16) - open/close 64331 (time 1746071968.67 total 426.20 last 208.88) total: 64768 open/close in 427.94 seconds: 151.35 ops/second - /unlink 3984 (time 1746071981.53 total 10.00 last 398.39) - /unlink 8325 (time 1746071991.54 total 20.00 last 433.96) - /unlink 10000 (time 1746071996.87 total 25.33 last 314.34) - /unlink 13130 (time 1746072006.87 total 35.34 last 312.83) - /unlink 15780 (time 1746072016.87 total 45.34 last 264.97) - /unlink 19669 (time 1746072026.87 total 55.34 last 388.86) - /unlink 23931 (time 1746072036.88 total 65.34 last 426.15) - /unlink 26362 (time 1746072046.88 total 75.34 last 243.05) - /unlink 29361 (time 1746072056.88 total 85.34 last 299.90) - /unlink 30000 (time 1746072059.36 total 87.82 last 257.54) - /unlink 32753 (time 1746072069.36 total 97.83 last 275.24) - /unlink 37358 (time 1746072079.36 total 107.83 last 460.32) - /unlink 40000 (time 1746072087.11 total 115.58 last 341.05) - /unlink 43805 (time 1746072097.11 total 125.58 last 380.49) - /unlink 46955 (time 1746072107.11 total 135.58 last 314.97) - /unlink 50000 (time 1746072114.77 total 143.24 last 397.66) - /unlink 53435 (time 1746072124.77 total 153.24 last 343.40) - /unlink 57235 (time 1746072134.78 total 163.24 last 379.91) - /unlink 60000 (time 1746072140.85 total 169.32 last 454.96) - /unlink 64582 (time 1746072150.85 total 179.32 last 458.16) total: 64768 /unlink in 179.78 seconds: 360.25 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 2593 (time 1746072178.24 total 10.00 last 259.20) - open/close 4727 (time 1746072188.24 total 20.01 last 213.35) - open/close 7206 (time 1746072198.24 total 30.01 last 247.74) - open/close 10000 (time 1746072206.96 total 38.72 last 320.72) - open/close 13359 (time 1746072216.96 total 48.73 last 335.85) - open/close 15793 (time 1746072226.96 total 58.73 last 243.40) - open/close 18160 (time 1746072236.97 total 68.74 last 236.43) - open/close 20000 (time 1746072245.64 total 77.41 last 212.10) - open/close 22181 (time 1746072255.65 total 87.41 last 218.06) - open/close 24540 (time 1746072265.65 total 97.42 last 235.86) - open/close 26295 (time 1746072275.65 total 107.42 last 175.44) - open/close 27731 (time 1746072285.66 total 117.42 last 143.54) - open/close 29658 (time 1746072295.66 total 127.43 last 192.62) - open/close 31572 (time 1746072305.66 total 137.43 last 191.36) - open/close 33482 (time 1746072315.66 total 147.43 last 190.98) - open/close 35364 (time 1746072325.66 total 157.43 last 188.17) - open/close 38360 (time 1746072335.67 total 167.43 last 299.54) - open/close 40000 (time 1746072342.05 total 173.82 last 256.99) - open/close 41981 (time 1746072352.05 total 183.82 last 198.03) - open/close 44480 (time 1746072362.05 total 193.82 last 249.89) - open/close 47043 (time 1746072372.05 total 203.82 last 256.26) - open/close 49394 (time 1746072382.05 total 213.82 last 235.09) - open/close 50000 (time 1746072384.52 total 216.29 last 245.64) - open/close 52202 (time 1746072394.53 total 226.30 last 220.02) - open/close 54310 (time 1746072404.53 total 236.30 last 210.69) - open/close 56818 (time 1746072414.54 total 246.30 last 250.73) - open/close 59005 (time 1746072424.54 total 256.31 last 218.63) - open/close 60000 (time 1746072428.26 total 260.02 last 267.75) - open/close 62800 (time 1746072438.26 total 270.02 last 280.00) total: 64768 open/close in 276.99 seconds: 233.83 ops/second - /unlink 3963 (time 1746072456.45 total 10.00 last 396.20) - /unlink 8894 (time 1746072466.45 total 20.00 last 493.08) - /unlink 10000 (time 1746072469.41 total 22.96 last 373.67) - /unlink 14705 (time 1746072479.41 total 32.96 last 470.43) - /unlink 20000 (time 1746072487.50 total 41.05 last 655.03) - /unlink 25373 (time 1746072497.50 total 51.05 last 537.15) - /unlink 30000 (time 1746072504.42 total 57.98 last 667.99) - /unlink 36734 (time 1746072514.43 total 67.98 last 673.36) - /unlink 40000 (time 1746072518.42 total 71.97 last 818.08) - /unlink 46700 (time 1746072528.42 total 81.97 last 669.99) - /unlink 50000 (time 1746072534.44 total 87.99 last 548.15) - /unlink 55189 (time 1746072544.44 total 97.99 last 518.82) - /unlink 60000 (time 1746072552.38 total 105.94 last 605.67) total: 64768 /unlink in 112.93 seconds: 573.51 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 2553 (time 1746072589.25 total 10.00 last 255.27) - open/close 5547 (time 1746072599.25 total 20.00 last 299.34) - open/close 7949 (time 1746072609.25 total 30.00 last 240.19) - open/close 9444 (time 1746072619.26 total 40.01 last 149.34) - open/close 10000 (time 1746072622.61 total 43.36 last 166.21) - open/close 12291 (time 1746072632.61 total 53.36 last 229.03) - open/close 15144 (time 1746072642.61 total 63.36 last 285.29) - open/close 17948 (time 1746072652.61 total 73.36 last 280.37) - open/close 20000 (time 1746072661.31 total 82.07 last 235.80) - open/close 22698 (time 1746072671.32 total 92.07 last 269.77) - open/close 25625 (time 1746072681.32 total 102.07 last 292.56) - open/close 29264 (time 1746072691.32 total 112.07 last 363.88) - open/close 30000 (time 1746072693.59 total 114.34 last 324.55) - open/close 32302 (time 1746072703.59 total 124.34 last 230.19) - open/close 34688 (time 1746072713.59 total 134.34 last 238.54) - open/close 36611 (time 1746072723.59 total 144.35 last 192.26) - open/close 38453 (time 1746072733.60 total 154.35 last 184.16) - open/close 39913 (time 1746072743.60 total 164.35 last 145.89) - open/close 41273 (time 1746072753.61 total 174.36 last 135.97) - open/close 43183 (time 1746072763.61 total 184.36 last 190.96) - open/close 44932 (time 1746072773.61 total 194.37 last 174.77) - open/close 46897 (time 1746072783.62 total 204.37 last 196.46) - open/close 48721 (time 1746072793.62 total 214.37 last 182.35) - open/close 50000 (time 1746072801.15 total 221.90 last 169.88) - open/close 51654 (time 1746072811.16 total 231.91 last 165.27) - open/close 53400 (time 1746072821.16 total 241.91 last 174.51) - open/close 55230 (time 1746072831.17 total 251.92 last 182.91) - open/close 56839 (time 1746072841.17 total 261.92 last 160.88) - open/close 58463 (time 1746072851.18 total 271.93 last 162.27) - open/close 60000 (time 1746072859.12 total 279.87 last 193.45) - open/close 62611 (time 1746072869.12 total 289.87 last 261.08) total: 64768 open/close in 299.85 seconds: 216.00 ops/second - /unlink 3049 (time 1746072890.65 total 10.01 last 304.74) - /unlink 6274 (time 1746072900.65 total 20.01 last 322.31) - /unlink 9353 (time 1746072910.66 total 30.01 last 307.87) - /unlink 12701 (time 1746072920.66 total 40.01 last 334.73) - /unlink 15736 (time 1746072930.66 total 50.02 last 303.44) - /unlink 18471 (time 1746072940.66 total 60.02 last 273.43) - /unlink 20000 (time 1746072946.53 total 65.88 last 260.75) - /unlink 23199 (time 1746072956.53 total 75.88 last 319.90) - /unlink 27084 (time 1746072966.53 total 85.89 last 388.40) - /unlink 30000 (time 1746072973.80 total 93.15 last 401.14) - /unlink 33024 (time 1746072983.80 total 103.15 last 302.40) - /unlink 35867 (time 1746072993.80 total 113.16 last 284.29) - /unlink 39019 (time 1746073003.80 total 123.16 last 315.18) - /unlink 40000 (time 1746073007.00 total 126.36 last 306.09) - /unlink 42981 (time 1746073017.00 total 136.36 last 298.09) - /unlink 45991 (time 1746073027.01 total 146.36 last 300.95) - /unlink 49118 (time 1746073037.01 total 156.36 last 312.70) - /unlink 50000 (time 1746073039.19 total 158.55 last 403.00) - /unlink 52778 (time 1746073049.20 total 168.55 last 277.74) - /unlink 55450 (time 1746073059.20 total 178.55 last 267.18) - /unlink 58957 (time 1746073069.20 total 188.56 last 350.61) - /unlink 60000 (time 1746073072.58 total 191.93 last 308.90) - /unlink 63270 (time 1746073082.58 total 201.93 last 326.95) total: 64768 /unlink in 206.64 seconds: 313.44 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1452 (time 1746073115.17 total 10.00 last 145.16) - open/close 2805 (time 1746073125.17 total 20.01 last 135.22) - open/close 4432 (time 1746073135.18 total 30.02 last 162.58) - open/close 6235 (time 1746073145.18 total 40.02 last 180.28) - open/close 8237 (time 1746073155.18 total 50.02 last 200.18) - open/close 9531 (time 1746073165.19 total 60.02 last 129.37) - open/close 10000 (time 1746073169.12 total 63.95 last 119.33) - open/close 11323 (time 1746073179.12 total 73.96 last 132.21) - open/close 12587 (time 1746073189.13 total 83.97 last 126.26) - open/close 13694 (time 1746073199.14 total 93.98 last 110.63) - open/close 14774 (time 1746073209.15 total 103.99 last 107.89) - open/close 15942 (time 1746073219.15 total 113.99 last 116.80) - open/close 17399 (time 1746073229.15 total 123.99 last 145.67) - open/close 18739 (time 1746073239.16 total 133.99 last 133.94) - open/close 20000 (time 1746073247.51 total 142.35 last 150.94) - open/close 21524 (time 1746073257.51 total 152.35 last 152.38) - open/close 23185 (time 1746073267.52 total 162.35 last 166.00) - open/close 24709 (time 1746073277.52 total 172.36 last 152.36) - open/close 26304 (time 1746073287.53 total 182.36 last 159.40) - open/close 27847 (time 1746073297.54 total 192.37 last 154.14) - open/close 29087 (time 1746073307.55 total 202.38 last 123.90) - open/close 30000 (time 1746073313.68 total 208.52 last 148.76) - open/close 31723 (time 1746073323.68 total 218.52 last 172.29) - open/close 33192 (time 1746073333.69 total 228.53 last 146.80) - open/close 34342 (time 1746073343.69 total 238.53 last 114.97) - open/close 35678 (time 1746073353.71 total 248.54 last 133.42) - open/close 36819 (time 1746073363.71 total 258.55 last 114.01) - open/close 38034 (time 1746073373.73 total 268.56 last 121.37) - open/close 39453 (time 1746073383.73 total 278.57 last 141.82) - open/close 40000 (time 1746073387.45 total 282.29 last 146.91) - open/close 41321 (time 1746073397.45 total 292.29 last 132.09) - open/close 42795 (time 1746073407.46 total 302.29 last 147.34) - open/close 44016 (time 1746073417.46 total 312.30 last 122.05) - open/close 45329 (time 1746073427.46 total 322.30 last 131.28) - open/close 46704 (time 1746073437.48 total 332.31 last 137.30) - open/close 47896 (time 1746073447.48 total 342.32 last 119.16) - open/close 49212 (time 1746073457.49 total 352.32 last 131.51) - open/close 50000 (time 1746073462.55 total 357.38 last 155.65) - open/close 51129 (time 1746073472.55 total 367.39 last 112.90) - open/close 52428 (time 1746073482.55 total 377.39 last 129.87) - open/close 53781 (time 1746073492.56 total 387.40 last 135.16) - open/close 54864 (time 1746073502.56 total 397.40 last 108.28) - open/close 55933 (time 1746073512.60 total 407.44 last 106.51) - open/close 57031 (time 1746073522.62 total 417.45 last 109.63) - open/close 58137 (time 1746073532.62 total 427.45 last 110.57) - open/close 59289 (time 1746073542.63 total 437.46 last 115.12) - open/close 60000 (time 1746073549.46 total 444.29 last 104.11) - open/close 61135 (time 1746073559.47 total 454.30 last 113.39) - open/close 62144 (time 1746073569.47 total 464.31 last 100.82) - open/close 63209 (time 1746073579.48 total 474.31 last 106.44) - open/close 64408 (time 1746073589.49 total 484.32 last 119.80) total: 64768 open/close in 487.25 seconds: 132.93 ops/second - /unlink 1817 (time 1746073604.20 total 10.00 last 181.68) - /unlink 3813 (time 1746073614.21 total 20.00 last 199.55) - /unlink 6541 (time 1746073624.21 total 30.01 last 272.72) - /unlink 8766 (time 1746073634.21 total 40.01 last 222.44) - /unlink 10000 (time 1746073639.42 total 45.22 last 237.03) - /unlink 12466 (time 1746073649.42 total 55.22 last 246.50) - /unlink 14344 (time 1746073659.42 total 65.22 last 187.79) - /unlink 16712 (time 1746073669.42 total 75.22 last 236.77) - /unlink 18953 (time 1746073679.43 total 85.23 last 223.98) - /unlink 20000 (time 1746073684.09 total 89.89 last 224.75) - /unlink 22553 (time 1746073694.09 total 99.89 last 255.21) - /unlink 25096 (time 1746073704.09 total 109.89 last 254.25) - /unlink 27322 (time 1746073714.09 total 119.89 last 222.58) - /unlink 29634 (time 1746073724.09 total 129.89 last 231.18) - /unlink 31727 (time 1746073734.10 total 139.89 last 209.26) - /unlink 33963 (time 1746073744.10 total 149.90 last 223.55) - /unlink 36088 (time 1746073754.10 total 159.90 last 212.47) - /unlink 37894 (time 1746073764.11 total 169.91 last 180.36) - /unlink 39646 (time 1746073774.11 total 179.91 last 175.17) - /unlink 40000 (time 1746073776.18 total 181.98 last 171.15) - /unlink 42183 (time 1746073786.19 total 191.98 last 218.25) - /unlink 43955 (time 1746073796.19 total 201.98 last 177.17) - /unlink 46465 (time 1746073806.19 total 211.98 last 250.99) - /unlink 48789 (time 1746073816.19 total 221.99 last 232.38) - /unlink 50000 (time 1746073821.44 total 227.23 last 230.77) - /unlink 51810 (time 1746073831.44 total 237.24 last 180.95) - /unlink 53498 (time 1746073841.44 total 247.24 last 168.72) - /unlink 55324 (time 1746073851.45 total 257.25 last 182.51) - /unlink 57580 (time 1746073861.45 total 267.25 last 225.55) - /unlink 59967 (time 1746073871.45 total 277.25 last 238.59) - /unlink 62367 (time 1746073881.46 total 287.25 last 239.96) - /unlink 64684 (time 1746073891.46 total 297.26 last 231.61) total: 64768 /unlink in 297.69 seconds: 217.57 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg217-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg217-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg217-server stop mds service on oleg217-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg217-server unloading modules via unload_modules_local on: 'oleg217-server' oleg217-server: oleg217-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg217-client: oleg217-client: ssh exited with exit code 2 pdsh@oleg217-client: oleg217-server: ssh exited with exit code 2 pdsh@oleg217-client: oleg217-client: ssh exited with exit code 2 pdsh@oleg217-client: oleg217-server: ssh exited with exit code 2 PASS 106 (3052s) Stopping clients: oleg217-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg217-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg217-client: oleg217-server: ssh exited with exit code 2 oleg217-server: oleg217-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg217-server' oleg217-server: oleg217-server.virtnet: executing load_modules_local oleg217-server: Loading modules from /home/green/git/lustre-release/lustre oleg217-server: detected 4 online CPUs by sysfs oleg217-server: MODOPTS_LIBCFS= oleg217-server: Force libcfs to create 2 CPU partitions oleg217-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg217-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg217-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3160 sec ========== 00:32:42 (1746073962) === conf-sanity: start cleanup 00:32:43 (1746073963) === === conf-sanity: finish cleanup 00:32:45 (1746073965) ===