-----============= acceptance-small: conf-sanity ============----- Tue Jun 17 12:45:03 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg103-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg103-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg103-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg103-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg103-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg103-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg103-server oleg103-server: oleg103-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg103-server' oleg103-server: oleg103-server.virtnet: executing load_modules_local oleg103-server: Loading modules from /home/green/git/lustre-release/lustre oleg103-server: detected 4 online CPUs by sysfs oleg103-server: MODOPTS_LIBCFS= oleg103-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg103-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg103-server: oleg103-server.virtnet: executing set_default_debug -1 all pdsh@oleg103-client: oleg103-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg103-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg103-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg103-server: oleg103-server.virtnet: executing set_default_debug -1 all pdsh@oleg103-client: oleg103-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg103-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg103-server: oleg103-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg103-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg103-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg103-server stop mds service on oleg103-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg103-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 12:46:24 (1750178784) Stopping clients: oleg103-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg103-client.virtnet /mnt/lustre2 (opts:-f) oleg103-server: oleg103-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg103-server' oleg103-server: oleg103-server.virtnet: executing load_modules_local oleg103-server: Loading modules from /home/green/git/lustre-release/lustre oleg103-server: detected 4 online CPUs by sysfs oleg103-server: MODOPTS_LIBCFS= oleg103-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg103-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg103-server: oleg103-server.virtnet: executing set_default_debug -1 all pdsh@oleg103-client: oleg103-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg103-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg103-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg103-server: oleg103-server.virtnet: executing set_default_debug -1 all pdsh@oleg103-client: oleg103-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg103-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg103-client.virtnet: -o user_xattr,flock 192.168.201.103@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1388 (time 1750178839.28 total 10.01 last 138.68) - open/close 2808 (time 1750178849.29 total 20.01 last 141.94) - open/close 4248 (time 1750178859.29 total 30.02 last 143.94) - open/close 5891 (time 1750178869.30 total 40.02 last 164.23) - open/close 7290 (time 1750178879.30 total 50.03 last 139.79) - open/close 8750 (time 1750178889.31 total 60.03 last 145.91) - open/close 10000 (time 1750178897.98 total 68.70 last 144.18) - open/close 11418 (time 1750178907.99 total 78.71 last 141.70) - open/close 12789 (time 1750178917.99 total 88.72 last 137.03) - open/close 14163 (time 1750178928.00 total 98.72 last 137.29) - open/close 15425 (time 1750178938.01 total 108.73 last 126.08) - open/close 16811 (time 1750178948.02 total 118.74 last 138.52) - open/close 18192 (time 1750178958.02 total 128.74 last 138.09) - open/close 19646 (time 1750178968.02 total 138.75 last 145.32) - open/close 20000 (time 1750178970.34 total 141.07 last 152.51) - open/close 21579 (time 1750178980.35 total 151.07 last 157.83) - open/close 23310 (time 1750178990.35 total 161.07 last 173.09) - open/close 25108 (time 1750179000.35 total 171.07 last 179.74) - open/close 27329 (time 1750179010.35 total 181.08 last 222.05) - open/close 29141 (time 1750179020.36 total 191.08 last 181.15) - open/close 30000 (time 1750179025.54 total 196.26 last 165.68) - open/close 31682 (time 1750179035.55 total 206.27 last 168.11) - open/close 33262 (time 1750179045.55 total 216.28 last 157.90) - open/close 34985 (time 1750179055.56 total 226.28 last 172.18) - open/close 37094 (time 1750179065.56 total 236.28 last 210.88) - open/close 39478 (time 1750179075.56 total 246.29 last 238.32) - open/close 40000 (time 1750179077.68 total 248.41 last 246.16) - open/close 42394 (time 1750179087.68 total 258.41 last 239.39) - open/close 45085 (time 1750179097.69 total 268.41 last 269.02) - open/close 47654 (time 1750179107.69 total 278.41 last 256.87) - open/close 49930 (time 1750179117.69 total 288.42 last 227.45) - open/close 52166 (time 1750179127.70 total 298.42 last 223.57) - open/close 54514 (time 1750179137.70 total 308.42 last 234.78) - open/close 56415 (time 1750179147.70 total 318.42 last 190.05) - open/close 58375 (time 1750179157.70 total 328.43 last 195.96) - open/close 60000 (time 1750179165.10 total 335.82 last 219.66) - open/close 61973 (time 1750179175.10 total 345.83 last 197.25) - open/close 64119 (time 1750179185.10 total 355.83 last 214.58) total: 64768 open/close in 358.60 seconds: 180.61 ops/second - /unlink 2633 (time 1750179199.05 total 10.01 last 263.16) - /unlink 4340 (time 1750179209.05 total 20.01 last 170.66) - /unlink 6288 (time 1750179219.05 total 30.01 last 194.78) - /unlink 8597 (time 1750179229.05 total 40.01 last 230.89) - /unlink 10000 (time 1750179236.07 total 47.03 last 199.84) - /unlink 12129 (time 1750179246.08 total 57.03 last 212.79) - /unlink 14304 (time 1750179256.08 total 67.04 last 217.45) - /unlink 16084 (time 1750179266.08 total 77.04 last 177.92) - /unlink 17968 (time 1750179276.08 total 87.04 last 188.38) - /unlink 20000 (time 1750179285.88 total 96.84 last 207.33) - /unlink 22309 (time 1750179295.89 total 106.85 last 230.80) - /unlink 24804 (time 1750179305.89 total 116.85 last 249.45) - /unlink 27431 (time 1750179315.89 total 126.85 last 262.61) - /unlink 29967 (time 1750179325.90 total 136.85 last 253.57) - /unlink 32809 (time 1750179335.90 total 146.85 last 284.19) - /unlink 36041 (time 1750179345.90 total 156.86 last 323.15) - /unlink 38715 (time 1750179355.90 total 166.86 last 267.38) - /unlink 40000 (time 1750179361.09 total 172.05 last 247.45) - /unlink 42590 (time 1750179371.09 total 182.05 last 258.94) - /unlink 45055 (time 1750179381.10 total 192.05 last 246.45) - /unlink 47353 (time 1750179391.10 total 202.06 last 229.70) - /unlink 49819 (time 1750179401.10 total 212.06 last 246.60) - /unlink 52070 (time 1750179411.10 total 222.06 last 225.06) - /unlink 54438 (time 1750179421.10 total 232.06 last 236.75) - /unlink 57166 (time 1750179431.10 total 242.06 last 272.78) - /unlink 59946 (time 1750179441.11 total 252.06 last 277.97) - /unlink 63074 (time 1750179451.11 total 262.06 last 312.79) total: 64768 /unlink in 266.30 seconds: 243.22 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1659 (time 1750179484.81 total 10.00 last 165.88) - open/close 3470 (time 1750179494.81 total 20.00 last 181.06) - open/close 5261 (time 1750179504.82 total 30.01 last 179.05) - open/close 7397 (time 1750179514.82 total 40.01 last 213.57) - open/close 9166 (time 1750179524.82 total 50.01 last 176.86) - open/close 10000 (time 1750179529.66 total 54.85 last 172.45) - open/close 11585 (time 1750179539.66 total 64.85 last 158.44) - open/close 13187 (time 1750179549.66 total 74.86 last 160.11) - open/close 14789 (time 1750179559.67 total 84.86 last 160.14) - open/close 16316 (time 1750179569.67 total 94.86 last 152.68) - open/close 17980 (time 1750179579.67 total 104.86 last 166.33) - open/close 19603 (time 1750179589.68 total 114.87 last 162.21) - open/close 20000 (time 1750179592.16 total 117.36 last 159.76) - open/close 21537 (time 1750179602.17 total 127.36 last 153.59) - open/close 23107 (time 1750179612.17 total 137.37 last 156.96) - open/close 24935 (time 1750179622.17 total 147.37 last 182.78) - open/close 27028 (time 1750179632.18 total 157.37 last 209.26) - open/close 29208 (time 1750179642.18 total 167.37 last 217.87) - open/close 30000 (time 1750179646.55 total 171.74 last 181.48) - open/close 31632 (time 1750179656.55 total 181.74 last 163.16) - open/close 33345 (time 1750179666.55 total 191.74 last 171.29) - open/close 35172 (time 1750179676.55 total 201.74 last 182.65) - open/close 37054 (time 1750179686.55 total 211.75 last 188.17) - open/close 38793 (time 1750179696.56 total 221.75 last 173.88) - open/close 40000 (time 1750179702.89 total 228.08 last 190.58) - open/close 41977 (time 1750179712.89 total 238.08 last 197.65) - open/close 43864 (time 1750179722.89 total 248.08 last 188.70) - open/close 45639 (time 1750179732.90 total 258.09 last 177.40) - open/close 47782 (time 1750179742.90 total 268.09 last 214.29) - open/close 49863 (time 1750179752.90 total 278.09 last 208.03) - open/close 51822 (time 1750179762.90 total 288.09 last 195.86) - open/close 53623 (time 1750179772.91 total 298.10 last 179.99) - open/close 55597 (time 1750179782.91 total 308.11 last 197.28) - open/close 57523 (time 1750179792.92 total 318.11 last 192.59) - open/close 59266 (time 1750179802.92 total 328.11 last 174.27) - open/close 60000 (time 1750179806.81 total 332.00 last 188.56) - open/close 61962 (time 1750179816.82 total 342.01 last 196.09) - open/close 63901 (time 1750179826.82 total 352.01 last 193.88) total: 64768 open/close in 355.83 seconds: 182.02 ops/second - /unlink 3873 (time 1750179841.70 total 10.00 last 387.21) - /unlink 7042 (time 1750179851.70 total 20.00 last 316.83) - /unlink 8871 (time 1750179861.70 total 30.01 last 182.81) - /unlink 10000 (time 1750179868.68 total 36.98 last 161.90) - /unlink 12046 (time 1750179878.68 total 46.99 last 204.52) - /unlink 14017 (time 1750179888.69 total 56.99 last 197.02) - /unlink 16007 (time 1750179898.69 total 66.99 last 199.00) - /unlink 18013 (time 1750179908.69 total 76.99 last 200.52) - /unlink 20000 (time 1750179918.48 total 86.78 last 203.04) - /unlink 22212 (time 1750179928.48 total 96.79 last 221.08) - /unlink 24372 (time 1750179938.48 total 106.79 last 215.95) - /unlink 26599 (time 1750179948.49 total 116.79 last 222.66) - /unlink 29452 (time 1750179958.49 total 126.79 last 285.29) - /unlink 32533 (time 1750179968.49 total 136.79 last 308.06) - /unlink 35358 (time 1750179978.49 total 146.79 last 282.47) - /unlink 37907 (time 1750179988.49 total 156.79 last 254.89) - /unlink 40000 (time 1750179996.86 total 165.16 last 250.15) - /unlink 42575 (time 1750180006.86 total 175.16 last 257.45) - /unlink 45026 (time 1750180016.86 total 185.16 last 245.06) - /unlink 48043 (time 1750180026.87 total 195.17 last 301.40) - /unlink 50000 (time 1750180035.10 total 203.41 last 237.73) - /unlink 52332 (time 1750180045.10 total 213.41 last 233.15) - /unlink 54693 (time 1750180055.10 total 223.41 last 236.09) - /unlink 56704 (time 1750180065.11 total 233.41 last 201.02) - /unlink 58796 (time 1750180075.11 total 243.42 last 209.14) - /unlink 60000 (time 1750180080.72 total 249.03 last 214.59) - /unlink 62319 (time 1750180090.72 total 259.03 last 231.89) - /unlink 64421 (time 1750180100.72 total 269.03 last 210.18) total: 64768 /unlink in 270.69 seconds: 239.27 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1391 (time 1750180131.36 total 10.01 last 139.00) - open/close 2694 (time 1750180141.36 total 20.01 last 130.28) - open/close 4133 (time 1750180151.36 total 30.01 last 143.86) - open/close 5568 (time 1750180161.37 total 40.02 last 143.41) - open/close 6956 (time 1750180171.37 total 50.02 last 138.77) - open/close 8250 (time 1750180181.38 total 60.03 last 129.32) - open/close 9596 (time 1750180191.38 total 70.03 last 134.56) - open/close 10000 (time 1750180194.35 total 72.99 last 136.26) - open/close 11489 (time 1750180204.36 total 83.00 last 148.74) - open/close 13016 (time 1750180214.37 total 93.01 last 152.55) - open/close 14440 (time 1750180224.37 total 103.02 last 142.29) - open/close 15918 (time 1750180234.38 total 113.03 last 147.67) - open/close 17401 (time 1750180244.39 total 123.04 last 148.23) - open/close 18893 (time 1750180254.39 total 133.04 last 149.13) - open/close 20000 (time 1750180261.87 total 140.52 last 148.02) - open/close 21374 (time 1750180271.88 total 150.53 last 137.31) - open/close 22728 (time 1750180281.88 total 160.53 last 135.37) - open/close 24185 (time 1750180291.88 total 170.53 last 145.64) - open/close 25666 (time 1750180301.88 total 180.53 last 148.10) - open/close 27341 (time 1750180311.89 total 190.54 last 167.39) - open/close 28911 (time 1750180321.90 total 200.55 last 156.86) - open/close 30000 (time 1750180329.37 total 208.02 last 145.77) - open/close 31607 (time 1750180339.37 total 218.02 last 160.66) - open/close 33061 (time 1750180349.38 total 228.03 last 145.33) - open/close 34446 (time 1750180359.38 total 238.03 last 138.48) - open/close 35764 (time 1750180369.38 total 248.03 last 131.76) - open/close 37164 (time 1750180379.39 total 258.03 last 139.94) - open/close 38513 (time 1750180389.39 total 268.04 last 134.86) - open/close 39918 (time 1750180399.40 total 278.04 last 140.42) - open/close 41363 (time 1750180409.40 total 288.05 last 144.44) - open/close 42865 (time 1750180419.40 total 298.05 last 150.17) - open/close 44281 (time 1750180429.41 total 308.06 last 141.45) - open/close 45683 (time 1750180439.42 total 318.06 last 140.13) - open/close 46953 (time 1750180449.43 total 328.07 last 126.88) - open/close 48286 (time 1750180459.43 total 338.08 last 133.24) - open/close 49862 (time 1750180469.43 total 348.08 last 157.54) - open/close 51173 (time 1750180479.44 total 358.09 last 131.00) - open/close 52661 (time 1750180489.45 total 368.09 last 148.75) - open/close 54232 (time 1750180499.45 total 378.10 last 157.04) - open/close 55913 (time 1750180509.46 total 388.11 last 167.91) - open/close 57496 (time 1750180519.46 total 398.11 last 158.27) - open/close 59416 (time 1750180529.46 total 408.11 last 191.98) - open/close 60000 (time 1750180532.61 total 411.26 last 185.50) - open/close 61620 (time 1750180542.62 total 421.26 last 161.93) - open/close 63289 (time 1750180552.62 total 431.27 last 166.85) total: 64768 open/close in 439.95 seconds: 147.22 ops/second - /unlink 3047 (time 1750180572.77 total 10.00 last 304.63) - /unlink 6026 (time 1750180582.77 total 20.00 last 297.84) - /unlink 8761 (time 1750180592.77 total 30.00 last 273.49) - /unlink 10000 (time 1750180596.65 total 33.88 last 319.49) - /unlink 12758 (time 1750180606.65 total 43.88 last 275.80) - /unlink 15814 (time 1750180616.65 total 53.88 last 305.57) - /unlink 19328 (time 1750180626.66 total 63.89 last 351.14) - /unlink 20000 (time 1750180628.82 total 66.05 last 310.82) - /unlink 23217 (time 1750180638.82 total 76.05 last 321.70) - /unlink 26128 (time 1750180648.82 total 86.06 last 291.02) - /unlink 29260 (time 1750180658.82 total 96.06 last 313.15) - /unlink 30000 (time 1750180661.63 total 98.86 last 263.85) - /unlink 32898 (time 1750180671.63 total 108.87 last 289.66) - /unlink 35845 (time 1750180681.63 total 118.87 last 294.67) - /unlink 38664 (time 1750180691.64 total 128.87 last 281.83) - /unlink 40000 (time 1750180696.62 total 133.85 last 268.10) - /unlink 42587 (time 1750180706.62 total 143.86 last 258.64) - /unlink 45281 (time 1750180716.62 total 153.86 last 269.34) - /unlink 48137 (time 1750180726.63 total 163.86 last 285.52) - /unlink 50000 (time 1750180734.17 total 171.40 last 246.98) - /unlink 52553 (time 1750180744.17 total 181.41 last 255.26) - /unlink 55359 (time 1750180754.17 total 191.41 last 280.54) - /unlink 57928 (time 1750180764.18 total 201.41 last 256.86) - /unlink 60000 (time 1750180772.04 total 209.27 last 263.60) - /unlink 62656 (time 1750180782.04 total 219.27 last 265.58) total: 64768 /unlink in 227.47 seconds: 284.73 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1468 (time 1750180817.52 total 10.00 last 146.79) - open/close 3150 (time 1750180827.52 total 20.00 last 168.19) - open/close 4758 (time 1750180837.52 total 30.00 last 160.78) - open/close 6513 (time 1750180847.52 total 40.01 last 175.43) - open/close 8338 (time 1750180857.52 total 50.01 last 182.49) - open/close 9957 (time 1750180867.53 total 60.01 last 161.86) - open/close 11457 (time 1750180877.53 total 70.01 last 149.99) - open/close 13007 (time 1750180887.53 total 80.01 last 154.97) - open/close 14731 (time 1750180897.53 total 90.02 last 172.33) - open/close 16254 (time 1750180907.53 total 100.02 last 152.28) - open/close 17775 (time 1750180917.54 total 110.02 last 151.99) - open/close 19263 (time 1750180927.54 total 120.03 last 148.77) - open/close 20000 (time 1750180932.66 total 125.14 last 144.10) - open/close 21457 (time 1750180942.66 total 135.14 last 145.69) - open/close 22860 (time 1750180952.66 total 145.15 last 140.22) - open/close 24215 (time 1750180962.66 total 155.15 last 135.49) - open/close 25816 (time 1750180972.67 total 165.16 last 159.97) - open/close 27294 (time 1750180982.68 total 175.16 last 147.76) - open/close 28813 (time 1750180992.68 total 185.16 last 151.83) - open/close 30000 (time 1750181000.48 total 192.97 last 152.11) - open/close 31465 (time 1750181010.49 total 202.97 last 146.47) - open/close 32970 (time 1750181020.49 total 212.97 last 150.43) - open/close 34528 (time 1750181030.50 total 222.98 last 155.70) - open/close 36091 (time 1750181040.50 total 232.98 last 156.27) - open/close 37910 (time 1750181050.50 total 242.99 last 181.85) - open/close 39573 (time 1750181060.51 total 252.99 last 166.23) - open/close 40000 (time 1750181063.21 total 255.69 last 158.13) - open/close 41553 (time 1750181073.22 total 265.70 last 155.15) - open/close 43187 (time 1750181083.22 total 275.70 last 163.37) - open/close 44630 (time 1750181093.22 total 285.70 last 144.30) - open/close 46141 (time 1750181103.22 total 295.70 last 151.10) - open/close 47733 (time 1750181113.22 total 305.71 last 159.14) - open/close 49351 (time 1750181123.22 total 315.71 last 161.79) - open/close 50000 (time 1750181126.98 total 319.46 last 172.86) - open/close 51744 (time 1750181136.98 total 329.47 last 174.31) - open/close 53338 (time 1750181146.98 total 339.47 last 159.37) - open/close 54833 (time 1750181156.99 total 349.47 last 149.40) - open/close 56248 (time 1750181167.00 total 359.48 last 141.40) - open/close 57525 (time 1750181177.00 total 369.48 last 127.68) - open/close 58853 (time 1750181187.00 total 379.49 last 132.77) - open/close 60000 (time 1750181194.88 total 387.36 last 145.65) - open/close 61373 (time 1750181204.88 total 397.36 last 137.27) - open/close 62731 (time 1750181214.88 total 407.37 last 135.74) - open/close 64132 (time 1750181224.89 total 417.37 last 140.06) total: 64768 open/close in 422.09 seconds: 153.44 ops/second - /unlink 2153 (time 1750181241.68 total 10.00 last 215.27) - /unlink 4407 (time 1750181251.68 total 20.00 last 225.33) - /unlink 6843 (time 1750181261.68 total 30.01 last 243.51) - /unlink 9097 (time 1750181271.68 total 40.01 last 225.38) - /unlink 10000 (time 1750181275.69 total 44.01 last 225.53) - /unlink 12462 (time 1750181285.69 total 54.01 last 246.17) - /unlink 14920 (time 1750181295.70 total 64.02 last 245.66) - /unlink 17342 (time 1750181305.70 total 74.02 last 242.15) - /unlink 19950 (time 1750181315.70 total 84.02 last 260.74) - /unlink 22315 (time 1750181325.70 total 94.03 last 236.46) - /unlink 24514 (time 1750181335.70 total 104.03 last 219.89) - /unlink 26947 (time 1750181345.70 total 114.03 last 243.23) - /unlink 29197 (time 1750181355.71 total 124.03 last 224.91) - /unlink 30000 (time 1750181359.26 total 127.58 last 226.27) - /unlink 32467 (time 1750181369.26 total 137.59 last 246.61) - /unlink 35505 (time 1750181379.26 total 147.59 last 303.79) - /unlink 38973 (time 1750181389.26 total 157.59 last 346.79) - /unlink 40000 (time 1750181392.03 total 160.35 last 371.01) - /unlink 43051 (time 1750181402.03 total 170.36 last 305.03) - /unlink 45766 (time 1750181412.03 total 180.36 last 271.48) - /unlink 48607 (time 1750181422.04 total 190.36 last 284.02) - /unlink 50000 (time 1750181426.59 total 194.92 last 305.64) - /unlink 53345 (time 1750181436.60 total 204.92 last 334.40) - /unlink 56747 (time 1750181446.60 total 214.92 last 340.12) - /unlink 59643 (time 1750181456.60 total 224.93 last 289.54) - /unlink 63038 (time 1750181466.60 total 234.93 last 339.45) total: 64768 /unlink in 239.09 seconds: 270.89 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1822 (time 1750181502.99 total 10.00 last 182.16) - open/close 3684 (time 1750181513.00 total 20.00 last 186.18) - open/close 5538 (time 1750181523.00 total 30.01 last 185.33) - open/close 7235 (time 1750181533.01 total 40.01 last 169.60) - open/close 8914 (time 1750181543.01 total 50.02 last 167.76) - open/close 10000 (time 1750181548.80 total 55.81 last 187.59) - open/close 11572 (time 1750181558.80 total 65.81 last 157.19) - open/close 13170 (time 1750181568.81 total 75.82 last 159.72) - open/close 14946 (time 1750181578.81 total 85.82 last 177.52) - open/close 16517 (time 1750181588.82 total 95.83 last 157.01) - open/close 18097 (time 1750181598.82 total 105.83 last 158.00) - open/close 19733 (time 1750181608.82 total 115.83 last 163.53) - open/close 21312 (time 1750181618.83 total 125.83 last 157.85) - open/close 22816 (time 1750181628.83 total 135.84 last 150.31) - open/close 24396 (time 1750181638.84 total 145.84 last 157.94) - open/close 26025 (time 1750181648.84 total 155.85 last 162.82) - open/close 27425 (time 1750181658.84 total 165.85 last 139.96) - open/close 28863 (time 1750181668.85 total 175.85 last 143.76) - open/close 30000 (time 1750181676.34 total 183.35 last 151.65) - open/close 31399 (time 1750181686.35 total 193.36 last 139.84) - open/close 32889 (time 1750181696.36 total 203.37 last 148.83) - open/close 34398 (time 1750181706.36 total 213.37 last 150.83) - open/close 35930 (time 1750181716.37 total 223.37 last 153.18) - open/close 37339 (time 1750181726.37 total 233.38 last 140.86) - open/close 38836 (time 1750181736.37 total 243.38 last 149.70) - open/close 40000 (time 1750181742.30 total 249.31 last 196.34) - open/close 42079 (time 1750181752.30 total 259.31 last 207.88) - open/close 43688 (time 1750181762.30 total 269.31 last 160.84) - open/close 45457 (time 1750181772.31 total 279.31 last 176.83) - open/close 47045 (time 1750181782.31 total 289.31 last 158.79) - open/close 49210 (time 1750181792.31 total 299.32 last 216.43) - open/close 50000 (time 1750181796.60 total 303.61 last 184.12) - open/close 51791 (time 1750181806.60 total 313.61 last 179.04) - open/close 53639 (time 1750181816.61 total 323.61 last 184.78) - open/close 55687 (time 1750181826.61 total 333.62 last 204.68) - open/close 57405 (time 1750181836.61 total 343.62 last 171.77) - open/close 58955 (time 1750181846.62 total 353.62 last 154.95) - open/close 60000 (time 1750181852.56 total 359.57 last 175.69) - open/close 61854 (time 1750181862.56 total 369.57 last 185.40) - open/close 63550 (time 1750181872.57 total 379.58 last 169.52) total: 64768 open/close in 387.58 seconds: 167.11 ops/second - /unlink 3427 (time 1750181892.18 total 10.00 last 342.65) - /unlink 6435 (time 1750181902.18 total 20.01 last 300.68) - /unlink 9384 (time 1750181912.18 total 30.01 last 294.89) - /unlink 10000 (time 1750181914.34 total 32.16 last 286.41) - /unlink 12794 (time 1750181924.34 total 42.16 last 279.39) - /unlink 16003 (time 1750181934.34 total 52.16 last 320.83) - /unlink 18833 (time 1750181944.34 total 62.16 last 282.90) - /unlink 20000 (time 1750181948.78 total 66.60 last 262.71) - /unlink 22758 (time 1750181958.79 total 76.61 last 275.74) - /unlink 25440 (time 1750181968.79 total 86.61 last 268.13) - /unlink 28634 (time 1750181978.79 total 96.61 last 319.33) - /unlink 30000 (time 1750181982.52 total 100.34 last 366.40) - /unlink 33082 (time 1750181992.52 total 110.34 last 308.13) - /unlink 36133 (time 1750182002.52 total 120.34 last 305.09) - /unlink 39043 (time 1750182012.53 total 130.35 last 290.82) - /unlink 40000 (time 1750182016.86 total 134.68 last 220.86) - /unlink 42321 (time 1750182026.87 total 144.69 last 231.98) - /unlink 44813 (time 1750182036.87 total 154.69 last 249.17) - /unlink 47402 (time 1750182046.87 total 164.69 last 258.87) - /unlink 50000 (time 1750182056.02 total 173.84 last 283.99) - /unlink 53403 (time 1750182066.02 total 183.84 last 340.24) - /unlink 56535 (time 1750182076.02 total 193.84 last 313.13) - /unlink 59720 (time 1750182086.02 total 203.84 last 318.41) - /unlink 62427 (time 1750182096.03 total 213.85 last 270.59) total: 64768 /unlink in 222.82 seconds: 290.67 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg103-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg103-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg103-server stop mds service on oleg103-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg103-server unloading modules via unload_modules_local on: 'oleg103-server' oleg103-server: oleg103-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg103-client: oleg103-client: ssh exited with exit code 2 pdsh@oleg103-client: oleg103-server: ssh exited with exit code 2 pdsh@oleg103-client: oleg103-client: ssh exited with exit code 2 pdsh@oleg103-client: oleg103-server: ssh exited with exit code 2 PASS 106 (3353s) Stopping clients: oleg103-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg103-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg103-client: oleg103-server: ssh exited with exit code 2 oleg103-server: oleg103-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg103-server' oleg103-server: oleg103-server.virtnet: executing load_modules_local oleg103-server: Loading modules from /home/green/git/lustre-release/lustre oleg103-server: detected 4 online CPUs by sysfs oleg103-server: MODOPTS_LIBCFS= oleg103-server: Force libcfs to create 2 CPU partitions oleg103-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg103-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3460 sec ========== 13:42:44 (1750182164) === conf-sanity: start cleanup 13:42:45 (1750182165) === === conf-sanity: finish cleanup 13:42:46 (1750182166) ===