-----============= acceptance-small: conf-sanity ============----- Wed May 29 16:01:13 EDT 2024 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 excepting tests: 32newtarball 110 Stopping clients: oleg115-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg115-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg115-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg115-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg115-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg115-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg115-server oleg115-server: oleg115-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory loading modules on: 'oleg115-server' oleg115-server: oleg115-server.virtnet: executing load_modules_local oleg115-server: Loading modules from /home/green/git/lustre-release/lustre oleg115-server: detected 4 online CPUs by sysfs oleg115-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg115-server Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg115-server: oleg115-server.virtnet: executing set_default_debug -1 all pdsh@oleg115-client: oleg115-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg115-server Starting mds2: -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg115-server: oleg115-server.virtnet: executing set_default_debug -1 all pdsh@oleg115-client: oleg115-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg115-server Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg115-server: oleg115-server.virtnet: executing set_default_debug -1 all pdsh@oleg115-client: oleg115-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg115-server: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg115-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg115-server: oleg115-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg115-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg115-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg115-server stop mds service on oleg115-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg115-server stop mds service on oleg115-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg115-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 16:02:59 (1717012979) Stopping clients: oleg115-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg115-client.virtnet /mnt/lustre2 (opts:-f) oleg115-server: oleg115-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory loading modules on: 'oleg115-server' oleg115-server: oleg115-server.virtnet: executing load_modules_local oleg115-server: Loading modules from /home/green/git/lustre-release/lustre oleg115-server: detected 4 online CPUs by sysfs oleg115-server: Force libcfs to create 2 CPU partitions oleg115-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg115-server Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg115-server: oleg115-server.virtnet: executing set_default_debug -1 all pdsh@oleg115-client: oleg115-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg115-server Starting mds2: -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg115-server: oleg115-server.virtnet: executing set_default_debug -1 all pdsh@oleg115-client: oleg115-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg115-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg115-server Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg115-server: oleg115-server.virtnet: executing set_default_debug -1 all pdsh@oleg115-client: oleg115-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg115-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg115-client.virtnet: -o user_xattr,flock oleg115-server@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 274 79718 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 32896 368 32528 2% /mnt/lustre[OST:0] filesystem_summary: 33057 529 32528 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1457 (time 1717013041.36 total 10.00 last 145.64) - open/close 2809 (time 1717013051.37 total 20.01 last 135.06) - open/close 4142 (time 1717013061.37 total 30.02 last 133.25) - open/close 5575 (time 1717013071.37 total 40.02 last 143.26) - open/close 7034 (time 1717013081.37 total 50.02 last 145.89) - open/close 8668 (time 1717013091.38 total 60.02 last 163.37) - open/close 10000 (time 1717013099.60 total 68.25 last 161.94) - open/close 11572 (time 1717013109.61 total 78.26 last 157.02) - open/close 13339 (time 1717013119.62 total 88.26 last 176.64) - open/close 14843 (time 1717013129.62 total 98.27 last 150.39) - open/close 16673 (time 1717013139.62 total 108.27 last 182.93) - open/close 18383 (time 1717013149.63 total 118.28 last 170.89) - open/close 19989 (time 1717013159.63 total 128.28 last 160.57) - open/close 21555 (time 1717013169.63 total 138.28 last 156.54) - open/close 23085 (time 1717013179.63 total 148.28 last 152.96) - open/close 24622 (time 1717013189.64 total 158.29 last 153.67) - open/close 26142 (time 1717013199.64 total 168.29 last 151.93) - open/close 27636 (time 1717013209.65 total 178.30 last 149.30) - open/close 29365 (time 1717013219.65 total 188.30 last 172.80) - open/close 30000 (time 1717013223.38 total 192.03 last 170.43) - open/close 31543 (time 1717013233.38 total 202.03 last 154.28) open(/mnt/lustre/d106.conf-sanity/f-32560) error: No space left on device total: 32560 open/close in 209.33 seconds: 155.55 ops/second UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 32835 47157 42% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 32896 32896 0 100% /mnt/lustre[OST:0] filesystem_summary: 33090 33090 0 100% /mnt/lustre - /unlink 2376 (time 1717013252.48 total 10.00 last 237.59) - /unlink 5113 (time 1717013262.48 total 20.00 last 273.66) - /unlink 7295 (time 1717013272.48 total 30.00 last 218.13) - /unlink 9570 (time 1717013282.48 total 40.01 last 227.49) - /unlink 10000 (time 1717013284.52 total 42.04 last 211.23) - /unlink 12431 (time 1717013294.52 total 52.04 last 243.07) - /unlink 14860 (time 1717013304.52 total 62.04 last 242.84) - /unlink 17394 (time 1717013314.53 total 72.05 last 253.31) - /unlink 19838 (time 1717013324.53 total 82.05 last 244.38) - /unlink 22345 (time 1717013334.53 total 92.05 last 250.62) - /unlink 24806 (time 1717013344.53 total 102.06 last 246.01) - /unlink 27492 (time 1717013354.54 total 112.06 last 268.54) - /unlink 29962 (time 1717013364.54 total 122.06 last 246.95) - /unlink 32489 (time 1717013374.54 total 132.06 last 252.70) unlink(/mnt/lustre/d106.conf-sanity/f-32560) error: No such file or directory total: 32560 /unlink in 132.33 seconds: 246.05 ops/second Waiting for MDT destroys to complete - open/close 1436 (time 1717013396.42 total 10.00 last 143.53) - open/close 2800 (time 1717013406.42 total 20.01 last 136.35) - open/close 4171 (time 1717013416.42 total 30.01 last 137.09) - open/close 5362 (time 1717013426.43 total 40.01 last 119.05) - open/close 6695 (time 1717013436.44 total 50.02 last 133.15) - open/close 8003 (time 1717013446.45 total 60.03 last 130.69) - open/close 9217 (time 1717013456.45 total 70.03 last 121.38) - open/close 10000 (time 1717013462.61 total 76.20 last 126.96) - open/close 11418 (time 1717013472.62 total 86.20 last 141.77) - open/close 12813 (time 1717013482.62 total 96.21 last 139.40) - open/close 14079 (time 1717013492.63 total 106.21 last 126.54) - open/close 15405 (time 1717013502.63 total 116.22 last 132.51) - open/close 16757 (time 1717013512.64 total 126.23 last 135.12) - open/close 18206 (time 1717013522.64 total 136.23 last 144.88) - open/close 19534 (time 1717013532.65 total 146.24 last 132.65) - open/close 20000 (time 1717013536.16 total 149.74 last 132.97) - open/close 21298 (time 1717013546.17 total 159.75 last 129.68) - open/close 22619 (time 1717013556.18 total 169.76 last 131.98) - open/close 24024 (time 1717013566.18 total 179.77 last 140.43) - open/close 25316 (time 1717013576.18 total 189.77 last 129.19) - open/close 26857 (time 1717013586.19 total 199.77 last 154.03) - open/close 28292 (time 1717013596.19 total 209.78 last 143.45) - open/close 29618 (time 1717013606.20 total 219.78 last 132.51) - open/close 30000 (time 1717013608.94 total 222.52 last 139.46) - open/close 31253 (time 1717013618.95 total 232.53 last 125.19) - open/close 32544 (time 1717013628.95 total 242.54 last 129.01) open(/mnt/lustre/d106.conf-sanity/f-32560) error: No space left on device total: 32560 open/close in 242.66 seconds: 134.18 ops/second UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 32835 47157 42% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 32896 32896 0 100% /mnt/lustre[OST:0] filesystem_summary: 33090 33090 0 100% /mnt/lustre - /unlink 2417 (time 1717013640.62 total 10.00 last 241.64) - /unlink 4834 (time 1717013650.62 total 20.00 last 241.66) - /unlink 7556 (time 1717013660.62 total 30.00 last 272.17) - /unlink 10000 (time 1717013670.37 total 39.76 last 250.64) - /unlink 12510 (time 1717013680.38 total 49.76 last 250.87) - /unlink 14684 (time 1717013690.38 total 59.76 last 217.40) - /unlink 17002 (time 1717013700.38 total 69.76 last 231.78) - /unlink 19606 (time 1717013710.38 total 79.76 last 260.40) - /unlink 22318 (time 1717013720.38 total 89.76 last 271.14) - /unlink 25114 (time 1717013730.39 total 99.77 last 279.50) - /unlink 27837 (time 1717013740.39 total 109.77 last 272.20) - /unlink 30000 (time 1717013747.98 total 117.36 last 284.92) unlink(/mnt/lustre/d106.conf-sanity/f-32560) error: No such file or directory total: 32560 /unlink in 126.66 seconds: 257.06 ops/second Waiting for MDT destroys to complete - open/close 1518 (time 1717013772.20 total 10.00 last 151.73) - open/close 3069 (time 1717013782.21 total 20.01 last 155.03) - open/close 4620 (time 1717013792.21 total 30.01 last 155.03) - open/close 6141 (time 1717013802.21 total 40.01 last 152.09) - open/close 7740 (time 1717013812.21 total 50.02 last 159.86) - open/close 9225 (time 1717013822.22 total 60.02 last 148.43) - open/close 10000 (time 1717013827.70 total 65.50 last 141.39) - open/close 11634 (time 1717013837.70 total 75.51 last 163.35) - open/close 13437 (time 1717013847.71 total 85.51 last 180.25) - open/close 14760 (time 1717013857.71 total 95.51 last 132.22) - open/close 15880 (time 1717013867.72 total 105.52 last 111.89) - open/close 17188 (time 1717013877.72 total 115.53 last 130.76) - open/close 18450 (time 1717013887.73 total 125.53 last 126.16) - open/close 19841 (time 1717013897.73 total 135.53 last 139.10) - open/close 21358 (time 1717013907.73 total 145.53 last 151.67) - open/close 22835 (time 1717013917.73 total 155.53 last 147.70) - open/close 24153 (time 1717013927.73 total 165.53 last 131.78) - open/close 25437 (time 1717013937.73 total 175.54 last 128.39) - open/close 26821 (time 1717013947.74 total 185.54 last 138.34) - open/close 28133 (time 1717013957.74 total 195.54 last 131.16) - open/close 29465 (time 1717013967.74 total 205.54 last 133.19) - open/close 30000 (time 1717013971.78 total 209.58 last 132.46) - open/close 31428 (time 1717013981.78 total 219.58 last 142.76) open(/mnt/lustre/d106.conf-sanity/f-32560) error: No space left on device total: 32560 open/close in 228.70 seconds: 142.37 ops/second UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 32835 47157 42% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 32896 32896 0 100% /mnt/lustre[OST:0] filesystem_summary: 33090 33090 0 100% /mnt/lustre - /unlink 2142 (time 1717014003.02 total 10.00 last 214.16) - /unlink 4489 (time 1717014013.02 total 20.01 last 234.58) - /unlink 6976 (time 1717014023.03 total 30.01 last 248.61) - /unlink 9488 (time 1717014033.03 total 40.01 last 251.14) - /unlink 12023 (time 1717014043.03 total 50.01 last 253.48) - /unlink 14682 (time 1717014053.03 total 60.01 last 265.90) - /unlink 17403 (time 1717014063.03 total 70.02 last 272.01) - /unlink 20000 (time 1717014072.61 total 79.59 last 271.20) - /unlink 22655 (time 1717014082.61 total 89.60 last 265.41) - /unlink 25236 (time 1717014092.62 total 99.60 last 258.00) - /unlink 27858 (time 1717014102.62 total 109.60 last 262.12) - /unlink 30000 (time 1717014111.75 total 118.73 last 234.75) unlink(/mnt/lustre/d106.conf-sanity/f-32560) error: No such file or directory total: 32560 /unlink in 128.66 seconds: 253.06 ops/second Waiting for MDT destroys to complete - open/close 1508 (time 1717014143.66 total 10.01 last 150.65) - open/close 2872 (time 1717014153.66 total 20.02 last 136.32) - open/close 4177 (time 1717014163.67 total 30.02 last 130.43) - open/close 5499 (time 1717014173.68 total 40.03 last 132.13) - open/close 6716 (time 1717014183.68 total 50.03 last 121.69) - open/close 8053 (time 1717014193.69 total 60.04 last 133.57) - open/close 9275 (time 1717014203.69 total 70.04 last 122.18) - open/close 10000 (time 1717014209.64 total 75.99 last 121.80) - open/close 11346 (time 1717014219.64 total 85.99 last 134.56) - open/close 12697 (time 1717014229.64 total 95.99 last 135.10) - open/close 14129 (time 1717014239.65 total 106.00 last 143.16) - open/close 15458 (time 1717014249.65 total 116.00 last 132.85) - open/close 16900 (time 1717014259.66 total 126.01 last 144.10) - open/close 18300 (time 1717014269.66 total 136.01 last 139.93) - open/close 19788 (time 1717014279.67 total 146.02 last 148.76) - open/close 21202 (time 1717014289.67 total 156.02 last 141.30) - open/close 22815 (time 1717014299.67 total 166.02 last 161.27) - open/close 24362 (time 1717014309.68 total 176.03 last 154.68) - open/close 25641 (time 1717014319.69 total 186.04 last 127.75) - open/close 27071 (time 1717014329.69 total 196.04 last 142.94) - open/close 28689 (time 1717014339.69 total 206.04 last 161.78) - open/close 30000 (time 1717014348.27 total 214.62 last 152.87) - open/close 31480 (time 1717014358.27 total 224.62 last 148.00) open(/mnt/lustre/d106.conf-sanity/f-32560) error: No space left on device total: 32560 open/close in 233.57 seconds: 139.40 ops/second UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 32835 47157 42% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 32896 32896 0 100% /mnt/lustre[OST:0] filesystem_summary: 33090 33090 0 100% /mnt/lustre - /unlink 2127 (time 1717014379.58 total 10.00 last 212.63) - /unlink 4731 (time 1717014389.58 total 20.00 last 260.38) - /unlink 7463 (time 1717014399.58 total 30.01 last 273.16) - /unlink 10000 (time 1717014409.26 total 39.68 last 262.14) - /unlink 12445 (time 1717014419.26 total 49.69 last 244.41) - /unlink 14829 (time 1717014429.27 total 59.69 last 238.37) - /unlink 18102 (time 1717014439.27 total 69.69 last 327.20) - /unlink 20000 (time 1717014446.23 total 76.65 last 272.75) - /unlink 22360 (time 1717014456.23 total 86.65 last 235.93) - /unlink 25110 (time 1717014466.23 total 96.65 last 274.98) - /unlink 28152 (time 1717014476.23 total 106.66 last 304.13) - /unlink 30000 (time 1717014484.01 total 114.44 last 237.56) - /unlink 32205 (time 1717014494.02 total 124.44 last 220.42) unlink(/mnt/lustre/d106.conf-sanity/f-32560) error: No such file or directory total: 32560 /unlink in 126.01 seconds: 258.40 ops/second Waiting for MDT destroys to complete - open/close 1344 (time 1717014514.84 total 10.01 last 134.32) - open/close 2712 (time 1717014524.85 total 20.01 last 136.72) - open/close 4158 (time 1717014534.86 total 30.02 last 144.51) - open/close 5614 (time 1717014544.86 total 40.03 last 145.50) - open/close 6977 (time 1717014554.86 total 50.03 last 136.29) - open/close 8453 (time 1717014564.87 total 60.03 last 147.57) - open/close 9892 (time 1717014574.87 total 70.03 last 143.82) - open/close 11345 (time 1717014584.87 total 80.03 last 145.30) - open/close 12816 (time 1717014594.87 total 90.04 last 147.08) - open/close 14246 (time 1717014604.87 total 100.04 last 143.00) - open/close 15696 (time 1717014614.88 total 110.04 last 144.93) - open/close 17219 (time 1717014624.88 total 120.04 last 152.30) - open/close 18661 (time 1717014634.88 total 130.05 last 144.14) - open/close 20000 (time 1717014643.26 total 138.42 last 159.79) - open/close 21615 (time 1717014653.27 total 148.43 last 161.38) - open/close 23048 (time 1717014663.28 total 158.44 last 143.22) - open/close 24576 (time 1717014673.29 total 168.45 last 152.61) - open/close 26096 (time 1717014683.29 total 178.46 last 151.91) - open/close 27501 (time 1717014693.30 total 188.46 last 140.45) - open/close 28957 (time 1717014703.30 total 198.46 last 145.58) - open/close 30000 (time 1717014710.59 total 205.75 last 143.15) - open/close 31425 (time 1717014720.59 total 215.75 last 142.46) open(/mnt/lustre/d106.conf-sanity/f-32560) error: No space left on device total: 32560 open/close in 224.06 seconds: 145.32 ops/second UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 32835 47157 42% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 32896 32896 0 100% /mnt/lustre[OST:0] filesystem_summary: 33090 33090 0 100% /mnt/lustre - /unlink 2660 (time 1717014740.71 total 10.00 last 265.98) - /unlink 5203 (time 1717014750.71 total 20.00 last 254.25) - /unlink 8020 (time 1717014760.71 total 30.01 last 281.57) - /unlink 10000 (time 1717014767.32 total 36.61 last 299.72) - /unlink 12838 (time 1717014777.32 total 46.62 last 283.70) - /unlink 15891 (time 1717014787.32 total 56.62 last 305.29) - /unlink 18918 (time 1717014797.33 total 66.62 last 302.67) - /unlink 20000 (time 1717014800.79 total 70.09 last 312.11) - /unlink 23653 (time 1717014810.79 total 80.09 last 365.27) - /unlink 27472 (time 1717014820.80 total 90.09 last 381.78) - /unlink 30000 (time 1717014826.75 total 96.05 last 424.42) unlink(/mnt/lustre/d106.conf-sanity/f-32560) error: No such file or directory total: 32560 /unlink in 102.69 seconds: 317.06 ops/second Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg115-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg115-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg115-server stop mds service on oleg115-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg115-server stop mds service on oleg115-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg115-server unloading modules on: 'oleg115-server' oleg115-server: oleg115-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg115-client: oleg115-client: ssh exited with exit code 2 pdsh@oleg115-client: oleg115-server: ssh exited with exit code 2 pdsh@oleg115-client: oleg115-client: ssh exited with exit code 2 pdsh@oleg115-client: oleg115-server: ssh exited with exit code 2 PASS 106 (1890s) Stopping clients: oleg115-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg115-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg115-client: oleg115-server: ssh exited with exit code 2 oleg115-server: oleg115-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg115-server' oleg115-server: oleg115-server.virtnet: executing load_modules_local oleg115-server: Loading modules from /home/green/git/lustre-release/lustre oleg115-server: detected 4 online CPUs by sysfs oleg115-server: Force libcfs to create 2 CPU partitions oleg115-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg115-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 2024 sec ========== 16:34:58 (1717014898) === conf-sanity: start cleanup 16:34:59 (1717014899) === === conf-sanity: finish cleanup 16:34:59 (1717014899) ===