-----============= acceptance-small: conf-sanity ============----- Tue Jun 17 12:44:12 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg249-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball Stopping clients: oleg249-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg249-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg249-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg249-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg249-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg249-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg249-server oleg249-server: oleg249-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg249-server' oleg249-server: oleg249-server.virtnet: executing load_modules_local oleg249-server: Loading modules from /home/green/git/lustre-release/lustre oleg249-server: detected 4 online CPUs by sysfs oleg249-server: MODOPTS_LIBCFS= oleg249-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg249-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg249-server: oleg249-server.virtnet: executing set_default_debug -1 all pdsh@oleg249-client: oleg249-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg249-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg249-server: oleg249-server.virtnet: executing set_default_debug -1 all pdsh@oleg249-client: oleg249-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg249-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg249-server: oleg249-server.virtnet: executing set_default_debug -1 all pdsh@oleg249-client: oleg249-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg249-server: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg249-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg249-server: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg249-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg249-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg249-server stop mds service on oleg249-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg249-server stop mds service on oleg249-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg249-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 12:45:47 (1750178747) Stopping clients: oleg249-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg249-client.virtnet /mnt/lustre2 (opts:-f) oleg249-server: oleg249-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg249-server' oleg249-server: oleg249-server.virtnet: executing load_modules_local oleg249-server: Loading modules from /home/green/git/lustre-release/lustre oleg249-server: detected 4 online CPUs by sysfs oleg249-server: MODOPTS_LIBCFS= oleg249-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg249-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg249-server: oleg249-server.virtnet: executing set_default_debug -1 all pdsh@oleg249-client: oleg249-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg249-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg249-server: oleg249-server.virtnet: executing set_default_debug -1 all pdsh@oleg249-client: oleg249-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg249-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg249-server: oleg249-server.virtnet: executing set_default_debug -1 all pdsh@oleg249-client: oleg249-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg249-client.virtnet: -o user_xattr,flock 192.168.202.149@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 274 79718 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 65808 368 65440 1% /mnt/lustre[OST:0] filesystem_summary: 65969 529 65440 1% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 3731 (time 1750178791.02 total 10.00 last 373.04) - open/close 7552 (time 1750178801.02 total 20.00 last 382.03) - open/close 10000 (time 1750178809.68 total 28.66 last 282.79) - open/close 13052 (time 1750178819.68 total 38.66 last 305.05) - open/close 16663 (time 1750178829.68 total 48.67 last 361.08) - open/close 20000 (time 1750178839.66 total 58.64 last 334.54) - open/close 23473 (time 1750178849.66 total 68.64 last 347.27) - open/close 26537 (time 1750178859.66 total 78.64 last 306.39) - open/close 29018 (time 1750178869.66 total 88.64 last 248.02) - open/close 30000 (time 1750178872.25 total 91.23 last 379.31) - open/close 34596 (time 1750178882.25 total 101.24 last 459.52) - open/close 39122 (time 1750178892.26 total 111.24 last 452.54) - open/close 40000 (time 1750178894.61 total 113.59 last 373.23) - open/close 44739 (time 1750178904.61 total 123.59 last 473.90) - open/close 48903 (time 1750178914.61 total 133.59 last 416.33) - open/close 50000 (time 1750178917.21 total 136.19 last 422.18) - open/close 54963 (time 1750178927.21 total 146.19 last 496.21) - open/close 59023 (time 1750178937.21 total 156.19 last 405.94) - open/close 60000 (time 1750178939.52 total 158.50 last 423.00) - open/close 64245 (time 1750178949.52 total 168.50 last 424.47) total: 64768 open/close in 169.55 seconds: 382.01 ops/second - /unlink 7617 (time 1750178961.44 total 10.00 last 761.65) - /unlink 10000 (time 1750178964.33 total 12.89 last 824.11) - /unlink 18176 (time 1750178974.33 total 22.89 last 817.60) - /unlink 20000 (time 1750178976.43 total 24.99 last 870.74) - /unlink 29508 (time 1750178986.43 total 34.99 last 950.72) - /unlink 39548 (time 1750178996.43 total 44.99 last 1003.96) - /unlink 47671 (time 1750179006.43 total 54.99 last 812.28) - /unlink 50000 (time 1750179009.83 total 58.39 last 684.62) - /unlink 57922 (time 1750179019.83 total 68.39 last 792.17) - /unlink 60000 (time 1750179022.80 total 71.36 last 698.93) total: 64768 /unlink in 79.10 seconds: 818.76 ops/second Waiting for MDT destroys to complete - open/close 3967 (time 1750179124.36 total 10.00 last 396.58) - open/close 7720 (time 1750179134.36 total 20.00 last 375.27) - open/close 10000 (time 1750179141.47 total 27.12 last 320.43) - open/close 13052 (time 1750179151.48 total 37.12 last 305.13) - open/close 16088 (time 1750179161.48 total 47.12 last 303.59) - open/close 20000 (time 1750179171.37 total 57.01 last 395.46) - open/close 23619 (time 1750179181.37 total 67.02 last 361.80) - open/close 26780 (time 1750179191.37 total 77.02 last 315.99) - open/close 28468 (time 1750179201.39 total 87.03 last 168.58) - open/close 29590 (time 1750179211.39 total 97.04 last 112.18) - open/close 30000 (time 1750179215.03 total 100.68 last 112.47) - open/close 31134 (time 1750179225.04 total 110.68 last 113.36) - open/close 32884 (time 1750179235.04 total 120.69 last 174.92) - open/close 34380 (time 1750179245.05 total 130.69 last 149.55) - open/close 36059 (time 1750179255.05 total 140.69 last 167.88) - open/close 37525 (time 1750179265.05 total 150.69 last 146.59) - open/close 38800 (time 1750179275.05 total 160.70 last 127.41) - open/close 40000 (time 1750179283.81 total 169.46 last 137.06) - open/close 41374 (time 1750179293.81 total 179.46 last 137.37) - open/close 42972 (time 1750179303.81 total 189.46 last 159.75) - open/close 44854 (time 1750179313.82 total 199.46 last 188.19) - open/close 46814 (time 1750179323.83 total 209.48 last 195.74) - open/close 48241 (time 1750179333.83 total 219.48 last 142.69) - open/close 49582 (time 1750179343.84 total 229.48 last 134.02) - open/close 50000 (time 1750179346.74 total 232.39 last 143.89) - open/close 51314 (time 1750179356.74 total 242.39 last 131.36) - open/close 52676 (time 1750179366.75 total 252.39 last 136.17) - open/close 54086 (time 1750179376.75 total 262.39 last 140.97) - open/close 55440 (time 1750179386.75 total 272.40 last 135.33) - open/close 56715 (time 1750179396.76 total 282.40 last 127.47) - open/close 57785 (time 1750179406.76 total 292.40 last 106.99) - open/close 59081 (time 1750179416.76 total 302.40 last 129.58) - open/close 60000 (time 1750179423.90 total 309.54 last 128.73) - open/close 61345 (time 1750179433.90 total 319.54 last 134.49) - open/close 62666 (time 1750179443.90 total 329.55 last 132.05) - open/close 63903 (time 1750179453.90 total 339.55 last 123.66) total: 64768 open/close in 346.36 seconds: 187.00 ops/second - /unlink 2153 (time 1750179473.07 total 10.00 last 215.22) - /unlink 4322 (time 1750179483.07 total 20.00 last 216.88) - /unlink 6419 (time 1750179493.07 total 30.01 last 209.64) - /unlink 8623 (time 1750179503.08 total 40.01 last 220.37) - /unlink 10000 (time 1750179509.50 total 46.43 last 214.49) - /unlink 12238 (time 1750179519.50 total 56.43 last 223.77) - /unlink 14525 (time 1750179529.50 total 66.43 last 228.62) - /unlink 16868 (time 1750179539.50 total 76.44 last 234.23) - /unlink 19294 (time 1750179549.50 total 86.44 last 242.59) - /unlink 20000 (time 1750179552.63 total 89.57 last 225.60) - /unlink 22207 (time 1750179562.63 total 99.57 last 220.69) - /unlink 24599 (time 1750179572.64 total 109.57 last 239.13) - /unlink 26878 (time 1750179582.64 total 119.57 last 227.83) - /unlink 29218 (time 1750179592.64 total 129.57 last 233.98) - /unlink 30000 (time 1750179595.70 total 132.63 last 255.73) - /unlink 32309 (time 1750179605.70 total 142.63 last 230.89) - /unlink 34719 (time 1750179615.70 total 152.63 last 240.98) - /unlink 37079 (time 1750179625.70 total 162.64 last 235.92) - /unlink 39772 (time 1750179635.71 total 172.64 last 269.18) - /unlink 42329 (time 1750179645.71 total 182.64 last 255.67) - /unlink 44952 (time 1750179655.71 total 192.65 last 262.15) - /unlink 47838 (time 1750179665.72 total 202.65 last 288.58) - /unlink 50000 (time 1750179673.49 total 210.42 last 278.23) - /unlink 52802 (time 1750179683.49 total 220.42 last 280.18) - /unlink 55431 (time 1750179693.49 total 230.42 last 262.81) - /unlink 58252 (time 1750179703.49 total 240.43 last 282.04) - /unlink 60000 (time 1750179709.43 total 246.36 last 294.57) - /unlink 63111 (time 1750179719.43 total 256.36 last 311.07) total: 64768 /unlink in 261.05 seconds: 248.11 ops/second Waiting for MDT destroys to complete - open/close 1484 (time 1750179746.12 total 10.00 last 148.37) - open/close 3170 (time 1750179756.13 total 20.01 last 168.53) - open/close 4657 (time 1750179766.13 total 30.01 last 148.68) - open/close 6772 (time 1750179776.13 total 40.01 last 211.41) - open/close 8990 (time 1750179786.13 total 50.01 last 221.77) - open/close 10000 (time 1750179789.97 total 53.85 last 262.94) - open/close 12337 (time 1750179799.98 total 63.86 last 233.62) - open/close 14242 (time 1750179809.98 total 73.86 last 190.48) - open/close 15743 (time 1750179819.98 total 83.86 last 150.02) - open/close 17068 (time 1750179829.99 total 93.87 last 132.45) - open/close 18651 (time 1750179839.99 total 103.87 last 158.26) - open/close 20000 (time 1750179849.93 total 113.81 last 135.66) - open/close 21149 (time 1750179859.94 total 123.82 last 114.85) - open/close 22288 (time 1750179869.94 total 133.82 last 113.85) - open/close 23546 (time 1750179879.95 total 143.83 last 125.76) - open/close 25056 (time 1750179889.95 total 153.83 last 150.90) - open/close 26499 (time 1750179899.95 total 163.83 last 144.28) - open/close 27817 (time 1750179909.96 total 173.84 last 131.73) - open/close 29175 (time 1750179919.96 total 183.84 last 135.77) - open/close 30000 (time 1750179926.11 total 189.99 last 134.22) - open/close 31343 (time 1750179936.11 total 199.99 last 134.25) - open/close 33311 (time 1750179946.12 total 210.00 last 196.64) - open/close 34953 (time 1750179956.12 total 220.00 last 164.13) - open/close 36422 (time 1750179966.13 total 230.01 last 146.85) - open/close 38306 (time 1750179976.13 total 240.01 last 188.39) - open/close 39745 (time 1750179986.13 total 250.01 last 143.85) - open/close 40000 (time 1750179988.22 total 252.10 last 122.14) - open/close 41528 (time 1750179998.22 total 262.10 last 152.75) - open/close 42816 (time 1750180008.23 total 272.11 last 128.69) - open/close 44313 (time 1750180018.23 total 282.11 last 149.70) - open/close 45616 (time 1750180028.23 total 292.11 last 130.29) - open/close 47032 (time 1750180038.23 total 302.11 last 141.58) - open/close 48639 (time 1750180048.23 total 312.11 last 160.69) - open/close 50000 (time 1750180057.96 total 321.84 last 139.93) - open/close 51628 (time 1750180067.96 total 331.84 last 162.76) - open/close 53062 (time 1750180077.97 total 341.85 last 143.24) - open/close 54448 (time 1750180087.98 total 351.86 last 138.48) - open/close 55833 (time 1750180097.98 total 361.86 last 138.48) - open/close 57402 (time 1750180107.99 total 371.87 last 156.79) - open/close 58885 (time 1750180117.99 total 381.87 last 148.26) - open/close 60000 (time 1750180125.23 total 389.11 last 154.05) - open/close 61516 (time 1750180135.23 total 399.11 last 151.57) - open/close 62677 (time 1750180145.24 total 409.12 last 116.04) - open/close 63815 (time 1750180155.24 total 419.12 last 113.76) total: 64768 open/close in 427.19 seconds: 151.61 ops/second - /unlink 1976 (time 1750180175.05 total 10.01 last 197.47) - /unlink 3909 (time 1750180185.06 total 20.01 last 193.29) - /unlink 5977 (time 1750180195.06 total 30.01 last 206.69) - /unlink 8045 (time 1750180205.06 total 40.02 last 206.74) - /unlink 9977 (time 1750180215.07 total 50.02 last 193.07) - /unlink 12219 (time 1750180225.07 total 60.02 last 224.15) - /unlink 14372 (time 1750180235.07 total 70.03 last 215.27) - /unlink 16276 (time 1750180245.08 total 80.03 last 190.34) - /unlink 18333 (time 1750180255.08 total 90.03 last 205.64) - /unlink 20000 (time 1750180262.91 total 97.86 last 212.85) - /unlink 22191 (time 1750180272.91 total 107.87 last 219.05) - /unlink 24915 (time 1750180282.92 total 117.87 last 272.33) - /unlink 27743 (time 1750180292.92 total 127.87 last 282.71) - /unlink 30000 (time 1750180300.51 total 135.46 last 297.23) - /unlink 33058 (time 1750180310.52 total 145.47 last 305.73) - /unlink 36249 (time 1750180320.52 total 155.47 last 318.96) - /unlink 39600 (time 1750180330.52 total 165.48 last 334.97) - /unlink 42309 (time 1750180340.53 total 175.48 last 270.86) - /unlink 45344 (time 1750180350.53 total 185.48 last 303.48) - /unlink 48218 (time 1750180360.53 total 195.48 last 287.32) - /unlink 50000 (time 1750180366.34 total 201.29 last 306.83) - /unlink 52828 (time 1750180376.34 total 211.29 last 282.73) - /unlink 55434 (time 1750180386.34 total 221.29 last 260.53) - /unlink 58166 (time 1750180396.34 total 231.30 last 273.14) - /unlink 60000 (time 1750180402.28 total 237.23 last 308.97) - /unlink 63683 (time 1750180412.28 total 247.23 last 368.26) total: 64768 /unlink in 249.50 seconds: 259.59 ops/second Waiting for MDT destroys to complete - open/close 1986 (time 1750180437.49 total 10.00 last 198.59) - open/close 3543 (time 1750180447.49 total 20.00 last 155.66) - open/close 5399 (time 1750180457.50 total 30.00 last 185.58) - open/close 6941 (time 1750180467.50 total 40.01 last 154.07) - open/close 8467 (time 1750180477.51 total 50.02 last 152.55) - open/close 9720 (time 1750180487.52 total 60.03 last 125.16) - open/close 10000 (time 1750180489.82 total 62.33 last 121.43) - open/close 11324 (time 1750180499.83 total 72.34 last 132.34) - open/close 13389 (time 1750180509.83 total 82.34 last 206.46) - open/close 15216 (time 1750180519.83 total 92.34 last 182.68) - open/close 16808 (time 1750180529.83 total 102.34 last 159.15) - open/close 18259 (time 1750180539.84 total 112.35 last 145.02) - open/close 19950 (time 1750180549.84 total 122.35 last 169.01) - open/close 21550 (time 1750180559.86 total 132.36 last 159.82) - open/close 23465 (time 1750180569.86 total 142.37 last 191.46) - open/close 25071 (time 1750180579.86 total 152.37 last 160.51) - open/close 26731 (time 1750180589.86 total 162.37 last 165.98) - open/close 28353 (time 1750180599.87 total 172.38 last 162.14) - open/close 30000 (time 1750180608.63 total 181.14 last 187.98) - open/close 32287 (time 1750180618.63 total 191.14 last 228.65) - open/close 34443 (time 1750180628.63 total 201.14 last 215.55) - open/close 36849 (time 1750180638.64 total 211.14 last 240.57) - open/close 39109 (time 1750180648.64 total 221.15 last 225.99) - open/close 40000 (time 1750180651.79 total 224.30 last 282.79) - open/close 42041 (time 1750180661.79 total 234.30 last 204.01) - open/close 43771 (time 1750180671.79 total 244.30 last 172.98) - open/close 45373 (time 1750180681.80 total 254.30 last 160.16) - open/close 47030 (time 1750180691.80 total 264.31 last 165.68) - open/close 48411 (time 1750180701.80 total 274.31 last 138.05) - open/close 50000 (time 1750180710.50 total 283.01 last 182.55) - open/close 52039 (time 1750180720.51 total 293.02 last 203.80) - open/close 53370 (time 1750180730.52 total 303.02 last 133.03) - open/close 54965 (time 1750180740.52 total 313.03 last 159.46) - open/close 56803 (time 1750180750.52 total 323.03 last 183.79) - open/close 58523 (time 1750180760.52 total 333.03 last 171.96) - open/close 60000 (time 1750180767.09 total 339.60 last 224.70) - open/close 62002 (time 1750180777.10 total 349.61 last 200.10) - open/close 63696 (time 1750180787.10 total 359.61 last 169.32) total: 64768 open/close in 364.68 seconds: 177.60 ops/second - /unlink 3969 (time 1750180803.33 total 10.00 last 396.87) - /unlink 7679 (time 1750180813.33 total 20.00 last 370.92) - /unlink 10000 (time 1750180820.20 total 26.87 last 337.89) - /unlink 13036 (time 1750180830.21 total 36.87 last 303.51) - /unlink 15622 (time 1750180840.21 total 46.88 last 258.55) - /unlink 18452 (time 1750180850.21 total 56.88 last 282.89) - /unlink 20000 (time 1750180855.81 total 62.48 last 276.53) - /unlink 24055 (time 1750180865.81 total 72.48 last 405.43) - /unlink 27624 (time 1750180875.81 total 82.48 last 356.87) - /unlink 30000 (time 1750180881.45 total 88.12 last 421.28) - /unlink 33683 (time 1750180891.45 total 98.12 last 368.21) - /unlink 37179 (time 1750180901.45 total 108.12 last 349.54) - /unlink 40000 (time 1750180910.84 total 117.51 last 300.48) - /unlink 42915 (time 1750180920.84 total 127.51 last 291.46) - /unlink 46003 (time 1750180930.85 total 137.52 last 308.73) - /unlink 48534 (time 1750180940.85 total 147.52 last 253.05) - /unlink 50000 (time 1750180946.92 total 153.58 last 241.66) - /unlink 52340 (time 1750180956.92 total 163.59 last 233.91) - /unlink 54755 (time 1750180966.92 total 173.59 last 241.43) - /unlink 57222 (time 1750180976.92 total 183.59 last 246.64) - /unlink 59455 (time 1750180986.93 total 193.60 last 223.27) - /unlink 60000 (time 1750180989.13 total 195.80 last 247.08) - /unlink 62391 (time 1750180999.14 total 205.81 last 239.01) - /unlink 64563 (time 1750181009.14 total 215.81 last 217.19) total: 64768 /unlink in 216.66 seconds: 298.94 ops/second Waiting for MDT destroys to complete - open/close 1208 (time 1750181033.76 total 10.00 last 120.75) - open/close 2576 (time 1750181043.77 total 20.01 last 136.71) - open/close 3925 (time 1750181053.78 total 30.02 last 134.83) - open/close 5207 (time 1750181063.78 total 40.02 last 128.13) - open/close 6613 (time 1750181073.79 total 50.03 last 140.48) - open/close 8038 (time 1750181083.79 total 60.03 last 142.47) - open/close 9457 (time 1750181093.79 total 70.03 last 141.87) - open/close 10000 (time 1750181097.29 total 73.53 last 155.41) - open/close 11317 (time 1750181107.29 total 83.53 last 131.66) - open/close 12786 (time 1750181117.29 total 93.53 last 146.87) - open/close 14086 (time 1750181127.30 total 103.54 last 129.89) - open/close 15458 (time 1750181137.30 total 113.54 last 137.17) - open/close 16908 (time 1750181147.31 total 123.55 last 144.90) - open/close 18141 (time 1750181157.31 total 133.56 last 123.24) - open/close 19493 (time 1750181167.32 total 143.56 last 135.13) - open/close 20000 (time 1750181171.06 total 147.30 last 135.70) - open/close 21206 (time 1750181181.06 total 157.30 last 120.59) - open/close 22452 (time 1750181191.06 total 167.30 last 124.58) - open/close 23792 (time 1750181201.06 total 177.30 last 133.94) - open/close 25174 (time 1750181211.07 total 187.31 last 138.08) - open/close 26699 (time 1750181221.07 total 197.32 last 152.44) - open/close 27954 (time 1750181231.08 total 207.32 last 125.44) - open/close 29402 (time 1750181241.08 total 217.32 last 144.78) - open/close 30000 (time 1750181245.62 total 221.86 last 131.80) - open/close 31085 (time 1750181255.63 total 231.87 last 108.37) - open/close 32269 (time 1750181265.63 total 241.87 last 118.38) - open/close 33523 (time 1750181275.64 total 251.88 last 125.33) - open/close 34726 (time 1750181285.64 total 261.88 last 120.21) - open/close 35998 (time 1750181295.64 total 271.89 last 127.19) - open/close 37289 (time 1750181305.65 total 281.89 last 129.06) - open/close 38748 (time 1750181315.65 total 291.89 last 145.89) - open/close 40000 (time 1750181323.60 total 299.84 last 157.39) - open/close 41783 (time 1750181333.61 total 309.85 last 178.23) - open/close 43191 (time 1750181343.61 total 319.85 last 140.71) - open/close 44757 (time 1750181353.63 total 329.87 last 156.42) - open/close 46025 (time 1750181363.63 total 339.87 last 126.72) - open/close 47402 (time 1750181373.64 total 349.88 last 137.64) - open/close 48729 (time 1750181383.64 total 359.88 last 132.65) - open/close 49931 (time 1750181393.64 total 369.89 last 120.14) - open/close 51174 (time 1750181403.65 total 379.89 last 124.27) - open/close 52331 (time 1750181413.65 total 389.89 last 115.68) - open/close 53454 (time 1750181423.66 total 399.90 last 112.23) - open/close 54656 (time 1750181433.66 total 409.90 last 120.16) - open/close 55898 (time 1750181443.66 total 419.90 last 124.18) - open/close 57096 (time 1750181453.67 total 429.91 last 119.74) - open/close 58382 (time 1750181463.67 total 439.91 last 128.56) - open/close 59731 (time 1750181473.68 total 449.92 last 134.79) - open/close 60000 (time 1750181476.28 total 452.52 last 103.22) - open/close 61218 (time 1750181486.29 total 462.54 last 121.65) - open/close 62486 (time 1750181496.30 total 472.54 last 126.74) - open/close 63718 (time 1750181506.30 total 482.54 last 123.15) total: 64768 open/close in 491.37 seconds: 131.81 ops/second - /unlink 1979 (time 1750181527.13 total 10.01 last 197.69) - /unlink 4155 (time 1750181537.13 total 20.01 last 217.57) - /unlink 6226 (time 1750181547.13 total 30.01 last 207.05) - /unlink 8246 (time 1750181557.13 total 40.02 last 201.97) - /unlink 10000 (time 1750181565.04 total 47.92 last 221.87) - /unlink 12045 (time 1750181575.04 total 57.93 last 204.43) - /unlink 13990 (time 1750181585.04 total 67.93 last 194.46) - /unlink 16160 (time 1750181595.04 total 77.93 last 216.99) - /unlink 18305 (time 1750181605.05 total 87.93 last 214.43) - /unlink 20000 (time 1750181612.29 total 95.17 last 234.00) - /unlink 22450 (time 1750181622.29 total 105.17 last 244.99) - /unlink 24535 (time 1750181632.33 total 115.22 last 207.60) - /unlink 26734 (time 1750181642.33 total 125.22 last 219.87) - /unlink 29020 (time 1750181652.34 total 135.22 last 228.56) - /unlink 30000 (time 1750181657.06 total 139.94 last 207.53) - /unlink 32431 (time 1750181667.06 total 149.94 last 243.09) - /unlink 35498 (time 1750181677.06 total 159.94 last 306.67) - /unlink 37704 (time 1750181687.06 total 169.95 last 220.58) - /unlink 39950 (time 1750181697.06 total 179.95 last 224.51) - /unlink 42042 (time 1750181707.07 total 189.95 last 209.13) - /unlink 44201 (time 1750181717.07 total 199.96 last 215.82) - /unlink 46624 (time 1750181727.08 total 209.96 last 242.13) - /unlink 49083 (time 1750181737.08 total 219.96 last 245.88) - /unlink 50000 (time 1750181741.18 total 224.07 last 223.56) - /unlink 52362 (time 1750181751.18 total 234.07 last 236.11) - /unlink 54868 (time 1750181761.19 total 244.07 last 250.50) - /unlink 57277 (time 1750181771.19 total 254.08 last 240.78) - /unlink 59999 (time 1750181781.20 total 264.08 last 272.14) - /unlink 62186 (time 1750181791.20 total 274.08 last 218.62) - /unlink 64469 (time 1750181801.20 total 284.09 last 228.25) total: 64768 /unlink in 285.52 seconds: 226.84 ops/second Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg249-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg249-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg249-server stop mds service on oleg249-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg249-server stop mds service on oleg249-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg249-server unloading modules via unload_modules_local on: 'oleg249-server' oleg249-server: oleg249-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg249-client: oleg249-client: ssh exited with exit code 2 pdsh@oleg249-client: oleg249-server: ssh exited with exit code 2 pdsh@oleg249-client: oleg249-server: ssh exited with exit code 2 pdsh@oleg249-client: oleg249-client: ssh exited with exit code 2 PASS 106 (3099s) Stopping clients: oleg249-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg249-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg249-client: oleg249-server: ssh exited with exit code 2 oleg249-server: oleg249-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg249-server' oleg249-server: oleg249-server.virtnet: executing load_modules_local oleg249-server: Loading modules from /home/green/git/lustre-release/lustre oleg249-server: detected 4 online CPUs by sysfs oleg249-server: MODOPTS_LIBCFS= oleg249-server: Force libcfs to create 2 CPU partitions oleg249-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg249-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 3232 sec ========== 13:38:05 (1750181885) === conf-sanity: start cleanup 13:38:06 (1750181886) === === conf-sanity: finish cleanup 13:38:08 (1750181888) ===