-----============= acceptance-small: conf-sanity ============----- Thu May 1 17:06:17 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg435-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg435-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg435-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg435-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg435-server oleg435-server: oleg435-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg435-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg435-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg435-server: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg435-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 17:07:24 (1746133644) Stopping clients: oleg435-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg435-client.virtnet /mnt/lustre2 (opts:-f) oleg435-server: oleg435-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg435-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg435-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1836 (time 1746133692.71 total 10.00 last 183.57) - open/close 3189 (time 1746133702.71 total 20.00 last 135.28) - open/close 4675 (time 1746133712.71 total 30.01 last 148.56) - open/close 6308 (time 1746133722.72 total 40.01 last 163.25) - open/close 8466 (time 1746133732.72 total 50.01 last 215.75) - open/close 10000 (time 1746133740.10 total 57.39 last 207.81) - open/close 11724 (time 1746133750.10 total 67.39 last 172.39) - open/close 13790 (time 1746133760.10 total 77.40 last 206.54) - open/close 15830 (time 1746133770.10 total 87.40 last 203.99) - open/close 17540 (time 1746133780.11 total 97.40 last 170.93) - open/close 19774 (time 1746133790.11 total 107.40 last 223.34) - open/close 22005 (time 1746133800.11 total 117.41 last 223.04) - open/close 24388 (time 1746133810.11 total 127.41 last 238.29) - open/close 26945 (time 1746133820.12 total 137.41 last 255.57) - open/close 28933 (time 1746133830.12 total 147.41 last 198.76) - open/close 30000 (time 1746133836.45 total 153.74 last 168.58) - open/close 32391 (time 1746133846.45 total 163.75 last 239.03) - open/close 34726 (time 1746133856.45 total 173.75 last 233.47) - open/close 36598 (time 1746133866.46 total 183.75 last 187.17) - open/close 38461 (time 1746133876.46 total 193.76 last 186.17) - open/close 39593 (time 1746133886.47 total 203.76 last 113.14) - open/close 40000 (time 1746133889.70 total 206.99 last 126.01) - open/close 41820 (time 1746133899.70 total 216.99 last 181.94) - open/close 43461 (time 1746133909.71 total 227.00 last 164.04) - open/close 44960 (time 1746133919.71 total 237.00 last 149.89) - open/close 46137 (time 1746133929.71 total 247.01 last 117.60) - open/close 47486 (time 1746133939.72 total 257.01 last 134.89) - open/close 49212 (time 1746133949.72 total 267.01 last 172.58) - open/close 50000 (time 1746133952.60 total 269.89 last 273.22) - open/close 52801 (time 1746133962.61 total 279.90 last 279.88) - open/close 55487 (time 1746133972.61 total 289.91 last 268.49) - open/close 57410 (time 1746133982.61 total 299.91 last 192.27) - open/close 59213 (time 1746133992.62 total 309.91 last 180.21) - open/close 60000 (time 1746133998.71 total 316.00 last 129.31) - open/close 61476 (time 1746134008.71 total 326.00 last 147.53) - open/close 62955 (time 1746134018.72 total 336.01 last 147.83) - open/close 64520 (time 1746134028.72 total 346.01 last 156.48) total: 64768 open/close in 347.38 seconds: 186.45 ops/second - /unlink 2364 (time 1746134041.29 total 10.00 last 236.39) - /unlink 5053 (time 1746134051.29 total 20.00 last 268.84) - /unlink 8395 (time 1746134061.30 total 30.01 last 333.95) - /unlink 10000 (time 1746134066.76 total 35.47 last 293.79) - /unlink 13220 (time 1746134076.76 total 45.47 last 321.98) - /unlink 16729 (time 1746134086.76 total 55.48 last 350.77) - /unlink 20000 (time 1746134096.54 total 65.25 last 334.72) - /unlink 24513 (time 1746134106.54 total 75.25 last 451.28) - /unlink 27178 (time 1746134116.54 total 85.25 last 266.44) - /unlink 29602 (time 1746134126.54 total 95.25 last 242.35) - /unlink 30000 (time 1746134128.75 total 97.46 last 180.52) - /unlink 32409 (time 1746134138.75 total 107.46 last 240.87) - /unlink 34824 (time 1746134148.75 total 117.46 last 241.49) - /unlink 36927 (time 1746134158.75 total 127.46 last 210.25) - /unlink 39118 (time 1746134168.75 total 137.46 last 219.07) - /unlink 40000 (time 1746134171.80 total 140.51 last 289.68) - /unlink 42615 (time 1746134181.80 total 150.51 last 261.50) - /unlink 45815 (time 1746134191.80 total 160.51 last 319.98) - /unlink 49334 (time 1746134201.80 total 170.52 last 351.71) - /unlink 50000 (time 1746134204.51 total 173.23 last 245.70) - /unlink 52192 (time 1746134214.52 total 183.23 last 219.13) - /unlink 54195 (time 1746134224.52 total 193.23 last 200.30) - /unlink 56215 (time 1746134234.52 total 203.23 last 201.94) - /unlink 58966 (time 1746134244.52 total 213.23 last 275.05) - /unlink 60000 (time 1746134246.84 total 215.55 last 446.00) - /unlink 63546 (time 1746134256.84 total 225.55 last 354.60) total: 64768 /unlink in 229.36 seconds: 282.38 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1656 (time 1746134288.32 total 10.00 last 165.56) - open/close 4078 (time 1746134298.33 total 20.01 last 242.11) - open/close 6461 (time 1746134308.33 total 30.01 last 238.29) - open/close 8709 (time 1746134318.33 total 40.01 last 224.75) - open/close 10000 (time 1746134325.47 total 47.14 last 180.92) - open/close 12512 (time 1746134335.47 total 57.15 last 251.15) - open/close 14612 (time 1746134345.48 total 67.15 last 209.83) - open/close 16271 (time 1746134355.48 total 77.16 last 165.78) - open/close 17758 (time 1746134365.49 total 87.17 last 148.63) - open/close 19034 (time 1746134375.49 total 97.17 last 127.56) - open/close 20000 (time 1746134382.69 total 104.36 last 134.26) - open/close 22021 (time 1746134392.69 total 114.37 last 202.08) - open/close 23259 (time 1746134402.69 total 124.37 last 123.75) - open/close 24554 (time 1746134412.69 total 134.37 last 129.46) - open/close 26223 (time 1746134422.70 total 144.37 last 166.88) - open/close 28087 (time 1746134432.70 total 154.38 last 186.37) - open/close 29631 (time 1746134442.70 total 164.38 last 154.39) - open/close 31519 (time 1746134452.70 total 174.38 last 188.78) - open/close 33155 (time 1746134462.71 total 184.39 last 163.42) - open/close 34520 (time 1746134472.71 total 194.39 last 136.49) - open/close 35753 (time 1746134482.71 total 204.39 last 123.28) - open/close 36915 (time 1746134492.72 total 214.40 last 116.12) - open/close 38318 (time 1746134502.72 total 224.40 last 140.26) - open/close 39595 (time 1746134512.73 total 234.41 last 127.58) - open/close 40000 (time 1746134516.62 total 238.30 last 104.06) - open/close 41227 (time 1746134526.62 total 248.30 last 122.70) - open/close 42408 (time 1746134536.63 total 258.31 last 117.99) - open/close 43697 (time 1746134546.64 total 268.32 last 128.84) - open/close 45296 (time 1746134556.64 total 278.32 last 159.82) - open/close 46700 (time 1746134566.66 total 288.34 last 140.20) - open/close 48121 (time 1746134576.66 total 298.34 last 142.02) - open/close 49643 (time 1746134586.67 total 308.34 last 152.14) - open/close 50000 (time 1746134588.92 total 310.60 last 158.45) - open/close 51625 (time 1746134598.92 total 320.60 last 162.44) - open/close 52857 (time 1746134608.92 total 330.60 last 123.20) - open/close 54285 (time 1746134618.93 total 340.61 last 142.71) - open/close 55458 (time 1746134628.94 total 350.61 last 117.23) - open/close 56559 (time 1746134638.94 total 360.62 last 110.09) - open/close 57746 (time 1746134648.94 total 370.62 last 118.63) - open/close 58924 (time 1746134658.96 total 380.63 last 117.64) - open/close 60000 (time 1746134668.69 total 390.37 last 110.57) - open/close 61419 (time 1746134678.69 total 400.37 last 141.90) - open/close 63055 (time 1746134688.69 total 410.37 last 163.54) - open/close 64483 (time 1746134698.69 total 420.37 last 142.77) total: 64768 open/close in 422.68 seconds: 153.23 ops/second - /unlink 2012 (time 1746134712.81 total 10.00 last 201.16) - /unlink 4437 (time 1746134722.81 total 20.00 last 242.48) - /unlink 6896 (time 1746134732.81 total 30.00 last 245.86) - /unlink 9152 (time 1746134742.81 total 40.00 last 225.60) - /unlink 10000 (time 1746134746.73 total 43.93 last 216.20) - /unlink 11840 (time 1746134756.73 total 53.93 last 183.95) - /unlink 13869 (time 1746134766.74 total 63.93 last 202.86) - /unlink 16516 (time 1746134776.74 total 73.93 last 264.68) - /unlink 19561 (time 1746134786.74 total 83.94 last 304.31) - /unlink 23181 (time 1746134796.75 total 93.94 last 361.88) - /unlink 28194 (time 1746134806.75 total 103.94 last 501.30) - /unlink 30000 (time 1746134809.85 total 107.05 last 580.98) - /unlink 33457 (time 1746134819.85 total 117.05 last 345.70) - /unlink 36414 (time 1746134829.86 total 127.05 last 295.67) - /unlink 38917 (time 1746134839.86 total 137.05 last 250.22) - /unlink 40000 (time 1746134845.24 total 142.44 last 201.16) - /unlink 42066 (time 1746134855.24 total 152.44 last 206.57) - /unlink 43890 (time 1746134865.25 total 162.44 last 182.35) - /unlink 45510 (time 1746134875.25 total 172.44 last 161.98) - /unlink 47415 (time 1746134885.26 total 182.45 last 190.32) - /unlink 49194 (time 1746134895.26 total 192.45 last 177.89) - /unlink 50000 (time 1746134899.76 total 196.96 last 178.80) - /unlink 51878 (time 1746134909.77 total 206.96 last 187.79) - /unlink 53743 (time 1746134919.77 total 216.96 last 186.49) - /unlink 55687 (time 1746134929.77 total 226.96 last 194.38) - /unlink 57644 (time 1746134939.77 total 236.97 last 195.58) - /unlink 59491 (time 1746134949.78 total 246.97 last 184.63) - /unlink 60000 (time 1746134952.45 total 249.64 last 190.54) - /unlink 62842 (time 1746134962.45 total 259.65 last 284.15) total: 64768 /unlink in 266.87 seconds: 242.69 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1514 (time 1746135000.93 total 10.00 last 151.34) - open/close 3021 (time 1746135010.93 total 20.01 last 150.64) - open/close 4874 (time 1746135020.94 total 30.01 last 185.19) - open/close 6459 (time 1746135030.94 total 40.01 last 158.49) - open/close 7728 (time 1746135040.95 total 50.02 last 126.80) - open/close 9331 (time 1746135050.96 total 60.03 last 160.13) - open/close 10000 (time 1746135054.73 total 63.81 last 177.25) - open/close 11357 (time 1746135064.73 total 73.81 last 135.67) - open/close 12744 (time 1746135074.74 total 83.82 last 138.63) - open/close 14356 (time 1746135084.74 total 93.82 last 161.19) - open/close 16291 (time 1746135094.74 total 103.82 last 193.46) - open/close 17757 (time 1746135104.74 total 113.82 last 146.57) - open/close 19057 (time 1746135114.75 total 123.82 last 129.97) - open/close 20000 (time 1746135122.33 total 131.40 last 124.40) - open/close 21366 (time 1746135132.33 total 141.41 last 136.51) - open/close 22888 (time 1746135142.34 total 151.41 last 152.15) - open/close 24459 (time 1746135152.34 total 161.41 last 157.10) - open/close 26359 (time 1746135162.34 total 171.41 last 190.00) - open/close 28351 (time 1746135172.34 total 181.42 last 199.10) - open/close 30000 (time 1746135180.20 total 189.27 last 209.94) - open/close 31639 (time 1746135190.21 total 199.28 last 163.73) - open/close 33188 (time 1746135200.21 total 209.28 last 154.88) - open/close 34494 (time 1746135210.21 total 219.29 last 130.55) - open/close 36001 (time 1746135220.22 total 229.29 last 150.65) - open/close 37314 (time 1746135230.22 total 239.29 last 131.27) - open/close 38845 (time 1746135240.22 total 249.30 last 153.01) - open/close 40000 (time 1746135247.82 total 256.89 last 152.15) - open/close 41386 (time 1746135257.82 total 266.89 last 138.59) - open/close 43039 (time 1746135267.82 total 276.89 last 165.28) - open/close 44938 (time 1746135277.82 total 286.90 last 189.85) - open/close 46958 (time 1746135287.82 total 296.90 last 201.97) - open/close 48576 (time 1746135297.83 total 306.90 last 161.72) - open/close 50000 (time 1746135306.81 total 315.89 last 158.47) - open/close 51840 (time 1746135316.85 total 325.93 last 183.27) - open/close 53613 (time 1746135326.85 total 335.93 last 177.28) - open/close 54829 (time 1746135336.86 total 345.93 last 121.57) - open/close 56685 (time 1746135346.86 total 355.94 last 185.54) - open/close 58304 (time 1746135356.87 total 365.95 last 161.75) - open/close 59366 (time 1746135366.87 total 375.95 last 106.15) - open/close 60000 (time 1746135372.29 total 381.36 last 117.13) - open/close 61297 (time 1746135382.29 total 391.37 last 129.65) - open/close 62503 (time 1746135392.30 total 401.37 last 120.54) - open/close 63748 (time 1746135402.30 total 411.38 last 124.40) total: 64768 open/close in 418.43 seconds: 154.79 ops/second - /unlink 1952 (time 1746135420.86 total 10.00 last 195.19) - /unlink 4012 (time 1746135430.86 total 20.00 last 205.94) - /unlink 6269 (time 1746135440.87 total 30.01 last 225.64) - /unlink 9017 (time 1746135450.87 total 40.01 last 274.74) - /unlink 10000 (time 1746135455.82 total 44.96 last 198.55) - /unlink 12129 (time 1746135465.82 total 54.96 last 212.89) - /unlink 14273 (time 1746135475.82 total 64.96 last 214.34) - /unlink 16821 (time 1746135485.83 total 74.96 last 254.76) - /unlink 19028 (time 1746135495.83 total 84.97 last 220.63) - /unlink 20000 (time 1746135500.38 total 89.52 last 213.42) - /unlink 22772 (time 1746135510.39 total 99.53 last 277.11) - /unlink 25288 (time 1746135520.39 total 109.53 last 251.57) - /unlink 27443 (time 1746135530.39 total 119.53 last 215.46) - /unlink 29971 (time 1746135540.39 total 129.53 last 252.77) - /unlink 32421 (time 1746135550.39 total 139.53 last 244.92) - /unlink 34816 (time 1746135560.40 total 149.54 last 239.38) - /unlink 37226 (time 1746135570.40 total 159.54 last 240.97) - /unlink 39839 (time 1746135580.40 total 169.54 last 261.27) - /unlink 42744 (time 1746135590.40 total 179.54 last 290.49) - /unlink 45487 (time 1746135600.41 total 189.54 last 274.20) - /unlink 48051 (time 1746135610.41 total 199.54 last 256.40) - /unlink 50000 (time 1746135618.27 total 207.41 last 247.82) - /unlink 52627 (time 1746135628.28 total 217.42 last 262.47) - /unlink 56110 (time 1746135638.28 total 227.42 last 348.20) - /unlink 59225 (time 1746135648.28 total 237.42 last 311.48) - /unlink 60000 (time 1746135651.50 total 240.64 last 240.79) - /unlink 62008 (time 1746135661.51 total 250.65 last 200.68) - /unlink 64202 (time 1746135671.51 total 260.65 last 219.29) total: 64768 /unlink in 262.75 seconds: 246.50 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1418 (time 1746135700.82 total 10.00 last 141.78) - open/close 3001 (time 1746135710.83 total 20.01 last 158.21) - open/close 4388 (time 1746135720.83 total 30.01 last 138.69) - open/close 5877 (time 1746135730.84 total 40.01 last 148.84) - open/close 7215 (time 1746135740.84 total 50.02 last 133.76) - open/close 8999 (time 1746135750.85 total 60.03 last 178.19) - open/close 10000 (time 1746135758.25 total 67.43 last 135.28) - open/close 11273 (time 1746135768.25 total 77.43 last 127.30) - open/close 12803 (time 1746135778.25 total 87.43 last 152.98) - open/close 13978 (time 1746135788.26 total 97.43 last 117.44) - open/close 15065 (time 1746135798.26 total 107.44 last 108.68) - open/close 16343 (time 1746135808.26 total 117.44 last 127.79) - open/close 17623 (time 1746135818.26 total 127.44 last 127.98) - open/close 19038 (time 1746135828.26 total 137.44 last 141.47) - open/close 20000 (time 1746135837.38 total 146.55 last 105.57) - open/close 21205 (time 1746135847.38 total 156.56 last 120.40) - open/close 22438 (time 1746135857.39 total 166.57 last 123.23) - open/close 23518 (time 1746135867.40 total 176.57 last 107.93) - open/close 24560 (time 1746135877.41 total 186.59 last 104.07) - open/close 25791 (time 1746135887.41 total 196.59 last 123.08) - open/close 26799 (time 1746135897.41 total 206.59 last 100.76) - open/close 28206 (time 1746135907.41 total 216.59 last 140.69) - open/close 29252 (time 1746135917.42 total 226.59 last 104.58) - open/close 30000 (time 1746135925.00 total 234.18 last 98.64) - open/close 31085 (time 1746135935.00 total 244.18 last 108.48) - open/close 32214 (time 1746135945.00 total 254.18 last 112.88) - open/close 33467 (time 1746135955.01 total 264.18 last 125.23) - open/close 34654 (time 1746135965.01 total 274.18 last 118.70) - open/close 35967 (time 1746135975.01 total 284.19 last 131.28) - open/close 37088 (time 1746135985.02 total 294.19 last 112.03) - open/close 38390 (time 1746135995.02 total 304.19 last 130.16) - open/close 39637 (time 1746136005.02 total 314.20 last 124.68) - open/close 40000 (time 1746136008.10 total 317.28 last 117.90) - open/close 41103 (time 1746136018.10 total 327.28 last 110.25) - open/close 42263 (time 1746136028.11 total 337.28 last 115.95) - open/close 43351 (time 1746136038.11 total 347.29 last 108.74) - open/close 44515 (time 1746136048.12 total 357.29 last 116.34) - open/close 45638 (time 1746136058.12 total 367.30 last 112.23) - open/close 46830 (time 1746136068.14 total 377.32 last 119.01) - open/close 47983 (time 1746136078.14 total 387.32 last 115.27) - open/close 49021 (time 1746136088.14 total 397.32 last 103.79) - open/close 50000 (time 1746136097.16 total 406.33 last 108.62) - open/close 51187 (time 1746136107.16 total 416.34 last 118.66) - open/close 52311 (time 1746136117.17 total 426.34 last 112.34) - open/close 53356 (time 1746136127.17 total 436.35 last 104.47) - open/close 54630 (time 1746136137.17 total 446.35 last 127.38) - open/close 56035 (time 1746136147.18 total 456.35 last 140.42) - open/close 57250 (time 1746136157.18 total 466.36 last 121.46) - open/close 59260 (time 1746136167.18 total 476.36 last 200.96) - open/close 60000 (time 1746136171.18 total 480.35 last 185.27) - open/close 61582 (time 1746136181.18 total 490.35 last 158.17) - open/close 63033 (time 1746136191.18 total 500.36 last 145.05) - open/close 64307 (time 1746136201.18 total 510.36 last 127.36) total: 64768 open/close in 514.45 seconds: 125.90 ops/second - /unlink 1986 (time 1746136217.19 total 10.00 last 198.55) - /unlink 4066 (time 1746136227.19 total 20.00 last 208.00) - /unlink 5881 (time 1746136237.19 total 30.01 last 181.38) - /unlink 7599 (time 1746136247.19 total 40.01 last 171.78) - /unlink 9685 (time 1746136257.19 total 50.01 last 208.58) - /unlink 11285 (time 1746136267.20 total 60.01 last 159.94) - /unlink 13100 (time 1746136277.20 total 70.02 last 181.46) - /unlink 14918 (time 1746136287.20 total 80.02 last 181.74) - /unlink 16578 (time 1746136297.21 total 90.02 last 165.95) - /unlink 18363 (time 1746136307.21 total 100.03 last 178.47) - /unlink 20000 (time 1746136315.12 total 107.94 last 206.90) - /unlink 21967 (time 1746136325.12 total 117.94 last 196.68) - /unlink 23872 (time 1746136335.12 total 127.94 last 190.50) - /unlink 26525 (time 1746136345.12 total 137.94 last 265.27) - /unlink 28338 (time 1746136355.13 total 147.95 last 181.18) - /unlink 30000 (time 1746136364.27 total 157.08 last 181.87) - /unlink 32150 (time 1746136374.27 total 167.09 last 214.97) - /unlink 33842 (time 1746136384.27 total 177.09 last 169.12) - /unlink 35532 (time 1746136394.28 total 187.10 last 168.91) - /unlink 37355 (time 1746136404.28 total 197.10 last 182.28) - /unlink 39279 (time 1746136414.29 total 207.10 last 192.28) - /unlink 40000 (time 1746136418.45 total 211.26 last 173.30) - /unlink 41595 (time 1746136428.45 total 221.27 last 159.45) - /unlink 43053 (time 1746136438.46 total 231.27 last 145.70) - /unlink 44653 (time 1746136448.47 total 241.28 last 159.87) - /unlink 46299 (time 1746136458.47 total 251.29 last 164.54) - /unlink 47874 (time 1746136468.47 total 261.29 last 157.47) - /unlink 49728 (time 1746136478.48 total 271.29 last 185.30) - /unlink 51463 (time 1746136488.48 total 281.30 last 173.44) - /unlink 53628 (time 1746136498.48 total 291.30 last 216.43) - /unlink 55721 (time 1746136508.49 total 301.30 last 209.25) - /unlink 57931 (time 1746136518.49 total 311.31 last 220.91) - /unlink 59966 (time 1746136528.49 total 321.31 last 203.42) - /unlink 62139 (time 1746136538.49 total 331.31 last 217.29) - /unlink 64115 (time 1746136548.49 total 341.31 last 197.59) total: 64768 /unlink in 344.76 seconds: 187.86 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1069 (time 1746136582.71 total 10.01 last 106.83) - open/close 2190 (time 1746136592.71 total 20.01 last 112.06) - open/close 3431 (time 1746136602.71 total 30.01 last 124.06) - open/close 4497 (time 1746136612.71 total 40.01 last 106.59) - open/close 5676 (time 1746136622.72 total 50.02 last 117.89) - open/close 6790 (time 1746136632.72 total 60.02 last 111.35) - open/close 7832 (time 1746136642.72 total 70.02 last 104.16) - open/close 8937 (time 1746136652.73 total 80.03 last 110.46) - open/close 10000 (time 1746136661.43 total 88.73 last 122.16) - open/close 11204 (time 1746136671.43 total 98.73 last 120.38) - open/close 12404 (time 1746136681.44 total 108.74 last 119.91) - open/close 13735 (time 1746136691.44 total 118.75 last 133.00) - open/close 15076 (time 1746136701.45 total 128.75 last 134.01) - open/close 16332 (time 1746136711.46 total 138.76 last 125.55) - open/close 17584 (time 1746136721.46 total 148.76 last 125.13) - open/close 18782 (time 1746136731.47 total 158.77 last 119.71) - open/close 20000 (time 1746136739.80 total 167.10 last 146.13) - open/close 21242 (time 1746136749.81 total 177.11 last 124.14) - open/close 22710 (time 1746136759.81 total 187.11 last 146.75) - open/close 23953 (time 1746136769.81 total 197.11 last 124.27) - open/close 25360 (time 1746136779.82 total 207.12 last 140.63) - open/close 26819 (time 1746136789.82 total 217.12 last 145.86) - open/close 28484 (time 1746136799.83 total 227.13 last 166.44) - open/close 29846 (time 1746136809.83 total 237.13 last 136.17) - open/close 31223 (time 1746136819.83 total 247.13 last 137.68) - open/close 32371 (time 1746136829.83 total 257.13 last 114.73) - open/close 33561 (time 1746136839.84 total 267.14 last 118.98) - open/close 34807 (time 1746136849.84 total 277.14 last 124.53) - open/close 35899 (time 1746136859.84 total 287.14 last 109.17) - open/close 36872 (time 1746136869.85 total 297.15 last 97.25) - open/close 38162 (time 1746136879.86 total 307.16 last 128.91) - open/close 39328 (time 1746136889.86 total 317.16 last 116.53) - open/close 40000 (time 1746136895.45 total 322.75 last 120.32) - open/close 41156 (time 1746136905.45 total 332.75 last 115.59) - open/close 42118 (time 1746136915.45 total 342.75 last 96.15) - open/close 43328 (time 1746136925.45 total 352.75 last 121.00) - open/close 44582 (time 1746136935.45 total 362.75 last 125.39) - open/close 45881 (time 1746136945.46 total 372.76 last 129.83) - open/close 47175 (time 1746136955.46 total 382.76 last 129.33) - open/close 48269 (time 1746136965.47 total 392.77 last 109.34) - open/close 49383 (time 1746136975.48 total 402.78 last 111.29) - open/close 50000 (time 1746136980.87 total 408.17 last 114.44) - open/close 51179 (time 1746136990.87 total 418.17 last 117.89) - open/close 52069 (time 1746137000.88 total 428.18 last 88.92) - open/close 53064 (time 1746137010.88 total 438.18 last 99.48) - open/close 54153 (time 1746137020.88 total 448.18 last 108.90) - open/close 55446 (time 1746137030.89 total 458.19 last 129.26) - open/close 56670 (time 1746137040.89 total 468.19 last 122.36) - open/close 57995 (time 1746137050.90 total 478.20 last 132.40) - open/close 59378 (time 1746137060.90 total 488.20 last 138.24) - open/close 60000 (time 1746137065.82 total 493.12 last 126.46) - open/close 61265 (time 1746137075.82 total 503.13 last 126.43) - open/close 62636 (time 1746137085.83 total 513.13 last 137.08) - open/close 63822 (time 1746137095.83 total 523.13 last 118.55) total: 64768 open/close in 530.66 seconds: 122.05 ops/second - /unlink 2004 (time 1746137114.86 total 10.00 last 200.33) - /unlink 3843 (time 1746137124.86 total 20.01 last 183.79) - /unlink 5616 (time 1746137134.86 total 30.01 last 177.27) - /unlink 7557 (time 1746137144.87 total 40.01 last 194.03) - /unlink 9649 (time 1746137154.87 total 50.02 last 209.10) - /unlink 11646 (time 1746137164.87 total 60.02 last 199.68) - /unlink 13836 (time 1746137174.87 total 70.02 last 218.95) - /unlink 16103 (time 1746137184.88 total 80.03 last 226.64) - /unlink 18805 (time 1746137194.88 total 90.03 last 270.15) - /unlink 20000 (time 1746137200.51 total 95.65 last 212.36) - /unlink 22094 (time 1746137210.51 total 105.65 last 209.38) - /unlink 24538 (time 1746137220.51 total 115.66 last 244.29) - /unlink 26722 (time 1746137230.51 total 125.66 last 218.40) - /unlink 28960 (time 1746137240.51 total 135.66 last 223.79) - /unlink 30000 (time 1746137244.62 total 139.77 last 252.86) - /unlink 32124 (time 1746137254.63 total 149.78 last 212.34) - /unlink 34659 (time 1746137264.63 total 159.78 last 253.44) - /unlink 37343 (time 1746137274.63 total 169.78 last 268.32) - /unlink 39493 (time 1746137284.63 total 179.78 last 214.99) - /unlink 40000 (time 1746137286.82 total 181.97 last 231.83) - /unlink 42547 (time 1746137296.82 total 191.97 last 254.60) - /unlink 44997 (time 1746137306.83 total 201.97 last 244.98) - /unlink 47451 (time 1746137316.83 total 211.97 last 245.39) - /unlink 49913 (time 1746137326.83 total 221.98 last 246.11) - /unlink 52421 (time 1746137336.83 total 231.98 last 250.75) - /unlink 54687 (time 1746137346.83 total 241.98 last 226.53) - /unlink 57364 (time 1746137356.83 total 251.98 last 267.68) - /unlink 59856 (time 1746137366.84 total 261.98 last 249.18) - /unlink 62112 (time 1746137376.84 total 271.98 last 225.59) - /unlink 64767 (time 1746137386.84 total 281.99 last 265.44) total: 64768 /unlink in 281.99 seconds: 229.68 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg435-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server unloading modules via unload_modules_local on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 PASS 106 (3776s) Stopping clients: oleg435-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg435-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 oleg435-server: oleg435-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg435-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3871 sec ========== 18:10:49 (1746137449) === conf-sanity: start cleanup 18:10:49 (1746137449) === === conf-sanity: finish cleanup 18:10:51 (1746137451) ===