-----============= acceptance-small: conf-sanity ============----- Thu May 1 18:58:54 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg216-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg216-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg216-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg216-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg216-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg216-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg216-server oleg216-server: oleg216-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory loading modules on: 'oleg216-server' oleg216-server: oleg216-server.virtnet: executing load_modules_local oleg216-server: Loading modules from /home/green/git/lustre-release/lustre oleg216-server: detected 4 online CPUs by sysfs oleg216-server: MODOPTS_LIBCFS= oleg216-server: Force libcfs to create 2 CPU partitions oleg216-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg216-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg216-server: oleg216-server.virtnet: executing set_default_debug -1 all pdsh@oleg216-client: oleg216-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg216-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg216-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg216-server: oleg216-server.virtnet: executing set_default_debug -1 all pdsh@oleg216-client: oleg216-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg216-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg216-server: oleg216-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg216-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg216-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg216-server stop mds service on oleg216-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg216-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 19:00:22 (1746140422) Stopping clients: oleg216-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg216-client.virtnet /mnt/lustre2 (opts:-f) oleg216-server: oleg216-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory loading modules on: 'oleg216-server' oleg216-server: oleg216-server.virtnet: executing load_modules_local oleg216-server: Loading modules from /home/green/git/lustre-release/lustre oleg216-server: detected 4 online CPUs by sysfs oleg216-server: MODOPTS_LIBCFS= oleg216-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg216-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg216-server: oleg216-server.virtnet: executing set_default_debug -1 all pdsh@oleg216-client: oleg216-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg216-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg216-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg216-server: oleg216-server.virtnet: executing set_default_debug -1 all pdsh@oleg216-client: oleg216-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg216-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg216-client.virtnet: -o user_xattr,flock 192.168.202.116@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32648 392 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1102 (time 1746140489.90 total 10.00 last 110.16) - open/close 2257 (time 1746140499.91 total 20.01 last 115.43) - open/close 3446 (time 1746140509.91 total 30.01 last 118.88) - open/close 4558 (time 1746140519.91 total 40.01 last 111.20) - open/close 5662 (time 1746140529.92 total 50.02 last 110.35) - open/close 6935 (time 1746140539.92 total 60.02 last 127.28) - open/close 8210 (time 1746140549.92 total 70.02 last 127.48) - open/close 9488 (time 1746140559.92 total 80.02 last 127.79) - open/close 10000 (time 1746140563.82 total 83.92 last 131.21) - open/close 11553 (time 1746140573.83 total 93.93 last 155.12) - open/close 12942 (time 1746140583.84 total 103.94 last 138.87) - open/close 14099 (time 1746140593.84 total 113.94 last 115.67) - open/close 15352 (time 1746140603.84 total 123.94 last 125.27) - open/close 16504 (time 1746140613.85 total 133.95 last 115.15) - open/close 17791 (time 1746140623.85 total 143.95 last 128.66) - open/close 19250 (time 1746140633.85 total 153.96 last 145.81) - open/close 20000 (time 1746140638.85 total 158.95 last 150.27) - open/close 21267 (time 1746140648.86 total 168.96 last 126.46) - open/close 22521 (time 1746140658.87 total 178.97 last 125.32) - open/close 23897 (time 1746140668.87 total 188.97 last 137.55) - open/close 25277 (time 1746140678.88 total 198.98 last 137.91) - open/close 26584 (time 1746140688.88 total 208.98 last 130.68) - open/close 28113 (time 1746140698.89 total 218.99 last 152.78) - open/close 29433 (time 1746140708.89 total 228.99 last 131.97) - open/close 30000 (time 1746140713.06 total 233.16 last 135.97) - open/close 31132 (time 1746140723.07 total 243.17 last 113.16) - open/close 32588 (time 1746140733.07 total 253.17 last 145.56) - open/close 34438 (time 1746140743.08 total 263.18 last 184.84) - open/close 35927 (time 1746140753.08 total 273.18 last 148.83) - open/close 37305 (time 1746140763.08 total 283.18 last 137.79) - open/close 38659 (time 1746140773.09 total 293.19 last 135.36) - open/close 40000 (time 1746140782.33 total 302.43 last 145.00) - open/close 41414 (time 1746140792.33 total 312.43 last 141.39) - open/close 42981 (time 1746140802.34 total 322.44 last 156.64) - open/close 44438 (time 1746140812.34 total 332.44 last 145.65) - open/close 45871 (time 1746140822.35 total 342.45 last 143.25) - open/close 47265 (time 1746140832.35 total 352.45 last 139.39) - open/close 48529 (time 1746140842.35 total 362.45 last 126.34) - open/close 49977 (time 1746140852.35 total 372.45 last 144.77) - open/close 51356 (time 1746140862.35 total 382.45 last 137.90) - open/close 52859 (time 1746140872.36 total 392.46 last 150.24) - open/close 54206 (time 1746140882.36 total 402.46 last 134.68) - open/close 55465 (time 1746140892.36 total 412.46 last 125.83) - open/close 56660 (time 1746140902.37 total 422.47 last 119.46) - open/close 57948 (time 1746140912.37 total 432.47 last 128.79) - open/close 59270 (time 1746140922.37 total 442.47 last 132.15) - open/close 60000 (time 1746140927.71 total 447.81 last 136.86) - open/close 61421 (time 1746140937.71 total 457.81 last 142.05) - open/close 62771 (time 1746140947.72 total 467.82 last 134.93) - open/close 64522 (time 1746140957.72 total 477.82 last 175.09) total: 64768 open/close in 479.03 seconds: 135.21 ops/second - /unlink 2647 (time 1746140970.14 total 10.00 last 264.70) - /unlink 5175 (time 1746140980.15 total 20.01 last 252.61) - /unlink 7480 (time 1746140990.15 total 30.01 last 230.47) - /unlink 9574 (time 1746141000.15 total 40.01 last 209.38) - /unlink 11662 (time 1746141010.16 total 50.01 last 208.73) - /unlink 13762 (time 1746141020.16 total 60.02 last 209.96) - /unlink 15947 (time 1746141030.16 total 70.02 last 218.48) - /unlink 17976 (time 1746141040.16 total 80.02 last 202.79) - /unlink 20000 (time 1746141048.86 total 88.72 last 232.64) - /unlink 22359 (time 1746141058.87 total 98.72 last 235.83) - /unlink 24714 (time 1746141068.87 total 108.73 last 235.37) - /unlink 27170 (time 1746141078.87 total 118.73 last 245.58) - /unlink 29592 (time 1746141088.88 total 128.73 last 242.16) - /unlink 32035 (time 1746141098.88 total 138.73 last 244.24) - /unlink 34558 (time 1746141108.88 total 148.73 last 252.30) - /unlink 37139 (time 1746141118.88 total 158.74 last 258.07) - /unlink 39424 (time 1746141128.88 total 168.74 last 228.40) - /unlink 40000 (time 1746141131.48 total 171.34 last 221.79) - /unlink 42091 (time 1746141141.48 total 181.34 last 209.08) - /unlink 44238 (time 1746141151.48 total 191.34 last 214.67) - /unlink 46468 (time 1746141161.48 total 201.34 last 223.00) - /unlink 48403 (time 1746141171.49 total 211.34 last 193.45) - /unlink 50000 (time 1746141179.46 total 219.32 last 200.18) - /unlink 52209 (time 1746141189.47 total 229.32 last 220.84) - /unlink 54484 (time 1746141199.47 total 239.32 last 227.46) - /unlink 56066 (time 1746141209.47 total 249.33 last 158.15) - /unlink 57957 (time 1746141219.47 total 259.33 last 189.06) - /unlink 59972 (time 1746141229.48 total 269.33 last 201.41) - /unlink 61993 (time 1746141239.48 total 279.34 last 201.99) - /unlink 64215 (time 1746141249.49 total 289.35 last 222.06) total: 64768 /unlink in 292.08 seconds: 221.75 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1234 (time 1746141279.05 total 10.00 last 123.37) - open/close 2730 (time 1746141289.06 total 20.01 last 149.56) - open/close 4260 (time 1746141299.06 total 30.01 last 152.93) - open/close 5718 (time 1746141309.06 total 40.01 last 145.79) - open/close 7173 (time 1746141319.06 total 50.01 last 145.47) - open/close 8730 (time 1746141329.07 total 60.02 last 155.61) - open/close 10000 (time 1746141337.28 total 68.23 last 154.70) - open/close 11503 (time 1746141347.29 total 78.23 last 150.21) - open/close 13127 (time 1746141357.29 total 88.24 last 162.35) - open/close 15123 (time 1746141367.29 total 98.24 last 199.57) - open/close 16654 (time 1746141377.29 total 108.24 last 153.07) - open/close 18161 (time 1746141387.29 total 118.24 last 150.67) - open/close 19686 (time 1746141397.30 total 128.25 last 152.44) - open/close 20000 (time 1746141399.41 total 130.36 last 148.64) - open/close 21355 (time 1746141409.41 total 140.36 last 135.49) - open/close 22781 (time 1746141419.41 total 150.36 last 142.59) - open/close 24385 (time 1746141429.42 total 160.37 last 160.32) - open/close 26033 (time 1746141439.42 total 170.37 last 164.78) - open/close 28445 (time 1746141449.47 total 180.42 last 240.04) - open/close 30000 (time 1746141456.71 total 187.66 last 214.73) - open/close 31714 (time 1746141466.71 total 197.66 last 171.38) - open/close 33370 (time 1746141476.71 total 207.66 last 165.53) - open/close 35153 (time 1746141486.71 total 217.66 last 178.30) - open/close 37004 (time 1746141496.72 total 227.67 last 185.00) - open/close 38481 (time 1746141506.72 total 237.67 last 147.68) - open/close 40000 (time 1746141515.57 total 246.51 last 171.73) - open/close 41647 (time 1746141525.57 total 256.52 last 164.58) - open/close 43875 (time 1746141535.57 total 266.52 last 222.78) - open/close 45818 (time 1746141545.58 total 276.53 last 194.24) - open/close 47595 (time 1746141555.58 total 286.53 last 177.65) - open/close 49127 (time 1746141565.58 total 296.53 last 153.15) - open/close 50000 (time 1746141571.32 total 302.27 last 152.11) - open/close 51601 (time 1746141581.34 total 312.28 last 159.89) - open/close 53201 (time 1746141591.34 total 322.29 last 159.98) - open/close 54797 (time 1746141601.34 total 332.29 last 159.57) - open/close 56681 (time 1746141611.34 total 342.29 last 188.35) - open/close 58659 (time 1746141621.34 total 352.29 last 197.79) - open/close 60000 (time 1746141628.30 total 359.25 last 192.63) - open/close 61708 (time 1746141638.31 total 369.26 last 170.75) - open/close 63353 (time 1746141648.31 total 379.26 last 164.47) total: 64768 open/close in 389.14 seconds: 166.44 ops/second - /unlink 2911 (time 1746141669.77 total 10.00 last 291.06) - /unlink 5593 (time 1746141679.77 total 20.00 last 268.18) - /unlink 8386 (time 1746141689.77 total 30.00 last 279.23) - /unlink 10000 (time 1746141695.63 total 35.86 last 275.72) - /unlink 12613 (time 1746141705.63 total 45.86 last 261.17) - /unlink 15528 (time 1746141715.63 total 55.86 last 291.49) - /unlink 17921 (time 1746141725.63 total 65.86 last 239.28) - /unlink 20000 (time 1746141733.18 total 73.41 last 275.61) - /unlink 22214 (time 1746141743.18 total 83.41 last 221.34) - /unlink 24771 (time 1746141753.18 total 93.41 last 255.61) - /unlink 27260 (time 1746141763.18 total 103.42 last 248.87) - /unlink 29623 (time 1746141773.19 total 113.42 last 236.24) - /unlink 31781 (time 1746141783.19 total 123.42 last 215.71) - /unlink 33771 (time 1746141793.19 total 133.42 last 198.98) - /unlink 35928 (time 1746141803.19 total 143.42 last 215.65) - /unlink 38088 (time 1746141813.20 total 153.43 last 215.90) - /unlink 39992 (time 1746141823.21 total 163.44 last 190.21) - /unlink 41851 (time 1746141833.21 total 173.44 last 185.84) - /unlink 44122 (time 1746141843.21 total 183.45 last 227.04) - /unlink 46040 (time 1746141853.22 total 193.45 last 191.78) - /unlink 47984 (time 1746141863.22 total 203.45 last 194.40) - /unlink 50000 (time 1746141872.67 total 212.90 last 213.18) - /unlink 52217 (time 1746141882.67 total 222.90 last 221.69) - /unlink 54406 (time 1746141892.68 total 232.91 last 218.84) - /unlink 57122 (time 1746141902.68 total 242.91 last 271.56) - /unlink 59791 (time 1746141912.68 total 252.91 last 266.83) - /unlink 63113 (time 1746141922.68 total 262.91 last 332.17) total: 64768 /unlink in 268.67 seconds: 241.07 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1317 (time 1746141960.52 total 10.04 last 131.12) - open/close 2767 (time 1746141970.53 total 20.05 last 144.85) - open/close 4184 (time 1746141980.53 total 30.06 last 141.62) - open/close 5640 (time 1746141990.54 total 40.06 last 145.55) - open/close 7178 (time 1746142000.54 total 50.07 last 153.76) - open/close 8477 (time 1746142010.54 total 60.07 last 129.83) - open/close 9761 (time 1746142020.55 total 70.08 last 128.27) - open/close 11274 (time 1746142030.55 total 80.08 last 151.30) - open/close 12908 (time 1746142040.56 total 90.09 last 163.26) - open/close 14257 (time 1746142050.56 total 100.09 last 134.90) - open/close 15796 (time 1746142060.57 total 110.10 last 153.80) - open/close 17276 (time 1746142070.57 total 120.10 last 147.97) - open/close 19068 (time 1746142080.58 total 130.11 last 179.08) - open/close 20000 (time 1746142085.48 total 135.00 last 190.28) - open/close 21489 (time 1746142095.48 total 145.01 last 148.81) - open/close 22836 (time 1746142105.49 total 155.01 last 134.64) - open/close 24086 (time 1746142115.49 total 165.02 last 124.94) - open/close 25518 (time 1746142125.49 total 175.02 last 143.17) - open/close 27435 (time 1746142135.50 total 185.02 last 191.62) - open/close 29080 (time 1746142145.50 total 195.03 last 164.38) - open/close 30000 (time 1746142150.67 total 200.20 last 177.97) - open/close 31574 (time 1746142160.68 total 210.20 last 157.33) - open/close 33089 (time 1746142170.69 total 220.21 last 151.38) - open/close 34577 (time 1746142180.69 total 230.22 last 148.76) - open/close 36047 (time 1746142190.69 total 240.22 last 146.99) - open/close 37543 (time 1746142200.69 total 250.22 last 149.59) - open/close 38922 (time 1746142210.69 total 260.22 last 137.87) - open/close 40000 (time 1746142217.59 total 267.11 last 156.37) - open/close 41357 (time 1746142227.59 total 277.12 last 135.60) - open/close 42816 (time 1746142237.59 total 287.12 last 145.90) - open/close 44116 (time 1746142247.60 total 297.13 last 129.89) - open/close 45550 (time 1746142257.61 total 307.14 last 143.30) - open/close 46900 (time 1746142267.61 total 317.14 last 134.95) - open/close 48085 (time 1746142277.62 total 327.14 last 118.46) - open/close 49238 (time 1746142287.62 total 337.15 last 115.26) - open/close 50000 (time 1746142294.66 total 344.19 last 108.18) - open/close 51115 (time 1746142304.67 total 354.20 last 111.41) - open/close 52182 (time 1746142314.67 total 364.20 last 106.68) - open/close 53344 (time 1746142324.67 total 374.20 last 116.18) - open/close 54778 (time 1746142334.68 total 384.21 last 143.31) - open/close 56090 (time 1746142344.68 total 394.21 last 131.15) - open/close 57471 (time 1746142354.69 total 404.22 last 137.97) - open/close 58851 (time 1746142364.70 total 414.22 last 137.96) - open/close 60000 (time 1746142373.13 total 422.66 last 136.24) - open/close 61269 (time 1746142383.14 total 432.67 last 126.81) - open/close 62767 (time 1746142393.14 total 442.67 last 149.78) - open/close 64191 (time 1746142403.15 total 452.68 last 142.26) total: 64768 open/close in 456.58 seconds: 141.86 ops/second - /unlink 2148 (time 1746142418.63 total 10.00 last 214.80) - /unlink 4266 (time 1746142428.64 total 20.01 last 211.69) - /unlink 6435 (time 1746142438.64 total 30.01 last 216.80) - /unlink 8368 (time 1746142448.64 total 40.01 last 193.29) - /unlink 10000 (time 1746142456.12 total 47.49 last 218.18) - /unlink 12532 (time 1746142466.12 total 57.49 last 253.19) - /unlink 15173 (time 1746142476.13 total 67.49 last 264.03) - /unlink 18157 (time 1746142486.13 total 77.50 last 298.28) - /unlink 20000 (time 1746142492.74 total 84.11 last 278.88) - /unlink 22935 (time 1746142502.74 total 94.11 last 293.45) - /unlink 26113 (time 1746142512.74 total 104.11 last 317.72) - /unlink 28655 (time 1746142522.74 total 114.11 last 254.19) - /unlink 30000 (time 1746142528.58 total 119.95 last 230.30) - /unlink 32284 (time 1746142538.59 total 129.95 last 228.33) - /unlink 34591 (time 1746142548.59 total 139.96 last 230.57) - /unlink 37169 (time 1746142558.59 total 149.96 last 257.78) - /unlink 40000 (time 1746142568.45 total 159.82 last 287.14) - /unlink 42890 (time 1746142578.46 total 169.82 last 288.93) - /unlink 45898 (time 1746142588.46 total 179.82 last 300.77) - /unlink 48605 (time 1746142598.46 total 189.83 last 270.55) - /unlink 50000 (time 1746142604.40 total 195.77 last 234.84) - /unlink 51988 (time 1746142614.40 total 205.77 last 198.75) - /unlink 53809 (time 1746142624.41 total 215.77 last 182.07) - /unlink 55664 (time 1746142634.41 total 225.77 last 185.48) - /unlink 58321 (time 1746142644.41 total 235.78 last 265.67) - /unlink 60000 (time 1746142650.05 total 241.42 last 297.70) - /unlink 62628 (time 1746142660.05 total 251.42 last 262.77) total: 64768 /unlink in 260.20 seconds: 248.92 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1424 (time 1746142700.59 total 10.01 last 142.31) - open/close 2804 (time 1746142710.59 total 20.01 last 137.97) - open/close 4211 (time 1746142720.59 total 30.01 last 140.67) - open/close 5790 (time 1746142730.59 total 40.01 last 157.90) - open/close 7238 (time 1746142740.59 total 50.01 last 144.79) - open/close 8612 (time 1746142750.60 total 60.02 last 137.31) - open/close 10000 (time 1746142760.22 total 69.64 last 144.32) - open/close 11368 (time 1746142770.22 total 79.64 last 136.79) - open/close 12843 (time 1746142780.22 total 89.64 last 147.46) - open/close 14297 (time 1746142790.22 total 99.64 last 145.39) - open/close 15645 (time 1746142800.22 total 109.64 last 134.75) - open/close 16944 (time 1746142810.23 total 119.65 last 129.85) - open/close 18404 (time 1746142820.23 total 129.65 last 145.97) - open/close 19779 (time 1746142830.23 total 139.65 last 137.48) - open/close 21085 (time 1746142840.23 total 149.65 last 130.55) - open/close 22514 (time 1746142850.23 total 159.65 last 142.89) - open/close 24023 (time 1746142860.24 total 169.66 last 150.82) - open/close 25331 (time 1746142870.24 total 179.66 last 130.75) - open/close 26824 (time 1746142880.25 total 189.67 last 149.28) - open/close 28248 (time 1746142890.25 total 199.67 last 142.33) - open/close 29708 (time 1746142900.25 total 209.67 last 145.96) - open/close 31195 (time 1746142910.25 total 219.67 last 148.67) - open/close 32691 (time 1746142920.26 total 229.68 last 149.57) - open/close 33850 (time 1746142930.26 total 239.68 last 115.88) - open/close 35092 (time 1746142940.26 total 249.68 last 124.20) - open/close 36400 (time 1746142950.26 total 259.68 last 130.76) - open/close 37789 (time 1746142960.26 total 269.68 last 138.87) - open/close 39162 (time 1746142970.27 total 279.69 last 137.23) - open/close 40000 (time 1746142976.24 total 285.66 last 140.45) - open/close 41250 (time 1746142986.24 total 295.66 last 124.96) - open/close 42375 (time 1746142996.24 total 305.66 last 112.44) - open/close 43758 (time 1746143006.25 total 315.67 last 138.24) - open/close 45268 (time 1746143016.25 total 325.67 last 150.97) - open/close 46468 (time 1746143026.26 total 335.68 last 119.89) - open/close 47764 (time 1746143036.26 total 345.68 last 129.55) - open/close 49105 (time 1746143046.27 total 355.69 last 134.07) - open/close 50000 (time 1746143054.05 total 363.47 last 115.02) - open/close 51228 (time 1746143064.05 total 373.47 last 122.78) - open/close 52438 (time 1746143074.06 total 383.48 last 120.92) - open/close 53640 (time 1746143084.06 total 393.48 last 120.18) - open/close 54740 (time 1746143094.07 total 403.49 last 109.89) - open/close 56002 (time 1746143104.07 total 413.49 last 126.16) - open/close 57228 (time 1746143114.09 total 423.51 last 122.41) - open/close 58465 (time 1746143124.09 total 433.51 last 123.63) - open/close 59761 (time 1746143134.10 total 443.52 last 129.49) - open/close 60970 (time 1746143144.10 total 453.52 last 120.86) - open/close 62107 (time 1746143154.11 total 463.53 last 113.64) - open/close 63195 (time 1746143164.11 total 473.53 last 108.76) - open/close 64308 (time 1746143174.12 total 483.54 last 111.24) total: 64768 open/close in 487.35 seconds: 132.90 ops/second - /unlink 1860 (time 1746143190.07 total 10.00 last 185.95) - /unlink 4040 (time 1746143200.07 total 20.01 last 217.94) - /unlink 5862 (time 1746143210.08 total 30.01 last 182.05) - /unlink 7697 (time 1746143220.08 total 40.02 last 183.45) - /unlink 9644 (time 1746143230.09 total 50.02 last 194.63) - /unlink 10000 (time 1746143232.16 total 52.09 last 171.99) - /unlink 12131 (time 1746143242.16 total 62.09 last 213.08) - /unlink 14085 (time 1746143252.17 total 72.10 last 195.18) - /unlink 16022 (time 1746143262.17 total 82.11 last 193.64) - /unlink 17905 (time 1746143272.17 total 92.11 last 188.27) - /unlink 19805 (time 1746143282.18 total 102.11 last 189.99) - /unlink 21759 (time 1746143292.18 total 112.11 last 195.35) - /unlink 23763 (time 1746143302.18 total 122.12 last 200.25) - /unlink 25614 (time 1746143312.19 total 132.12 last 184.99) - /unlink 27531 (time 1746143322.20 total 142.13 last 191.54) - /unlink 29387 (time 1746143332.20 total 152.13 last 185.59) - /unlink 30000 (time 1746143336.01 total 155.94 last 160.92) - /unlink 31872 (time 1746143346.02 total 165.95 last 187.02) - /unlink 33747 (time 1746143356.02 total 175.95 last 187.47) - /unlink 35583 (time 1746143366.02 total 185.96 last 183.53) - /unlink 37472 (time 1746143376.02 total 195.96 last 188.89) - /unlink 39243 (time 1746143386.03 total 205.96 last 177.05) - /unlink 40000 (time 1746143389.98 total 209.92 last 191.31) - /unlink 41585 (time 1746143399.98 total 219.92 last 158.49) - /unlink 43409 (time 1746143409.99 total 229.92 last 182.35) - /unlink 44926 (time 1746143420.00 total 239.93 last 151.51) - /unlink 46638 (time 1746143430.00 total 249.93 last 171.16) - /unlink 48510 (time 1746143440.01 total 259.94 last 187.07) - /unlink 50000 (time 1746143447.33 total 267.26 last 203.65) - /unlink 52123 (time 1746143457.33 total 277.26 last 212.24) - /unlink 54216 (time 1746143467.33 total 287.26 last 209.29) - /unlink 56541 (time 1746143477.33 total 297.27 last 232.41) - /unlink 58458 (time 1746143487.33 total 307.27 last 191.69) - /unlink 60000 (time 1746143495.04 total 314.97 last 200.04) - /unlink 62294 (time 1746143505.05 total 324.98 last 229.32) - /unlink 64409 (time 1746143515.05 total 334.98 last 211.42) total: 64768 /unlink in 336.98 seconds: 192.20 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1239 (time 1746143548.12 total 10.01 last 123.81) - open/close 2511 (time 1746143558.13 total 20.01 last 127.13) - open/close 3719 (time 1746143568.13 total 30.02 last 120.74) - open/close 4909 (time 1746143578.14 total 40.03 last 118.89) - open/close 6128 (time 1746143588.15 total 50.03 last 121.83) - open/close 7280 (time 1746143598.15 total 60.03 last 115.20) - open/close 8599 (time 1746143608.16 total 70.04 last 131.83) - open/close 9903 (time 1746143618.16 total 80.05 last 130.29) - open/close 11184 (time 1746143628.17 total 90.05 last 128.05) - open/close 12497 (time 1746143638.17 total 100.05 last 131.25) - open/close 13860 (time 1746143648.18 total 110.06 last 136.19) - open/close 15124 (time 1746143658.18 total 120.06 last 126.40) - open/close 16356 (time 1746143668.18 total 130.06 last 123.18) - open/close 17545 (time 1746143678.18 total 140.07 last 118.86) - open/close 18547 (time 1746143688.19 total 150.08 last 100.09) - open/close 19654 (time 1746143698.20 total 160.08 last 110.66) - open/close 20000 (time 1746143700.98 total 162.86 last 124.56) - open/close 21234 (time 1746143710.98 total 172.86 last 123.34) - open/close 22523 (time 1746143720.98 total 182.87 last 128.85) - open/close 23701 (time 1746143731.00 total 192.88 last 117.67) - open/close 24925 (time 1746143741.00 total 202.88 last 122.38) - open/close 26241 (time 1746143751.00 total 212.88 last 131.55) - open/close 27510 (time 1746143761.01 total 222.89 last 126.79) - open/close 29007 (time 1746143771.02 total 232.90 last 149.59) - open/close 30000 (time 1746143778.11 total 239.99 last 140.06) - open/close 31345 (time 1746143788.11 total 249.99 last 134.43) - open/close 32792 (time 1746143798.11 total 260.00 last 144.69) - open/close 34294 (time 1746143808.11 total 270.00 last 150.19) - open/close 35838 (time 1746143818.12 total 280.00 last 154.31) - open/close 37445 (time 1746143828.12 total 290.01 last 160.65) - open/close 38868 (time 1746143838.13 total 300.02 last 142.12) - open/close 40000 (time 1746143846.57 total 308.45 last 134.21) - open/close 41328 (time 1746143856.57 total 318.46 last 132.76) - open/close 42697 (time 1746143866.58 total 328.46 last 136.86) - open/close 44124 (time 1746143876.58 total 338.46 last 142.67) - open/close 45716 (time 1746143886.58 total 348.46 last 159.16) - open/close 47468 (time 1746143896.58 total 358.47 last 175.13) - open/close 49208 (time 1746143906.59 total 368.47 last 173.95) - open/close 50000 (time 1746143912.11 total 374.00 last 143.29) - open/close 51377 (time 1746143922.12 total 384.01 last 137.59) - open/close 52745 (time 1746143932.12 total 394.01 last 136.79) - open/close 54095 (time 1746143942.13 total 404.01 last 134.94) - open/close 55543 (time 1746143952.13 total 414.01 last 144.77) - open/close 56959 (time 1746143962.13 total 424.01 last 141.58) - open/close 58478 (time 1746143972.14 total 434.02 last 151.84) - open/close 59817 (time 1746143982.14 total 444.03 last 133.78) - open/close 61344 (time 1746143992.15 total 454.03 last 152.62) - open/close 62827 (time 1746144002.15 total 464.04 last 148.23) - open/close 64280 (time 1746144012.16 total 474.04 last 145.22) total: 64768 open/close in 477.97 seconds: 135.51 ops/second - /unlink 2153 (time 1746144027.83 total 10.01 last 215.16) - /unlink 4156 (time 1746144037.83 total 20.01 last 200.25) - /unlink 6148 (time 1746144047.83 total 30.01 last 199.18) - /unlink 8119 (time 1746144057.84 total 40.02 last 196.95) - /unlink 9727 (time 1746144067.84 total 50.02 last 160.78) - /unlink 11604 (time 1746144077.85 total 60.02 last 187.66) - /unlink 13563 (time 1746144087.85 total 70.03 last 195.81) - /unlink 15372 (time 1746144097.86 total 80.03 last 180.80) - /unlink 17427 (time 1746144107.86 total 90.03 last 205.44) - /unlink 19507 (time 1746144117.86 total 100.04 last 207.96) - /unlink 21751 (time 1746144127.86 total 110.04 last 224.40) - /unlink 23671 (time 1746144137.86 total 120.04 last 191.96) - /unlink 25938 (time 1746144147.86 total 130.04 last 226.66) - /unlink 28263 (time 1746144157.87 total 140.04 last 232.42) - /unlink 30000 (time 1746144166.67 total 148.84 last 197.43) - /unlink 32194 (time 1746144176.67 total 158.84 last 219.36) - /unlink 34344 (time 1746144186.67 total 168.84 last 214.99) - /unlink 36615 (time 1746144196.67 total 178.85 last 227.07) - /unlink 38949 (time 1746144206.68 total 188.85 last 233.28) - /unlink 40000 (time 1746144212.49 total 194.66 last 180.86) - /unlink 42086 (time 1746144222.49 total 204.66 last 208.58) - /unlink 44228 (time 1746144232.49 total 214.67 last 214.14) - /unlink 46640 (time 1746144242.49 total 224.67 last 241.12) - /unlink 49414 (time 1746144252.50 total 234.67 last 277.35) - /unlink 50000 (time 1746144254.86 total 237.04 last 247.85) - /unlink 52164 (time 1746144264.86 total 247.04 last 216.32) - /unlink 54470 (time 1746144274.87 total 257.04 last 230.51) - /unlink 56900 (time 1746144284.88 total 267.05 last 242.81) - /unlink 59238 (time 1746144294.88 total 277.05 last 233.71) - /unlink 60000 (time 1746144298.21 total 280.39 last 228.53) - /unlink 62009 (time 1746144308.22 total 290.39 last 200.87) - /unlink 64072 (time 1746144318.22 total 300.39 last 206.24) total: 64768 /unlink in 304.26 seconds: 212.87 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg216-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg216-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg216-server stop mds service on oleg216-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg216-server unloading modules via unload_modules_local on: 'oleg216-server' oleg216-server: oleg216-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg216-client: oleg216-client: ssh exited with exit code 2 pdsh@oleg216-client: oleg216-server: ssh exited with exit code 2 pdsh@oleg216-client: oleg216-client: ssh exited with exit code 2 pdsh@oleg216-client: oleg216-server: ssh exited with exit code 2 PASS 106 (3933s) Stopping clients: oleg216-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg216-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg216-client: oleg216-server: ssh exited with exit code 2 oleg216-server: oleg216-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg216-server' oleg216-server: oleg216-server.virtnet: executing load_modules_local oleg216-server: Loading modules from /home/green/git/lustre-release/lustre oleg216-server: detected 4 online CPUs by sysfs oleg216-server: MODOPTS_LIBCFS= oleg216-server: Force libcfs to create 2 CPU partitions oleg216-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg216-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 4050 sec ========== 20:06:25 (1746144385) === conf-sanity: start cleanup 20:06:26 (1746144386) === === conf-sanity: finish cleanup 20:06:28 (1746144388) ===