-----============= acceptance-small: conf-sanity ============----- Thu May 1 17:44:47 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg401-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg401-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg401-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg401-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg401-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg401-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg401-server oleg401-server: oleg401-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg401-server' oleg401-server: oleg401-server.virtnet: executing load_modules_local oleg401-server: Loading modules from /home/green/git/lustre-release/lustre oleg401-server: detected 4 online CPUs by sysfs oleg401-server: MODOPTS_LIBCFS= oleg401-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg401-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg401-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg401-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg401-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg401-server: oleg401-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg401-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg401-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg401-server stop mds service on oleg401-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg401-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 17:46:07 (1746135967) Stopping clients: oleg401-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg401-client.virtnet /mnt/lustre2 (opts:-f) oleg401-server: oleg401-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg401-server' oleg401-server: oleg401-server.virtnet: executing load_modules_local oleg401-server: Loading modules from /home/green/git/lustre-release/lustre oleg401-server: detected 4 online CPUs by sysfs oleg401-server: MODOPTS_LIBCFS= oleg401-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg401-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg401-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg401-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg401-server: oleg401-server.virtnet: executing set_default_debug -1 all pdsh@oleg401-client: oleg401-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg401-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg401-client.virtnet: -o user_xattr,flock 192.168.204.101@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1385 (time 1746136016.61 total 10.00 last 138.44) - open/close 2906 (time 1746136026.61 total 20.01 last 152.08) - open/close 4304 (time 1746136036.62 total 30.01 last 139.72) - open/close 5656 (time 1746136046.62 total 40.01 last 135.19) - open/close 6949 (time 1746136056.63 total 50.02 last 129.18) - open/close 8416 (time 1746136066.63 total 60.02 last 146.69) - open/close 9842 (time 1746136076.63 total 70.03 last 142.56) - open/close 11197 (time 1746136086.64 total 80.03 last 135.43) - open/close 12525 (time 1746136096.64 total 90.03 last 132.75) - open/close 14000 (time 1746136106.66 total 100.05 last 147.27) - open/close 15397 (time 1746136116.67 total 110.06 last 139.62) - open/close 16565 (time 1746136126.67 total 120.06 last 116.79) - open/close 17867 (time 1746136136.67 total 130.06 last 130.14) - open/close 19467 (time 1746136146.68 total 140.07 last 159.90) - open/close 20000 (time 1746136150.13 total 143.52 last 154.36) - open/close 21603 (time 1746136160.13 total 153.52 last 160.29) - open/close 23786 (time 1746136170.13 total 163.52 last 218.24) - open/close 25736 (time 1746136180.14 total 173.53 last 194.86) - open/close 27288 (time 1746136190.15 total 183.54 last 155.07) - open/close 28746 (time 1746136200.15 total 193.54 last 145.78) - open/close 29831 (time 1746136210.16 total 203.55 last 108.37) - open/close 31208 (time 1746136220.17 total 213.56 last 137.63) - open/close 32552 (time 1746136230.17 total 223.56 last 134.35) - open/close 33670 (time 1746136240.19 total 233.59 last 111.54) - open/close 34955 (time 1746136250.20 total 243.59 last 128.39) - open/close 36283 (time 1746136260.21 total 253.60 last 132.77) - open/close 37368 (time 1746136270.21 total 263.60 last 108.41) - open/close 38489 (time 1746136280.22 total 273.61 last 112.03) - open/close 39600 (time 1746136290.23 total 283.62 last 111.00) - open/close 40000 (time 1746136293.69 total 287.08 last 115.62) - open/close 41104 (time 1746136303.70 total 297.09 last 110.31) - open/close 42436 (time 1746136313.70 total 307.09 last 133.20) - open/close 43736 (time 1746136323.70 total 317.09 last 129.97) - open/close 44981 (time 1746136333.70 total 327.09 last 124.46) - open/close 46740 (time 1746136343.71 total 337.10 last 175.83) - open/close 47935 (time 1746136353.71 total 347.10 last 119.50) - open/close 49216 (time 1746136363.71 total 357.10 last 128.04) - open/close 50000 (time 1746136368.77 total 362.16 last 155.08) - open/close 51379 (time 1746136378.77 total 372.17 last 137.81) - open/close 52639 (time 1746136388.78 total 382.17 last 125.98) - open/close 53962 (time 1746136398.78 total 392.18 last 132.19) - open/close 55353 (time 1746136408.79 total 402.18 last 139.05) - open/close 56594 (time 1746136418.79 total 412.18 last 124.07) - open/close 57733 (time 1746136428.79 total 422.19 last 113.86) - open/close 58923 (time 1746136438.80 total 432.19 last 118.98) - open/close 60000 (time 1746136446.97 total 440.36 last 131.83) - open/close 61361 (time 1746136456.97 total 450.36 last 136.08) - open/close 62665 (time 1746136466.97 total 460.36 last 130.35) - open/close 63915 (time 1746136476.99 total 470.38 last 124.78) total: 64768 open/close in 477.54 seconds: 135.63 ops/second - /unlink 2190 (time 1746136496.05 total 10.00 last 218.94) - /unlink 4657 (time 1746136506.05 total 20.00 last 246.68) - /unlink 6955 (time 1746136516.05 total 30.00 last 229.78) - /unlink 9105 (time 1746136526.06 total 40.01 last 214.83) - /unlink 10000 (time 1746136530.07 total 44.02 last 223.42) - /unlink 12409 (time 1746136540.07 total 54.02 last 240.88) - /unlink 14461 (time 1746136550.07 total 64.02 last 205.08) - /unlink 16355 (time 1746136560.08 total 74.03 last 189.35) - /unlink 18190 (time 1746136570.08 total 84.03 last 183.39) - /unlink 20000 (time 1746136579.01 total 92.96 last 202.82) - /unlink 21861 (time 1746136589.01 total 102.96 last 186.03) - /unlink 24121 (time 1746136599.01 total 112.96 last 225.96) - /unlink 26116 (time 1746136609.01 total 122.96 last 199.47) - /unlink 27992 (time 1746136619.01 total 132.97 last 187.58) - /unlink 30000 (time 1746136628.50 total 142.45 last 211.75) - /unlink 31804 (time 1746136638.50 total 152.45 last 180.33) - /unlink 33563 (time 1746136648.50 total 162.46 last 175.85) - /unlink 35761 (time 1746136658.51 total 172.47 last 219.57) - /unlink 38028 (time 1746136668.52 total 182.47 last 226.68) - /unlink 40000 (time 1746136677.80 total 191.75 last 212.39) - /unlink 42326 (time 1746136687.80 total 201.75 last 232.59) - /unlink 44974 (time 1746136697.80 total 211.75 last 264.71) - /unlink 47476 (time 1746136707.81 total 221.76 last 250.07) - /unlink 49761 (time 1746136717.82 total 231.77 last 228.35) - /unlink 51856 (time 1746136727.82 total 241.77 last 209.47) - /unlink 54321 (time 1746136737.82 total 251.77 last 246.44) - /unlink 56756 (time 1746136747.82 total 261.77 last 243.41) - /unlink 59462 (time 1746136757.82 total 271.78 last 270.53) - /unlink 62113 (time 1746136767.83 total 281.78 last 265.08) total: 64768 /unlink in 291.19 seconds: 222.43 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1690 (time 1746136803.90 total 10.00 last 168.96) - open/close 3129 (time 1746136813.90 total 20.00 last 143.87) - open/close 4533 (time 1746136823.90 total 30.01 last 140.38) - open/close 5885 (time 1746136833.91 total 40.01 last 135.14) - open/close 7166 (time 1746136843.91 total 50.01 last 128.06) - open/close 8396 (time 1746136853.92 total 60.02 last 122.92) - open/close 9490 (time 1746136863.93 total 70.04 last 109.22) - open/close 10000 (time 1746136869.01 total 75.12 last 100.36) - open/close 11388 (time 1746136879.02 total 85.12 last 138.77) - open/close 12693 (time 1746136889.02 total 95.12 last 130.49) - open/close 13991 (time 1746136899.02 total 105.12 last 129.75) - open/close 15225 (time 1746136909.02 total 115.13 last 123.34) - open/close 16324 (time 1746136919.03 total 125.14 last 109.84) - open/close 17762 (time 1746136929.04 total 135.14 last 143.70) - open/close 19083 (time 1746136939.04 total 145.14 last 132.09) - open/close 20000 (time 1746136944.97 total 151.08 last 154.55) - open/close 21418 (time 1746136954.97 total 161.08 last 141.77) - open/close 22715 (time 1746136964.98 total 171.08 last 129.64) - open/close 23945 (time 1746136974.98 total 181.08 last 122.99) - open/close 25235 (time 1746136984.99 total 191.09 last 128.93) - open/close 26417 (time 1746136995.00 total 201.10 last 118.07) - open/close 27541 (time 1746137005.00 total 211.10 last 112.39) - open/close 28639 (time 1746137015.00 total 221.11 last 109.74) - open/close 30000 (time 1746137024.73 total 230.84 last 139.90) - open/close 31374 (time 1746137034.73 total 240.84 last 137.36) - open/close 32768 (time 1746137044.74 total 250.84 last 139.38) - open/close 34270 (time 1746137054.74 total 260.84 last 150.13) - open/close 35729 (time 1746137064.75 total 270.85 last 145.83) - open/close 37151 (time 1746137074.75 total 280.85 last 142.18) - open/close 38689 (time 1746137084.75 total 290.85 last 153.79) - open/close 40000 (time 1746137094.13 total 300.23 last 139.75) - open/close 41428 (time 1746137104.13 total 310.24 last 142.73) - open/close 42726 (time 1746137114.14 total 320.25 last 129.66) - open/close 43948 (time 1746137124.15 total 330.25 last 122.17) - open/close 45169 (time 1746137134.15 total 340.26 last 122.05) - open/close 46519 (time 1746137144.16 total 350.26 last 134.94) - open/close 47870 (time 1746137154.16 total 360.26 last 135.05) - open/close 49116 (time 1746137164.16 total 370.26 last 124.59) - open/close 50000 (time 1746137171.38 total 377.49 last 122.36) - open/close 51452 (time 1746137181.39 total 387.49 last 145.19) - open/close 53228 (time 1746137191.39 total 397.49 last 177.54) - open/close 54875 (time 1746137201.39 total 407.50 last 164.60) - open/close 56162 (time 1746137211.40 total 417.50 last 128.65) - open/close 57756 (time 1746137221.40 total 427.51 last 159.31) - open/close 59243 (time 1746137231.40 total 437.51 last 148.70) - open/close 60000 (time 1746137237.14 total 443.24 last 132.07) - open/close 61549 (time 1746137247.14 total 453.25 last 154.83) - open/close 62995 (time 1746137257.14 total 463.25 last 144.58) - open/close 64656 (time 1746137267.14 total 473.25 last 166.08) total: 64768 open/close in 473.86 seconds: 136.68 ops/second - /unlink 3073 (time 1746137278.93 total 10.00 last 307.24) - /unlink 5610 (time 1746137288.93 total 20.00 last 253.65) - /unlink 8638 (time 1746137298.94 total 30.01 last 302.75) - /unlink 10000 (time 1746137303.39 total 34.46 last 305.87) - /unlink 12630 (time 1746137313.39 total 44.46 last 262.96) - /unlink 15155 (time 1746137323.39 total 54.46 last 252.44) - /unlink 17960 (time 1746137333.39 total 64.46 last 280.50) - /unlink 20000 (time 1746137340.71 total 71.78 last 278.65) - /unlink 22643 (time 1746137350.72 total 81.79 last 264.14) - /unlink 25203 (time 1746137360.73 total 91.80 last 255.76) - /unlink 27987 (time 1746137370.73 total 101.80 last 278.28) - /unlink 30000 (time 1746137378.39 total 109.46 last 262.90) - /unlink 32840 (time 1746137388.39 total 119.46 last 283.99) - /unlink 35462 (time 1746137398.39 total 129.46 last 262.19) - /unlink 38059 (time 1746137408.39 total 139.46 last 259.68) - /unlink 40000 (time 1746137415.34 total 146.41 last 279.50) - /unlink 42915 (time 1746137425.34 total 156.41 last 291.33) - /unlink 45310 (time 1746137435.35 total 166.42 last 239.35) - /unlink 47758 (time 1746137445.35 total 176.42 last 244.74) - /unlink 50000 (time 1746137454.35 total 185.41 last 249.29) - /unlink 52491 (time 1746137464.35 total 195.42 last 249.05) - /unlink 55072 (time 1746137474.35 total 205.42 last 258.08) - /unlink 57568 (time 1746137484.35 total 215.42 last 249.60) - /unlink 60000 (time 1746137493.01 total 224.08 last 280.62) - /unlink 62678 (time 1746137503.01 total 234.08 last 267.78) total: 64768 /unlink in 240.20 seconds: 269.64 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1807 (time 1746137539.95 total 10.00 last 180.68) - open/close 3367 (time 1746137549.96 total 20.00 last 155.96) - open/close 4719 (time 1746137559.96 total 30.01 last 135.16) - open/close 6113 (time 1746137569.97 total 40.01 last 139.31) - open/close 7557 (time 1746137579.97 total 50.02 last 144.35) - open/close 9122 (time 1746137589.98 total 60.02 last 156.42) - open/close 10000 (time 1746137595.46 total 65.50 last 160.17) - open/close 11372 (time 1746137605.46 total 75.51 last 137.14) - open/close 12928 (time 1746137615.46 total 85.51 last 155.58) - open/close 14379 (time 1746137625.47 total 95.51 last 145.04) - open/close 15977 (time 1746137635.47 total 105.52 last 159.71) - open/close 17342 (time 1746137645.48 total 115.53 last 136.40) - open/close 18749 (time 1746137655.48 total 125.53 last 140.69) - open/close 20000 (time 1746137663.89 total 133.93 last 148.81) - open/close 21179 (time 1746137673.89 total 143.94 last 117.85) - open/close 22588 (time 1746137683.90 total 153.94 last 140.81) - open/close 24067 (time 1746137693.90 total 163.95 last 147.89) - open/close 25695 (time 1746137703.91 total 173.95 last 162.66) - open/close 27286 (time 1746137713.91 total 183.96 last 159.08) - open/close 28685 (time 1746137723.91 total 193.96 last 139.87) - open/close 30000 (time 1746137731.74 total 201.79 last 167.89) - open/close 31491 (time 1746137741.75 total 211.80 last 149.01) - open/close 32926 (time 1746137751.76 total 221.80 last 143.37) - open/close 34767 (time 1746137761.76 total 231.81 last 184.07) - open/close 36268 (time 1746137771.76 total 241.81 last 150.08) - open/close 37850 (time 1746137781.76 total 251.81 last 158.16) - open/close 39425 (time 1746137791.77 total 261.82 last 157.41) - open/close 40000 (time 1746137794.39 total 264.44 last 219.11) - open/close 41595 (time 1746137804.40 total 274.44 last 159.44) - open/close 42986 (time 1746137814.40 total 284.45 last 139.03) - open/close 44307 (time 1746137824.41 total 294.46 last 132.00) - open/close 45679 (time 1746137834.41 total 304.46 last 137.19) - open/close 47123 (time 1746137844.42 total 314.47 last 144.28) - open/close 48808 (time 1746137854.42 total 324.47 last 168.49) - open/close 50000 (time 1746137861.89 total 331.94 last 159.58) - open/close 51351 (time 1746137871.89 total 341.94 last 135.09) - open/close 52757 (time 1746137881.90 total 351.95 last 140.45) - open/close 54310 (time 1746137891.90 total 361.95 last 155.23) - open/close 55806 (time 1746137901.91 total 371.95 last 149.58) - open/close 57262 (time 1746137911.91 total 381.95 last 145.59) - open/close 58927 (time 1746137921.91 total 391.96 last 166.46) - open/close 60000 (time 1746137928.02 total 398.06 last 175.73) - open/close 61905 (time 1746137938.02 total 408.06 last 190.46) - open/close 63662 (time 1746137948.02 total 418.07 last 175.65) total: 64768 open/close in 423.45 seconds: 152.95 ops/second - /unlink 3181 (time 1746137964.62 total 10.00 last 318.06) - /unlink 7115 (time 1746137974.62 total 20.00 last 393.38) - /unlink 10000 (time 1746137982.49 total 27.87 last 366.68) - /unlink 13315 (time 1746137992.49 total 37.87 last 331.47) - /unlink 16179 (time 1746138002.49 total 47.88 last 286.25) - /unlink 19374 (time 1746138012.50 total 57.88 last 319.47) - /unlink 22316 (time 1746138022.50 total 67.88 last 294.15) - /unlink 25966 (time 1746138032.50 total 77.88 last 364.89) - /unlink 28605 (time 1746138042.50 total 87.88 last 263.82) - /unlink 30000 (time 1746138048.34 total 93.72 last 238.90) - /unlink 32978 (time 1746138058.34 total 103.72 last 297.76) - /unlink 35390 (time 1746138068.35 total 113.73 last 241.15) - /unlink 37718 (time 1746138078.35 total 123.73 last 232.80) - /unlink 40000 (time 1746138084.95 total 130.33 last 345.72) - /unlink 42930 (time 1746138094.95 total 140.33 last 292.98) - /unlink 45871 (time 1746138104.95 total 150.33 last 294.08) - /unlink 49238 (time 1746138114.95 total 160.33 last 336.58) - /unlink 50000 (time 1746138117.90 total 163.28 last 258.66) - /unlink 53402 (time 1746138127.90 total 173.29 last 339.95) - /unlink 57339 (time 1746138137.91 total 183.29 last 393.64) - /unlink 60000 (time 1746138147.72 total 193.10 last 271.18) - /unlink 62869 (time 1746138157.72 total 203.10 last 286.88) total: 64768 /unlink in 210.00 seconds: 308.42 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1481 (time 1746138195.89 total 10.00 last 148.04) - open/close 3003 (time 1746138205.89 total 20.01 last 152.17) - open/close 4714 (time 1746138215.89 total 30.01 last 171.02) - open/close 6451 (time 1746138225.90 total 40.01 last 173.63) - open/close 8480 (time 1746138235.90 total 50.02 last 202.87) - open/close 10000 (time 1746138242.77 total 56.89 last 221.27) - open/close 12021 (time 1746138252.77 total 66.89 last 202.03) - open/close 13830 (time 1746138262.77 total 76.89 last 180.87) - open/close 15666 (time 1746138272.78 total 86.89 last 183.54) - open/close 17396 (time 1746138282.78 total 96.90 last 172.96) - open/close 19064 (time 1746138292.79 total 106.90 last 166.68) - open/close 20000 (time 1746138299.79 total 113.91 last 133.65) - open/close 21378 (time 1746138309.79 total 123.91 last 137.79) - open/close 22796 (time 1746138319.80 total 133.91 last 141.72) - open/close 24134 (time 1746138329.80 total 143.92 last 133.70) - open/close 25344 (time 1746138339.81 total 153.92 last 120.97) - open/close 26816 (time 1746138349.81 total 163.93 last 147.16) - open/close 28452 (time 1746138359.81 total 173.93 last 163.58) - open/close 30000 (time 1746138369.50 total 183.62 last 159.71) - open/close 31403 (time 1746138379.51 total 193.63 last 140.22) - open/close 32978 (time 1746138389.51 total 203.63 last 157.47) - open/close 34824 (time 1746138399.51 total 213.63 last 184.58) - open/close 36774 (time 1746138409.51 total 223.63 last 194.97) - open/close 38707 (time 1746138419.51 total 233.63 last 193.26) - open/close 40000 (time 1746138427.61 total 241.73 last 159.74) - open/close 41865 (time 1746138437.61 total 251.73 last 186.48) - open/close 43696 (time 1746138447.61 total 261.73 last 183.08) - open/close 45380 (time 1746138457.62 total 271.73 last 168.32) - open/close 46991 (time 1746138467.62 total 281.74 last 161.06) - open/close 48900 (time 1746138477.62 total 291.74 last 190.81) - open/close 50000 (time 1746138485.63 total 299.75 last 137.30) - open/close 51436 (time 1746138495.64 total 309.75 last 143.57) - open/close 52853 (time 1746138505.64 total 319.76 last 141.64) - open/close 54203 (time 1746138515.64 total 329.76 last 134.99) - open/close 55617 (time 1746138525.65 total 339.76 last 141.32) - open/close 56959 (time 1746138535.65 total 349.77 last 134.18) - open/close 58291 (time 1746138545.65 total 359.77 last 133.13) - open/close 59686 (time 1746138555.66 total 369.78 last 139.39) - open/close 60000 (time 1746138558.40 total 372.52 last 114.54) - open/close 61384 (time 1746138568.41 total 382.52 last 138.35) - open/close 62977 (time 1746138578.42 total 392.54 last 159.07) - open/close 64258 (time 1746138588.43 total 402.54 last 128.01) total: 64768 open/close in 406.04 seconds: 159.51 ops/second - /unlink 2101 (time 1746138603.41 total 10.00 last 210.06) - /unlink 4639 (time 1746138613.41 total 20.01 last 253.71) - /unlink 6676 (time 1746138623.42 total 30.01 last 203.67) - /unlink 9659 (time 1746138633.42 total 40.01 last 298.11) - /unlink 12096 (time 1746138643.43 total 50.02 last 243.61) - /unlink 15590 (time 1746138653.43 total 60.02 last 349.29) - /unlink 18260 (time 1746138663.43 total 70.02 last 266.97) - /unlink 20000 (time 1746138670.64 total 77.23 last 241.37) - /unlink 22866 (time 1746138680.64 total 87.23 last 286.54) - /unlink 25123 (time 1746138690.64 total 97.23 last 225.65) - /unlink 27336 (time 1746138700.64 total 107.23 last 221.29) - /unlink 29496 (time 1746138710.65 total 117.24 last 215.94) - /unlink 30000 (time 1746138713.37 total 119.96 last 184.81) - /unlink 31754 (time 1746138723.38 total 129.97 last 175.34) - /unlink 33735 (time 1746138733.38 total 139.97 last 198.06) - /unlink 36091 (time 1746138743.38 total 149.97 last 235.58) - /unlink 38396 (time 1746138753.38 total 159.97 last 230.49) - /unlink 40000 (time 1746138760.20 total 166.79 last 235.20) - /unlink 42270 (time 1746138770.20 total 176.79 last 226.93) - /unlink 44708 (time 1746138780.20 total 186.80 last 243.77) - /unlink 47102 (time 1746138790.21 total 196.80 last 239.33) - /unlink 49225 (time 1746138800.21 total 206.80 last 212.22) - /unlink 50000 (time 1746138804.13 total 210.72 last 197.78) - /unlink 52249 (time 1746138814.13 total 220.72 last 224.84) - /unlink 54369 (time 1746138824.13 total 230.72 last 211.97) - /unlink 56410 (time 1746138834.14 total 240.73 last 204.07) - /unlink 58753 (time 1746138844.14 total 250.73 last 234.26) - /unlink 60000 (time 1746138849.31 total 255.90 last 241.31) - /unlink 62152 (time 1746138859.31 total 265.90 last 215.18) - /unlink 64447 (time 1746138869.31 total 275.90 last 229.40) total: 64768 /unlink in 277.63 seconds: 233.29 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1176 (time 1746138903.03 total 10.01 last 117.49) - open/close 2361 (time 1746138913.03 total 20.01 last 118.48) - open/close 3473 (time 1746138923.03 total 30.01 last 111.17) - open/close 4646 (time 1746138933.04 total 40.02 last 117.25) - open/close 6036 (time 1746138943.05 total 50.03 last 138.84) - open/close 7385 (time 1746138953.05 total 60.03 last 134.84) - open/close 8762 (time 1746138963.06 total 70.04 last 137.58) - open/close 10000 (time 1746138972.70 total 79.68 last 128.41) - open/close 11047 (time 1746138982.70 total 89.68 last 104.69) - open/close 12093 (time 1746138992.71 total 99.69 last 104.51) - open/close 13262 (time 1746139002.72 total 109.70 last 116.82) - open/close 14518 (time 1746139012.72 total 119.70 last 125.57) - open/close 15746 (time 1746139022.72 total 129.71 last 122.75) - open/close 17188 (time 1746139032.72 total 139.71 last 144.19) - open/close 18486 (time 1746139042.73 total 149.71 last 129.77) - open/close 19702 (time 1746139052.73 total 159.72 last 121.50) - open/close 20000 (time 1746139055.38 total 162.36 last 112.84) - open/close 21132 (time 1746139065.38 total 172.36 last 113.14) - open/close 22252 (time 1746139075.38 total 182.36 last 111.98) - open/close 23375 (time 1746139085.39 total 192.37 last 112.27) - open/close 24861 (time 1746139095.39 total 202.37 last 148.56) - open/close 26178 (time 1746139105.39 total 212.38 last 131.63) - open/close 27656 (time 1746139115.40 total 222.38 last 147.74) - open/close 29645 (time 1746139125.40 total 232.38 last 198.85) - open/close 31218 (time 1746139135.41 total 242.39 last 157.18) - open/close 32467 (time 1746139145.42 total 252.40 last 124.78) - open/close 33851 (time 1746139155.42 total 262.40 last 138.36) - open/close 35348 (time 1746139165.43 total 272.41 last 149.59) - open/close 36779 (time 1746139175.43 total 282.41 last 143.05) - open/close 37927 (time 1746139185.43 total 292.42 last 114.76) - open/close 39162 (time 1746139195.43 total 302.42 last 123.50) - open/close 40000 (time 1746139201.41 total 308.39 last 140.30) - open/close 41251 (time 1746139211.41 total 318.39 last 125.09) - open/close 42462 (time 1746139221.41 total 328.40 last 121.01) - open/close 43778 (time 1746139231.42 total 338.40 last 131.57) - open/close 45120 (time 1746139241.42 total 348.40 last 134.20) - open/close 46558 (time 1746139251.42 total 358.40 last 143.74) - open/close 47737 (time 1746139261.43 total 368.42 last 117.77) - open/close 49184 (time 1746139271.44 total 378.42 last 144.63) - open/close 50000 (time 1746139277.02 total 384.00 last 146.29) - open/close 51627 (time 1746139287.02 total 394.00 last 162.64) - open/close 52827 (time 1746139297.07 total 404.05 last 119.41) - open/close 53857 (time 1746139307.07 total 414.06 last 102.96) - open/close 54960 (time 1746139317.08 total 424.06 last 110.23) - open/close 56302 (time 1746139327.08 total 434.06 last 134.16) - open/close 57738 (time 1746139337.08 total 444.07 last 143.59) - open/close 58991 (time 1746139347.09 total 454.07 last 125.25) - open/close 60000 (time 1746139355.43 total 462.41 last 120.92) - open/close 61143 (time 1746139365.44 total 472.42 last 114.23) - open/close 62241 (time 1746139375.44 total 482.42 last 109.79) - open/close 63462 (time 1746139385.44 total 492.42 last 122.05) - open/close 64690 (time 1746139395.45 total 502.43 last 122.75) total: 64768 open/close in 502.98 seconds: 128.77 ops/second - /unlink 2407 (time 1746139407.62 total 10.00 last 240.67) - /unlink 4671 (time 1746139417.62 total 20.00 last 226.34) - /unlink 6795 (time 1746139427.62 total 30.01 last 212.35) - /unlink 8658 (time 1746139437.63 total 40.01 last 186.21) - /unlink 10000 (time 1746139445.31 total 47.69 last 174.77) - /unlink 11728 (time 1746139455.32 total 57.70 last 172.64) - /unlink 13471 (time 1746139465.32 total 67.70 last 174.23) - /unlink 15379 (time 1746139475.32 total 77.70 last 190.78) - /unlink 17017 (time 1746139485.32 total 87.70 last 163.79) - /unlink 18674 (time 1746139495.32 total 97.70 last 165.70) - /unlink 20000 (time 1746139503.02 total 105.41 last 172.18) - /unlink 21882 (time 1746139513.02 total 115.41 last 188.16) - /unlink 24106 (time 1746139523.03 total 125.41 last 222.37) - /unlink 26186 (time 1746139533.03 total 135.41 last 207.93) - /unlink 28327 (time 1746139543.03 total 145.42 last 213.97) - /unlink 30000 (time 1746139550.86 total 153.24 last 213.84) - /unlink 32797 (time 1746139560.86 total 163.24 last 279.67) - /unlink 35215 (time 1746139570.86 total 173.24 last 241.76) - /unlink 37735 (time 1746139580.86 total 183.25 last 251.94) - /unlink 40000 (time 1746139587.89 total 190.28 last 322.19) - /unlink 42246 (time 1746139597.90 total 200.28 last 224.54) - /unlink 43787 (time 1746139607.90 total 210.28 last 154.08) - /unlink 45923 (time 1746139617.90 total 220.29 last 213.44) - /unlink 48314 (time 1746139627.91 total 230.29 last 238.94) - /unlink 50000 (time 1746139636.07 total 238.45 last 206.68) - /unlink 51906 (time 1746139646.07 total 248.46 last 190.54) - /unlink 54235 (time 1746139656.07 total 258.46 last 232.86) - /unlink 56394 (time 1746139666.08 total 268.46 last 215.81) - /unlink 58542 (time 1746139676.08 total 278.46 last 214.71) - /unlink 60000 (time 1746139682.27 total 284.66 last 235.43) - /unlink 62110 (time 1746139692.28 total 294.66 last 210.89) - /unlink 64142 (time 1746139702.28 total 304.67 last 203.11) total: 64768 /unlink in 307.85 seconds: 210.39 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg401-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg401-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg401-server stop mds service on oleg401-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg401-server unloading modules via unload_modules_local on: 'oleg401-server' oleg401-server: oleg401-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg401-client: oleg401-client: ssh exited with exit code 2 pdsh@oleg401-client: oleg401-server: ssh exited with exit code 2 pdsh@oleg401-client: oleg401-client: ssh exited with exit code 2 pdsh@oleg401-client: oleg401-server: ssh exited with exit code 2 PASS 106 (3775s) Stopping clients: oleg401-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg401-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg401-client: oleg401-server: ssh exited with exit code 2 oleg401-server: oleg401-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg401-server' oleg401-server: oleg401-server.virtnet: executing load_modules_local oleg401-server: Loading modules from /home/green/git/lustre-release/lustre oleg401-server: detected 4 online CPUs by sysfs oleg401-server: MODOPTS_LIBCFS= oleg401-server: Force libcfs to create 2 CPU partitions oleg401-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg401-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3887 sec ========== 18:49:34 (1746139774) === conf-sanity: start cleanup 18:49:35 (1746139775) === === conf-sanity: finish cleanup 18:49:36 (1746139776) ===