== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 19:19:26 (1746141566) Stopping clients: oleg257-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg257-client.virtnet /mnt/lustre2 (opts:-f) oleg257-server: oleg257-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg257-server' oleg257-server: oleg257-server.virtnet: executing load_modules_local oleg257-server: Loading modules from /home/green/git/lustre-release/lustre oleg257-server: detected 4 online CPUs by sysfs oleg257-server: MODOPTS_LIBCFS= oleg257-server: Force libcfs to create 2 CPU partitions oleg257-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg257-server Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg257-server: oleg257-server.virtnet: executing set_default_debug -1 all pdsh@oleg257-client: oleg257-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg257-server Starting mds2: -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg257-server: oleg257-server.virtnet: executing set_default_debug -1 all pdsh@oleg257-client: oleg257-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg257-server Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg257-server: oleg257-server.virtnet: executing set_default_debug -1 all pdsh@oleg257-client: oleg257-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg257-client.virtnet: -o user_xattr,flock 192.168.202.157@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 274 79718 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 65808 368 65440 1% /mnt/lustre[OST:0] filesystem_summary: 65969 529 65440 1% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1896 (time 1746141624.18 total 10.00 last 189.57) - open/close 3630 (time 1746141634.18 total 20.00 last 173.36) - open/close 5220 (time 1746141644.19 total 30.01 last 158.84) - open/close 6766 (time 1746141654.19 total 40.02 last 154.57) - open/close 8296 (time 1746141664.20 total 50.02 last 152.94) - open/close 10000 (time 1746141673.93 total 59.75 last 175.11) - open/close 11535 (time 1746141683.94 total 69.76 last 153.35) - open/close 13214 (time 1746141693.94 total 79.76 last 167.87) - open/close 14750 (time 1746141703.94 total 89.77 last 153.53) - open/close 16473 (time 1746141713.95 total 99.77 last 172.23) - open/close 17929 (time 1746141723.95 total 109.77 last 145.59) - open/close 19600 (time 1746141733.96 total 119.78 last 166.95) - open/close 20000 (time 1746141737.30 total 123.13 last 119.49) - open/close 21347 (time 1746141747.30 total 133.13 last 134.70) - open/close 22841 (time 1746141757.31 total 143.13 last 149.36) - open/close 24344 (time 1746141767.32 total 153.14 last 150.17) - open/close 25735 (time 1746141777.32 total 163.14 last 139.07) - open/close 27012 (time 1746141787.32 total 173.15 last 127.64) - open/close 28348 (time 1746141797.32 total 183.15 last 133.59) - open/close 29657 (time 1746141807.33 total 193.15 last 130.81) - open/close 30000 (time 1746141809.36 total 195.18 last 169.29) - open/close 31273 (time 1746141819.37 total 205.20 last 127.10) - open/close 32474 (time 1746141829.39 total 215.21 last 119.90) - open/close 33794 (time 1746141839.39 total 225.22 last 131.94) - open/close 35100 (time 1746141849.40 total 235.22 last 130.55) - open/close 36401 (time 1746141859.41 total 245.23 last 129.97) - open/close 37711 (time 1746141869.41 total 255.23 last 130.95) - open/close 39004 (time 1746141879.41 total 265.24 last 129.27) - open/close 40000 (time 1746141887.59 total 273.41 last 121.82) - open/close 41444 (time 1746141897.59 total 283.41 last 144.37) - open/close 42796 (time 1746141907.60 total 293.42 last 135.12) - open/close 44489 (time 1746141917.60 total 303.42 last 169.30) - open/close 46027 (time 1746141927.60 total 313.42 last 153.74) - open/close 47396 (time 1746141937.60 total 323.43 last 136.87) - open/close 48805 (time 1746141947.61 total 333.43 last 140.87) - open/close 50000 (time 1746141956.56 total 342.39 last 133.43) - open/close 51329 (time 1746141966.56 total 352.39 last 132.88) - open/close 52761 (time 1746141976.57 total 362.39 last 143.18) - open/close 54128 (time 1746141986.57 total 372.39 last 136.70) - open/close 55639 (time 1746141996.57 total 382.39 last 151.10) - open/close 56978 (time 1746142006.57 total 392.39 last 133.90) - open/close 58213 (time 1746142016.57 total 402.40 last 123.42) - open/close 59621 (time 1746142026.58 total 412.40 last 140.75) - open/close 60000 (time 1746142029.02 total 414.84 last 155.12) - open/close 61540 (time 1746142039.03 total 424.85 last 153.90) - open/close 62936 (time 1746142049.03 total 434.85 last 139.57) - open/close 64405 (time 1746142059.03 total 444.85 last 146.88) total: 64768 open/close in 446.79 seconds: 144.96 ops/second - /unlink 2600 (time 1746142072.49 total 10.00 last 259.99) - /unlink 5586 (time 1746142082.50 total 20.00 last 298.48) - /unlink 8331 (time 1746142092.50 total 30.01 last 274.35) - /unlink 10000 (time 1746142099.79 total 37.30 last 228.93) - /unlink 12185 (time 1746142109.80 total 47.30 last 218.45) - /unlink 14410 (time 1746142119.80 total 57.30 last 222.49) - /unlink 17180 (time 1746142129.80 total 67.30 last 276.95) - /unlink 20000 (time 1746142138.63 total 76.14 last 319.28) - /unlink 22920 (time 1746142148.63 total 86.14 last 291.97) - /unlink 25529 (time 1746142158.63 total 96.14 last 260.84) - /unlink 28132 (time 1746142168.64 total 106.14 last 260.24) - /unlink 30000 (time 1746142175.34 total 112.85 last 278.51) - /unlink 32481 (time 1746142185.34 total 122.85 last 248.09) - /unlink 34833 (time 1746142195.35 total 132.85 last 235.13) - /unlink 37436 (time 1746142205.35 total 142.86 last 260.21) - /unlink 39863 (time 1746142215.35 total 152.86 last 242.67) - /unlink 42274 (time 1746142225.35 total 162.86 last 241.03) - /unlink 44764 (time 1746142235.36 total 172.86 last 248.94) - /unlink 47024 (time 1746142245.36 total 182.87 last 225.90) - /unlink 49574 (time 1746142255.36 total 192.87 last 254.91) - /unlink 51937 (time 1746142265.36 total 202.87 last 236.30) - /unlink 54188 (time 1746142275.37 total 212.87 last 225.03) - /unlink 56481 (time 1746142285.37 total 222.88 last 229.17) - /unlink 58566 (time 1746142295.38 total 232.88 last 208.45) - /unlink 60000 (time 1746142302.08 total 239.59 last 213.93) - /unlink 62124 (time 1746142312.08 total 249.59 last 212.33) - /unlink 64337 (time 1746142322.08 total 259.59 last 221.29) total: 64768 /unlink in 261.35 seconds: 247.82 ops/second Waiting for MDT destroys to complete - open/close 1423 (time 1746142346.93 total 10.00 last 142.28) - open/close 2817 (time 1746142356.94 total 20.01 last 139.35) - open/close 4348 (time 1746142366.94 total 30.01 last 153.07) - open/close 5710 (time 1746142376.94 total 40.01 last 136.18) - open/close 7205 (time 1746142386.94 total 50.01 last 149.48) - open/close 8892 (time 1746142396.95 total 60.02 last 168.61) - open/close 10000 (time 1746142404.73 total 67.80 last 142.30) - open/close 11447 (time 1746142414.75 total 77.81 last 144.52) - open/close 12887 (time 1746142424.75 total 87.82 last 143.96) - open/close 14301 (time 1746142434.75 total 97.82 last 141.33) - open/close 15734 (time 1746142444.76 total 107.83 last 143.19) - open/close 17165 (time 1746142454.76 total 117.83 last 143.06) - open/close 18752 (time 1746142464.76 total 127.83 last 158.68) - open/close 20000 (time 1746142473.09 total 136.16 last 149.84) - open/close 21651 (time 1746142483.10 total 146.16 last 165.05) - open/close 23376 (time 1746142493.10 total 156.17 last 172.35) - open/close 25093 (time 1746142503.11 total 166.18 last 171.61) - open/close 26921 (time 1746142513.11 total 176.18 last 182.72) - open/close 28484 (time 1746142523.12 total 186.19 last 156.25) - open/close 29806 (time 1746142533.12 total 196.19 last 132.17) - open/close 31213 (time 1746142543.13 total 206.19 last 140.61) - open/close 32588 (time 1746142553.13 total 216.20 last 137.49) - open/close 34239 (time 1746142563.13 total 226.20 last 165.10) - open/close 35905 (time 1746142573.13 total 236.20 last 166.48) - open/close 37522 (time 1746142583.14 total 246.21 last 161.65) - open/close 39126 (time 1746142593.14 total 256.21 last 160.36) - open/close 40000 (time 1746142599.03 total 262.10 last 148.46) - open/close 41368 (time 1746142609.03 total 272.10 last 136.70) - open/close 42440 (time 1746142619.04 total 282.11 last 107.14) - open/close 43557 (time 1746142629.04 total 292.11 last 111.68) - open/close 44807 (time 1746142639.05 total 302.11 last 124.95) - open/close 46362 (time 1746142649.05 total 312.12 last 155.45) - open/close 47862 (time 1746142659.06 total 322.13 last 149.85) - open/close 49239 (time 1746142669.06 total 332.13 last 137.68) - open/close 50000 (time 1746142673.74 total 336.81 last 162.65) - open/close 51353 (time 1746142683.75 total 346.81 last 135.21) - open/close 52667 (time 1746142693.76 total 356.83 last 131.23) - open/close 54045 (time 1746142703.76 total 366.83 last 137.72) - open/close 55282 (time 1746142713.77 total 376.84 last 123.60) - open/close 56726 (time 1746142723.77 total 386.84 last 144.40) - open/close 58244 (time 1746142733.78 total 396.85 last 151.67) - open/close 59608 (time 1746142743.78 total 406.85 last 136.35) - open/close 60000 (time 1746142746.87 total 409.94 last 127.01) - open/close 61336 (time 1746142756.87 total 419.94 last 133.57) - open/close 62609 (time 1746142766.88 total 429.94 last 127.27) - open/close 64026 (time 1746142776.89 total 439.96 last 141.54) total: 64768 open/close in 444.93 seconds: 145.57 ops/second - /unlink 2377 (time 1746142793.39 total 10.00 last 237.61) - /unlink 4460 (time 1746142803.40 total 20.00 last 208.28) - /unlink 6779 (time 1746142813.40 total 30.01 last 231.76) - /unlink 9152 (time 1746142823.40 total 40.01 last 237.28) - /unlink 10000 (time 1746142826.80 total 43.41 last 249.30) - /unlink 12390 (time 1746142836.81 total 53.41 last 238.97) - /unlink 14784 (time 1746142846.81 total 63.42 last 239.32) - /unlink 17427 (time 1746142856.81 total 73.42 last 264.19) - /unlink 19594 (time 1746142866.82 total 83.43 last 216.57) - /unlink 22051 (time 1746142876.82 total 93.43 last 245.61) - /unlink 24452 (time 1746142886.82 total 103.43 last 240.05) - /unlink 26861 (time 1746142896.83 total 113.44 last 240.83) - /unlink 29369 (time 1746142906.83 total 123.44 last 250.78) - /unlink 30000 (time 1746142909.32 total 125.93 last 252.80) - /unlink 32711 (time 1746142919.33 total 135.94 last 271.01) - /unlink 34858 (time 1746142929.33 total 145.94 last 214.67) - /unlink 36965 (time 1746142939.33 total 155.94 last 210.62) - /unlink 39414 (time 1746142949.34 total 165.95 last 244.79) - /unlink 40000 (time 1746142951.71 total 168.32 last 246.81) - /unlink 42551 (time 1746142961.71 total 178.32 last 255.09) - /unlink 44927 (time 1746142971.71 total 188.32 last 237.55) - /unlink 47274 (time 1746142981.71 total 198.32 last 234.69) - /unlink 49743 (time 1746142991.71 total 208.32 last 246.90) - /unlink 52029 (time 1746143001.71 total 218.32 last 228.59) - /unlink 54788 (time 1746143011.72 total 228.33 last 275.87) - /unlink 57327 (time 1746143021.72 total 238.33 last 253.90) - /unlink 59554 (time 1746143031.72 total 248.33 last 222.69) - /unlink 61935 (time 1746143041.72 total 258.33 last 238.06) - /unlink 64440 (time 1746143051.73 total 268.34 last 250.25) total: 64768 /unlink in 269.89 seconds: 239.98 ops/second Waiting for MDT destroys to complete - open/close 1452 (time 1746143074.81 total 10.02 last 144.97) - open/close 2840 (time 1746143084.81 total 20.02 last 138.80) - open/close 3982 (time 1746143094.81 total 30.02 last 114.11) - open/close 5260 (time 1746143104.81 total 40.02 last 127.79) - open/close 6584 (time 1746143114.82 total 50.03 last 132.38) - open/close 8048 (time 1746143124.82 total 60.03 last 146.28) - open/close 9482 (time 1746143134.82 total 70.04 last 143.39) - open/close 10000 (time 1746143138.45 total 73.66 last 142.92) - open/close 11381 (time 1746143148.45 total 83.66 last 138.08) - open/close 12570 (time 1746143158.45 total 93.67 last 118.86) - open/close 13928 (time 1746143168.46 total 103.67 last 135.77) - open/close 15187 (time 1746143178.46 total 113.67 last 125.86) - open/close 16523 (time 1746143188.46 total 123.67 last 133.58) - open/close 17994 (time 1746143198.47 total 133.68 last 146.99) - open/close 19278 (time 1746143208.47 total 143.68 last 128.34) - open/close 20000 (time 1746143213.97 total 149.18 last 131.26) - open/close 21416 (time 1746143223.98 total 159.19 last 141.57) - open/close 22882 (time 1746143233.98 total 169.19 last 146.50) - open/close 24304 (time 1746143243.99 total 179.20 last 142.06) - open/close 25659 (time 1746143254.00 total 189.21 last 135.39) - open/close 27066 (time 1746143264.00 total 199.21 last 140.65) - open/close 28401 (time 1746143274.01 total 209.22 last 133.48) - open/close 29736 (time 1746143284.01 total 219.22 last 133.42) - open/close 31086 (time 1746143294.02 total 229.23 last 134.94) - open/close 32438 (time 1746143304.03 total 239.24 last 135.06) - open/close 33801 (time 1746143314.03 total 249.24 last 136.23) - open/close 35214 (time 1746143324.03 total 259.24 last 141.29) - open/close 36593 (time 1746143334.04 total 269.25 last 137.87) - open/close 37852 (time 1746143344.04 total 279.25 last 125.88) - open/close 39169 (time 1746143354.04 total 289.25 last 131.62) - open/close 40000 (time 1746143360.60 total 295.81 last 126.69) - open/close 41310 (time 1746143370.61 total 305.82 last 130.92) - open/close 42695 (time 1746143380.61 total 315.82 last 138.45) - open/close 44070 (time 1746143390.62 total 325.83 last 137.44) - open/close 45282 (time 1746143400.62 total 335.83 last 121.16) - open/close 46607 (time 1746143410.62 total 345.83 last 132.43) - open/close 47767 (time 1746143420.63 total 355.84 last 115.97) - open/close 48993 (time 1746143430.63 total 365.84 last 122.53) - open/close 50000 (time 1746143438.15 total 373.36 last 133.98) - open/close 51346 (time 1746143448.15 total 383.36 last 134.55) - open/close 52721 (time 1746143458.16 total 393.37 last 137.43) - open/close 54109 (time 1746143468.16 total 403.37 last 138.76) - open/close 55559 (time 1746143478.16 total 413.37 last 144.97) - open/close 56861 (time 1746143488.16 total 423.37 last 130.17) - open/close 58166 (time 1746143498.17 total 433.38 last 130.44) - open/close 59587 (time 1746143508.20 total 443.41 last 141.68) - open/close 60000 (time 1746143511.17 total 446.38 last 138.81) - open/close 61362 (time 1746143521.18 total 456.39 last 136.14) - open/close 62785 (time 1746143531.18 total 466.39 last 142.27) - open/close 64182 (time 1746143541.18 total 476.39 last 139.69) total: 64768 open/close in 480.79 seconds: 134.71 ops/second - /unlink 2014 (time 1746143557.71 total 10.00 last 201.35) - /unlink 4034 (time 1746143567.71 total 20.01 last 201.93) - /unlink 6132 (time 1746143577.72 total 30.01 last 209.74) - /unlink 8311 (time 1746143587.72 total 40.01 last 217.89) - /unlink 10000 (time 1746143595.56 total 47.85 last 215.38) - /unlink 12273 (time 1746143605.56 total 57.86 last 227.16) - /unlink 14724 (time 1746143615.57 total 67.86 last 245.06) - /unlink 17146 (time 1746143625.57 total 77.86 last 242.14) - /unlink 19572 (time 1746143635.57 total 87.86 last 242.51) - /unlink 22002 (time 1746143645.57 total 97.87 last 242.99) - /unlink 24388 (time 1746143655.57 total 107.87 last 238.58) - /unlink 26743 (time 1746143665.58 total 117.87 last 235.43) - /unlink 29202 (time 1746143675.58 total 127.87 last 245.83) - /unlink 30000 (time 1746143679.19 total 131.49 last 220.84) - /unlink 31993 (time 1746143689.19 total 141.49 last 199.29) - /unlink 34084 (time 1746143699.20 total 151.49 last 209.01) - /unlink 36325 (time 1746143709.20 total 161.49 last 224.08) - /unlink 38735 (time 1746143719.20 total 171.49 last 240.94) - /unlink 40000 (time 1746143724.83 total 177.12 last 224.74) - /unlink 42209 (time 1746143734.83 total 187.12 last 220.87) - /unlink 44657 (time 1746143744.83 total 197.13 last 244.71) - /unlink 47138 (time 1746143754.84 total 207.13 last 248.07) - /unlink 49514 (time 1746143764.84 total 217.13 last 237.57) - /unlink 52269 (time 1746143774.84 total 227.13 last 275.49) - /unlink 54846 (time 1746143784.84 total 237.13 last 257.69) - /unlink 57490 (time 1746143794.84 total 247.13 last 264.32) - /unlink 60000 (time 1746143803.74 total 256.04 last 281.92) - /unlink 62868 (time 1746143813.75 total 266.04 last 286.70) total: 64768 /unlink in 273.13 seconds: 237.14 ops/second Waiting for MDT destroys to complete - open/close 1462 (time 1746143845.16 total 10.01 last 146.11) - open/close 3007 (time 1746143855.16 total 20.01 last 154.42) - open/close 4408 (time 1746143865.17 total 30.02 last 140.03) - open/close 5887 (time 1746143875.17 total 40.02 last 147.88) - open/close 7534 (time 1746143885.17 total 50.02 last 164.67) - open/close 9339 (time 1746143895.17 total 60.02 last 180.48) - open/close 10000 (time 1746143899.05 total 63.90 last 170.47) - open/close 11754 (time 1746143909.05 total 73.90 last 175.35) - open/close 13193 (time 1746143919.05 total 83.90 last 143.89) - open/close 14727 (time 1746143929.05 total 93.90 last 153.36) - open/close 16209 (time 1746143939.06 total 103.91 last 148.10) - open/close 17682 (time 1746143949.06 total 113.91 last 147.29) - open/close 19134 (time 1746143959.06 total 123.91 last 145.18) - open/close 20000 (time 1746143963.75 total 128.60 last 184.62) - open/close 21518 (time 1746143973.76 total 138.61 last 151.77) - open/close 23058 (time 1746143983.76 total 148.61 last 153.94) - open/close 24668 (time 1746143993.76 total 158.61 last 160.99) - open/close 26239 (time 1746144003.76 total 168.61 last 157.09) - open/close 27771 (time 1746144013.77 total 178.62 last 153.12) - open/close 29211 (time 1746144023.77 total 188.62 last 143.99) - open/close 30000 (time 1746144029.24 total 194.09 last 144.22) - open/close 31470 (time 1746144039.24 total 204.09 last 146.95) - open/close 32885 (time 1746144049.25 total 214.10 last 141.41) - open/close 34346 (time 1746144059.25 total 224.10 last 145.99) - open/close 35601 (time 1746144069.26 total 234.11 last 125.43) - open/close 36993 (time 1746144079.28 total 244.13 last 138.99) - open/close 38385 (time 1746144089.28 total 254.13 last 139.12) - open/close 39648 (time 1746144099.29 total 264.14 last 126.23) - open/close 40000 (time 1746144102.06 total 266.91 last 126.80) - open/close 41390 (time 1746144112.07 total 276.92 last 138.97) - open/close 42848 (time 1746144122.07 total 286.92 last 145.80) - open/close 44208 (time 1746144132.07 total 296.92 last 135.90) - open/close 45591 (time 1746144142.08 total 306.93 last 138.22) - open/close 46977 (time 1746144152.08 total 316.93 last 138.59) - open/close 48437 (time 1746144162.08 total 326.93 last 145.92) - open/close 49811 (time 1746144172.09 total 336.94 last 137.38) - open/close 51143 (time 1746144182.09 total 346.94 last 133.10) - open/close 52620 (time 1746144192.10 total 356.95 last 147.59) - open/close 54157 (time 1746144202.11 total 366.96 last 153.61) - open/close 55501 (time 1746144212.11 total 376.96 last 134.34) - open/close 56933 (time 1746144222.12 total 386.97 last 143.12) - open/close 58406 (time 1746144232.12 total 396.97 last 147.28) - open/close 59943 (time 1746144242.12 total 406.97 last 153.68) - open/close 61544 (time 1746144252.12 total 416.97 last 160.05) - open/close 63012 (time 1746144262.13 total 426.98 last 146.76) - open/close 64436 (time 1746144272.13 total 436.98 last 142.37) total: 64768 open/close in 439.06 seconds: 147.51 ops/second - /unlink 2555 (time 1746144285.89 total 10.00 last 255.42) - /unlink 4949 (time 1746144295.89 total 20.00 last 239.38) - /unlink 7152 (time 1746144305.89 total 30.01 last 220.26) - /unlink 9343 (time 1746144315.90 total 40.01 last 218.99) - /unlink 10000 (time 1746144318.58 total 42.70 last 244.52) - /unlink 12568 (time 1746144328.58 total 52.70 last 256.73) - /unlink 15162 (time 1746144338.59 total 62.70 last 259.34) - /unlink 18064 (time 1746144348.59 total 72.71 last 290.06) - /unlink 20000 (time 1746144355.40 total 79.52 last 284.38) - /unlink 22815 (time 1746144365.40 total 89.52 last 281.48) - /unlink 25406 (time 1746144375.40 total 99.52 last 259.06) - /unlink 27987 (time 1746144385.40 total 109.52 last 258.09) - /unlink 30000 (time 1746144394.03 total 118.14 last 233.40) - /unlink 32718 (time 1746144404.03 total 128.15 last 271.61) - /unlink 35115 (time 1746144414.04 total 138.15 last 239.66) - /unlink 37552 (time 1746144424.04 total 148.15 last 243.64) - /unlink 40000 (time 1746144433.92 total 158.03 last 247.83) - /unlink 42369 (time 1746144443.92 total 168.03 last 236.86) - /unlink 44716 (time 1746144453.92 total 178.03 last 234.70) - /unlink 47394 (time 1746144463.92 total 188.03 last 267.78) - /unlink 50000 (time 1746144473.10 total 197.21 last 283.89) - /unlink 53517 (time 1746144483.10 total 207.21 last 351.68) - /unlink 57582 (time 1746144493.10 total 217.22 last 406.44) - /unlink 60000 (time 1746144499.02 total 223.14 last 408.19) - /unlink 64250 (time 1746144509.03 total 233.14 last 424.92) total: 64768 /unlink in 234.67 seconds: 276.00 ops/second Waiting for MDT destroys to complete - open/close 1583 (time 1746144529.91 total 10.00 last 158.24) - open/close 3495 (time 1746144539.91 total 20.01 last 191.15) - open/close 5182 (time 1746144549.92 total 30.01 last 168.67) - open/close 6808 (time 1746144559.93 total 40.02 last 162.45) - open/close 8614 (time 1746144569.93 total 50.02 last 180.58) - open/close 10000 (time 1746144578.90 total 59.00 last 154.38) - open/close 11577 (time 1746144588.91 total 69.00 last 157.58) - open/close 13105 (time 1746144598.92 total 79.01 last 152.72) - open/close 14775 (time 1746144608.92 total 89.01 last 166.98) - open/close 16434 (time 1746144618.92 total 99.01 last 165.87) - open/close 18136 (time 1746144628.92 total 109.01 last 170.16) - open/close 19950 (time 1746144638.93 total 119.02 last 181.33) - open/close 21724 (time 1746144648.93 total 129.02 last 177.37) - open/close 23888 (time 1746144658.93 total 139.02 last 216.31) - open/close 25395 (time 1746144668.94 total 149.03 last 150.60) - open/close 26942 (time 1746144678.94 total 159.03 last 154.70) - open/close 28566 (time 1746144688.94 total 169.04 last 162.32) - open/close 30000 (time 1746144697.68 total 177.77 last 164.21) - open/close 31575 (time 1746144707.68 total 187.77 last 157.47) - open/close 33153 (time 1746144717.68 total 197.77 last 157.76) - open/close 34812 (time 1746144727.68 total 207.78 last 165.86) - open/close 36499 (time 1746144737.69 total 217.78 last 168.66) - open/close 38332 (time 1746144747.69 total 227.78 last 183.28) - open/close 39939 (time 1746144757.69 total 237.78 last 160.66) - open/close 41349 (time 1746144767.69 total 247.78 last 140.97) - open/close 43018 (time 1746144777.69 total 257.78 last 166.88) - open/close 44687 (time 1746144787.70 total 267.79 last 166.86) - open/close 46339 (time 1746144797.70 total 277.79 last 165.11) - open/close 47768 (time 1746144807.71 total 287.80 last 142.80) - open/close 49156 (time 1746144817.71 total 297.81 last 138.72) - open/close 50000 (time 1746144823.72 total 303.81 last 140.63) - open/close 51464 (time 1746144833.72 total 313.81 last 146.39) - open/close 53052 (time 1746144843.72 total 323.81 last 158.70) - open/close 54565 (time 1746144853.73 total 333.82 last 151.26) - open/close 55992 (time 1746144863.73 total 343.82 last 142.70) - open/close 57328 (time 1746144873.74 total 353.83 last 133.47) - open/close 58737 (time 1746144883.74 total 363.83 last 140.89) - open/close 60000 (time 1746144892.15 total 372.24 last 150.15) - open/close 61342 (time 1746144902.15 total 382.24 last 134.14) - open/close 62923 (time 1746144912.15 total 392.24 last 158.09) - open/close 64359 (time 1746144922.15 total 402.25 last 143.58) total: 64768 open/close in 405.31 seconds: 159.80 ops/second - /unlink 2303 (time 1746144936.98 total 10.00 last 230.30) - /unlink 4734 (time 1746144946.98 total 20.00 last 243.05) - /unlink 7275 (time 1746144956.98 total 30.01 last 253.97) - /unlink 9670 (time 1746144966.98 total 40.01 last 239.49) - /unlink 12016 (time 1746144976.99 total 50.01 last 234.51) - /unlink 14489 (time 1746144986.99 total 60.02 last 247.19) - /unlink 16955 (time 1746144996.99 total 70.02 last 246.56) - /unlink 19340 (time 1746145007.00 total 80.02 last 238.45) - /unlink 20000 (time 1746145009.38 total 82.40 last 276.78) - /unlink 22583 (time 1746145019.38 total 92.41 last 258.19) - /unlink 25061 (time 1746145029.39 total 102.41 last 247.79) - /unlink 27532 (time 1746145039.39 total 112.41 last 247.04) - /unlink 29996 (time 1746145049.39 total 122.41 last 246.35) - /unlink 32364 (time 1746145059.39 total 132.42 last 236.72) - /unlink 34908 (time 1746145069.40 total 142.42 last 254.34) - /unlink 37279 (time 1746145079.40 total 152.42 last 237.06) - /unlink 40000 (time 1746145088.37 total 161.40 last 303.16) - /unlink 42872 (time 1746145098.37 total 171.40 last 287.18) - /unlink 45592 (time 1746145108.37 total 181.40 last 271.96) - /unlink 48080 (time 1746145118.38 total 191.40 last 248.78) - /unlink 50000 (time 1746145125.12 total 198.14 last 284.77) - /unlink 52976 (time 1746145135.14 total 208.16 last 297.09) - /unlink 55855 (time 1746145145.14 total 218.16 last 287.80) - /unlink 58855 (time 1746145155.14 total 228.17 last 299.89) - /unlink 60000 (time 1746145159.36 total 232.38 last 271.80) - /unlink 62705 (time 1746145169.36 total 242.38 last 270.46) total: 64768 /unlink in 250.68 seconds: 258.37 ops/second Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg257-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg257-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg257-server stop mds service on oleg257-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg257-server stop mds service on oleg257-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg257-server unloading modules via unload_modules_local on: 'oleg257-server' oleg257-server: oleg257-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg257-client: oleg257-client: ssh exited with exit code 2 pdsh@oleg257-client: oleg257-server: ssh exited with exit code 2 pdsh@oleg257-client: oleg257-client: ssh exited with exit code 2 pdsh@oleg257-client: oleg257-server: ssh exited with exit code 2