== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 18:25:02 (1745619902) Stopping clients: oleg137-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg137-client.virtnet /mnt/lustre2 (opts:-f) oleg137-server: oleg137-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory loading modules on: 'oleg137-server' oleg137-server: oleg137-server.virtnet: executing load_modules_local oleg137-server: Loading modules from /home/green/git/lustre-release/lustre oleg137-server: detected 4 online CPUs by sysfs oleg137-server: MODOPTS_LIBCFS= oleg137-server: Force libcfs to create 2 CPU partitions oleg137-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg137-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg137-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg137-server: oleg137-server.virtnet: executing set_default_debug -1 all pdsh@oleg137-client: oleg137-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg137-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg137-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg137-server: oleg137-server.virtnet: executing set_default_debug -1 all pdsh@oleg137-client: oleg137-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg137-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg137-client.virtnet: -o user_xattr,flock 192.168.201.137@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1202 (time 1745619975.44 total 10.01 last 120.06) - open/close 2359 (time 1745619985.44 total 20.01 last 115.66) - open/close 3600 (time 1745619995.45 total 30.02 last 124.06) - open/close 4908 (time 1745620005.45 total 40.02 last 130.77) - open/close 6176 (time 1745620015.46 total 50.03 last 126.73) - open/close 7412 (time 1745620025.46 total 60.03 last 123.56) - open/close 8611 (time 1745620035.47 total 70.04 last 119.80) - open/close 9803 (time 1745620045.47 total 80.04 last 119.20) - open/close 10911 (time 1745620055.47 total 90.04 last 110.74) - open/close 12068 (time 1745620065.48 total 100.05 last 115.67) - open/close 13268 (time 1745620075.48 total 110.05 last 119.95) - open/close 14491 (time 1745620085.48 total 120.05 last 122.29) - open/close 15621 (time 1745620095.48 total 130.05 last 112.99) - open/close 16901 (time 1745620105.49 total 140.06 last 127.95) - open/close 18293 (time 1745620115.49 total 150.06 last 139.18) - open/close 19663 (time 1745620125.49 total 160.06 last 136.94) - open/close 20000 (time 1745620128.36 total 162.93 last 117.31) - open/close 21170 (time 1745620138.37 total 172.94 last 116.94) - open/close 22314 (time 1745620148.38 total 182.95 last 114.25) - open/close 23322 (time 1745620158.39 total 192.96 last 100.73) - open/close 24614 (time 1745620168.39 total 202.96 last 129.18) - open/close 26262 (time 1745620178.39 total 212.96 last 164.80) - open/close 27733 (time 1745620188.39 total 222.96 last 147.08) - open/close 29098 (time 1745620198.39 total 232.96 last 136.48) - open/close 30000 (time 1745620204.59 total 239.16 last 145.51) - open/close 31391 (time 1745620214.59 total 249.16 last 139.09) - open/close 32784 (time 1745620224.60 total 259.17 last 139.16) - open/close 33986 (time 1745620234.61 total 269.18 last 120.16) - open/close 35332 (time 1745620244.61 total 279.18 last 134.50) - open/close 36536 (time 1745620254.62 total 289.19 last 120.35) - open/close 37921 (time 1745620264.62 total 299.19 last 138.43) - open/close 39542 (time 1745620274.63 total 309.20 last 162.01) - open/close 40000 (time 1745620277.42 total 311.99 last 164.19) - open/close 41783 (time 1745620287.42 total 321.99 last 178.23) - open/close 43293 (time 1745620297.42 total 331.99 last 150.98) - open/close 44983 (time 1745620307.43 total 342.00 last 168.93) - open/close 46522 (time 1745620317.44 total 352.01 last 153.75) - open/close 47952 (time 1745620327.44 total 362.01 last 142.99) - open/close 49739 (time 1745620337.44 total 372.01 last 178.60) - open/close 51336 (time 1745620347.45 total 382.02 last 159.57) - open/close 52891 (time 1745620357.45 total 392.02 last 155.48) - open/close 54353 (time 1745620367.46 total 402.03 last 146.08) - open/close 55943 (time 1745620377.47 total 412.04 last 158.94) - open/close 57395 (time 1745620387.47 total 422.04 last 145.20) - open/close 58851 (time 1745620397.47 total 432.04 last 145.59) - open/close 60000 (time 1745620404.63 total 439.20 last 160.41) - open/close 61549 (time 1745620414.63 total 449.20 last 154.83) - open/close 63129 (time 1745620424.63 total 459.20 last 158.00) total: 64768 open/close in 468.41 seconds: 138.27 ops/second - /unlink 3340 (time 1745620445.44 total 10.00 last 333.94) - /unlink 6161 (time 1745620455.44 total 20.00 last 282.08) - /unlink 8957 (time 1745620465.44 total 30.00 last 279.58) - /unlink 10000 (time 1745620469.49 total 34.05 last 257.77) - /unlink 12335 (time 1745620479.50 total 44.06 last 233.30) - /unlink 14696 (time 1745620489.50 total 54.06 last 236.03) - /unlink 17181 (time 1745620499.50 total 64.06 last 248.49) - /unlink 19952 (time 1745620509.50 total 74.06 last 277.05) - /unlink 22375 (time 1745620519.50 total 84.07 last 242.25) - /unlink 24551 (time 1745620529.51 total 94.07 last 217.53) - /unlink 27179 (time 1745620539.51 total 104.07 last 262.79) - /unlink 29550 (time 1745620549.52 total 114.08 last 236.91) - /unlink 30000 (time 1745620554.23 total 118.79 last 95.38) - /unlink 31073 (time 1745620564.24 total 128.80 last 107.28) - /unlink 32090 (time 1745620574.25 total 138.81 last 101.60) - /unlink 33404 (time 1745620584.25 total 148.81 last 131.39) - /unlink 35009 (time 1745620594.25 total 158.81 last 160.48) - /unlink 36592 (time 1745620604.25 total 168.81 last 158.29) - /unlink 38139 (time 1745620614.25 total 178.81 last 154.69) - /unlink 39622 (time 1745620624.26 total 188.82 last 148.20) - /unlink 40000 (time 1745620626.53 total 191.09 last 166.01) - /unlink 41385 (time 1745620636.54 total 201.10 last 138.38) - /unlink 42674 (time 1745620646.54 total 211.10 last 128.90) - /unlink 43939 (time 1745620656.55 total 221.11 last 126.44) - /unlink 45371 (time 1745620666.55 total 231.11 last 143.14) - /unlink 46751 (time 1745620676.55 total 241.11 last 137.98) - /unlink 48190 (time 1745620686.56 total 251.12 last 143.77) - /unlink 49899 (time 1745620696.56 total 261.12 last 170.89) - /unlink 51712 (time 1745620706.56 total 271.12 last 181.29) - /unlink 53507 (time 1745620716.56 total 281.12 last 179.49) - /unlink 55023 (time 1745620726.57 total 291.13 last 151.55) - /unlink 56707 (time 1745620736.57 total 301.13 last 168.35) - /unlink 58247 (time 1745620746.57 total 311.14 last 153.93) - /unlink 60000 (time 1745620756.30 total 320.86 last 180.23) - /unlink 61469 (time 1745620766.30 total 330.86 last 146.89) - /unlink 62897 (time 1745620776.31 total 340.87 last 142.75) - /unlink 64299 (time 1745620786.31 total 350.88 last 140.08) total: 64768 /unlink in 354.37 seconds: 182.77 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 882 (time 1745620820.40 total 10.00 last 88.19) - open/close 1887 (time 1745620830.41 total 20.00 last 100.49) - open/close 2877 (time 1745620840.41 total 30.01 last 98.94) - open/close 3801 (time 1745620850.42 total 40.02 last 92.32) - open/close 4712 (time 1745620860.42 total 50.02 last 91.10) - open/close 5548 (time 1745620870.44 total 60.03 last 83.47) - open/close 6354 (time 1745620880.45 total 70.04 last 80.53) - open/close 7142 (time 1745620890.45 total 80.05 last 78.73) - open/close 8017 (time 1745620900.46 total 90.06 last 87.42) - open/close 8954 (time 1745620910.47 total 100.06 last 93.68) - open/close 9928 (time 1745620920.47 total 110.06 last 97.38) - open/close 10870 (time 1745620930.47 total 120.07 last 94.17) - open/close 11896 (time 1745620940.47 total 130.07 last 102.59) - open/close 12791 (time 1745620950.47 total 140.07 last 89.49) - open/close 13765 (time 1745620960.48 total 150.07 last 97.39) - open/close 14611 (time 1745620970.49 total 160.09 last 84.47) - open/close 15437 (time 1745620980.49 total 170.09 last 82.60) - open/close 16291 (time 1745620990.50 total 180.10 last 85.28) - open/close 17177 (time 1745621000.51 total 190.11 last 88.51) - open/close 17913 (time 1745621010.52 total 200.12 last 73.53) - open/close 18807 (time 1745621020.53 total 210.12 last 89.39) - open/close 19709 (time 1745621030.53 total 220.12 last 90.18) - open/close 20000 (time 1745621034.23 total 223.82 last 78.67) - open/close 20858 (time 1745621044.23 total 233.83 last 85.76) - open/close 21770 (time 1745621054.24 total 243.83 last 91.16) - open/close 22606 (time 1745621064.24 total 253.84 last 83.57) - open/close 23467 (time 1745621074.25 total 263.85 last 86.01) - open/close 24319 (time 1745621084.25 total 273.85 last 85.18) - open/close 25175 (time 1745621094.26 total 283.85 last 85.55) - open/close 26061 (time 1745621104.26 total 293.86 last 88.57) - open/close 26937 (time 1745621114.27 total 303.87 last 87.51) - open/close 27810 (time 1745621124.28 total 313.87 last 87.25) - open/close 28681 (time 1745621134.28 total 323.88 last 87.08) - open/close 29535 (time 1745621144.29 total 333.88 last 85.33) - open/close 30000 (time 1745621150.39 total 339.99 last 76.15) - open/close 30855 (time 1745621160.40 total 350.00 last 85.43) - open/close 31734 (time 1745621170.40 total 360.00 last 87.88) - open/close 32574 (time 1745621180.42 total 370.02 last 83.86) - open/close 33392 (time 1745621190.43 total 380.03 last 81.72) - open/close 34197 (time 1745621200.43 total 390.03 last 80.49) - open/close 35024 (time 1745621210.44 total 400.03 last 82.65) - open/close 35885 (time 1745621220.44 total 410.03 last 86.10) - open/close 36767 (time 1745621230.44 total 420.04 last 88.13) - open/close 37681 (time 1745621240.45 total 430.04 last 91.37) - open/close 38668 (time 1745621250.45 total 440.04 last 98.69) - open/close 39576 (time 1745621260.45 total 450.05 last 90.78) - open/close 40000 (time 1745621264.95 total 454.55 last 94.16) - open/close 41031 (time 1745621274.96 total 464.55 last 103.06) - open/close 42003 (time 1745621284.96 total 474.56 last 97.15) - open/close 43018 (time 1745621294.97 total 484.56 last 101.46) - open/close 43877 (time 1745621304.97 total 494.56 last 85.90) - open/close 44750 (time 1745621314.97 total 504.57 last 87.26) - open/close 45594 (time 1745621324.98 total 514.57 last 84.36) - open/close 46536 (time 1745621334.98 total 524.58 last 94.19) - open/close 47476 (time 1745621344.98 total 534.58 last 93.96) - open/close 48435 (time 1745621354.99 total 544.59 last 95.84) - open/close 49320 (time 1745621364.99 total 554.59 last 88.46) - open/close 50000 (time 1745621372.05 total 561.65 last 96.33) - open/close 50983 (time 1745621382.07 total 571.66 last 98.18) - open/close 51922 (time 1745621392.07 total 581.66 last 93.90) - open/close 52904 (time 1745621402.07 total 591.67 last 98.14) - open/close 53786 (time 1745621412.08 total 601.68 last 88.11) - open/close 54669 (time 1745621422.08 total 611.68 last 88.29) - open/close 55615 (time 1745621432.09 total 621.68 last 94.57) - open/close 56625 (time 1745621442.09 total 631.68 last 100.98) - open/close 57724 (time 1745621452.09 total 641.69 last 109.88) - open/close 58685 (time 1745621462.10 total 651.70 last 96.02) - open/close 59687 (time 1745621472.10 total 661.70 last 100.17) - open/close 60000 (time 1745621475.26 total 664.86 last 99.08) - open/close 61000 (time 1745621485.29 total 674.88 last 99.74) - open/close 61938 (time 1745621495.29 total 684.89 last 93.78) - open/close 62843 (time 1745621505.30 total 694.89 last 90.43) - open/close 63836 (time 1745621515.30 total 704.90 last 99.23) - open/close 64736 (time 1745621525.31 total 714.91 last 89.90) total: 64768 open/close in 715.24 seconds: 90.55 ops/second - /unlink 1012 (time 1745621538.49 total 10.01 last 101.11) - /unlink 1849 (time 1745621548.49 total 20.01 last 83.70) - /unlink 2823 (time 1745621558.52 total 30.04 last 97.09) - /unlink 3784 (time 1745621568.53 total 40.05 last 96.00) - /unlink 4640 (time 1745621578.53 total 50.05 last 85.58) - /unlink 5503 (time 1745621588.53 total 60.06 last 86.26) - /unlink 6379 (time 1745621598.54 total 70.06 last 87.59) - /unlink 7344 (time 1745621608.54 total 80.06 last 96.49) - /unlink 8240 (time 1745621618.55 total 90.08 last 89.45) - /unlink 9225 (time 1745621628.55 total 100.08 last 98.50) - /unlink 10000 (time 1745621636.00 total 107.52 last 104.07) - /unlink 11073 (time 1745621646.00 total 117.53 last 107.27) - /unlink 12062 (time 1745621656.01 total 127.53 last 98.88) - /unlink 12977 (time 1745621666.01 total 137.53 last 91.46) - /unlink 13952 (time 1745621676.01 total 147.54 last 97.49) - /unlink 14850 (time 1745621686.02 total 157.54 last 89.77) - /unlink 15731 (time 1745621696.02 total 167.54 last 88.07) - /unlink 16822 (time 1745621706.02 total 177.55 last 109.06) - /unlink 17711 (time 1745621716.03 total 187.55 last 88.85) - /unlink 18549 (time 1745621726.04 total 197.56 last 83.75) - /unlink 19489 (time 1745621736.05 total 207.57 last 93.88) - /unlink 20000 (time 1745621741.84 total 213.37 last 88.20) - /unlink 20855 (time 1745621751.85 total 223.37 last 85.45) - /unlink 21723 (time 1745621761.86 total 233.38 last 86.72) - /unlink 22628 (time 1745621771.86 total 243.38 last 90.48) - /unlink 23619 (time 1745621781.87 total 253.39 last 98.98) - /unlink 24610 (time 1745621791.89 total 263.41 last 98.90) - /unlink 25457 (time 1745621801.90 total 273.42 last 84.66) - /unlink 26345 (time 1745621811.90 total 283.42 last 88.76) - /unlink 27334 (time 1745621821.91 total 293.43 last 98.85) - /unlink 28365 (time 1745621831.91 total 303.43 last 103.10) - /unlink 29408 (time 1745621841.92 total 313.44 last 104.16) - /unlink 30000 (time 1745621848.25 total 319.77 last 93.54) - /unlink 31079 (time 1745621858.26 total 329.78 last 107.81) - /unlink 32098 (time 1745621868.26 total 339.78 last 101.84) - /unlink 33178 (time 1745621878.27 total 349.79 last 107.95) - /unlink 34238 (time 1745621888.27 total 359.79 last 105.95) - /unlink 35335 (time 1745621898.27 total 369.80 last 109.68) - /unlink 36391 (time 1745621908.27 total 379.80 last 105.60) - /unlink 37404 (time 1745621918.29 total 389.81 last 101.15) - /unlink 38470 (time 1745621928.29 total 399.81 last 106.58) - /unlink 39563 (time 1745621938.31 total 409.83 last 109.12) - /unlink 40000 (time 1745621942.81 total 414.34 last 96.96) - /unlink 41127 (time 1745621952.82 total 424.34 last 112.66) - /unlink 42190 (time 1745621962.82 total 434.35 last 106.24) - /unlink 43269 (time 1745621972.84 total 444.36 last 107.75) - /unlink 44446 (time 1745621982.84 total 454.37 last 117.63) - /unlink 45582 (time 1745621992.84 total 464.37 last 113.58) - /unlink 46904 (time 1745622002.85 total 474.37 last 132.18) - /unlink 48063 (time 1745622012.85 total 484.37 last 115.86) - /unlink 49114 (time 1745622022.86 total 494.38 last 105.03) - /unlink 50000 (time 1745622031.46 total 502.99 last 102.92) - /unlink 51142 (time 1745622041.47 total 512.99 last 114.19) - /unlink 52357 (time 1745622051.48 total 523.00 last 121.36) - /unlink 53613 (time 1745622061.48 total 533.00 last 125.56) - /unlink 54797 (time 1745622071.49 total 543.01 last 118.32) - /unlink 55992 (time 1745622081.49 total 553.01 last 119.50) - /unlink 57253 (time 1745622091.49 total 563.02 last 126.03) - /unlink 58580 (time 1745622101.49 total 573.02 last 132.70) - /unlink 59760 (time 1745622111.50 total 583.02 last 117.94) - /unlink 60000 (time 1745622113.84 total 585.37 last 102.38) - /unlink 61219 (time 1745622123.84 total 595.37 last 121.89) - /unlink 62480 (time 1745622133.84 total 605.37 last 126.08) - /unlink 63681 (time 1745622143.86 total 615.38 last 119.96) - /unlink 64745 (time 1745622153.86 total 625.38 last 106.34) total: 64768 /unlink in 625.59 seconds: 103.53 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1439 (time 1745622182.46 total 10.01 last 143.80) - open/close 2847 (time 1745622192.46 total 20.01 last 140.78) - open/close 4200 (time 1745622202.47 total 30.02 last 135.19) - open/close 5436 (time 1745622212.48 total 40.03 last 123.47) - open/close 6738 (time 1745622222.49 total 50.03 last 130.14) - open/close 8034 (time 1745622232.49 total 60.04 last 129.52) - open/close 9359 (time 1745622242.50 total 70.05 last 132.38) - open/close 10000 (time 1745622248.05 total 75.60 last 115.47) - open/close 11476 (time 1745622258.06 total 85.60 last 147.55) - open/close 12947 (time 1745622268.06 total 95.60 last 147.07) - open/close 14388 (time 1745622278.06 total 105.61 last 144.03) - open/close 15675 (time 1745622288.07 total 115.61 last 128.66) - open/close 16953 (time 1745622298.07 total 125.61 last 127.75) - open/close 18078 (time 1745622308.07 total 135.62 last 112.48) - open/close 19305 (time 1745622318.08 total 145.63 last 122.57) - open/close 20000 (time 1745622324.12 total 151.67 last 115.11) - open/close 21160 (time 1745622334.13 total 161.67 last 115.94) - open/close 22337 (time 1745622344.13 total 171.67 last 117.69) - open/close 23665 (time 1745622354.13 total 181.68 last 132.75) - open/close 24893 (time 1745622364.14 total 191.68 last 122.73) - open/close 26174 (time 1745622374.14 total 201.69 last 128.02) - open/close 27406 (time 1745622384.14 total 211.69 last 123.16) - open/close 28711 (time 1745622394.15 total 221.70 last 130.41) - open/close 30000 (time 1745622403.64 total 231.18 last 135.90) - open/close 31309 (time 1745622413.64 total 241.19 last 130.83) - open/close 32528 (time 1745622423.65 total 251.19 last 121.84) - open/close 33793 (time 1745622433.65 total 261.20 last 126.39) - open/close 34865 (time 1745622443.66 total 271.20 last 107.17) - open/close 35876 (time 1745622453.66 total 281.21 last 101.08) - open/close 37110 (time 1745622463.66 total 291.21 last 123.34) - open/close 38379 (time 1745622473.67 total 301.21 last 126.87) - open/close 39664 (time 1745622483.67 total 311.22 last 128.47) - open/close 40000 (time 1745622485.91 total 313.46 last 149.89) - open/close 41405 (time 1745622495.92 total 323.47 last 140.37) - open/close 42866 (time 1745622505.92 total 333.47 last 146.06) - open/close 44389 (time 1745622515.92 total 343.47 last 152.29) - open/close 45846 (time 1745622525.93 total 353.47 last 145.67) - open/close 47125 (time 1745622535.94 total 363.48 last 127.79) - open/close 48505 (time 1745622545.94 total 373.49 last 137.94) - open/close 49835 (time 1745622555.95 total 383.49 last 132.93) - open/close 51130 (time 1745622565.95 total 393.50 last 129.43) - open/close 52284 (time 1745622575.95 total 403.50 last 115.36) - open/close 53614 (time 1745622585.96 total 413.50 last 132.93) - open/close 54776 (time 1745622595.96 total 423.51 last 116.18) - open/close 55949 (time 1745622605.97 total 433.51 last 117.25) - open/close 57130 (time 1745622615.98 total 443.52 last 117.95) - open/close 58195 (time 1745622625.98 total 453.53 last 106.44) - open/close 59281 (time 1745622635.99 total 463.53 last 108.58) - open/close 60000 (time 1745622642.98 total 470.52 last 102.84) - open/close 60934 (time 1745622652.98 total 480.52 last 93.39) - open/close 61852 (time 1745622662.99 total 490.53 last 91.73) - open/close 62905 (time 1745622672.99 total 500.54 last 105.22) - open/close 64149 (time 1745622682.99 total 510.54 last 124.39) total: 64768 open/close in 516.03 seconds: 125.51 ops/second - /unlink 1695 (time 1745622700.71 total 10.00 last 169.42) - /unlink 3429 (time 1745622710.71 total 20.01 last 173.33) - /unlink 5287 (time 1745622720.72 total 30.01 last 185.73) - /unlink 7258 (time 1745622730.72 total 40.01 last 197.08) - /unlink 9166 (time 1745622740.72 total 50.01 last 190.79) - /unlink 10000 (time 1745622745.51 total 54.81 last 174.09) - /unlink 11596 (time 1745622755.52 total 64.82 last 159.42) - /unlink 13407 (time 1745622765.52 total 74.82 last 181.10) - /unlink 15339 (time 1745622775.52 total 84.82 last 193.16) - /unlink 17303 (time 1745622785.52 total 94.82 last 196.40) - /unlink 19373 (time 1745622795.53 total 104.82 last 206.94) - /unlink 20000 (time 1745622799.00 total 108.30 last 180.48) - /unlink 21831 (time 1745622809.00 total 118.30 last 183.08) - /unlink 23830 (time 1745622819.00 total 128.30 last 199.90) - /unlink 25693 (time 1745622829.00 total 138.30 last 186.27) - /unlink 27510 (time 1745622839.00 total 148.30 last 181.69) - /unlink 29160 (time 1745622849.01 total 158.30 last 164.94) - /unlink 30000 (time 1745622853.78 total 163.07 last 176.08) - /unlink 31650 (time 1745622863.78 total 173.08 last 164.96) - /unlink 33452 (time 1745622873.79 total 183.08 last 180.08) - /unlink 35237 (time 1745622883.79 total 193.09 last 178.39) - /unlink 37150 (time 1745622893.80 total 203.09 last 191.22) - /unlink 39218 (time 1745622903.80 total 213.10 last 206.75) - /unlink 40000 (time 1745622907.45 total 216.75 last 214.12) - /unlink 41803 (time 1745622917.45 total 226.75 last 180.29) - /unlink 43728 (time 1745622927.45 total 236.75 last 192.45) - /unlink 45542 (time 1745622937.46 total 246.76 last 181.30) - /unlink 47370 (time 1745622947.46 total 256.76 last 182.79) - /unlink 49300 (time 1745622957.46 total 266.76 last 192.97) - /unlink 50000 (time 1745622961.54 total 270.83 last 171.72) - /unlink 51779 (time 1745622971.54 total 280.84 last 177.83) - /unlink 53468 (time 1745622981.55 total 290.84 last 168.85) - /unlink 55310 (time 1745622991.55 total 300.84 last 184.16) - /unlink 57080 (time 1745623001.55 total 310.85 last 176.93) - /unlink 59051 (time 1745623011.55 total 320.85 last 197.08) - /unlink 60000 (time 1745623015.68 total 324.97 last 230.08) - /unlink 62240 (time 1745623025.68 total 334.98 last 223.93) - /unlink 64456 (time 1745623035.68 total 344.98 last 221.60) total: 64768 /unlink in 346.41 seconds: 186.97 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1548 (time 1745623068.13 total 10.01 last 154.70) - open/close 3176 (time 1745623078.13 total 20.01 last 162.76) - open/close 4957 (time 1745623088.13 total 30.01 last 178.06) - open/close 6427 (time 1745623098.13 total 40.01 last 146.95) - open/close 7916 (time 1745623108.14 total 50.02 last 148.85) - open/close 9300 (time 1745623118.14 total 60.02 last 138.37) - open/close 10000 (time 1745623122.78 total 64.66 last 150.89) - open/close 11541 (time 1745623132.78 total 74.66 last 154.04) - open/close 12978 (time 1745623142.78 total 84.67 last 143.66) - open/close 14404 (time 1745623152.79 total 94.67 last 142.53) - open/close 15897 (time 1745623162.79 total 104.67 last 149.27) - open/close 17348 (time 1745623172.79 total 114.67 last 145.10) - open/close 18873 (time 1745623182.79 total 124.68 last 152.47) - open/close 20000 (time 1745623189.85 total 131.73 last 159.66) - open/close 21771 (time 1745623199.86 total 141.74 last 177.06) - open/close 23399 (time 1745623209.86 total 151.74 last 162.76) - open/close 25352 (time 1745623219.86 total 161.74 last 195.19) - open/close 27198 (time 1745623229.87 total 171.75 last 184.54) - open/close 28914 (time 1745623239.87 total 181.75 last 171.49) - open/close 30000 (time 1745623245.84 total 187.72 last 182.05) - open/close 32070 (time 1745623255.84 total 197.72 last 206.98) - open/close 33807 (time 1745623265.85 total 207.73 last 173.58) - open/close 35250 (time 1745623275.85 total 217.73 last 144.29) - open/close 36631 (time 1745623285.85 total 227.74 last 138.00) - open/close 38274 (time 1745623295.86 total 237.74 last 164.23) - open/close 39906 (time 1745623305.86 total 247.74 last 163.16) - open/close 41794 (time 1745623315.86 total 257.74 last 188.79) - open/close 43346 (time 1745623325.87 total 267.75 last 155.14) - open/close 45063 (time 1745623335.87 total 277.75 last 171.69) - open/close 46965 (time 1745623345.87 total 287.75 last 190.20) - open/close 48738 (time 1745623355.87 total 297.75 last 177.24) - open/close 50000 (time 1745623362.47 total 304.35 last 191.25) - open/close 52181 (time 1745623372.47 total 314.35 last 218.07) - open/close 54657 (time 1745623382.47 total 324.35 last 247.51) - open/close 56743 (time 1745623392.47 total 334.35 last 208.59) - open/close 58608 (time 1745623402.48 total 344.36 last 186.46) - open/close 60000 (time 1745623411.15 total 353.03 last 160.49) - open/close 61561 (time 1745623421.15 total 363.03 last 156.09) - open/close 63157 (time 1745623431.16 total 373.04 last 159.47) - open/close 64747 (time 1745623441.16 total 383.04 last 158.95) total: 64768 open/close in 383.18 seconds: 169.03 ops/second - /unlink 2962 (time 1745623452.98 total 10.00 last 296.17) - /unlink 6109 (time 1745623462.99 total 20.01 last 314.53) - /unlink 8852 (time 1745623472.99 total 30.01 last 274.26) - /unlink 10000 (time 1745623477.15 total 34.16 last 276.18) - /unlink 13253 (time 1745623487.15 total 44.17 last 325.17) - /unlink 16892 (time 1745623497.15 total 54.17 last 363.87) - /unlink 19923 (time 1745623507.15 total 64.17 last 303.04) - /unlink 24000 (time 1745623517.16 total 74.17 last 407.63) - /unlink 26823 (time 1745623527.16 total 84.18 last 282.13) - /unlink 29953 (time 1745623537.16 total 94.18 last 312.96) - /unlink 33495 (time 1745623547.16 total 104.18 last 354.19) - /unlink 36382 (time 1745623557.16 total 114.18 last 288.67) - /unlink 39510 (time 1745623567.17 total 124.19 last 312.67) - /unlink 42570 (time 1745623577.17 total 134.19 last 306.00) - /unlink 45290 (time 1745623587.17 total 144.19 last 272.00) - /unlink 47341 (time 1745623597.17 total 154.19 last 205.05) - /unlink 49491 (time 1745623607.17 total 164.19 last 214.99) - /unlink 50000 (time 1745623609.53 total 166.54 last 216.03) - /unlink 52801 (time 1745623619.53 total 176.55 last 280.08) - /unlink 55667 (time 1745623629.53 total 186.55 last 286.60) - /unlink 58442 (time 1745623639.53 total 196.55 last 277.44) - /unlink 60000 (time 1745623645.73 total 202.75 last 251.38) - /unlink 62524 (time 1745623655.73 total 212.75 last 252.36) total: 64768 /unlink in 220.46 seconds: 293.79 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1409 (time 1745623690.05 total 10.01 last 140.81) - open/close 2947 (time 1745623700.05 total 20.01 last 153.72) - open/close 4547 (time 1745623710.06 total 30.02 last 159.88) - open/close 6319 (time 1745623720.06 total 40.02 last 177.15) - open/close 7975 (time 1745623730.06 total 50.02 last 165.60) - open/close 9677 (time 1745623740.06 total 60.02 last 170.18) - open/close 11288 (time 1745623750.07 total 70.03 last 160.99) - open/close 13071 (time 1745623760.08 total 80.04 last 178.18) - open/close 15201 (time 1745623770.08 total 90.04 last 212.88) - open/close 17035 (time 1745623780.09 total 100.05 last 183.18) - open/close 18569 (time 1745623790.10 total 110.06 last 153.32) - open/close 20000 (time 1745623798.74 total 118.70 last 165.57) - open/close 21938 (time 1745623808.75 total 128.71 last 193.69) - open/close 23586 (time 1745623818.75 total 138.71 last 164.74) - open/close 25549 (time 1745623828.75 total 148.71 last 196.24) - open/close 27360 (time 1745623838.76 total 158.72 last 180.96) - open/close 29230 (time 1745623848.76 total 168.72 last 186.95) - open/close 30000 (time 1745623852.11 total 172.07 last 230.15) - open/close 32013 (time 1745623862.12 total 182.08 last 201.16) - open/close 33993 (time 1745623872.12 total 192.08 last 198.00) - open/close 36139 (time 1745623882.12 total 202.08 last 214.56) - open/close 38036 (time 1745623892.12 total 212.08 last 189.67) - open/close 40000 (time 1745623901.94 total 221.90 last 200.10) - open/close 42009 (time 1745623911.94 total 231.90 last 200.81) - open/close 43834 (time 1745623921.94 total 241.90 last 182.44) - open/close 45532 (time 1745623931.95 total 251.91 last 169.70) - open/close 47500 (time 1745623941.95 total 261.91 last 196.72) - open/close 49183 (time 1745623951.96 total 271.92 last 168.22) - open/close 50000 (time 1745623956.28 total 276.24 last 188.92) - open/close 52262 (time 1745623966.28 total 286.24 last 226.17) - open/close 54109 (time 1745623976.29 total 296.25 last 184.66) - open/close 56023 (time 1745623986.29 total 306.25 last 191.36) - open/close 58146 (time 1745623996.29 total 316.25 last 212.25) - open/close 60000 (time 1745624005.49 total 325.45 last 201.60) - open/close 62191 (time 1745624015.49 total 335.45 last 219.09) - open/close 64175 (time 1745624025.49 total 345.45 last 198.39) total: 64768 open/close in 348.75 seconds: 185.71 ops/second - /unlink 2884 (time 1745624040.19 total 10.00 last 288.39) - /unlink 6104 (time 1745624050.19 total 20.00 last 321.94) - /unlink 9400 (time 1745624060.19 total 30.01 last 329.46) - /unlink 10000 (time 1745624062.71 total 32.53 last 238.09) - /unlink 13557 (time 1745624072.71 total 42.53 last 355.69) - /unlink 17114 (time 1745624082.71 total 52.53 last 355.63) - /unlink 20000 (time 1745624091.40 total 61.21 last 332.26) - /unlink 22851 (time 1745624101.40 total 71.22 last 285.06) - /unlink 26533 (time 1745624111.40 total 81.22 last 368.17) - /unlink 29876 (time 1745624121.40 total 91.22 last 334.29) - /unlink 32540 (time 1745624131.40 total 101.22 last 266.38) - /unlink 36071 (time 1745624141.41 total 111.22 last 353.05) - /unlink 39633 (time 1745624151.41 total 121.22 last 356.13) - /unlink 43558 (time 1745624161.41 total 131.22 last 392.39) - /unlink 46811 (time 1745624171.41 total 141.22 last 325.30) - /unlink 49835 (time 1745624181.41 total 151.23 last 302.34) - /unlink 52603 (time 1745624191.41 total 161.23 last 276.73) - /unlink 55622 (time 1745624201.42 total 171.23 last 301.79) - /unlink 58876 (time 1745624211.42 total 181.23 last 325.37) - /unlink 60000 (time 1745624215.45 total 185.27 last 278.53) - /unlink 63571 (time 1745624225.46 total 195.27 last 357.03) total: 64768 /unlink in 198.90 seconds: 325.63 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg137-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg137-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg137-server stop mds service on oleg137-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg137-server unloading modules via unload_modules_local on: 'oleg137-server' oleg137-server: oleg137-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg137-client: oleg137-client: ssh exited with exit code 2 pdsh@oleg137-client: oleg137-server: ssh exited with exit code 2 pdsh@oleg137-client: oleg137-client: ssh exited with exit code 2 pdsh@oleg137-client: oleg137-server: ssh exited with exit code 2