== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 20:06:26 (1712275586) Stopping clients: oleg428-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg428-client.virtnet /mnt/lustre2 (opts:-f) oleg428-server: oleg428-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg428-server' oleg428-server: oleg428-server.virtnet: executing load_modules_local oleg428-server: Loading modules from /home/green/git/lustre-release/lustre oleg428-server: detected 4 online CPUs by sysfs oleg428-server: Force libcfs to create 2 CPU partitions oleg428-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg428-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg428-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg428-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg428-server: oleg428-server.virtnet: executing set_default_debug -1 all pdsh@oleg428-client: oleg428-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg428-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg428-client.virtnet: -o user_xattr,flock oleg428-server@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 16134 326 15808 3% /mnt/lustre[OST:0] filesystem_summary: 16137 329 15808 3% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 3979 (time 1712275620.45 total 10.00 last 397.83) - open/close 7726 (time 1712275630.45 total 20.00 last 374.66) - open/close 10000 (time 1712275638.71 total 28.26 last 275.42) - open/close 12141 (time 1712275648.71 total 38.26 last 214.08) - open/close 14182 (time 1712275658.71 total 48.26 last 204.02) - open/close 15605 (time 1712275668.72 total 58.27 last 142.25) - open/close 17314 (time 1712275678.72 total 68.27 last 170.88) - open/close 19027 (time 1712275688.72 total 78.27 last 171.21) - open/close 20000 (time 1712275695.62 total 85.17 last 141.02) - open/close 21482 (time 1712275705.63 total 95.18 last 148.05) - open/close 22937 (time 1712275715.63 total 105.19 last 145.47) - open/close 24350 (time 1712275725.64 total 115.19 last 141.28) - open/close 25614 (time 1712275735.64 total 125.19 last 126.39) - open/close 26733 (time 1712275745.64 total 135.19 last 111.85) - open/close 27626 (time 1712275755.65 total 145.20 last 89.26) - open/close 28628 (time 1712275765.65 total 155.20 last 100.15) - open/close 29768 (time 1712275775.66 total 165.21 last 113.94) - open/close 30000 (time 1712275778.21 total 167.76 last 90.80) - open/close 30924 (time 1712275788.21 total 177.77 last 92.37) - open/close 31977 (time 1712275798.22 total 187.77 last 105.28) - open/close 32943 (time 1712275808.23 total 197.78 last 96.52) - open/close 34009 (time 1712275818.23 total 207.79 last 106.50) - open/close 35045 (time 1712275828.24 total 217.79 last 103.56) - open/close 36042 (time 1712275838.25 total 227.80 last 99.62) - open/close 36856 (time 1712275848.25 total 237.80 last 81.37) - open/close 37700 (time 1712275858.25 total 247.80 last 84.39) - open/close 38620 (time 1712275868.26 total 257.81 last 91.96) - open/close 39537 (time 1712275878.26 total 267.81 last 91.67) - open/close 40000 (time 1712275884.86 total 274.41 last 70.19) - open/close 40582 (time 1712275894.86 total 284.41 last 58.16) - open/close 41201 (time 1712275904.87 total 294.42 last 61.86) - open/close 41950 (time 1712275914.88 total 304.43 last 74.83) - open/close 42699 (time 1712275924.89 total 314.44 last 74.81) - open/close 43427 (time 1712275934.90 total 324.45 last 72.73) - open/close 43999 (time 1712275944.90 total 334.45 last 57.18) - open/close 44547 (time 1712275954.90 total 344.46 last 54.78) - open/close 45132 (time 1712275964.91 total 354.46 last 58.48) - open/close 45659 (time 1712275974.91 total 364.46 last 52.68) - open/close 46109 (time 1712275984.93 total 374.48 last 44.94) - open/close 46549 (time 1712275994.93 total 384.48 last 43.98) - open/close 47078 (time 1712276004.94 total 394.49 last 52.87) - open/close 47661 (time 1712276014.95 total 404.50 last 58.25) - open/close 48241 (time 1712276024.96 total 414.51 last 57.91) - open/close 48859 (time 1712276034.97 total 424.52 last 61.72) - open/close 49403 (time 1712276044.99 total 434.54 last 54.33) - open/close 50000 (time 1712276054.99 total 444.54 last 59.70) - open/close 50528 (time 1712276064.99 total 454.54 last 52.77) - open/close 51037 (time 1712276075.00 total 464.55 last 50.84) - open/close 51557 (time 1712276085.01 total 474.56 last 51.97) - open/close 52101 (time 1712276095.02 total 484.57 last 54.35) - open/close 52639 (time 1712276105.02 total 494.58 last 53.77) - open/close 53157 (time 1712276115.04 total 504.59 last 51.73) - open/close 53591 (time 1712276125.05 total 514.60 last 43.35) - open/close 54075 (time 1712276135.06 total 524.61 last 48.34) - open/close 54532 (time 1712276145.08 total 534.63 last 45.63) - open/close 54975 (time 1712276155.08 total 544.63 last 44.30) - open/close 55445 (time 1712276165.10 total 554.65 last 46.92) - open/close 56006 (time 1712276175.10 total 564.66 last 56.05) - open/close 56590 (time 1712276185.12 total 574.67 last 58.33) - open/close 57150 (time 1712276195.12 total 584.67 last 55.97) - open/close 57660 (time 1712276205.13 total 594.68 last 50.95) - open/close 58250 (time 1712276215.15 total 604.70 last 58.89) - open/close 58801 (time 1712276225.16 total 614.71 last 55.06) - open/close 59281 (time 1712276235.17 total 624.72 last 47.94) - open/close 59689 (time 1712276245.19 total 634.74 last 40.72) - open/close 60000 (time 1712276252.39 total 641.94 last 43.20) - open/close 60440 (time 1712276262.40 total 651.95 last 43.95) - open/close 60901 (time 1712276272.41 total 661.96 last 46.07) - open/close 61317 (time 1712276282.44 total 671.99 last 41.47) - open/close 61778 (time 1712276292.46 total 682.01 last 46.01) - open/close 62154 (time 1712276302.48 total 692.03 last 37.53) - open/close 62592 (time 1712276312.49 total 702.04 last 43.75) - open/close 62959 (time 1712276322.50 total 712.06 last 36.66) - open/close 63332 (time 1712276332.51 total 722.06 last 37.27) - open/close 63739 (time 1712276342.52 total 732.07 last 40.65) - open/close 64226 (time 1712276352.52 total 742.07 last 48.69) - open/close 64623 (time 1712276362.56 total 752.11 last 39.57) total: 64768 open/close in 755.76 seconds: 85.70 ops/second - /unlink 485 (time 1712276377.49 total 10.02 last 48.40) - /unlink 942 (time 1712276387.49 total 20.02 last 45.68) - /unlink 1397 (time 1712276397.51 total 30.04 last 45.43) - /unlink 1873 (time 1712276407.52 total 40.05 last 47.55) - /unlink 2345 (time 1712276417.53 total 50.06 last 47.16) - /unlink 2829 (time 1712276427.53 total 60.07 last 48.37) - /unlink 3333 (time 1712276437.54 total 70.07 last 50.38) - /unlink 3840 (time 1712276447.55 total 80.09 last 50.62) - /unlink 4354 (time 1712276457.56 total 90.09 last 51.38) - /unlink 4836 (time 1712276467.56 total 100.09 last 48.20) - /unlink 5329 (time 1712276477.56 total 110.09 last 49.28) - /unlink 5806 (time 1712276487.57 total 120.10 last 47.67) - /unlink 6307 (time 1712276497.57 total 130.11 last 50.06) - /unlink 6962 (time 1712276507.58 total 140.11 last 65.46) - /unlink 7846 (time 1712276517.59 total 150.12 last 88.32) - /unlink 8612 (time 1712276527.60 total 160.14 last 76.49) - /unlink 9396 (time 1712276537.61 total 170.14 last 78.38) - /unlink 10000 (time 1712276545.59 total 178.13 last 75.63) - /unlink 10747 (time 1712276555.60 total 188.14 last 74.61) - /unlink 11301 (time 1712276565.62 total 198.16 last 55.31) - /unlink 11930 (time 1712276575.63 total 208.17 last 62.83) - /unlink 12535 (time 1712276585.65 total 218.18 last 60.40) - /unlink 13108 (time 1712276595.65 total 228.19 last 57.27) - /unlink 13759 (time 1712276605.66 total 238.20 last 65.04) - /unlink 14355 (time 1712276615.67 total 248.20 last 59.57) - /unlink 15025 (time 1712276625.68 total 258.22 last 66.91) - /unlink 15730 (time 1712276635.69 total 268.22 last 70.47) - /unlink 16437 (time 1712276645.69 total 278.23 last 70.67) - /unlink 17129 (time 1712276655.69 total 288.23 last 69.18) - /unlink 17866 (time 1712276665.70 total 298.24 last 73.64) - /unlink 18632 (time 1712276675.70 total 308.24 last 76.60) - /unlink 19456 (time 1712276685.70 total 318.24 last 82.38) - /unlink 20000 (time 1712276692.30 total 324.83 last 82.49) - /unlink 20766 (time 1712276702.31 total 334.84 last 76.52) - /unlink 21521 (time 1712276712.31 total 344.85 last 75.47) - /unlink 22349 (time 1712276722.33 total 354.86 last 82.67) - /unlink 23144 (time 1712276732.34 total 364.87 last 79.45) - /unlink 23907 (time 1712276742.34 total 374.87 last 76.29) - /unlink 24732 (time 1712276752.34 total 384.88 last 82.47) - /unlink 25629 (time 1712276762.35 total 394.89 last 89.62) - /unlink 26518 (time 1712276772.36 total 404.89 last 88.82) - /unlink 27561 (time 1712276782.36 total 414.89 last 104.29) - /unlink 28665 (time 1712276792.36 total 424.90 last 110.36) - /unlink 29783 (time 1712276802.36 total 434.90 last 111.79) - /unlink 30000 (time 1712276804.46 total 437.00 last 103.38) - /unlink 30900 (time 1712276814.47 total 447.01 last 89.93) - /unlink 31868 (time 1712276824.48 total 457.01 last 96.75) - /unlink 33210 (time 1712276834.48 total 467.01 last 134.16) - /unlink 34868 (time 1712276844.48 total 477.01 last 165.78) - /unlink 36589 (time 1712276854.49 total 487.02 last 172.00) - /unlink 38418 (time 1712276864.49 total 497.02 last 182.89) - /unlink 40000 (time 1712276872.46 total 504.99 last 198.40) - /unlink 41435 (time 1712276882.46 total 515.00 last 143.47) - /unlink 42754 (time 1712276892.47 total 525.00 last 131.86) - /unlink 44032 (time 1712276902.47 total 535.01 last 127.71) - /unlink 45749 (time 1712276912.47 total 545.01 last 171.67) - /unlink 47284 (time 1712276922.48 total 555.01 last 153.46) - /unlink 48922 (time 1712276932.48 total 565.01 last 163.78) - /unlink 50000 (time 1712276938.06 total 570.59 last 193.25) - /unlink 51868 (time 1712276948.06 total 580.59 last 186.73) - /unlink 53804 (time 1712276958.06 total 590.60 last 193.51) - /unlink 56386 (time 1712276968.07 total 600.60 last 258.15) - /unlink 59040 (time 1712276978.07 total 610.60 last 265.37) - /unlink 60000 (time 1712276981.38 total 613.91 last 290.17) - /unlink 63481 (time 1712276991.38 total 623.91 last 348.06) total: 64768 /unlink in 626.99 seconds: 103.30 ops/second sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete - open/close 2515 (time 1712277021.58 total 10.00 last 251.49) - open/close 4645 (time 1712277031.58 total 20.00 last 212.95) - open/close 6614 (time 1712277041.59 total 30.00 last 196.87) - open/close 8428 (time 1712277051.59 total 40.01 last 181.33) - open/close 10000 (time 1712277060.43 total 48.85 last 177.71) - open/close 11653 (time 1712277070.44 total 58.86 last 165.25) - open/close 13370 (time 1712277080.44 total 68.86 last 171.65) - open/close 14818 (time 1712277090.45 total 78.86 last 144.73) - open/close 16396 (time 1712277100.45 total 88.87 last 157.72) - open/close 17824 (time 1712277110.46 total 98.88 last 142.70) - open/close 19110 (time 1712277120.46 total 108.88 last 128.59) - open/close 20000 (time 1712277126.82 total 115.24 last 139.88) - open/close 21473 (time 1712277136.82 total 125.24 last 147.26) - open/close 23466 (time 1712277146.82 total 135.24 last 199.28) - open/close 24677 (time 1712277156.83 total 145.25 last 121.03) - open/close 25774 (time 1712277166.83 total 155.25 last 109.66) - open/close 26838 (time 1712277176.84 total 165.26 last 106.36) - open/close 27929 (time 1712277186.84 total 175.26 last 109.04) - open/close 28935 (time 1712277196.85 total 185.27 last 100.53) - open/close 29938 (time 1712277206.85 total 195.27 last 100.27) - open/close 30932 (time 1712277216.85 total 205.27 last 99.39) - open/close 31907 (time 1712277226.86 total 215.28 last 97.46) - open/close 32839 (time 1712277236.86 total 225.28 last 93.13) - open/close 33927 (time 1712277246.87 total 235.29 last 108.76) - open/close 34988 (time 1712277256.87 total 245.29 last 106.09) - open/close 35958 (time 1712277266.88 total 255.30 last 96.91) - open/close 36884 (time 1712277276.88 total 265.30 last 92.57) - open/close 37837 (time 1712277286.90 total 275.32 last 95.16) - open/close 38659 (time 1712277296.90 total 285.32 last 82.18) - open/close 39625 (time 1712277306.90 total 295.32 last 96.57) - open/close 40000 (time 1712277311.27 total 299.69 last 85.78) - open/close 40794 (time 1712277321.28 total 309.70 last 79.36) - open/close 41582 (time 1712277331.28 total 319.70 last 78.76) - open/close 42413 (time 1712277341.29 total 329.71 last 83.07) - open/close 43167 (time 1712277351.29 total 339.71 last 75.38) - open/close 43922 (time 1712277361.30 total 349.72 last 75.45) - open/close 44705 (time 1712277371.30 total 359.72 last 78.25) - open/close 45437 (time 1712277381.32 total 369.74 last 73.10) - open/close 46235 (time 1712277391.32 total 379.74 last 79.78) - open/close 46957 (time 1712277401.33 total 389.75 last 72.09) - open/close 47689 (time 1712277411.34 total 399.76 last 73.18) - open/close 48399 (time 1712277421.35 total 409.77 last 70.93) - open/close 49180 (time 1712277431.36 total 419.78 last 77.98) - open/close 49932 (time 1712277441.38 total 429.79 last 75.11) - open/close 50581 (time 1712277451.39 total 439.81 last 64.80) - open/close 51214 (time 1712277461.40 total 449.82 last 63.22) - open/close 51840 (time 1712277471.40 total 459.82 last 62.59) - open/close 52462 (time 1712277481.41 total 469.83 last 62.19) - open/close 53049 (time 1712277491.41 total 479.83 last 58.66) - open/close 53710 (time 1712277501.42 total 489.84 last 66.04) - open/close 54341 (time 1712277511.43 total 499.85 last 63.07) - open/close 54896 (time 1712277521.43 total 509.85 last 55.48) - open/close 55443 (time 1712277531.45 total 519.87 last 54.60) - open/close 56008 (time 1712277541.46 total 529.88 last 56.44) - open/close 56567 (time 1712277551.47 total 539.89 last 55.82) - open/close 57138 (time 1712277561.49 total 549.91 last 57.02) - open/close 57688 (time 1712277571.49 total 559.91 last 54.97) - open/close 58405 (time 1712277581.50 total 569.91 last 71.68) - open/close 59058 (time 1712277591.50 total 579.92 last 65.26) - open/close 59595 (time 1712277601.52 total 589.94 last 53.62) - open/close 60000 (time 1712277608.23 total 596.65 last 60.29) - open/close 60648 (time 1712277618.24 total 606.66 last 64.78) - open/close 61225 (time 1712277628.25 total 616.67 last 57.62) - open/close 61768 (time 1712277638.25 total 626.67 last 54.30) - open/close 62420 (time 1712277648.26 total 636.68 last 65.16) - open/close 63120 (time 1712277658.27 total 646.69 last 69.88) - open/close 63771 (time 1712277668.28 total 656.70 last 65.03) - open/close 64477 (time 1712277678.30 total 666.72 last 70.50) total: 64768 open/close in 672.40 seconds: 96.32 ops/second - /unlink 647 (time 1712277694.80 total 10.01 last 64.61) - /unlink 1260 (time 1712277704.81 total 20.02 last 61.28) - /unlink 1913 (time 1712277714.82 total 30.04 last 65.18) - /unlink 2593 (time 1712277724.83 total 40.04 last 67.96) - /unlink 3259 (time 1712277734.85 total 50.06 last 66.50) - /unlink 4013 (time 1712277744.85 total 60.06 last 75.37) - /unlink 4723 (time 1712277754.87 total 70.08 last 70.89) - /unlink 5560 (time 1712277764.87 total 80.08 last 83.66) - /unlink 6301 (time 1712277774.88 total 90.09 last 74.03) - /unlink 7015 (time 1712277784.89 total 100.10 last 71.35) - /unlink 7895 (time 1712277794.89 total 110.11 last 87.94) - /unlink 8616 (time 1712277804.91 total 120.12 last 72.00) - /unlink 9415 (time 1712277814.91 total 130.12 last 79.87) - /unlink 10000 (time 1712277823.51 total 138.72 last 68.04) - /unlink 10829 (time 1712277833.52 total 148.73 last 82.80) - /unlink 11763 (time 1712277843.53 total 158.74 last 93.34) - /unlink 12660 (time 1712277853.54 total 168.75 last 89.64) - /unlink 13499 (time 1712277863.54 total 178.75 last 83.88) - /unlink 14270 (time 1712277873.54 total 188.75 last 77.10) - /unlink 15090 (time 1712277883.55 total 198.77 last 81.88) - /unlink 15980 (time 1712277893.56 total 208.77 last 88.96) - /unlink 16857 (time 1712277903.57 total 218.78 last 87.63) - /unlink 17763 (time 1712277913.57 total 228.78 last 90.55) - /unlink 18665 (time 1712277923.58 total 238.79 last 90.16) - /unlink 19576 (time 1712277933.58 total 248.79 last 91.09) - /unlink 20000 (time 1712277937.78 total 252.99 last 100.85) - /unlink 20999 (time 1712277947.78 total 262.99 last 99.89) - /unlink 21981 (time 1712277957.79 total 273.00 last 98.15) - /unlink 22989 (time 1712277967.80 total 283.01 last 100.69) - /unlink 24003 (time 1712277977.81 total 293.02 last 101.30) - /unlink 25037 (time 1712277987.82 total 303.03 last 103.31) - /unlink 26105 (time 1712277997.83 total 313.04 last 106.69) - /unlink 27262 (time 1712278007.83 total 323.04 last 115.68) - /unlink 28408 (time 1712278017.84 total 333.05 last 114.49) - /unlink 29521 (time 1712278027.84 total 343.05 last 111.27) - /unlink 30000 (time 1712278032.20 total 347.42 last 109.83) - /unlink 31137 (time 1712278042.21 total 357.42 last 113.59) - /unlink 32197 (time 1712278052.22 total 367.43 last 105.89) - /unlink 33468 (time 1712278062.23 total 377.44 last 127.01) - /unlink 34718 (time 1712278072.24 total 387.45 last 124.94) - /unlink 35931 (time 1712278082.24 total 397.45 last 121.22) - /unlink 37112 (time 1712278092.24 total 407.46 last 118.08) - /unlink 38269 (time 1712278102.25 total 417.46 last 115.65) - /unlink 39473 (time 1712278112.25 total 427.46 last 120.36) - /unlink 40000 (time 1712278116.40 total 431.62 last 126.93) - /unlink 41285 (time 1712278126.41 total 441.62 last 128.39) - /unlink 42597 (time 1712278136.41 total 451.62 last 131.20) - /unlink 43973 (time 1712278146.41 total 461.63 last 137.57) - /unlink 45429 (time 1712278156.42 total 471.63 last 145.54) - /unlink 46935 (time 1712278166.42 total 481.63 last 150.54) - /unlink 48474 (time 1712278176.43 total 491.64 last 153.86) - /unlink 50000 (time 1712278185.75 total 500.96 last 163.60) - /unlink 51796 (time 1712278195.75 total 510.97 last 179.57) - /unlink 53914 (time 1712278205.76 total 520.97 last 211.76) - /unlink 56398 (time 1712278215.76 total 530.97 last 248.39) - /unlink 58903 (time 1712278225.76 total 540.97 last 250.39) - /unlink 60000 (time 1712278229.53 total 544.74 last 291.48) - /unlink 63744 (time 1712278239.53 total 554.74 last 374.39) total: 64768 /unlink in 556.88 seconds: 116.30 ops/second sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete - open/close 2593 (time 1712278273.34 total 10.01 last 259.14) - open/close 4760 (time 1712278283.34 total 20.01 last 216.61) - open/close 6721 (time 1712278293.34 total 30.01 last 196.05) - open/close 8535 (time 1712278303.35 total 40.02 last 181.28) - open/close 10000 (time 1712278311.63 total 48.30 last 176.94) - open/close 11691 (time 1712278321.63 total 58.30 last 169.05) - open/close 13317 (time 1712278331.64 total 68.31 last 162.52) - open/close 14820 (time 1712278341.64 total 78.31 last 150.23) - open/close 16236 (time 1712278351.65 total 88.32 last 141.52) - open/close 17606 (time 1712278361.65 total 98.32 last 136.98) - open/close 18970 (time 1712278371.65 total 108.32 last 136.37) - open/close 20000 (time 1712278379.51 total 116.18 last 131.06) - open/close 21337 (time 1712278389.51 total 126.18 last 133.67) - open/close 22573 (time 1712278399.52 total 136.19 last 123.52) - open/close 23847 (time 1712278409.52 total 146.19 last 127.39) - open/close 25942 (time 1712278419.52 total 156.19 last 209.40) - open/close 27915 (time 1712278429.53 total 166.20 last 197.22) - open/close 29778 (time 1712278439.53 total 176.20 last 186.21) - open/close 31014 (time 1712278449.53 total 186.20 last 123.57) - open/close 32354 (time 1712278459.54 total 196.21 last 133.96) - open/close 33668 (time 1712278469.55 total 206.22 last 131.28) - open/close 34721 (time 1712278479.55 total 216.22 last 105.22) - open/close 35853 (time 1712278489.56 total 226.23 last 113.14) - open/close 37042 (time 1712278499.56 total 236.23 last 118.89) - open/close 37950 (time 1712278509.56 total 246.23 last 90.78) - open/close 38988 (time 1712278519.57 total 256.24 last 103.73) - open/close 39811 (time 1712278529.57 total 266.24 last 82.30) - open/close 40000 (time 1712278531.74 total 268.41 last 87.01) - open/close 40876 (time 1712278541.74 total 278.41 last 87.58) - open/close 41681 (time 1712278551.75 total 288.42 last 80.46) - open/close 42585 (time 1712278561.75 total 298.43 last 90.35) - open/close 43401 (time 1712278571.77 total 308.44 last 81.51) - open/close 44162 (time 1712278581.78 total 318.45 last 76.02) - open/close 44890 (time 1712278591.78 total 328.45 last 72.77) - open/close 45725 (time 1712278601.79 total 338.46 last 83.47) - open/close 46541 (time 1712278611.79 total 348.46 last 81.56) - open/close 47246 (time 1712278621.81 total 358.48 last 70.39) - open/close 47982 (time 1712278631.81 total 368.48 last 73.56) - open/close 48667 (time 1712278641.81 total 378.48 last 68.48) - open/close 49366 (time 1712278651.81 total 388.49 last 69.90) - open/close 50000 (time 1712278660.67 total 397.34 last 71.58) - open/close 50683 (time 1712278670.69 total 407.36 last 68.21) - open/close 51380 (time 1712278680.70 total 417.37 last 69.57) - open/close 52026 (time 1712278690.71 total 427.38 last 64.56) - open/close 52664 (time 1712278700.72 total 437.39 last 63.71) - open/close 53296 (time 1712278710.74 total 447.41 last 63.13) - open/close 53950 (time 1712278720.74 total 457.41 last 65.37) - open/close 54538 (time 1712278730.74 total 467.41 last 58.79) - open/close 55135 (time 1712278740.75 total 477.42 last 59.62) - open/close 55842 (time 1712278750.76 total 487.43 last 70.69) - open/close 56554 (time 1712278760.77 total 497.44 last 71.10) - open/close 57242 (time 1712278770.78 total 507.45 last 68.74) - open/close 57993 (time 1712278780.79 total 517.46 last 75.00) - open/close 58628 (time 1712278790.80 total 527.47 last 63.44) - open/close 59252 (time 1712278800.81 total 537.48 last 62.37) - open/close 59879 (time 1712278810.81 total 547.48 last 62.66) - open/close 60000 (time 1712278812.85 total 549.52 last 59.35) - open/close 60594 (time 1712278822.86 total 559.53 last 59.38) - open/close 61223 (time 1712278832.86 total 569.54 last 62.84) - open/close 61791 (time 1712278842.87 total 579.54 last 56.79) - open/close 62389 (time 1712278852.87 total 589.54 last 59.76) - open/close 63011 (time 1712278862.88 total 599.56 last 62.13) - open/close 63642 (time 1712278872.89 total 609.56 last 63.10) - open/close 64333 (time 1712278882.89 total 619.56 last 69.07) total: 64768 open/close in 627.43 seconds: 103.23 ops/second - /unlink 666 (time 1712278902.03 total 10.01 last 66.56) - /unlink 1289 (time 1712278912.04 total 20.02 last 62.24) - /unlink 1928 (time 1712278922.04 total 30.02 last 63.89) - /unlink 2545 (time 1712278932.05 total 40.03 last 61.62) - /unlink 3212 (time 1712278942.06 total 50.04 last 66.64) - /unlink 3883 (time 1712278952.07 total 60.05 last 67.04) - /unlink 4534 (time 1712278962.08 total 70.06 last 65.05) - /unlink 5278 (time 1712278972.09 total 80.07 last 74.33) - /unlink 5641 (time 1712278982.11 total 90.09 last 36.21) - /unlink 6146 (time 1712278992.11 total 100.09 last 50.48) - /unlink 6754 (time 1712279002.12 total 110.10 last 60.76) - /unlink 7288 (time 1712279012.13 total 120.11 last 53.32) - /unlink 7774 (time 1712279022.14 total 130.12 last 48.58) - /unlink 8355 (time 1712279032.14 total 140.12 last 58.10) - /unlink 9082 (time 1712279042.15 total 150.13 last 72.63) - /unlink 9695 (time 1712279052.15 total 160.14 last 61.27) - /unlink 10000 (time 1712279059.14 total 167.12 last 43.65) - /unlink 10449 (time 1712279069.16 total 177.14 last 44.81) - /unlink 10935 (time 1712279079.17 total 187.15 last 48.57) - /unlink 11413 (time 1712279089.17 total 197.15 last 47.78) - /unlink 11872 (time 1712279099.19 total 207.17 last 45.83) - /unlink 12442 (time 1712279109.19 total 217.17 last 56.98) - /unlink 13095 (time 1712279119.19 total 227.17 last 65.30) - /unlink 13756 (time 1712279129.21 total 237.19 last 66.00) - /unlink 14447 (time 1712279139.21 total 247.19 last 69.08) - /unlink 15196 (time 1712279149.22 total 257.20 last 74.83) - /unlink 15942 (time 1712279159.22 total 267.20 last 74.59) - /unlink 16718 (time 1712279169.23 total 277.21 last 77.52) - /unlink 17476 (time 1712279179.23 total 287.21 last 75.77) - /unlink 18238 (time 1712279189.24 total 297.22 last 76.17) - /unlink 19061 (time 1712279199.24 total 307.22 last 82.29) - /unlink 19881 (time 1712279209.25 total 317.23 last 81.92) - /unlink 20723 (time 1712279219.26 total 327.24 last 84.11) - /unlink 21575 (time 1712279229.26 total 337.24 last 85.19) - /unlink 22501 (time 1712279239.27 total 347.25 last 92.54) - /unlink 23387 (time 1712279249.27 total 357.25 last 88.54) - /unlink 24300 (time 1712279259.28 total 367.27 last 91.20) - /unlink 25202 (time 1712279269.29 total 377.27 last 90.16) - /unlink 25908 (time 1712279279.29 total 387.28 last 70.56) - /unlink 26349 (time 1712279289.30 total 397.28 last 44.06) - /unlink 26944 (time 1712279299.32 total 407.30 last 59.40) - /unlink 27718 (time 1712279309.32 total 417.30 last 77.39) - /unlink 28373 (time 1712279319.33 total 427.31 last 65.43) - /unlink 28958 (time 1712279329.33 total 437.32 last 58.49) - /unlink 29727 (time 1712279339.34 total 447.32 last 76.88) - /unlink 30000 (time 1712279341.98 total 449.96 last 103.14) - /unlink 30943 (time 1712279351.99 total 459.97 last 94.25) - /unlink 31760 (time 1712279362.00 total 469.98 last 81.63) - /unlink 32531 (time 1712279372.00 total 479.98 last 77.06) - /unlink 33497 (time 1712279382.00 total 489.99 last 96.58) - /unlink 34414 (time 1712279392.01 total 499.99 last 91.63) - /unlink 35555 (time 1712279402.01 total 510.00 last 114.07) - /unlink 36601 (time 1712279412.02 total 520.00 last 104.59) - /unlink 37559 (time 1712279422.02 total 530.00 last 95.75) - /unlink 38592 (time 1712279432.02 total 540.01 last 103.25) - /unlink 39806 (time 1712279442.03 total 550.01 last 121.38) - /unlink 41276 (time 1712279452.03 total 560.01 last 146.89) - /unlink 42836 (time 1712279462.04 total 570.02 last 155.93) - /unlink 44501 (time 1712279472.04 total 580.02 last 166.40) - /unlink 46270 (time 1712279482.04 total 590.03 last 176.88) - /unlink 48232 (time 1712279492.05 total 600.03 last 196.16) - /unlink 50000 (time 1712279500.45 total 608.43 last 210.50) - /unlink 52217 (time 1712279510.45 total 618.43 last 221.68) - /unlink 56051 (time 1712279520.45 total 628.43 last 383.38) - /unlink 60000 (time 1712279528.76 total 636.74 last 474.92) total: 64768 /unlink in 643.84 seconds: 100.60 ops/second sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete - open/close 4637 (time 1712279565.28 total 10.00 last 463.65) - open/close 8149 (time 1712279575.29 total 20.00 last 351.17) - open/close 10000 (time 1712279581.31 total 26.03 last 307.14) - open/close 12686 (time 1712279591.32 total 36.03 last 268.48) - open/close 15090 (time 1712279601.32 total 46.03 last 240.37) - open/close 16819 (time 1712279611.32 total 56.04 last 172.87) - open/close 18578 (time 1712279621.32 total 66.04 last 175.80) - open/close 20000 (time 1712279629.10 total 73.82 last 182.82) - open/close 21764 (time 1712279639.10 total 83.82 last 176.37) - open/close 23407 (time 1712279649.11 total 93.83 last 164.18) - open/close 24878 (time 1712279659.12 total 103.84 last 146.97) - open/close 26257 (time 1712279669.12 total 113.84 last 137.85) - open/close 27653 (time 1712279679.13 total 123.85 last 139.51) - open/close 29002 (time 1712279689.13 total 133.85 last 134.90) - open/close 30000 (time 1712279696.68 total 141.39 last 132.28) - open/close 31364 (time 1712279706.68 total 151.39 last 136.38) - open/close 32546 (time 1712279716.68 total 161.40 last 118.11) - open/close 33754 (time 1712279726.69 total 171.40 last 120.79) - open/close 34809 (time 1712279736.69 total 181.41 last 105.41) - open/close 35892 (time 1712279746.70 total 191.41 last 108.27) - open/close 36988 (time 1712279756.70 total 201.41 last 109.58) - open/close 37969 (time 1712279766.70 total 211.42 last 98.06) - open/close 38900 (time 1712279776.71 total 221.43 last 93.04) - open/close 39815 (time 1712279786.72 total 231.43 last 91.41) - open/close 40840 (time 1712279796.72 total 241.44 last 102.49) - open/close 41814 (time 1712279806.72 total 251.44 last 97.36) - open/close 42801 (time 1712279816.73 total 261.45 last 98.65) - open/close 43672 (time 1712279826.74 total 271.45 last 87.04) - open/close 44503 (time 1712279836.75 total 281.46 last 83.01) - open/close 45265 (time 1712279846.75 total 291.47 last 76.17) - open/close 46013 (time 1712279856.75 total 301.47 last 74.77) - open/close 46747 (time 1712279866.76 total 311.48 last 73.33) - open/close 47502 (time 1712279876.78 total 321.49 last 75.40) - open/close 48250 (time 1712279886.79 total 331.51 last 74.71) - open/close 48971 (time 1712279896.80 total 341.51 last 72.03) - open/close 49593 (time 1712279906.80 total 351.52 last 62.15) - open/close 50000 (time 1712279911.92 total 356.63 last 79.62) - open/close 50780 (time 1712279921.93 total 366.64 last 77.93) - open/close 51435 (time 1712279931.93 total 376.65 last 65.45) - open/close 52121 (time 1712279941.95 total 386.67 last 68.50) - open/close 52797 (time 1712279951.96 total 396.68 last 67.54) - open/close 53428 (time 1712279961.97 total 406.68 last 63.06) - open/close 54076 (time 1712279971.97 total 416.68 last 64.80) - open/close 54762 (time 1712279981.98 total 426.69 last 68.52) - open/close 55416 (time 1712279991.99 total 436.71 last 65.29) - open/close 56082 (time 1712280002.00 total 446.71 last 66.58) - open/close 56652 (time 1712280012.00 total 456.72 last 56.98) - open/close 57227 (time 1712280022.01 total 466.72 last 57.47) - open/close 57860 (time 1712280032.02 total 476.74 last 63.23) - open/close 58388 (time 1712280042.03 total 486.75 last 52.72) - open/close 58900 (time 1712280052.04 total 496.76 last 51.17) - open/close 59518 (time 1712280062.05 total 506.76 last 61.76) - open/close 60000 (time 1712280071.15 total 515.87 last 52.92) - open/close 60499 (time 1712280081.18 total 525.90 last 49.78) - open/close 61067 (time 1712280091.19 total 535.90 last 56.76) - open/close 61653 (time 1712280101.19 total 545.90 last 58.60) - open/close 62255 (time 1712280111.20 total 555.92 last 60.10) - open/close 62780 (time 1712280121.21 total 565.93 last 52.46) - open/close 63332 (time 1712280131.22 total 575.93 last 55.16) - open/close 63902 (time 1712280141.22 total 585.94 last 56.98) - open/close 64464 (time 1712280151.23 total 595.95 last 56.16) total: 64768 open/close in 601.35 seconds: 107.70 ops/second