-----============= acceptance-small: conf-sanity ============----- Fri Apr 5 18:46:33 EDT 2024 excepting tests: 32newtarball 110 Stopping clients: oleg358-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg358-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg358-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg358-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg358-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg358-server oleg358-server: oleg358-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg358-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug -1 all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg358-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg358-server: oleg358-server.virtnet: executing set_default_debug -1 all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg358-server: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg358-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg358-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg358-server stop mds service on oleg358-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg358-server debug_raw_pointers=Y debug_raw_pointers=Y == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 18:47:47 (1712357267) Stopping clients: oleg358-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg358-client.virtnet /mnt/lustre2 (opts:-f) oleg358-server: oleg358-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg358-server' oleg358-server: oleg358-server.virtnet: executing load_modules_local oleg358-server: Loading modules from /home/green/git/lustre-release/lustre oleg358-server: detected 4 online CPUs by sysfs oleg358-server: Force libcfs to create 2 CPU partitions oleg358-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg358-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg358-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug -1 all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg358-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg358-server: oleg358-server.virtnet: executing set_default_debug -1 all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg358-client.virtnet: -o user_xattr,flock oleg358-server@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 16134 326 15808 3% /mnt/lustre[OST:0] filesystem_summary: 16137 329 15808 3% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 689 (time 1712357320.98 total 15.85 last 43.47) - open/close 2013 (time 1712357330.99 total 25.85 last 132.32) - open/close 3426 (time 1712357340.99 total 35.86 last 141.23) - open/close 4776 (time 1712357350.99 total 45.86 last 134.97) - open/close 5850 (time 1712357361.00 total 55.87 last 107.33) - open/close 6835 (time 1712357371.00 total 65.87 last 98.47) - open/close 7904 (time 1712357381.01 total 75.88 last 106.82) - open/close 9179 (time 1712357391.01 total 85.88 last 127.48) - open/close 10000 (time 1712357396.58 total 91.45 last 147.35) - open/close 11570 (time 1712357406.59 total 101.46 last 156.93) - open/close 13012 (time 1712357416.59 total 111.46 last 144.20) - open/close 14309 (time 1712357426.59 total 121.46 last 129.62) - open/close 15292 (time 1712357436.60 total 131.46 last 98.29) - open/close 16563 (time 1712357446.60 total 141.47 last 127.02) - open/close 17811 (time 1712357456.61 total 151.48 last 124.74) - open/close 18939 (time 1712357466.61 total 161.48 last 112.78) - open/close 20000 (time 1712357475.47 total 170.34 last 119.76) - open/close 21289 (time 1712357485.47 total 180.34 last 128.84) - open/close 22422 (time 1712357495.48 total 190.35 last 113.21) - open/close 23443 (time 1712357505.48 total 200.35 last 102.07) - open/close 24418 (time 1712357515.49 total 210.36 last 97.44) - open/close 25390 (time 1712357525.49 total 220.36 last 97.19) - open/close 26458 (time 1712357535.50 total 230.37 last 106.70) - open/close 27619 (time 1712357545.50 total 240.37 last 116.08) - open/close 28646 (time 1712357555.51 total 250.37 last 102.65) - open/close 29473 (time 1712357565.51 total 260.38 last 82.64) - open/close 30000 (time 1712357571.90 total 266.77 last 82.54) - open/close 30906 (time 1712357581.90 total 276.77 last 90.58) - open/close 31850 (time 1712357591.90 total 286.77 last 94.37) - open/close 32795 (time 1712357601.91 total 296.78 last 94.41) - open/close 33650 (time 1712357611.92 total 306.79 last 85.45) - open/close 34462 (time 1712357621.93 total 316.79 last 81.14) - open/close 35330 (time 1712357631.93 total 326.80 last 86.76) - open/close 36178 (time 1712357641.94 total 336.81 last 84.72) - open/close 37019 (time 1712357651.95 total 346.82 last 84.02) - open/close 37817 (time 1712357661.95 total 356.82 last 79.79) - open/close 38640 (time 1712357671.96 total 366.82 last 82.26) - open/close 39410 (time 1712357681.96 total 376.83 last 76.97) - open/close 40000 (time 1712357689.33 total 384.20 last 80.03) - open/close 40757 (time 1712357699.34 total 394.21 last 75.61) - open/close 41505 (time 1712357709.35 total 404.21 last 74.78) - open/close 42269 (time 1712357719.35 total 414.21 last 76.40) - open/close 42977 (time 1712357729.35 total 424.22 last 70.74) - open/close 43647 (time 1712357739.36 total 434.23 last 66.93) - open/close 44342 (time 1712357749.36 total 444.23 last 69.50) - open/close 45018 (time 1712357759.37 total 454.24 last 67.57) - open/close 45723 (time 1712357769.37 total 464.24 last 70.47) - open/close 46451 (time 1712357779.39 total 474.26 last 72.69) - open/close 47130 (time 1712357789.40 total 484.27 last 67.84) - open/close 47818 (time 1712357799.40 total 494.27 last 68.77) - open/close 48476 (time 1712357809.41 total 504.28 last 65.73) - open/close 49138 (time 1712357819.43 total 514.30 last 66.10) - open/close 49767 (time 1712357829.44 total 524.31 last 62.83) - open/close 50000 (time 1712357833.26 total 528.13 last 61.00) - open/close 50634 (time 1712357843.27 total 538.14 last 63.34) - open/close 51279 (time 1712357853.27 total 548.14 last 64.50) - open/close 51925 (time 1712357863.27 total 558.14 last 64.59) - open/close 52502 (time 1712357873.28 total 568.15 last 57.62) - open/close 53004 (time 1712357883.30 total 578.17 last 50.11) - open/close 53462 (time 1712357893.32 total 588.19 last 45.71) - open/close 54005 (time 1712357903.32 total 598.19 last 54.29) - open/close 54551 (time 1712357913.34 total 608.21 last 54.52) - open/close 55062 (time 1712357923.35 total 618.22 last 51.04) - open/close 55548 (time 1712357933.35 total 628.22 last 48.58) - open/close 56056 (time 1712357943.35 total 638.22 last 50.80) - open/close 56571 (time 1712357953.37 total 648.23 last 51.44) - open/close 57089 (time 1712357963.37 total 658.24 last 51.79) - open/close 57573 (time 1712357973.37 total 668.24 last 48.38) - open/close 58026 (time 1712357983.37 total 678.24 last 45.30) - open/close 58433 (time 1712357993.39 total 688.25 last 40.64) - open/close 58861 (time 1712358003.39 total 698.25 last 42.80) - open/close 59291 (time 1712358013.41 total 708.28 last 42.91) - open/close 59718 (time 1712358023.42 total 718.29 last 42.64) - open/close 60000 (time 1712358029.96 total 724.83 last 43.10) - open/close 60433 (time 1712358039.97 total 734.84 last 43.27) - open/close 60852 (time 1712358049.98 total 744.85 last 41.84) - open/close 61280 (time 1712358060.00 total 754.87 last 42.73) - open/close 61689 (time 1712358070.01 total 764.88 last 40.85) - open/close 62085 (time 1712358080.03 total 774.90 last 39.54) - open/close 62491 (time 1712358090.03 total 784.90 last 40.57) - open/close 62894 (time 1712358100.04 total 794.91 last 40.27) - open/close 63281 (time 1712358110.05 total 804.92 last 38.65) - open/close 63672 (time 1712358120.06 total 814.93 last 39.08) - open/close 64059 (time 1712358130.07 total 824.94 last 38.66) - open/close 64493 (time 1712358140.08 total 834.94 last 43.38) total: 64768 open/close in 841.73 seconds: 76.95 ops/second - /unlink 319 (time 1712358157.70 total 10.02 last 31.83) - /unlink 666 (time 1712358167.72 total 20.04 last 34.64) - /unlink 985 (time 1712358177.74 total 30.06 last 31.83) - /unlink 1307 (time 1712358187.76 total 40.08 last 32.15) - /unlink 1635 (time 1712358197.78 total 50.10 last 32.71) - /unlink 1945 (time 1712358207.81 total 60.13 last 30.93) - /unlink 2299 (time 1712358217.83 total 70.15 last 35.33) - /unlink 2660 (time 1712358227.83 total 80.15 last 36.09) - /unlink 3017 (time 1712358237.86 total 90.18 last 35.58) - /unlink 3359 (time 1712358247.87 total 100.19 last 34.16) - /unlink 3700 (time 1712358257.88 total 110.20 last 34.08) - /unlink 4059 (time 1712358267.89 total 120.21 last 35.86) - /unlink 4461 (time 1712358277.91 total 130.23 last 40.13) - /unlink 4880 (time 1712358287.91 total 140.23 last 41.89) - /unlink 5351 (time 1712358297.92 total 150.24 last 47.04) - /unlink 5877 (time 1712358307.93 total 160.25 last 52.54) - /unlink 6421 (time 1712358317.95 total 170.27 last 54.32) - /unlink 6897 (time 1712358327.97 total 180.29 last 47.51) - /unlink 7378 (time 1712358337.97 total 190.29 last 48.07) - /unlink 7941 (time 1712358347.98 total 200.30 last 56.28) - /unlink 8577 (time 1712358357.98 total 210.30 last 63.57) - /unlink 9134 (time 1712358367.99 total 220.31 last 55.65) - /unlink 9590 (time 1712358377.99 total 230.32 last 45.58) - /unlink 10000 (time 1712358386.57 total 238.89 last 47.81) - /unlink 10494 (time 1712358396.57 total 248.89 last 49.38) - /unlink 11065 (time 1712358406.58 total 258.90 last 57.05) - /unlink 11528 (time 1712358416.61 total 268.93 last 46.19) - /unlink 11903 (time 1712358426.62 total 278.94 last 37.47) - /unlink 12367 (time 1712358436.63 total 288.95 last 46.32) - /unlink 12864 (time 1712358446.64 total 298.96 last 49.64) - /unlink 13395 (time 1712358456.66 total 308.98 last 53.00) - /unlink 13753 (time 1712358466.68 total 319.00 last 35.74) - /unlink 14230 (time 1712358476.70 total 329.03 last 47.59) - /unlink 14742 (time 1712358486.71 total 339.03 last 51.20) - /unlink 15248 (time 1712358496.72 total 349.04 last 50.54) - /unlink 15814 (time 1712358506.72 total 359.05 last 56.56) - /unlink 16392 (time 1712358516.73 total 369.05 last 57.79) - /unlink 17009 (time 1712358526.74 total 379.06 last 61.64) - /unlink 17568 (time 1712358536.74 total 389.06 last 55.85) - /unlink 18144 (time 1712358546.75 total 399.07 last 57.58) - /unlink 18824 (time 1712358556.75 total 409.07 last 67.96) - /unlink 19490 (time 1712358566.76 total 419.08 last 66.57) - /unlink 20000 (time 1712358573.46 total 425.78 last 76.15) - /unlink 20705 (time 1712358583.47 total 435.79 last 70.42) - /unlink 21337 (time 1712358593.47 total 445.79 last 63.17) - /unlink 22047 (time 1712358603.48 total 455.80 last 70.98) - /unlink 22784 (time 1712358613.49 total 465.81 last 73.62) - /unlink 23584 (time 1712358623.49 total 475.81 last 79.94) - /unlink 24417 (time 1712358633.50 total 485.82 last 83.24) - /unlink 25212 (time 1712358643.51 total 495.83 last 79.46) - /unlink 25994 (time 1712358653.51 total 505.83 last 78.18) - /unlink 26963 (time 1712358663.51 total 515.83 last 96.86) - /unlink 27956 (time 1712358673.52 total 525.84 last 99.23) - /unlink 29017 (time 1712358683.52 total 535.84 last 106.07) - /unlink 30000 (time 1712358692.64 total 544.96 last 107.78) - /unlink 31104 (time 1712358702.65 total 554.97 last 110.37) - /unlink 32216 (time 1712358712.65 total 564.97 last 111.11) - /unlink 33693 (time 1712358722.65 total 574.97 last 147.69) - /unlink 35202 (time 1712358732.66 total 584.98 last 150.86) - /unlink 36600 (time 1712358742.66 total 594.98 last 139.74) - /unlink 37980 (time 1712358752.67 total 604.99 last 137.92) - /unlink 39573 (time 1712358762.67 total 614.99 last 159.21) - /unlink 40000 (time 1712358765.27 total 617.59 last 164.19) - /unlink 41640 (time 1712358775.28 total 627.60 last 163.91) - /unlink 43321 (time 1712358785.28 total 637.61 last 168.01) - /unlink 45218 (time 1712358795.29 total 647.61 last 189.63) - /unlink 47001 (time 1712358805.29 total 657.61 last 178.30) - /unlink 49270 (time 1712358815.29 total 667.61 last 226.88) - /unlink 50000 (time 1712358818.02 total 670.34 last 267.45) - /unlink 52853 (time 1712358828.02 total 680.34 last 285.22) - /unlink 56260 (time 1712358838.02 total 690.34 last 340.65) - /unlink 60000 (time 1712358846.19 total 698.51 last 457.77) total: 64768 /unlink in 705.68 seconds: 91.78 ops/second sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete - open/close 4727 (time 1712358879.19 total 10.00 last 472.69) - open/close 8339 (time 1712358889.19 total 20.00 last 361.11) - open/close 10000 (time 1712358894.54 total 25.36 last 310.10) - open/close 12983 (time 1712358904.55 total 35.36 last 298.21) - open/close 15643 (time 1712358914.55 total 45.36 last 266.00) - open/close 18159 (time 1712358924.55 total 55.37 last 251.49) - open/close 20000 (time 1712358932.90 total 63.71 last 220.52) - open/close 22160 (time 1712358942.90 total 73.72 last 215.99) - open/close 24091 (time 1712358952.91 total 83.72 last 193.02) - open/close 25916 (time 1712358962.91 total 93.72 last 182.43) - open/close 27758 (time 1712358972.91 total 103.73 last 184.15) - open/close 29334 (time 1712358982.91 total 113.73 last 157.57) - open/close 30000 (time 1712358987.22 total 118.03 last 154.73) - open/close 31404 (time 1712358997.22 total 128.03 last 140.37) - open/close 32751 (time 1712359007.22 total 138.03 last 134.70) - open/close 34106 (time 1712359017.23 total 148.04 last 135.41) - open/close 35497 (time 1712359027.23 total 158.05 last 139.02) - open/close 36823 (time 1712359037.23 total 168.05 last 132.58) - open/close 37951 (time 1712359047.23 total 178.05 last 112.80) - open/close 39056 (time 1712359057.24 total 188.06 last 110.39) - open/close 39990 (time 1712359067.25 total 198.07 last 93.32) - open/close 40955 (time 1712359077.26 total 208.07 last 96.43) - open/close 41986 (time 1712359087.27 total 218.08 last 103.04) - open/close 42981 (time 1712359097.27 total 228.08 last 99.46) - open/close 43993 (time 1712359107.28 total 238.09 last 101.09) - open/close 44921 (time 1712359117.29 total 248.11 last 92.69) - open/close 45844 (time 1712359127.30 total 258.11 last 92.26) - open/close 46699 (time 1712359137.31 total 268.12 last 85.42) - open/close 47530 (time 1712359147.31 total 278.13 last 83.04) - open/close 48368 (time 1712359157.32 total 288.13 last 83.76) - open/close 49020 (time 1712359167.33 total 298.14 last 65.15) - open/close 49764 (time 1712359177.34 total 308.15 last 74.33) - open/close 50000 (time 1712359180.63 total 311.44 last 71.64) - open/close 50718 (time 1712359190.64 total 321.45 last 71.75) - open/close 51494 (time 1712359200.64 total 331.46 last 77.55) - open/close 52200 (time 1712359210.66 total 341.47 last 70.49) - open/close 52877 (time 1712359220.67 total 351.48 last 67.65) - open/close 53588 (time 1712359230.67 total 361.48 last 71.09) - open/close 54306 (time 1712359240.68 total 371.49 last 71.76) - open/close 54999 (time 1712359250.68 total 381.50 last 69.25) - open/close 55735 (time 1712359260.69 total 391.51 last 73.53) - open/close 56444 (time 1712359270.70 total 401.52 last 70.82) - open/close 57050 (time 1712359280.71 total 411.52 last 60.58) - open/close 57668 (time 1712359290.72 total 421.54 last 61.70) - open/close 58277 (time 1712359300.72 total 431.54 last 60.89) - open/close 58921 (time 1712359310.74 total 441.56 last 64.28) - open/close 59555 (time 1712359320.75 total 451.56 last 63.37) - open/close 60000 (time 1712359328.06 total 458.87 last 60.89) - open/close 60581 (time 1712359338.06 total 468.87 last 58.09) - open/close 61161 (time 1712359348.06 total 478.87 last 58.00) - open/close 61734 (time 1712359358.06 total 488.87 last 57.29) - open/close 62251 (time 1712359368.08 total 498.89 last 51.61) - open/close 62744 (time 1712359378.09 total 508.91 last 49.23) - open/close 63214 (time 1712359388.11 total 518.93 last 46.90) - open/close 63670 (time 1712359398.11 total 528.93 last 45.59) - open/close 64140 (time 1712359408.13 total 538.95 last 46.92) - open/close 64638 (time 1712359418.15 total 548.96 last 49.72) total: 64768 open/close in 551.61 seconds: 117.42 ops/second - /unlink 505 (time 1712359431.62 total 10.01 last 50.44) - /unlink 1026 (time 1712359441.64 total 20.03 last 52.01) - /unlink 1510 (time 1712359451.65 total 30.05 last 48.32) - /unlink 1940 (time 1712359461.66 total 40.06 last 42.96) - /unlink 2385 (time 1712359471.67 total 50.06 last 44.47) - /unlink 2796 (time 1712359481.68 total 60.08 last 41.04) - /unlink 3263 (time 1712359491.70 total 70.09 last 46.64) - /unlink 3759 (time 1712359501.70 total 80.10 last 49.56) - /unlink 4244 (time 1712359511.71 total 90.10 last 48.50) - /unlink 4740 (time 1712359521.71 total 100.10 last 49.60) - /unlink 5235 (time 1712359531.71 total 110.10 last 49.50) - /unlink 5682 (time 1712359541.73 total 120.12 last 44.61) - /unlink 6133 (time 1712359551.75 total 130.14 last 45.01) - /unlink 6620 (time 1712359561.76 total 140.16 last 48.62) - /unlink 7112 (time 1712359571.77 total 150.16 last 49.17) - /unlink 7601 (time 1712359581.77 total 160.17 last 48.88) - /unlink 8146 (time 1712359591.79 total 170.18 last 54.42) - /unlink 8697 (time 1712359601.80 total 180.19 last 55.04) - /unlink 9219 (time 1712359611.81 total 190.20 last 52.15) - /unlink 9786 (time 1712359621.83 total 200.22 last 56.60) - /unlink 10000 (time 1712359625.87 total 204.26 last 52.94) - /unlink 10542 (time 1712359635.87 total 214.26 last 54.18) - /unlink 11061 (time 1712359645.88 total 224.27 last 51.86) - /unlink 11582 (time 1712359655.90 total 234.29 last 51.98) - /unlink 12139 (time 1712359665.91 total 244.30 last 55.64) - /unlink 12734 (time 1712359675.93 total 254.32 last 59.39) - /unlink 13331 (time 1712359685.94 total 264.33 last 59.62) - /unlink 13901 (time 1712359695.95 total 274.34 last 56.96) - /unlink 14519 (time 1712359705.96 total 284.35 last 61.74) - /unlink 15135 (time 1712359715.97 total 294.36 last 61.55) - /unlink 15771 (time 1712359725.98 total 304.37 last 63.54) - /unlink 16447 (time 1712359735.98 total 314.37 last 67.57) - /unlink 17111 (time 1712359745.99 total 324.39 last 66.32) - /unlink 17877 (time 1712359756.00 total 334.39 last 76.59) - /unlink 18636 (time 1712359766.00 total 344.39 last 75.89) - /unlink 19448 (time 1712359776.01 total 354.40 last 81.11) - /unlink 20000 (time 1712359782.61 total 361.00 last 83.63) - /unlink 20745 (time 1712359792.61 total 371.00 last 74.49) - /unlink 21513 (time 1712359802.61 total 381.00 last 76.80) - /unlink 22342 (time 1712359812.62 total 391.01 last 82.84) - /unlink 23249 (time 1712359822.62 total 401.01 last 90.67) - /unlink 24104 (time 1712359832.62 total 411.02 last 85.47) - /unlink 24953 (time 1712359842.63 total 421.02 last 84.88) - /unlink 25852 (time 1712359852.64 total 431.03 last 89.81) - /unlink 26760 (time 1712359862.64 total 441.03 last 90.74) - /unlink 27727 (time 1712359872.65 total 451.04 last 96.60) - /unlink 28699 (time 1712359882.66 total 461.05 last 97.17) - /unlink 29652 (time 1712359892.66 total 471.05 last 95.25) - /unlink 30000 (time 1712359895.96 total 474.35 last 105.57) - /unlink 31092 (time 1712359905.96 total 484.35 last 109.17) - /unlink 32146 (time 1712359915.97 total 494.36 last 105.34) - /unlink 33312 (time 1712359925.97 total 504.36 last 116.60) - /unlink 34496 (time 1712359935.97 total 514.36 last 118.34) - /unlink 35824 (time 1712359945.98 total 524.37 last 132.70) - /unlink 37341 (time 1712359955.98 total 534.37 last 151.66) - /unlink 38753 (time 1712359965.98 total 544.38 last 141.14) - /unlink 40000 (time 1712359973.78 total 552.17 last 160.03) - /unlink 41603 (time 1712359983.78 total 562.17 last 160.23) - /unlink 43288 (time 1712359993.78 total 572.17 last 168.47) - /unlink 45026 (time 1712360003.78 total 582.18 last 173.78) - /unlink 46803 (time 1712360013.79 total 592.18 last 177.66) - /unlink 48896 (time 1712360023.79 total 602.18 last 209.19) - /unlink 50000 (time 1712360028.98 total 607.37 last 212.75) - /unlink 52153 (time 1712360038.98 total 617.38 last 215.22) - /unlink 54919 (time 1712360048.99 total 627.38 last 276.57) - /unlink 58284 (time 1712360058.99 total 637.38 last 336.42) - /unlink 60000 (time 1712360063.23 total 641.63 last 404.14) total: 64768 /unlink in 649.89 seconds: 99.66 ops/second sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete - open/close 2829 (time 1712360100.66 total 10.00 last 282.80) - open/close 5500 (time 1712360110.66 total 20.01 last 267.06) - open/close 8189 (time 1712360120.67 total 30.01 last 268.86) - open/close 10000 (time 1712360126.90 total 36.24 last 290.47) - open/close 12491 (time 1712360136.90 total 46.24 last 249.06) - open/close 14813 (time 1712360146.90 total 56.24 last 232.20) - open/close 16949 (time 1712360156.91 total 66.25 last 213.51) - open/close 18857 (time 1712360166.91 total 76.25 last 190.79) - open/close 20000 (time 1712360172.62 total 81.96 last 200.18) - open/close 21455 (time 1712360182.63 total 91.97 last 145.38) - open/close 23042 (time 1712360192.63 total 101.97 last 158.59) - open/close 24360 (time 1712360202.63 total 111.97 last 131.79) - open/close 25739 (time 1712360212.63 total 121.97 last 137.89) - open/close 27220 (time 1712360222.64 total 131.98 last 148.07) - open/close 28596 (time 1712360232.64 total 141.98 last 137.53) - open/close 29842 (time 1712360242.65 total 151.99 last 124.55) - open/close 30846 (time 1712360252.65 total 161.99 last 100.33) - open/close 32049 (time 1712360262.65 total 171.99 last 120.29) - open/close 33078 (time 1712360272.66 total 182.00 last 102.82) - open/close 34367 (time 1712360282.66 total 192.00 last 128.88) - open/close 35623 (time 1712360292.67 total 202.01 last 125.55) - open/close 36686 (time 1712360302.68 total 212.02 last 106.16) - open/close 37638 (time 1712360312.68 total 222.03 last 95.16) - open/close 38612 (time 1712360322.70 total 232.04 last 97.28) - open/close 39561 (time 1712360332.70 total 242.04 last 94.87) - open/close 40000 (time 1712360338.12 total 247.46 last 80.95) - open/close 40956 (time 1712360348.13 total 257.47 last 95.56) - open/close 41753 (time 1712360358.14 total 267.48 last 79.58) - open/close 42496 (time 1712360368.15 total 277.49 last 74.25) - open/close 43245 (time 1712360378.16 total 287.50 last 74.83) - open/close 44133 (time 1712360388.16 total 297.50 last 88.76) - open/close 44972 (time 1712360398.17 total 307.51 last 83.84) - open/close 45819 (time 1712360408.18 total 317.52 last 84.63) - open/close 46621 (time 1712360418.18 total 327.52 last 80.16) - open/close 47385 (time 1712360428.19 total 337.53 last 76.34) - open/close 48088 (time 1712360438.20 total 347.54 last 70.21) - open/close 48848 (time 1712360448.20 total 357.55 last 75.98) - open/close 49590 (time 1712360458.21 total 367.55 last 74.14) - open/close 50000 (time 1712360463.88 total 373.22 last 72.36) - open/close 50767 (time 1712360473.89 total 383.23 last 76.64) - open/close 51497 (time 1712360483.89 total 393.23 last 72.97) - open/close 52130 (time 1712360493.90 total 403.24 last 63.25) - open/close 52770 (time 1712360503.90 total 413.24 last 63.96) - open/close 53432 (time 1712360513.91 total 423.25 last 66.15) - open/close 54126 (time 1712360523.93 total 433.27 last 69.27) - open/close 54677 (time 1712360533.94 total 443.28 last 55.03) - open/close 55264 (time 1712360543.95 total 453.29 last 58.69) - open/close 55839 (time 1712360553.95 total 463.29 last 57.49) - open/close 56407 (time 1712360563.95 total 473.29 last 56.78) - open/close 56947 (time 1712360573.96 total 483.30 last 53.94) - open/close 57514 (time 1712360583.97 total 493.31 last 56.66) - open/close 58054 (time 1712360593.97 total 503.31 last 53.98) - open/close 58578 (time 1712360603.99 total 513.33 last 52.33) - open/close 59112 (time 1712360613.99 total 523.33 last 53.39) - open/close 59690 (time 1712360624.00 total 533.34 last 57.72) - open/close 60000 (time 1712360628.82 total 538.16 last 64.28) - open/close 60620 (time 1712360638.83 total 548.17 last 61.95) - open/close 61211 (time 1712360648.84 total 558.19 last 59.03) - open/close 61898 (time 1712360658.85 total 568.19 last 68.68) - open/close 62438 (time 1712360668.85 total 578.19 last 54.00) - open/close 62979 (time 1712360678.85 total 588.19 last 54.07) - open/close 63541 (time 1712360688.86 total 598.20 last 56.18) - open/close 64138 (time 1712360698.86 total 608.20 last 59.70) - open/close 64644 (time 1712360708.87 total 618.21 last 50.52) total: 64768 open/close in 620.45 seconds: 104.39 ops/second - /unlink 608 (time 1712360722.18 total 10.01 last 60.73) - /unlink 1461 (time 1712360732.19 total 20.02 last 85.19) - /unlink 2344 (time 1712360742.20 total 30.03 last 88.24) - /unlink 2994 (time 1712360752.20 total 40.04 last 64.97) - /unlink 3678 (time 1712360762.21 total 50.04 last 68.36) - /unlink 4492 (time 1712360772.22 total 60.05 last 81.33) - /unlink 5250 (time 1712360782.22 total 70.06 last 75.73) - /unlink 6064 (time 1712360792.24 total 80.07 last 81.30) - /unlink 6806 (time 1712360802.25 total 90.08 last 74.10) - /unlink 7563 (time 1712360812.25 total 100.09 last 75.66) - /unlink 8231 (time 1712360822.26 total 110.09 last 66.76) - /unlink 8889 (time 1712360832.27 total 120.10 last 65.75) - /unlink 9643 (time 1712360842.27 total 130.11 last 75.36) - /unlink 10000 (time 1712360847.37 total 135.21 last 70.02) - /unlink 10678 (time 1712360857.38 total 145.21 last 67.75) - /unlink 11460 (time 1712360867.39 total 155.23 last 78.12) - /unlink 12276 (time 1712360877.40 total 165.23 last 81.53) - /unlink 13187 (time 1712360887.41 total 175.24 last 91.01) - /unlink 13959 (time 1712360897.41 total 185.25 last 77.15) - /unlink 14718 (time 1712360907.42 total 195.26 last 75.83) - /unlink 15465 (time 1712360917.43 total 205.27 last 74.63) - /unlink 16211 (time 1712360927.44 total 215.27 last 74.56) - /unlink 17065 (time 1712360937.44 total 225.28 last 85.35) - /unlink 17945 (time 1712360947.45 total 235.29 last 87.93) - /unlink 18817 (time 1712360957.46 total 245.29 last 87.17) - /unlink 19771 (time 1712360967.46 total 255.29 last 95.36) - /unlink 20000 (time 1712360969.79 total 257.63 last 98.09) - /unlink 20913 (time 1712360979.80 total 267.64 last 91.21) - /unlink 21906 (time 1712360989.81 total 277.65 last 99.21) - /unlink 23075 (time 1712360999.81 total 287.65 last 116.88) - /unlink 23960 (time 1712361009.82 total 297.65 last 88.46) - /unlink 24845 (time 1712361019.82 total 307.66 last 88.49) - /unlink 25851 (time 1712361029.83 total 317.66 last 100.54) - /unlink 26815 (time 1712361039.83 total 327.66 last 96.40) - /unlink 27720 (time 1712361049.84 total 337.67 last 90.40) - /unlink 28625 (time 1712361059.85 total 347.68 last 90.40) - /unlink 29521 (time 1712361069.85 total 357.69 last 89.55) - /unlink 30000 (time 1712361075.00 total 362.84 last 93.00) - /unlink 31061 (time 1712361085.01 total 372.84 last 106.07) - /unlink 32048 (time 1712361095.01 total 382.85 last 98.67) - /unlink 32986 (time 1712361105.01 total 392.85 last 93.80) - /unlink 34084 (time 1712361115.02 total 402.86 last 109.69) - /unlink 35067 (time 1712361125.03 total 412.86 last 98.24) - /unlink 36086 (time 1712361135.03 total 422.87 last 101.85) - /unlink 37207 (time 1712361145.04 total 432.87 last 112.04) - /unlink 38358 (time 1712361155.04 total 442.88 last 115.01) - /unlink 39573 (time 1712361165.05 total 452.88 last 121.45) - /unlink 40000 (time 1712361167.74 total 455.57 last 158.69) - /unlink 41755 (time 1712361177.74 total 465.58 last 175.43) - /unlink 43737 (time 1712361187.75 total 475.58 last 198.14) - /unlink 45745 (time 1712361197.75 total 485.58 last 200.78) - /unlink 48009 (time 1712361207.75 total 495.58 last 226.35) - /unlink 50000 (time 1712361215.26 total 503.09 last 265.15) - /unlink 52955 (time 1712361225.26 total 513.10 last 295.40) - /unlink 56234 (time 1712361235.26 total 523.10 last 327.85) - /unlink 60000 (time 1712361244.70 total 532.54 last 398.90) total: 64768 /unlink in 541.67 seconds: 119.57 ops/second sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete - open/close 2301 (time 1712361283.57 total 10.00 last 230.07) - open/close 4445 (time 1712361293.57 total 20.00 last 214.35) - open/close 6354 (time 1712361303.57 total 30.00 last 190.89) - open/close 8233 (time 1712361313.57 total 40.00 last 187.89) - open/close 9916 (time 1712361323.57 total 50.01 last 168.28) - open/close 11537 (time 1712361333.58 total 60.01 last 162.02) - open/close 13056 (time 1712361343.58 total 70.01 last 151.85) - open/close 14557 (time 1712361353.58 total 80.02 last 150.05) - open/close 15921 (time 1712361363.59 total 90.03 last 136.30) - open/close 17300 (time 1712361373.60 total 100.03 last 137.83) - open/close 18651 (time 1712361383.60 total 110.03 last 135.06) - open/close 20000 (time 1712361392.31 total 118.74 last 154.93) - open/close 22060 (time 1712361402.31 total 128.74 last 205.91) - open/close 23233 (time 1712361412.31 total 138.75 last 117.25) - open/close 24766 (time 1712361422.32 total 148.75 last 153.24) - open/close 26145 (time 1712361432.33 total 158.76 last 137.78) - open/close 27266 (time 1712361442.33 total 168.77 last 112.02) - open/close 28395 (time 1712361452.34 total 178.77 last 112.85) - open/close 29434 (time 1712361462.35 total 188.78 last 103.83) - open/close 30000 (time 1712361467.57 total 194.00 last 108.42) - open/close 30959 (time 1712361477.57 total 204.01 last 95.84) - open/close 31945 (time 1712361487.57 total 214.01 last 98.57) - open/close 33038 (time 1712361497.58 total 224.01 last 109.25) - open/close 34438 (time 1712361507.58 total 234.02 last 139.95) - open/close 35494 (time 1712361517.58 total 244.02 last 105.58) - open/close 36513 (time 1712361527.59 total 254.02 last 101.87) - open/close 38025 (time 1712361537.59 total 264.03 last 151.15) - open/close 39242 (time 1712361547.59 total 274.03 last 121.67) - open/close 40000 (time 1712361557.07 total 283.51 last 79.96) - open/close 40805 (time 1712361567.08 total 293.52 last 80.41) - open/close 41716 (time 1712361577.09 total 303.53 last 91.04) - open/close 42502 (time 1712361587.10 total 313.54 last 78.51) - open/close 43233 (time 1712361597.11 total 323.55 last 73.02) - open/close 43951 (time 1712361607.12 total 333.56 last 71.73) - open/close 44612 (time 1712361617.13 total 343.57 last 66.03) - open/close 45334 (time 1712361627.14 total 353.57 last 72.19) - open/close 46070 (time 1712361637.15 total 363.58 last 73.52) - open/close 46796 (time 1712361647.15 total 373.58 last 72.57) - open/close 47629 (time 1712361657.15 total 383.59 last 83.27) - open/close 48634 (time 1712361667.16 total 393.59 last 100.46) - open/close 49391 (time 1712361677.16 total 403.60 last 75.66) - open/close 50000 (time 1712361686.75 total 413.18 last 63.54) - open/close 50608 (time 1712361696.76 total 423.19 last 60.74) - open/close 51177 (time 1712361706.77 total 433.20 last 56.85) - open/close 51758 (time 1712361716.79 total 443.22 last 57.99) - open/close 52350 (time 1712361726.79 total 453.22 last 59.20) - open/close 52922 (time 1712361736.79 total 463.23 last 57.15) - open/close 53524 (time 1712361746.80 total 473.23 last 60.17) - open/close 54068 (time 1712361756.80 total 483.23 last 54.39) - open/close 54669 (time 1712361766.80 total 493.24 last 60.08) - open/close 55452 (time 1712361776.81 total 503.24 last 78.29) - open/close 56070 (time 1712361786.81 total 513.25 last 61.76) - open/close 56692 (time 1712361796.83 total 523.26 last 62.09) - open/close 57334 (time 1712361806.85 total 533.28 last 64.09) - open/close 57897 (time 1712361816.85 total 543.29 last 56.28) - open/close 58431 (time 1712361826.86 total 553.29 last 53.37) - open/close 59163 (time 1712361836.87 total 563.31 last 73.09) - open/close 59818 (time 1712361846.88 total 573.32 last 65.42) - open/close 60000 (time 1712361849.90 total 576.34 last 60.27) - open/close 60660 (time 1712361859.91 total 586.34 last 65.98) - open/close 61321 (time 1712361869.92 total 596.35 last 66.04) - open/close 62026 (time 1712361879.92 total 606.35 last 70.47) - open/close 62815 (time 1712361889.93 total 616.37 last 78.81) - open/close 63418 (time 1712361899.94 total 626.37 last 60.26) - open/close 64066 (time 1712361909.94 total 636.37 last 64.80) - open/close 64747 (time 1712361919.94 total 646.38 last 68.09) total: 64768 open/close in 646.82 seconds: 100.13 ops/second