== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 17:07:24 (1746133644) Stopping clients: oleg435-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg435-client.virtnet /mnt/lustre2 (opts:-f) oleg435-server: oleg435-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg435-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg435-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1836 (time 1746133692.71 total 10.00 last 183.57) - open/close 3189 (time 1746133702.71 total 20.00 last 135.28) - open/close 4675 (time 1746133712.71 total 30.01 last 148.56) - open/close 6308 (time 1746133722.72 total 40.01 last 163.25) - open/close 8466 (time 1746133732.72 total 50.01 last 215.75) - open/close 10000 (time 1746133740.10 total 57.39 last 207.81) - open/close 11724 (time 1746133750.10 total 67.39 last 172.39) - open/close 13790 (time 1746133760.10 total 77.40 last 206.54) - open/close 15830 (time 1746133770.10 total 87.40 last 203.99) - open/close 17540 (time 1746133780.11 total 97.40 last 170.93) - open/close 19774 (time 1746133790.11 total 107.40 last 223.34) - open/close 22005 (time 1746133800.11 total 117.41 last 223.04) - open/close 24388 (time 1746133810.11 total 127.41 last 238.29) - open/close 26945 (time 1746133820.12 total 137.41 last 255.57) - open/close 28933 (time 1746133830.12 total 147.41 last 198.76) - open/close 30000 (time 1746133836.45 total 153.74 last 168.58) - open/close 32391 (time 1746133846.45 total 163.75 last 239.03) - open/close 34726 (time 1746133856.45 total 173.75 last 233.47) - open/close 36598 (time 1746133866.46 total 183.75 last 187.17) - open/close 38461 (time 1746133876.46 total 193.76 last 186.17) - open/close 39593 (time 1746133886.47 total 203.76 last 113.14) - open/close 40000 (time 1746133889.70 total 206.99 last 126.01) - open/close 41820 (time 1746133899.70 total 216.99 last 181.94) - open/close 43461 (time 1746133909.71 total 227.00 last 164.04) - open/close 44960 (time 1746133919.71 total 237.00 last 149.89) - open/close 46137 (time 1746133929.71 total 247.01 last 117.60) - open/close 47486 (time 1746133939.72 total 257.01 last 134.89) - open/close 49212 (time 1746133949.72 total 267.01 last 172.58) - open/close 50000 (time 1746133952.60 total 269.89 last 273.22) - open/close 52801 (time 1746133962.61 total 279.90 last 279.88) - open/close 55487 (time 1746133972.61 total 289.91 last 268.49) - open/close 57410 (time 1746133982.61 total 299.91 last 192.27) - open/close 59213 (time 1746133992.62 total 309.91 last 180.21) - open/close 60000 (time 1746133998.71 total 316.00 last 129.31) - open/close 61476 (time 1746134008.71 total 326.00 last 147.53) - open/close 62955 (time 1746134018.72 total 336.01 last 147.83) - open/close 64520 (time 1746134028.72 total 346.01 last 156.48) total: 64768 open/close in 347.38 seconds: 186.45 ops/second - /unlink 2364 (time 1746134041.29 total 10.00 last 236.39) - /unlink 5053 (time 1746134051.29 total 20.00 last 268.84) - /unlink 8395 (time 1746134061.30 total 30.01 last 333.95) - /unlink 10000 (time 1746134066.76 total 35.47 last 293.79) - /unlink 13220 (time 1746134076.76 total 45.47 last 321.98) - /unlink 16729 (time 1746134086.76 total 55.48 last 350.77) - /unlink 20000 (time 1746134096.54 total 65.25 last 334.72) - /unlink 24513 (time 1746134106.54 total 75.25 last 451.28) - /unlink 27178 (time 1746134116.54 total 85.25 last 266.44) - /unlink 29602 (time 1746134126.54 total 95.25 last 242.35) - /unlink 30000 (time 1746134128.75 total 97.46 last 180.52) - /unlink 32409 (time 1746134138.75 total 107.46 last 240.87) - /unlink 34824 (time 1746134148.75 total 117.46 last 241.49) - /unlink 36927 (time 1746134158.75 total 127.46 last 210.25) - /unlink 39118 (time 1746134168.75 total 137.46 last 219.07) - /unlink 40000 (time 1746134171.80 total 140.51 last 289.68) - /unlink 42615 (time 1746134181.80 total 150.51 last 261.50) - /unlink 45815 (time 1746134191.80 total 160.51 last 319.98) - /unlink 49334 (time 1746134201.80 total 170.52 last 351.71) - /unlink 50000 (time 1746134204.51 total 173.23 last 245.70) - /unlink 52192 (time 1746134214.52 total 183.23 last 219.13) - /unlink 54195 (time 1746134224.52 total 193.23 last 200.30) - /unlink 56215 (time 1746134234.52 total 203.23 last 201.94) - /unlink 58966 (time 1746134244.52 total 213.23 last 275.05) - /unlink 60000 (time 1746134246.84 total 215.55 last 446.00) - /unlink 63546 (time 1746134256.84 total 225.55 last 354.60) total: 64768 /unlink in 229.36 seconds: 282.38 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1656 (time 1746134288.32 total 10.00 last 165.56) - open/close 4078 (time 1746134298.33 total 20.01 last 242.11) - open/close 6461 (time 1746134308.33 total 30.01 last 238.29) - open/close 8709 (time 1746134318.33 total 40.01 last 224.75) - open/close 10000 (time 1746134325.47 total 47.14 last 180.92) - open/close 12512 (time 1746134335.47 total 57.15 last 251.15) - open/close 14612 (time 1746134345.48 total 67.15 last 209.83) - open/close 16271 (time 1746134355.48 total 77.16 last 165.78) - open/close 17758 (time 1746134365.49 total 87.17 last 148.63) - open/close 19034 (time 1746134375.49 total 97.17 last 127.56) - open/close 20000 (time 1746134382.69 total 104.36 last 134.26) - open/close 22021 (time 1746134392.69 total 114.37 last 202.08) - open/close 23259 (time 1746134402.69 total 124.37 last 123.75) - open/close 24554 (time 1746134412.69 total 134.37 last 129.46) - open/close 26223 (time 1746134422.70 total 144.37 last 166.88) - open/close 28087 (time 1746134432.70 total 154.38 last 186.37) - open/close 29631 (time 1746134442.70 total 164.38 last 154.39) - open/close 31519 (time 1746134452.70 total 174.38 last 188.78) - open/close 33155 (time 1746134462.71 total 184.39 last 163.42) - open/close 34520 (time 1746134472.71 total 194.39 last 136.49) - open/close 35753 (time 1746134482.71 total 204.39 last 123.28) - open/close 36915 (time 1746134492.72 total 214.40 last 116.12) - open/close 38318 (time 1746134502.72 total 224.40 last 140.26) - open/close 39595 (time 1746134512.73 total 234.41 last 127.58) - open/close 40000 (time 1746134516.62 total 238.30 last 104.06) - open/close 41227 (time 1746134526.62 total 248.30 last 122.70) - open/close 42408 (time 1746134536.63 total 258.31 last 117.99) - open/close 43697 (time 1746134546.64 total 268.32 last 128.84) - open/close 45296 (time 1746134556.64 total 278.32 last 159.82) - open/close 46700 (time 1746134566.66 total 288.34 last 140.20) - open/close 48121 (time 1746134576.66 total 298.34 last 142.02) - open/close 49643 (time 1746134586.67 total 308.34 last 152.14) - open/close 50000 (time 1746134588.92 total 310.60 last 158.45) - open/close 51625 (time 1746134598.92 total 320.60 last 162.44) - open/close 52857 (time 1746134608.92 total 330.60 last 123.20) - open/close 54285 (time 1746134618.93 total 340.61 last 142.71) - open/close 55458 (time 1746134628.94 total 350.61 last 117.23) - open/close 56559 (time 1746134638.94 total 360.62 last 110.09) - open/close 57746 (time 1746134648.94 total 370.62 last 118.63) - open/close 58924 (time 1746134658.96 total 380.63 last 117.64) - open/close 60000 (time 1746134668.69 total 390.37 last 110.57) - open/close 61419 (time 1746134678.69 total 400.37 last 141.90) - open/close 63055 (time 1746134688.69 total 410.37 last 163.54) - open/close 64483 (time 1746134698.69 total 420.37 last 142.77) total: 64768 open/close in 422.68 seconds: 153.23 ops/second - /unlink 2012 (time 1746134712.81 total 10.00 last 201.16) - /unlink 4437 (time 1746134722.81 total 20.00 last 242.48) - /unlink 6896 (time 1746134732.81 total 30.00 last 245.86) - /unlink 9152 (time 1746134742.81 total 40.00 last 225.60) - /unlink 10000 (time 1746134746.73 total 43.93 last 216.20) - /unlink 11840 (time 1746134756.73 total 53.93 last 183.95) - /unlink 13869 (time 1746134766.74 total 63.93 last 202.86) - /unlink 16516 (time 1746134776.74 total 73.93 last 264.68) - /unlink 19561 (time 1746134786.74 total 83.94 last 304.31) - /unlink 23181 (time 1746134796.75 total 93.94 last 361.88) - /unlink 28194 (time 1746134806.75 total 103.94 last 501.30) - /unlink 30000 (time 1746134809.85 total 107.05 last 580.98) - /unlink 33457 (time 1746134819.85 total 117.05 last 345.70) - /unlink 36414 (time 1746134829.86 total 127.05 last 295.67) - /unlink 38917 (time 1746134839.86 total 137.05 last 250.22) - /unlink 40000 (time 1746134845.24 total 142.44 last 201.16) - /unlink 42066 (time 1746134855.24 total 152.44 last 206.57) - /unlink 43890 (time 1746134865.25 total 162.44 last 182.35) - /unlink 45510 (time 1746134875.25 total 172.44 last 161.98) - /unlink 47415 (time 1746134885.26 total 182.45 last 190.32) - /unlink 49194 (time 1746134895.26 total 192.45 last 177.89) - /unlink 50000 (time 1746134899.76 total 196.96 last 178.80) - /unlink 51878 (time 1746134909.77 total 206.96 last 187.79) - /unlink 53743 (time 1746134919.77 total 216.96 last 186.49) - /unlink 55687 (time 1746134929.77 total 226.96 last 194.38) - /unlink 57644 (time 1746134939.77 total 236.97 last 195.58) - /unlink 59491 (time 1746134949.78 total 246.97 last 184.63) - /unlink 60000 (time 1746134952.45 total 249.64 last 190.54) - /unlink 62842 (time 1746134962.45 total 259.65 last 284.15) total: 64768 /unlink in 266.87 seconds: 242.69 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1514 (time 1746135000.93 total 10.00 last 151.34) - open/close 3021 (time 1746135010.93 total 20.01 last 150.64) - open/close 4874 (time 1746135020.94 total 30.01 last 185.19) - open/close 6459 (time 1746135030.94 total 40.01 last 158.49) - open/close 7728 (time 1746135040.95 total 50.02 last 126.80) - open/close 9331 (time 1746135050.96 total 60.03 last 160.13) - open/close 10000 (time 1746135054.73 total 63.81 last 177.25) - open/close 11357 (time 1746135064.73 total 73.81 last 135.67) - open/close 12744 (time 1746135074.74 total 83.82 last 138.63) - open/close 14356 (time 1746135084.74 total 93.82 last 161.19) - open/close 16291 (time 1746135094.74 total 103.82 last 193.46) - open/close 17757 (time 1746135104.74 total 113.82 last 146.57) - open/close 19057 (time 1746135114.75 total 123.82 last 129.97) - open/close 20000 (time 1746135122.33 total 131.40 last 124.40) - open/close 21366 (time 1746135132.33 total 141.41 last 136.51) - open/close 22888 (time 1746135142.34 total 151.41 last 152.15) - open/close 24459 (time 1746135152.34 total 161.41 last 157.10) - open/close 26359 (time 1746135162.34 total 171.41 last 190.00) - open/close 28351 (time 1746135172.34 total 181.42 last 199.10) - open/close 30000 (time 1746135180.20 total 189.27 last 209.94) - open/close 31639 (time 1746135190.21 total 199.28 last 163.73) - open/close 33188 (time 1746135200.21 total 209.28 last 154.88) - open/close 34494 (time 1746135210.21 total 219.29 last 130.55) - open/close 36001 (time 1746135220.22 total 229.29 last 150.65) - open/close 37314 (time 1746135230.22 total 239.29 last 131.27) - open/close 38845 (time 1746135240.22 total 249.30 last 153.01) - open/close 40000 (time 1746135247.82 total 256.89 last 152.15) - open/close 41386 (time 1746135257.82 total 266.89 last 138.59) - open/close 43039 (time 1746135267.82 total 276.89 last 165.28) - open/close 44938 (time 1746135277.82 total 286.90 last 189.85) - open/close 46958 (time 1746135287.82 total 296.90 last 201.97) - open/close 48576 (time 1746135297.83 total 306.90 last 161.72) - open/close 50000 (time 1746135306.81 total 315.89 last 158.47) - open/close 51840 (time 1746135316.85 total 325.93 last 183.27) - open/close 53613 (time 1746135326.85 total 335.93 last 177.28) - open/close 54829 (time 1746135336.86 total 345.93 last 121.57) - open/close 56685 (time 1746135346.86 total 355.94 last 185.54) - open/close 58304 (time 1746135356.87 total 365.95 last 161.75) - open/close 59366 (time 1746135366.87 total 375.95 last 106.15) - open/close 60000 (time 1746135372.29 total 381.36 last 117.13) - open/close 61297 (time 1746135382.29 total 391.37 last 129.65) - open/close 62503 (time 1746135392.30 total 401.37 last 120.54) - open/close 63748 (time 1746135402.30 total 411.38 last 124.40) total: 64768 open/close in 418.43 seconds: 154.79 ops/second - /unlink 1952 (time 1746135420.86 total 10.00 last 195.19) - /unlink 4012 (time 1746135430.86 total 20.00 last 205.94) - /unlink 6269 (time 1746135440.87 total 30.01 last 225.64) - /unlink 9017 (time 1746135450.87 total 40.01 last 274.74) - /unlink 10000 (time 1746135455.82 total 44.96 last 198.55) - /unlink 12129 (time 1746135465.82 total 54.96 last 212.89) - /unlink 14273 (time 1746135475.82 total 64.96 last 214.34) - /unlink 16821 (time 1746135485.83 total 74.96 last 254.76) - /unlink 19028 (time 1746135495.83 total 84.97 last 220.63) - /unlink 20000 (time 1746135500.38 total 89.52 last 213.42) - /unlink 22772 (time 1746135510.39 total 99.53 last 277.11) - /unlink 25288 (time 1746135520.39 total 109.53 last 251.57) - /unlink 27443 (time 1746135530.39 total 119.53 last 215.46) - /unlink 29971 (time 1746135540.39 total 129.53 last 252.77) - /unlink 32421 (time 1746135550.39 total 139.53 last 244.92) - /unlink 34816 (time 1746135560.40 total 149.54 last 239.38) - /unlink 37226 (time 1746135570.40 total 159.54 last 240.97) - /unlink 39839 (time 1746135580.40 total 169.54 last 261.27) - /unlink 42744 (time 1746135590.40 total 179.54 last 290.49) - /unlink 45487 (time 1746135600.41 total 189.54 last 274.20) - /unlink 48051 (time 1746135610.41 total 199.54 last 256.40) - /unlink 50000 (time 1746135618.27 total 207.41 last 247.82) - /unlink 52627 (time 1746135628.28 total 217.42 last 262.47) - /unlink 56110 (time 1746135638.28 total 227.42 last 348.20) - /unlink 59225 (time 1746135648.28 total 237.42 last 311.48) - /unlink 60000 (time 1746135651.50 total 240.64 last 240.79) - /unlink 62008 (time 1746135661.51 total 250.65 last 200.68) - /unlink 64202 (time 1746135671.51 total 260.65 last 219.29) total: 64768 /unlink in 262.75 seconds: 246.50 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1418 (time 1746135700.82 total 10.00 last 141.78) - open/close 3001 (time 1746135710.83 total 20.01 last 158.21) - open/close 4388 (time 1746135720.83 total 30.01 last 138.69) - open/close 5877 (time 1746135730.84 total 40.01 last 148.84) - open/close 7215 (time 1746135740.84 total 50.02 last 133.76) - open/close 8999 (time 1746135750.85 total 60.03 last 178.19) - open/close 10000 (time 1746135758.25 total 67.43 last 135.28) - open/close 11273 (time 1746135768.25 total 77.43 last 127.30) - open/close 12803 (time 1746135778.25 total 87.43 last 152.98) - open/close 13978 (time 1746135788.26 total 97.43 last 117.44) - open/close 15065 (time 1746135798.26 total 107.44 last 108.68) - open/close 16343 (time 1746135808.26 total 117.44 last 127.79) - open/close 17623 (time 1746135818.26 total 127.44 last 127.98) - open/close 19038 (time 1746135828.26 total 137.44 last 141.47) - open/close 20000 (time 1746135837.38 total 146.55 last 105.57) - open/close 21205 (time 1746135847.38 total 156.56 last 120.40) - open/close 22438 (time 1746135857.39 total 166.57 last 123.23) - open/close 23518 (time 1746135867.40 total 176.57 last 107.93) - open/close 24560 (time 1746135877.41 total 186.59 last 104.07) - open/close 25791 (time 1746135887.41 total 196.59 last 123.08) - open/close 26799 (time 1746135897.41 total 206.59 last 100.76) - open/close 28206 (time 1746135907.41 total 216.59 last 140.69) - open/close 29252 (time 1746135917.42 total 226.59 last 104.58) - open/close 30000 (time 1746135925.00 total 234.18 last 98.64) - open/close 31085 (time 1746135935.00 total 244.18 last 108.48) - open/close 32214 (time 1746135945.00 total 254.18 last 112.88) - open/close 33467 (time 1746135955.01 total 264.18 last 125.23) - open/close 34654 (time 1746135965.01 total 274.18 last 118.70) - open/close 35967 (time 1746135975.01 total 284.19 last 131.28) - open/close 37088 (time 1746135985.02 total 294.19 last 112.03) - open/close 38390 (time 1746135995.02 total 304.19 last 130.16) - open/close 39637 (time 1746136005.02 total 314.20 last 124.68) - open/close 40000 (time 1746136008.10 total 317.28 last 117.90) - open/close 41103 (time 1746136018.10 total 327.28 last 110.25) - open/close 42263 (time 1746136028.11 total 337.28 last 115.95) - open/close 43351 (time 1746136038.11 total 347.29 last 108.74) - open/close 44515 (time 1746136048.12 total 357.29 last 116.34) - open/close 45638 (time 1746136058.12 total 367.30 last 112.23) - open/close 46830 (time 1746136068.14 total 377.32 last 119.01) - open/close 47983 (time 1746136078.14 total 387.32 last 115.27) - open/close 49021 (time 1746136088.14 total 397.32 last 103.79) - open/close 50000 (time 1746136097.16 total 406.33 last 108.62) - open/close 51187 (time 1746136107.16 total 416.34 last 118.66) - open/close 52311 (time 1746136117.17 total 426.34 last 112.34) - open/close 53356 (time 1746136127.17 total 436.35 last 104.47) - open/close 54630 (time 1746136137.17 total 446.35 last 127.38) - open/close 56035 (time 1746136147.18 total 456.35 last 140.42) - open/close 57250 (time 1746136157.18 total 466.36 last 121.46) - open/close 59260 (time 1746136167.18 total 476.36 last 200.96) - open/close 60000 (time 1746136171.18 total 480.35 last 185.27) - open/close 61582 (time 1746136181.18 total 490.35 last 158.17) - open/close 63033 (time 1746136191.18 total 500.36 last 145.05) - open/close 64307 (time 1746136201.18 total 510.36 last 127.36) total: 64768 open/close in 514.45 seconds: 125.90 ops/second - /unlink 1986 (time 1746136217.19 total 10.00 last 198.55) - /unlink 4066 (time 1746136227.19 total 20.00 last 208.00) - /unlink 5881 (time 1746136237.19 total 30.01 last 181.38) - /unlink 7599 (time 1746136247.19 total 40.01 last 171.78) - /unlink 9685 (time 1746136257.19 total 50.01 last 208.58) - /unlink 11285 (time 1746136267.20 total 60.01 last 159.94) - /unlink 13100 (time 1746136277.20 total 70.02 last 181.46) - /unlink 14918 (time 1746136287.20 total 80.02 last 181.74) - /unlink 16578 (time 1746136297.21 total 90.02 last 165.95) - /unlink 18363 (time 1746136307.21 total 100.03 last 178.47) - /unlink 20000 (time 1746136315.12 total 107.94 last 206.90) - /unlink 21967 (time 1746136325.12 total 117.94 last 196.68) - /unlink 23872 (time 1746136335.12 total 127.94 last 190.50) - /unlink 26525 (time 1746136345.12 total 137.94 last 265.27) - /unlink 28338 (time 1746136355.13 total 147.95 last 181.18) - /unlink 30000 (time 1746136364.27 total 157.08 last 181.87) - /unlink 32150 (time 1746136374.27 total 167.09 last 214.97) - /unlink 33842 (time 1746136384.27 total 177.09 last 169.12) - /unlink 35532 (time 1746136394.28 total 187.10 last 168.91) - /unlink 37355 (time 1746136404.28 total 197.10 last 182.28) - /unlink 39279 (time 1746136414.29 total 207.10 last 192.28) - /unlink 40000 (time 1746136418.45 total 211.26 last 173.30) - /unlink 41595 (time 1746136428.45 total 221.27 last 159.45) - /unlink 43053 (time 1746136438.46 total 231.27 last 145.70) - /unlink 44653 (time 1746136448.47 total 241.28 last 159.87) - /unlink 46299 (time 1746136458.47 total 251.29 last 164.54) - /unlink 47874 (time 1746136468.47 total 261.29 last 157.47) - /unlink 49728 (time 1746136478.48 total 271.29 last 185.30) - /unlink 51463 (time 1746136488.48 total 281.30 last 173.44) - /unlink 53628 (time 1746136498.48 total 291.30 last 216.43) - /unlink 55721 (time 1746136508.49 total 301.30 last 209.25) - /unlink 57931 (time 1746136518.49 total 311.31 last 220.91) - /unlink 59966 (time 1746136528.49 total 321.31 last 203.42) - /unlink 62139 (time 1746136538.49 total 331.31 last 217.29) - /unlink 64115 (time 1746136548.49 total 341.31 last 197.59) total: 64768 /unlink in 344.76 seconds: 187.86 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1069 (time 1746136582.71 total 10.01 last 106.83) - open/close 2190 (time 1746136592.71 total 20.01 last 112.06) - open/close 3431 (time 1746136602.71 total 30.01 last 124.06) - open/close 4497 (time 1746136612.71 total 40.01 last 106.59) - open/close 5676 (time 1746136622.72 total 50.02 last 117.89) - open/close 6790 (time 1746136632.72 total 60.02 last 111.35) - open/close 7832 (time 1746136642.72 total 70.02 last 104.16) - open/close 8937 (time 1746136652.73 total 80.03 last 110.46) - open/close 10000 (time 1746136661.43 total 88.73 last 122.16) - open/close 11204 (time 1746136671.43 total 98.73 last 120.38) - open/close 12404 (time 1746136681.44 total 108.74 last 119.91) - open/close 13735 (time 1746136691.44 total 118.75 last 133.00) - open/close 15076 (time 1746136701.45 total 128.75 last 134.01) - open/close 16332 (time 1746136711.46 total 138.76 last 125.55) - open/close 17584 (time 1746136721.46 total 148.76 last 125.13) - open/close 18782 (time 1746136731.47 total 158.77 last 119.71) - open/close 20000 (time 1746136739.80 total 167.10 last 146.13) - open/close 21242 (time 1746136749.81 total 177.11 last 124.14) - open/close 22710 (time 1746136759.81 total 187.11 last 146.75) - open/close 23953 (time 1746136769.81 total 197.11 last 124.27) - open/close 25360 (time 1746136779.82 total 207.12 last 140.63) - open/close 26819 (time 1746136789.82 total 217.12 last 145.86) - open/close 28484 (time 1746136799.83 total 227.13 last 166.44) - open/close 29846 (time 1746136809.83 total 237.13 last 136.17) - open/close 31223 (time 1746136819.83 total 247.13 last 137.68) - open/close 32371 (time 1746136829.83 total 257.13 last 114.73) - open/close 33561 (time 1746136839.84 total 267.14 last 118.98) - open/close 34807 (time 1746136849.84 total 277.14 last 124.53) - open/close 35899 (time 1746136859.84 total 287.14 last 109.17) - open/close 36872 (time 1746136869.85 total 297.15 last 97.25) - open/close 38162 (time 1746136879.86 total 307.16 last 128.91) - open/close 39328 (time 1746136889.86 total 317.16 last 116.53) - open/close 40000 (time 1746136895.45 total 322.75 last 120.32) - open/close 41156 (time 1746136905.45 total 332.75 last 115.59) - open/close 42118 (time 1746136915.45 total 342.75 last 96.15) - open/close 43328 (time 1746136925.45 total 352.75 last 121.00) - open/close 44582 (time 1746136935.45 total 362.75 last 125.39) - open/close 45881 (time 1746136945.46 total 372.76 last 129.83) - open/close 47175 (time 1746136955.46 total 382.76 last 129.33) - open/close 48269 (time 1746136965.47 total 392.77 last 109.34) - open/close 49383 (time 1746136975.48 total 402.78 last 111.29) - open/close 50000 (time 1746136980.87 total 408.17 last 114.44) - open/close 51179 (time 1746136990.87 total 418.17 last 117.89) - open/close 52069 (time 1746137000.88 total 428.18 last 88.92) - open/close 53064 (time 1746137010.88 total 438.18 last 99.48) - open/close 54153 (time 1746137020.88 total 448.18 last 108.90) - open/close 55446 (time 1746137030.89 total 458.19 last 129.26) - open/close 56670 (time 1746137040.89 total 468.19 last 122.36) - open/close 57995 (time 1746137050.90 total 478.20 last 132.40) - open/close 59378 (time 1746137060.90 total 488.20 last 138.24) - open/close 60000 (time 1746137065.82 total 493.12 last 126.46) - open/close 61265 (time 1746137075.82 total 503.13 last 126.43) - open/close 62636 (time 1746137085.83 total 513.13 last 137.08) - open/close 63822 (time 1746137095.83 total 523.13 last 118.55) total: 64768 open/close in 530.66 seconds: 122.05 ops/second - /unlink 2004 (time 1746137114.86 total 10.00 last 200.33) - /unlink 3843 (time 1746137124.86 total 20.01 last 183.79) - /unlink 5616 (time 1746137134.86 total 30.01 last 177.27) - /unlink 7557 (time 1746137144.87 total 40.01 last 194.03) - /unlink 9649 (time 1746137154.87 total 50.02 last 209.10) - /unlink 11646 (time 1746137164.87 total 60.02 last 199.68) - /unlink 13836 (time 1746137174.87 total 70.02 last 218.95) - /unlink 16103 (time 1746137184.88 total 80.03 last 226.64) - /unlink 18805 (time 1746137194.88 total 90.03 last 270.15) - /unlink 20000 (time 1746137200.51 total 95.65 last 212.36) - /unlink 22094 (time 1746137210.51 total 105.65 last 209.38) - /unlink 24538 (time 1746137220.51 total 115.66 last 244.29) - /unlink 26722 (time 1746137230.51 total 125.66 last 218.40) - /unlink 28960 (time 1746137240.51 total 135.66 last 223.79) - /unlink 30000 (time 1746137244.62 total 139.77 last 252.86) - /unlink 32124 (time 1746137254.63 total 149.78 last 212.34) - /unlink 34659 (time 1746137264.63 total 159.78 last 253.44) - /unlink 37343 (time 1746137274.63 total 169.78 last 268.32) - /unlink 39493 (time 1746137284.63 total 179.78 last 214.99) - /unlink 40000 (time 1746137286.82 total 181.97 last 231.83) - /unlink 42547 (time 1746137296.82 total 191.97 last 254.60) - /unlink 44997 (time 1746137306.83 total 201.97 last 244.98) - /unlink 47451 (time 1746137316.83 total 211.97 last 245.39) - /unlink 49913 (time 1746137326.83 total 221.98 last 246.11) - /unlink 52421 (time 1746137336.83 total 231.98 last 250.75) - /unlink 54687 (time 1746137346.83 total 241.98 last 226.53) - /unlink 57364 (time 1746137356.83 total 251.98 last 267.68) - /unlink 59856 (time 1746137366.84 total 261.98 last 249.18) - /unlink 62112 (time 1746137376.84 total 271.98 last 225.59) - /unlink 64767 (time 1746137386.84 total 281.99 last 265.44) total: 64768 /unlink in 281.99 seconds: 229.68 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg435-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server unloading modules via unload_modules_local on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2