== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 22:19:46 (1746065986) Stopping clients: oleg250-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg250-client.virtnet /mnt/lustre2 (opts:-f) oleg250-server: oleg250-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg250-server' oleg250-server: oleg250-server.virtnet: executing load_modules_local oleg250-server: Loading modules from /home/green/git/lustre-release/lustre oleg250-server: detected 4 online CPUs by sysfs oleg250-server: MODOPTS_LIBCFS= oleg250-server: Force libcfs to create 2 CPU partitions oleg250-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg250-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg250-server: oleg250-server.virtnet: executing set_default_debug -1 all pdsh@oleg250-client: oleg250-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg250-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg250-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg250-server: oleg250-server.virtnet: executing set_default_debug -1 all pdsh@oleg250-client: oleg250-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg250-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg250-client.virtnet: -o user_xattr,flock 192.168.202.150@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1313 (time 1746066040.49 total 10.00 last 131.25) - open/close 2614 (time 1746066050.50 total 20.01 last 129.99) - open/close 3918 (time 1746066060.50 total 30.01 last 130.37) - open/close 5383 (time 1746066070.50 total 40.02 last 146.48) - open/close 6911 (time 1746066080.50 total 50.02 last 152.73) - open/close 8376 (time 1746066090.51 total 60.03 last 146.41) - open/close 9792 (time 1746066100.52 total 70.03 last 141.54) - open/close 11181 (time 1746066110.52 total 80.03 last 138.86) - open/close 12600 (time 1746066120.52 total 90.04 last 141.85) - open/close 13952 (time 1746066130.53 total 100.05 last 135.09) - open/close 15190 (time 1746066140.53 total 110.05 last 123.75) - open/close 16505 (time 1746066150.54 total 120.06 last 131.42) - open/close 17666 (time 1746066160.55 total 130.06 last 116.03) - open/close 19024 (time 1746066170.55 total 140.07 last 135.73) - open/close 20000 (time 1746066178.07 total 147.59 last 129.82) - open/close 21270 (time 1746066188.07 total 157.59 last 126.99) - open/close 22574 (time 1746066198.07 total 167.59 last 130.35) - open/close 23934 (time 1746066208.08 total 177.59 last 135.97) - open/close 25328 (time 1746066218.08 total 187.59 last 139.39) - open/close 26641 (time 1746066228.09 total 197.61 last 131.07) - open/close 27915 (time 1746066238.10 total 207.62 last 127.30) - open/close 29274 (time 1746066248.10 total 217.62 last 135.89) - open/close 30000 (time 1746066252.95 total 222.46 last 149.90) - open/close 31405 (time 1746066262.96 total 232.47 last 140.35) - open/close 32782 (time 1746066272.96 total 242.48 last 137.66) - open/close 34150 (time 1746066282.96 total 252.48 last 136.79) - open/close 35417 (time 1746066292.96 total 262.48 last 126.66) - open/close 36810 (time 1746066302.97 total 272.48 last 139.24) - open/close 38447 (time 1746066312.97 total 282.49 last 163.64) - open/close 39973 (time 1746066322.97 total 292.49 last 152.55) - open/close 41709 (time 1746066332.98 total 302.49 last 173.53) - open/close 43550 (time 1746066342.98 total 312.50 last 184.04) - open/close 45093 (time 1746066352.99 total 322.50 last 154.20) - open/close 46544 (time 1746066362.99 total 332.51 last 145.05) - open/close 48143 (time 1746066372.99 total 342.51 last 159.88) - open/close 49486 (time 1746066383.00 total 352.52 last 134.20) - open/close 50000 (time 1746066387.21 total 356.73 last 121.97) - open/close 51256 (time 1746066397.22 total 366.74 last 125.49) - open/close 52389 (time 1746066407.22 total 376.74 last 113.30) - open/close 53593 (time 1746066417.23 total 386.75 last 120.28) - open/close 54845 (time 1746066427.23 total 396.75 last 125.18) - open/close 56183 (time 1746066437.24 total 406.75 last 133.79) - open/close 57562 (time 1746066447.24 total 416.75 last 137.87) - open/close 59014 (time 1746066457.24 total 426.76 last 145.16) - open/close 60000 (time 1746066464.14 total 433.66 last 142.92) - open/close 61208 (time 1746066474.14 total 443.66 last 120.75) - open/close 62256 (time 1746066484.15 total 453.67 last 104.73) - open/close 63346 (time 1746066494.15 total 463.67 last 108.98) - open/close 64709 (time 1746066504.16 total 473.67 last 136.23) total: 64768 open/close in 474.13 seconds: 136.60 ops/second - /unlink 2015 (time 1746066516.37 total 10.00 last 201.42) - /unlink 4211 (time 1746066526.38 total 20.01 last 219.56) - /unlink 6467 (time 1746066536.38 total 30.01 last 225.51) - /unlink 8482 (time 1746066546.38 total 40.01 last 201.43) - /unlink 10000 (time 1746066555.16 total 48.79 last 172.93) - /unlink 11515 (time 1746066565.16 total 58.80 last 151.44) - /unlink 12975 (time 1746066575.17 total 68.80 last 145.99) - /unlink 14522 (time 1746066585.17 total 78.80 last 154.66) - /unlink 15972 (time 1746066595.17 total 88.80 last 144.99) - /unlink 17644 (time 1746066605.17 total 98.80 last 167.16) - /unlink 19461 (time 1746066615.17 total 108.80 last 181.69) - /unlink 20000 (time 1746066617.82 total 111.45 last 203.67) - /unlink 21711 (time 1746066627.82 total 121.45 last 171.05) - /unlink 23442 (time 1746066637.82 total 131.45 last 173.07) - /unlink 25283 (time 1746066647.83 total 141.46 last 184.05) - /unlink 27153 (time 1746066657.83 total 151.46 last 186.87) - /unlink 28842 (time 1746066667.84 total 161.47 last 168.76) - /unlink 30000 (time 1746066675.14 total 168.77 last 158.61) - /unlink 31872 (time 1746066685.14 total 178.77 last 187.15) - /unlink 34071 (time 1746066695.15 total 188.78 last 219.82) - /unlink 36155 (time 1746066705.15 total 198.78 last 208.34) - /unlink 38371 (time 1746066715.15 total 208.78 last 221.60) - /unlink 40000 (time 1746066721.58 total 215.21 last 253.51) - /unlink 42331 (time 1746066731.58 total 225.21 last 233.08) - /unlink 44379 (time 1746066741.58 total 235.21 last 204.75) - /unlink 46284 (time 1746066751.58 total 245.21 last 190.44) - /unlink 48253 (time 1746066761.59 total 255.22 last 196.85) - /unlink 50000 (time 1746066770.42 total 264.05 last 197.72) - /unlink 51898 (time 1746066780.42 total 274.05 last 189.76) - /unlink 53885 (time 1746066790.43 total 284.06 last 198.63) - /unlink 55811 (time 1746066800.43 total 294.06 last 192.54) - /unlink 57640 (time 1746066810.43 total 304.06 last 182.88) - /unlink 60000 (time 1746066819.69 total 313.33 last 254.76) - /unlink 62067 (time 1746066829.69 total 323.33 last 206.70) - /unlink 64169 (time 1746066839.70 total 333.33 last 210.19) total: 64768 /unlink in 336.21 seconds: 192.64 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1196 (time 1746066873.10 total 10.00 last 119.59) - open/close 2545 (time 1746066883.11 total 20.01 last 134.84) - open/close 3716 (time 1746066893.11 total 30.01 last 117.08) - open/close 4961 (time 1746066903.11 total 40.01 last 124.45) - open/close 6258 (time 1746066913.12 total 50.01 last 129.66) - open/close 7959 (time 1746066923.12 total 60.02 last 170.05) - open/close 9377 (time 1746066933.12 total 70.02 last 141.74) - open/close 10000 (time 1746066938.35 total 75.24 last 119.30) - open/close 11326 (time 1746066948.35 total 85.25 last 132.54) - open/close 12663 (time 1746066958.35 total 95.25 last 133.66) - open/close 13891 (time 1746066968.36 total 105.26 last 122.73) - open/close 15053 (time 1746066978.37 total 115.26 last 116.12) - open/close 16175 (time 1746066988.37 total 125.26 last 112.19) - open/close 17304 (time 1746066998.37 total 135.27 last 112.84) - open/close 18593 (time 1746067008.38 total 145.27 last 128.83) - open/close 20000 (time 1746067017.87 total 154.77 last 148.20) - open/close 21278 (time 1746067027.87 total 164.77 last 127.76) - open/close 22639 (time 1746067037.89 total 174.79 last 135.92) - open/close 23928 (time 1746067047.89 total 184.79 last 128.80) - open/close 25285 (time 1746067057.90 total 194.80 last 135.67) - open/close 26684 (time 1746067067.90 total 204.80 last 139.80) - open/close 27967 (time 1746067077.90 total 214.80 last 128.30) - open/close 29570 (time 1746067087.91 total 224.81 last 160.26) - open/close 30000 (time 1746067090.52 total 227.42 last 164.73) - open/close 31980 (time 1746067100.52 total 237.42 last 197.88) - open/close 33736 (time 1746067110.52 total 247.42 last 175.59) - open/close 35319 (time 1746067120.53 total 257.43 last 158.24) - open/close 36932 (time 1746067130.53 total 267.43 last 161.29) - open/close 38726 (time 1746067140.53 total 277.43 last 179.35) - open/close 40000 (time 1746067146.68 total 283.58 last 207.31) - open/close 41440 (time 1746067156.68 total 293.58 last 143.99) - open/close 42635 (time 1746067166.69 total 303.58 last 119.40) - open/close 43823 (time 1746067176.69 total 313.59 last 118.75) - open/close 45053 (time 1746067186.70 total 323.59 last 122.93) - open/close 46439 (time 1746067196.70 total 333.60 last 138.54) - open/close 47869 (time 1746067206.70 total 343.60 last 142.97) - open/close 49253 (time 1746067216.70 total 353.60 last 138.37) - open/close 50000 (time 1746067222.37 total 359.27 last 131.89) - open/close 51339 (time 1746067232.38 total 369.27 last 133.82) - open/close 52651 (time 1746067242.38 total 379.28 last 131.14) - open/close 54025 (time 1746067252.38 total 389.28 last 137.36) - open/close 55613 (time 1746067262.39 total 399.29 last 158.70) - open/close 56867 (time 1746067272.40 total 409.29 last 125.30) - open/close 58102 (time 1746067282.40 total 419.30 last 123.45) - open/close 59485 (time 1746067292.40 total 429.30 last 138.28) - open/close 60000 (time 1746067295.59 total 432.49 last 161.41) - open/close 61445 (time 1746067305.59 total 442.49 last 144.49) - open/close 62893 (time 1746067315.59 total 452.49 last 144.78) - open/close 64278 (time 1746067325.59 total 462.49 last 138.49) total: 64768 open/close in 466.23 seconds: 138.92 ops/second - /unlink 2940 (time 1746067340.77 total 10.00 last 294.00) - /unlink 6051 (time 1746067350.77 total 20.00 last 311.03) - /unlink 9269 (time 1746067360.78 total 30.00 last 321.74) - /unlink 10000 (time 1746067363.14 total 32.37 last 309.58) - /unlink 12514 (time 1746067373.14 total 42.37 last 251.38) - /unlink 14552 (time 1746067383.14 total 52.37 last 203.76) - /unlink 16858 (time 1746067393.14 total 62.37 last 230.58) - /unlink 19321 (time 1746067403.14 total 72.37 last 246.28) - /unlink 20000 (time 1746067406.24 total 75.47 last 219.19) - /unlink 22336 (time 1746067416.24 total 85.47 last 233.52) - /unlink 24553 (time 1746067426.25 total 95.48 last 221.49) - /unlink 27146 (time 1746067436.25 total 105.48 last 259.24) - /unlink 29731 (time 1746067446.26 total 115.48 last 258.48) - /unlink 32407 (time 1746067456.26 total 125.49 last 267.52) - /unlink 35294 (time 1746067466.26 total 135.49 last 288.66) - /unlink 37749 (time 1746067476.26 total 145.49 last 245.47) - /unlink 40000 (time 1746067485.02 total 154.25 last 257.01) - /unlink 42502 (time 1746067495.02 total 164.25 last 250.16) - /unlink 44836 (time 1746067505.02 total 174.25 last 233.40) - /unlink 47464 (time 1746067515.02 total 184.25 last 262.78) - /unlink 50000 (time 1746067523.22 total 192.45 last 309.25) - /unlink 52697 (time 1746067533.22 total 202.45 last 269.70) - /unlink 55458 (time 1746067543.22 total 212.45 last 276.04) - /unlink 57849 (time 1746067553.23 total 222.45 last 239.08) - /unlink 60000 (time 1746067560.51 total 229.74 last 295.20) - /unlink 62880 (time 1746067570.51 total 239.74 last 287.96) total: 64768 /unlink in 245.35 seconds: 263.99 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1610 (time 1746067604.13 total 10.01 last 160.83) - open/close 3273 (time 1746067614.14 total 20.02 last 166.15) - open/close 4943 (time 1746067624.14 total 30.03 last 166.90) - open/close 6656 (time 1746067634.14 total 40.03 last 171.29) - open/close 8319 (time 1746067644.15 total 50.03 last 166.23) - open/close 9776 (time 1746067654.16 total 60.04 last 145.53) - open/close 11384 (time 1746067664.16 total 70.04 last 160.76) - open/close 12813 (time 1746067674.16 total 80.05 last 142.89) - open/close 14784 (time 1746067684.16 total 90.05 last 197.09) - open/close 16784 (time 1746067694.17 total 100.06 last 199.81) - open/close 18482 (time 1746067704.18 total 110.06 last 169.69) - open/close 20000 (time 1746067713.42 total 119.31 last 164.18) - open/close 21870 (time 1746067723.43 total 129.31 last 186.94) - open/close 23823 (time 1746067733.43 total 139.31 last 195.28) - open/close 25710 (time 1746067743.43 total 149.31 last 188.69) - open/close 27413 (time 1746067753.44 total 159.32 last 170.19) - open/close 29124 (time 1746067763.44 total 169.33 last 171.00) - open/close 30000 (time 1746067767.74 total 173.62 last 203.93) - open/close 31953 (time 1746067777.74 total 183.62 last 195.29) - open/close 33790 (time 1746067787.74 total 193.62 last 183.69) - open/close 35751 (time 1746067797.74 total 203.62 last 196.08) - open/close 38164 (time 1746067807.74 total 213.62 last 241.28) - open/close 40000 (time 1746067816.65 total 222.53 last 206.14) - open/close 42043 (time 1746067826.65 total 232.54 last 204.17) - open/close 43724 (time 1746067836.66 total 242.54 last 167.97) - open/close 45620 (time 1746067846.67 total 252.55 last 189.41) - open/close 47021 (time 1746067856.67 total 262.56 last 140.04) - open/close 48641 (time 1746067866.68 total 272.56 last 161.92) - open/close 50000 (time 1746067874.97 total 280.85 last 163.90) - open/close 51653 (time 1746067884.98 total 290.87 last 165.13) - open/close 53316 (time 1746067894.99 total 300.87 last 166.22) - open/close 54983 (time 1746067904.99 total 310.87 last 166.68) - open/close 56545 (time 1746067914.99 total 320.87 last 156.16) - open/close 58064 (time 1746067924.99 total 330.88 last 151.87) - open/close 59447 (time 1746067935.00 total 340.88 last 138.21) - open/close 60000 (time 1746067938.89 total 344.78 last 142.03) - open/close 61424 (time 1746067948.90 total 354.78 last 142.34) - open/close 62984 (time 1746067958.90 total 364.78 last 155.93) - open/close 64703 (time 1746067968.91 total 374.79 last 171.79) total: 64768 open/close in 375.20 seconds: 172.62 ops/second - /unlink 2771 (time 1746067980.73 total 10.00 last 277.08) - /unlink 5825 (time 1746067990.73 total 20.00 last 305.37) - /unlink 8246 (time 1746068000.73 total 30.00 last 242.03) - /unlink 10000 (time 1746068007.96 total 37.24 last 242.43) - /unlink 12945 (time 1746068017.97 total 47.24 last 294.46) - /unlink 16920 (time 1746068027.97 total 57.24 last 397.43) - /unlink 20000 (time 1746068037.80 total 67.08 last 313.13) - /unlink 22415 (time 1746068047.81 total 77.08 last 241.42) - /unlink 25095 (time 1746068057.81 total 87.08 last 268.00) - /unlink 27725 (time 1746068067.81 total 97.08 last 262.95) - /unlink 30000 (time 1746068076.15 total 105.42 last 272.87) - /unlink 32796 (time 1746068086.15 total 115.42 last 279.59) - /unlink 35991 (time 1746068096.15 total 125.42 last 319.44) - /unlink 39062 (time 1746068106.15 total 135.43 last 307.06) - /unlink 40000 (time 1746068110.00 total 139.28 last 243.49) - /unlink 42448 (time 1746068120.00 total 149.28 last 244.78) - /unlink 45154 (time 1746068130.00 total 159.28 last 270.56) - /unlink 47761 (time 1746068140.01 total 169.28 last 260.60) - /unlink 50000 (time 1746068148.47 total 177.75 last 264.49) - /unlink 53325 (time 1746068158.47 total 187.75 last 332.49) - /unlink 56639 (time 1746068168.48 total 197.75 last 331.33) - /unlink 60000 (time 1746068176.34 total 205.62 last 427.18) - /unlink 63430 (time 1746068186.35 total 215.62 last 342.92) total: 64768 /unlink in 220.22 seconds: 294.10 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1606 (time 1746068220.20 total 10.01 last 160.49) - open/close 3222 (time 1746068230.20 total 20.01 last 161.57) - open/close 4656 (time 1746068240.21 total 30.01 last 143.32) - open/close 6768 (time 1746068250.21 total 40.02 last 211.18) - open/close 8731 (time 1746068260.22 total 50.02 last 196.16) - open/close 10000 (time 1746068267.54 total 57.34 last 173.38) - open/close 11867 (time 1746068277.54 total 67.34 last 186.65) - open/close 13570 (time 1746068287.54 total 77.35 last 170.26) - open/close 15157 (time 1746068297.54 total 87.35 last 158.70) - open/close 16615 (time 1746068307.55 total 97.35 last 145.75) - open/close 17834 (time 1746068317.55 total 107.35 last 121.86) - open/close 19127 (time 1746068327.57 total 117.37 last 129.07) - open/close 20000 (time 1746068333.25 total 123.06 last 153.51) - open/close 21517 (time 1746068343.25 total 133.06 last 151.69) - open/close 23513 (time 1746068353.26 total 143.06 last 199.54) - open/close 25590 (time 1746068363.26 total 153.06 last 207.68) - open/close 28126 (time 1746068373.26 total 163.06 last 253.57) - open/close 30000 (time 1746068380.14 total 169.94 last 272.48) - open/close 32799 (time 1746068390.14 total 179.94 last 279.82) - open/close 35175 (time 1746068400.14 total 189.95 last 237.59) - open/close 37371 (time 1746068410.14 total 199.95 last 219.58) - open/close 39623 (time 1746068420.14 total 209.95 last 225.15) - open/close 40000 (time 1746068422.49 total 212.30 last 160.61) - open/close 41913 (time 1746068432.49 total 222.30 last 191.29) - open/close 43756 (time 1746068442.50 total 232.30 last 184.23) - open/close 45597 (time 1746068452.50 total 242.30 last 184.08) - open/close 47278 (time 1746068462.50 total 252.30 last 168.08) - open/close 49394 (time 1746068472.50 total 262.30 last 211.58) - open/close 50000 (time 1746068475.68 total 265.49 last 190.19) - open/close 52313 (time 1746068485.69 total 275.49 last 231.29) - open/close 54170 (time 1746068495.69 total 285.49 last 185.63) - open/close 56571 (time 1746068505.69 total 295.49 last 240.09) - open/close 58985 (time 1746068515.69 total 305.50 last 241.35) - open/close 60000 (time 1746068520.23 total 310.03 last 223.71) - open/close 62670 (time 1746068530.23 total 320.04 last 266.86) total: 64768 open/close in 328.37 seconds: 197.24 ops/second - /unlink 4289 (time 1746068549.64 total 10.00 last 428.85) - /unlink 8310 (time 1746068559.64 total 20.00 last 402.09) - /unlink 10000 (time 1746068565.15 total 25.51 last 306.57) - /unlink 13226 (time 1746068575.15 total 35.52 last 322.57) - /unlink 16292 (time 1746068585.15 total 45.52 last 306.51) - /unlink 19037 (time 1746068595.15 total 55.52 last 274.47) - /unlink 20000 (time 1746068598.78 total 59.15 last 265.46) - /unlink 22826 (time 1746068608.78 total 69.15 last 282.59) - /unlink 26121 (time 1746068618.78 total 79.15 last 329.49) - /unlink 29096 (time 1746068628.78 total 89.15 last 297.47) - /unlink 30000 (time 1746068631.76 total 92.12 last 303.72) - /unlink 33707 (time 1746068641.76 total 102.12 last 370.69) - /unlink 36890 (time 1746068651.76 total 112.13 last 318.26) - /unlink 40000 (time 1746068661.22 total 121.58 last 328.82) - /unlink 42887 (time 1746068671.22 total 131.59 last 288.67) - /unlink 45109 (time 1746068681.22 total 141.59 last 222.19) - /unlink 47155 (time 1746068691.22 total 151.59 last 204.56) - /unlink 50000 (time 1746068700.57 total 160.93 last 304.41) - /unlink 53088 (time 1746068710.57 total 170.94 last 308.74) - /unlink 57057 (time 1746068720.57 total 180.94 last 396.84) - /unlink 60000 (time 1746068726.90 total 187.27 last 464.94) - /unlink 63431 (time 1746068736.90 total 197.27 last 343.03) total: 64768 /unlink in 200.66 seconds: 322.78 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1762 (time 1746068769.38 total 10.01 last 176.09) - open/close 3110 (time 1746068779.39 total 20.01 last 134.72) - open/close 4569 (time 1746068789.40 total 30.02 last 145.84) - open/close 6255 (time 1746068799.40 total 40.02 last 168.52) - open/close 7888 (time 1746068809.40 total 50.02 last 163.29) - open/close 9544 (time 1746068819.41 total 60.03 last 165.45) - open/close 10000 (time 1746068821.91 total 62.53 last 182.66) - open/close 11993 (time 1746068831.91 total 72.53 last 199.26) - open/close 14427 (time 1746068841.91 total 82.54 last 243.27) - open/close 16531 (time 1746068851.92 total 92.54 last 210.36) - open/close 18626 (time 1746068861.92 total 102.54 last 209.44) - open/close 20000 (time 1746068870.02 total 110.64 last 169.63) - open/close 21704 (time 1746068880.02 total 120.65 last 170.30) - open/close 23263 (time 1746068890.02 total 130.65 last 155.89) - open/close 25162 (time 1746068900.03 total 140.65 last 189.74) - open/close 26789 (time 1746068910.03 total 150.65 last 162.69) - open/close 28534 (time 1746068920.03 total 160.66 last 174.48) - open/close 30000 (time 1746068926.55 total 167.17 last 224.90) - open/close 31792 (time 1746068936.55 total 177.17 last 179.19) - open/close 33475 (time 1746068946.56 total 187.18 last 168.20) - open/close 35033 (time 1746068956.56 total 197.18 last 155.74) - open/close 36646 (time 1746068966.57 total 207.19 last 161.22) - open/close 38412 (time 1746068976.57 total 217.19 last 176.57) - open/close 40000 (time 1746068986.12 total 226.74 last 166.35) - open/close 42326 (time 1746068996.12 total 236.74 last 232.55) - open/close 44091 (time 1746069006.12 total 246.74 last 176.45) - open/close 45774 (time 1746069016.13 total 256.75 last 168.21) - open/close 47498 (time 1746069026.13 total 266.75 last 172.39) - open/close 49130 (time 1746069036.13 total 276.75 last 163.14) - open/close 50000 (time 1746069040.52 total 281.14 last 198.07) - open/close 51789 (time 1746069050.53 total 291.15 last 178.85) - open/close 53508 (time 1746069060.53 total 301.15 last 171.85) - open/close 55364 (time 1746069070.53 total 311.15 last 185.55) - open/close 56954 (time 1746069080.53 total 321.16 last 158.95) - open/close 58355 (time 1746069090.54 total 331.16 last 140.00) - open/close 59932 (time 1746069100.54 total 341.16 last 157.70) - open/close 61347 (time 1746069110.55 total 351.17 last 141.43) - open/close 62696 (time 1746069120.55 total 361.17 last 134.88) - open/close 63970 (time 1746069130.55 total 371.17 last 127.36) total: 64768 open/close in 376.97 seconds: 171.81 ops/second - /unlink 2336 (time 1746069147.93 total 10.00 last 233.60) - /unlink 4541 (time 1746069157.93 total 20.00 last 220.45) - /unlink 6430 (time 1746069167.94 total 30.01 last 188.79) - /unlink 8662 (time 1746069177.94 total 40.01 last 223.19) - /unlink 10000 (time 1746069184.16 total 46.23 last 215.14) - /unlink 12232 (time 1746069194.16 total 56.23 last 223.12) - /unlink 14667 (time 1746069204.16 total 66.23 last 243.48) - /unlink 17439 (time 1746069214.17 total 76.24 last 277.12) - /unlink 19716 (time 1746069224.17 total 86.24 last 227.63) - /unlink 22094 (time 1746069234.17 total 96.24 last 237.74) - /unlink 24403 (time 1746069244.18 total 106.24 last 230.84) - /unlink 27178 (time 1746069254.18 total 116.25 last 277.41) - /unlink 29803 (time 1746069264.18 total 126.25 last 262.48) - /unlink 32419 (time 1746069274.18 total 136.25 last 261.59) - /unlink 34951 (time 1746069284.18 total 146.25 last 253.08) - /unlink 37468 (time 1746069294.19 total 156.26 last 251.61) - /unlink 39568 (time 1746069304.19 total 166.26 last 209.93) - /unlink 40000 (time 1746069306.42 total 168.49 last 193.87) - /unlink 41981 (time 1746069316.42 total 178.49 last 198.08) - /unlink 44033 (time 1746069326.42 total 188.49 last 205.18) - /unlink 46380 (time 1746069336.43 total 198.50 last 234.54) - /unlink 49193 (time 1746069346.43 total 208.50 last 281.29) - /unlink 50000 (time 1746069349.61 total 211.68 last 253.86) - /unlink 52196 (time 1746069359.61 total 221.68 last 219.56) - /unlink 54380 (time 1746069369.61 total 231.68 last 218.31) - /unlink 56809 (time 1746069379.61 total 241.68 last 242.90) - /unlink 59499 (time 1746069389.61 total 251.68 last 268.99) - /unlink 60000 (time 1746069391.65 total 253.72 last 245.79) - /unlink 62521 (time 1746069401.66 total 263.72 last 252.00) total: 64768 /unlink in 272.59 seconds: 237.60 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg250-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg250-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg250-server stop mds service on oleg250-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg250-server unloading modules via unload_modules_local on: 'oleg250-server' oleg250-server: oleg250-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg250-client: oleg250-client: ssh exited with exit code 2 pdsh@oleg250-client: oleg250-server: ssh exited with exit code 2 pdsh@oleg250-client: oleg250-client: ssh exited with exit code 2 pdsh@oleg250-client: oleg250-server: ssh exited with exit code 2