== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 16:53:44 (1746132824) Stopping clients: oleg441-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg441-client.virtnet /mnt/lustre2 (opts:-f) oleg441-server: oleg441-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg441-server' oleg441-server: oleg441-server.virtnet: executing load_modules_local oleg441-server: Loading modules from /home/green/git/lustre-release/lustre oleg441-server: detected 4 online CPUs by sysfs oleg441-server: MODOPTS_LIBCFS= oleg441-server: Force libcfs to create 2 CPU partitions oleg441-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg441-server Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg441-server: oleg441-server.virtnet: executing set_default_debug -1 all pdsh@oleg441-client: oleg441-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg441-server Starting mds2: -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg441-server: oleg441-server.virtnet: executing set_default_debug -1 all pdsh@oleg441-client: oleg441-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg441-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg441-server Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg441-server: oleg441-server.virtnet: executing set_default_debug -1 all pdsh@oleg441-client: oleg441-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg441-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg441-client.virtnet: -o user_xattr,flock 192.168.204.141@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 274 79718 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 65808 368 65440 1% /mnt/lustre[OST:0] filesystem_summary: 65969 529 65440 1% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1215 (time 1746132881.84 total 10.00 last 121.50) - open/close 2410 (time 1746132891.85 total 20.00 last 119.45) - open/close 3699 (time 1746132901.85 total 30.00 last 128.90) - open/close 4971 (time 1746132911.85 total 40.01 last 127.18) - open/close 6283 (time 1746132921.85 total 50.01 last 131.16) - open/close 7874 (time 1746132931.85 total 60.01 last 159.05) - open/close 9532 (time 1746132941.86 total 70.01 last 165.76) - open/close 10000 (time 1746132944.74 total 72.89 last 162.48) - open/close 11611 (time 1746132954.74 total 82.90 last 161.06) - open/close 12847 (time 1746132964.74 total 92.90 last 123.54) - open/close 14086 (time 1746132974.74 total 102.90 last 123.89) - open/close 15394 (time 1746132984.76 total 112.92 last 130.59) - open/close 16633 (time 1746132994.76 total 122.92 last 123.88) - open/close 17953 (time 1746133004.76 total 132.92 last 131.98) - open/close 19354 (time 1746133014.77 total 142.92 last 140.07) - open/close 20000 (time 1746133019.88 total 148.04 last 126.27) - open/close 21243 (time 1746133029.88 total 158.04 last 124.27) - open/close 22395 (time 1746133039.89 total 168.05 last 115.14) - open/close 23514 (time 1746133049.90 total 178.05 last 111.84) - open/close 24718 (time 1746133059.90 total 188.06 last 120.33) - open/close 25954 (time 1746133069.90 total 198.06 last 123.55) - open/close 27076 (time 1746133079.91 total 208.07 last 112.18) - open/close 28258 (time 1746133089.91 total 218.07 last 118.19) - open/close 29527 (time 1746133099.91 total 228.07 last 126.84) - open/close 30000 (time 1746133103.33 total 231.49 last 138.23) - open/close 31340 (time 1746133113.33 total 241.49 last 133.99) - open/close 32575 (time 1746133123.34 total 251.50 last 123.47) - open/close 33779 (time 1746133133.34 total 261.50 last 120.32) - open/close 35233 (time 1746133143.35 total 271.50 last 145.36) - open/close 36406 (time 1746133153.35 total 281.51 last 117.25) - open/close 37742 (time 1746133163.35 total 291.51 last 133.60) - open/close 38935 (time 1746133173.36 total 301.51 last 119.24) - open/close 40000 (time 1746133182.93 total 311.09 last 111.20) - open/close 41012 (time 1746133192.95 total 321.10 last 101.08) - open/close 42073 (time 1746133202.95 total 331.10 last 106.10) - open/close 43122 (time 1746133212.95 total 341.11 last 104.86) - open/close 44081 (time 1746133222.95 total 351.11 last 95.90) - open/close 45196 (time 1746133232.95 total 361.11 last 111.44) - open/close 46285 (time 1746133242.96 total 371.12 last 108.82) - open/close 47441 (time 1746133252.96 total 381.12 last 115.58) - open/close 48587 (time 1746133262.96 total 391.12 last 114.60) - open/close 49763 (time 1746133272.97 total 401.12 last 117.57) - open/close 50000 (time 1746133275.11 total 403.26 last 110.74) - open/close 51126 (time 1746133285.11 total 413.27 last 112.55) - open/close 52387 (time 1746133295.12 total 423.28 last 125.98) - open/close 53510 (time 1746133305.12 total 433.28 last 112.29) - open/close 54693 (time 1746133315.12 total 443.28 last 118.28) - open/close 55954 (time 1746133325.13 total 453.28 last 126.06) - open/close 57191 (time 1746133335.13 total 463.28 last 123.69) - open/close 58424 (time 1746133345.13 total 473.29 last 123.28) - open/close 59780 (time 1746133355.13 total 483.29 last 135.57) - open/close 61186 (time 1746133365.13 total 493.29 last 140.57) - open/close 62393 (time 1746133375.13 total 503.29 last 120.70) - open/close 63441 (time 1746133385.14 total 513.30 last 104.72) - open/close 64518 (time 1746133395.15 total 523.31 last 107.62) total: 64768 open/close in 525.46 seconds: 123.26 ops/second - /unlink 2009 (time 1746133409.61 total 10.00 last 200.87) - /unlink 4030 (time 1746133419.61 total 20.00 last 202.04) - /unlink 6013 (time 1746133429.62 total 30.01 last 198.15) - /unlink 8254 (time 1746133439.62 total 40.01 last 224.04) - /unlink 10000 (time 1746133447.74 total 48.14 last 215.00) - /unlink 12440 (time 1746133457.74 total 58.14 last 244.00) - /unlink 14638 (time 1746133467.74 total 68.14 last 219.76) - /unlink 16704 (time 1746133477.75 total 78.14 last 206.44) - /unlink 18879 (time 1746133487.75 total 88.15 last 217.40) - /unlink 20000 (time 1746133493.36 total 93.76 last 199.88) - /unlink 22317 (time 1746133503.36 total 103.76 last 231.65) - /unlink 24292 (time 1746133513.37 total 113.76 last 197.44) - /unlink 26350 (time 1746133523.37 total 123.76 last 205.80) - /unlink 28822 (time 1746133533.37 total 133.77 last 247.14) - /unlink 30000 (time 1746133538.30 total 138.70 last 238.72) - /unlink 32559 (time 1746133548.31 total 148.70 last 255.86) - /unlink 35374 (time 1746133558.31 total 158.70 last 281.47) - /unlink 37671 (time 1746133568.31 total 168.71 last 229.55) - /unlink 39978 (time 1746133578.32 total 178.71 last 230.60) - /unlink 42117 (time 1746133588.32 total 188.71 last 213.90) - /unlink 44402 (time 1746133598.32 total 198.71 last 228.48) - /unlink 47439 (time 1746133608.32 total 208.71 last 303.68) - /unlink 50000 (time 1746133618.00 total 218.40 last 264.40) - /unlink 52778 (time 1746133628.01 total 228.41 last 277.62) - /unlink 56255 (time 1746133638.01 total 238.41 last 347.63) - /unlink 59610 (time 1746133648.01 total 248.41 last 335.48) - /unlink 62843 (time 1746133658.02 total 258.41 last 323.25) total: 64768 /unlink in 266.06 seconds: 243.44 ops/second Waiting for MDT destroys to complete - open/close 1450 (time 1746133684.56 total 10.00 last 144.96) - open/close 3302 (time 1746133694.56 total 20.00 last 185.16) - open/close 4726 (time 1746133704.56 total 30.01 last 142.36) - open/close 6283 (time 1746133714.57 total 40.01 last 155.63) - open/close 8017 (time 1746133724.57 total 50.02 last 173.34) - open/close 10000 (time 1746133733.39 total 58.84 last 224.78) - open/close 11975 (time 1746133743.40 total 68.84 last 197.37) - open/close 13853 (time 1746133753.40 total 78.85 last 187.77) - open/close 15904 (time 1746133763.42 total 88.86 last 204.74) - open/close 17952 (time 1746133773.42 total 98.86 last 204.77) - open/close 19709 (time 1746133783.42 total 108.86 last 175.69) - open/close 21948 (time 1746133793.42 total 118.87 last 223.85) - open/close 24339 (time 1746133803.42 total 128.87 last 239.07) - open/close 26919 (time 1746133813.43 total 138.87 last 257.98) - open/close 29251 (time 1746133823.43 total 148.87 last 233.13) - open/close 30000 (time 1746133826.89 total 152.34 last 216.23) - open/close 31880 (time 1746133836.90 total 162.34 last 187.92) - open/close 34221 (time 1746133846.90 total 172.34 last 234.05) - open/close 36549 (time 1746133856.90 total 182.35 last 232.75) - open/close 38422 (time 1746133866.90 total 192.35 last 187.25) - open/close 40000 (time 1746133875.28 total 200.72 last 188.49) - open/close 41452 (time 1746133885.28 total 210.72 last 145.19) - open/close 42992 (time 1746133895.28 total 220.72 last 154.00) - open/close 44797 (time 1746133905.28 total 230.72 last 180.43) - open/close 46322 (time 1746133915.29 total 240.74 last 152.34) - open/close 47682 (time 1746133925.30 total 250.74 last 135.94) - open/close 48969 (time 1746133935.30 total 260.74 last 128.66) - open/close 50000 (time 1746133941.70 total 267.14 last 161.18) - open/close 51927 (time 1746133951.70 total 277.14 last 192.69) - open/close 54904 (time 1746133961.70 total 287.14 last 297.62) - open/close 57473 (time 1746133971.70 total 297.14 last 256.88) - open/close 59589 (time 1746133981.70 total 307.15 last 211.50) - open/close 61542 (time 1746133991.71 total 317.15 last 195.26) - open/close 62949 (time 1746134001.72 total 327.16 last 140.56) - open/close 64493 (time 1746134011.72 total 337.16 last 154.36) total: 64768 open/close in 339.36 seconds: 190.85 ops/second - /unlink 2877 (time 1746134025.53 total 10.00 last 287.66) - /unlink 5907 (time 1746134035.53 total 20.00 last 302.90) - /unlink 7905 (time 1746134045.53 total 30.01 last 199.79) - /unlink 10000 (time 1746134052.26 total 36.74 last 311.24) - /unlink 13316 (time 1746134062.27 total 46.75 last 331.29) - /unlink 16466 (time 1746134072.27 total 56.75 last 315.00) - /unlink 19885 (time 1746134082.28 total 66.75 last 341.81) - /unlink 23260 (time 1746134092.28 total 76.75 last 337.43) - /unlink 27183 (time 1746134102.28 total 86.75 last 392.26) - /unlink 30000 (time 1746134108.65 total 93.12 last 442.28) - /unlink 32729 (time 1746134118.65 total 103.12 last 272.89) - /unlink 35131 (time 1746134128.65 total 113.13 last 240.06) - /unlink 37726 (time 1746134138.66 total 123.13 last 259.40) - /unlink 40000 (time 1746134147.89 total 132.37 last 246.23) - /unlink 42503 (time 1746134157.89 total 142.37 last 250.27) - /unlink 44904 (time 1746134167.90 total 152.37 last 239.99) - /unlink 47779 (time 1746134177.90 total 162.37 last 287.41) - /unlink 50000 (time 1746134185.20 total 169.67 last 304.28) - /unlink 53461 (time 1746134195.20 total 179.67 last 346.10) - /unlink 57011 (time 1746134205.20 total 189.68 last 354.92) - /unlink 59554 (time 1746134215.21 total 199.68 last 254.24) - /unlink 60000 (time 1746134217.64 total 202.11 last 183.29) - /unlink 62181 (time 1746134227.64 total 212.11 last 218.10) - /unlink 64538 (time 1746134237.64 total 222.12 last 235.62) total: 64768 /unlink in 222.83 seconds: 290.67 ops/second Waiting for MDT destroys to complete - open/close 2162 (time 1746134254.52 total 10.01 last 216.08) - open/close 3896 (time 1746134264.53 total 20.01 last 173.36) - open/close 5677 (time 1746134274.53 total 30.01 last 178.01) - open/close 7404 (time 1746134284.54 total 40.02 last 172.62) - open/close 9648 (time 1746134294.54 total 50.02 last 224.33) - open/close 12075 (time 1746134304.54 total 60.02 last 242.60) - open/close 14540 (time 1746134314.54 total 70.03 last 246.46) - open/close 16488 (time 1746134324.55 total 80.03 last 194.69) - open/close 18947 (time 1746134334.55 total 90.03 last 245.86) - open/close 20000 (time 1746134339.30 total 94.78 last 221.81) - open/close 22031 (time 1746134349.30 total 104.78 last 203.02) - open/close 23728 (time 1746134359.31 total 114.79 last 169.66) - open/close 25231 (time 1746134369.31 total 124.79 last 150.28) - open/close 26527 (time 1746134379.31 total 134.79 last 129.57) - open/close 28361 (time 1746134389.31 total 144.79 last 183.33) - open/close 29938 (time 1746134399.32 total 154.80 last 157.63) - open/close 31214 (time 1746134409.32 total 164.80 last 127.55) - open/close 32635 (time 1746134419.32 total 174.80 last 142.09) - open/close 34546 (time 1746134429.32 total 184.81 last 191.03) - open/close 36170 (time 1746134439.33 total 194.81 last 162.31) - open/close 37936 (time 1746134449.33 total 204.82 last 176.51) - open/close 39687 (time 1746134459.34 total 214.82 last 175.01) - open/close 40000 (time 1746134461.40 total 216.88 last 152.07) - open/close 41355 (time 1746134471.40 total 226.88 last 135.50) - open/close 42724 (time 1746134481.41 total 236.89 last 136.78) - open/close 43881 (time 1746134491.41 total 246.89 last 115.70) - open/close 45353 (time 1746134501.42 total 256.90 last 147.08) - open/close 46769 (time 1746134511.43 total 266.91 last 141.46) - open/close 48024 (time 1746134521.43 total 276.91 last 125.48) - open/close 49354 (time 1746134531.43 total 286.91 last 132.94) - open/close 50000 (time 1746134536.66 total 292.14 last 123.52) - open/close 51315 (time 1746134546.67 total 302.15 last 131.38) - open/close 52931 (time 1746134556.67 total 312.15 last 161.59) - open/close 54381 (time 1746134566.68 total 322.16 last 144.90) - open/close 55823 (time 1746134576.68 total 332.16 last 144.16) - open/close 57329 (time 1746134586.68 total 342.16 last 150.58) - open/close 59096 (time 1746134596.69 total 352.17 last 176.53) - open/close 60000 (time 1746134604.08 total 359.57 last 122.27) - open/close 61308 (time 1746134614.09 total 369.57 last 130.70) - open/close 62740 (time 1746134624.10 total 379.58 last 143.14) - open/close 63868 (time 1746134634.10 total 389.58 last 112.78) total: 64768 open/close in 396.60 seconds: 163.31 ops/second - /unlink 2207 (time 1746134652.90 total 10.01 last 220.56) - /unlink 4108 (time 1746134662.91 total 20.01 last 190.02) - /unlink 6218 (time 1746134672.91 total 30.01 last 210.99) - /unlink 9166 (time 1746134682.91 total 40.01 last 294.73) - /unlink 10000 (time 1746134686.23 total 43.33 last 251.59) - /unlink 12908 (time 1746134696.23 total 53.33 last 290.76) - /unlink 15022 (time 1746134706.23 total 63.33 last 211.33) - /unlink 17374 (time 1746134716.23 total 73.33 last 235.20) - /unlink 20000 (time 1746134726.23 total 83.33 last 262.55) - /unlink 22495 (time 1746134736.23 total 93.33 last 249.49) - /unlink 24917 (time 1746134746.24 total 103.34 last 242.14) - /unlink 26808 (time 1746134756.24 total 113.34 last 189.06) - /unlink 28879 (time 1746134766.24 total 123.34 last 207.01) - /unlink 30000 (time 1746134770.60 total 127.70 last 257.27) - /unlink 32920 (time 1746134780.60 total 137.70 last 291.92) - /unlink 35878 (time 1746134790.61 total 147.71 last 295.67) - /unlink 39925 (time 1746134800.61 total 157.71 last 404.69) - /unlink 45391 (time 1746134810.61 total 167.71 last 546.54) - /unlink 48792 (time 1746134820.61 total 177.71 last 340.01) - /unlink 50000 (time 1746134824.50 total 181.60 last 310.83) - /unlink 53225 (time 1746134834.50 total 191.60 last 322.43) - /unlink 55706 (time 1746134844.50 total 201.60 last 248.07) - /unlink 58012 (time 1746134854.50 total 211.61 last 230.52) - /unlink 60000 (time 1746134863.55 total 220.65 last 219.84) - /unlink 62174 (time 1746134873.55 total 230.65 last 217.39) - /unlink 64558 (time 1746134883.55 total 240.66 last 238.25) total: 64768 /unlink in 241.57 seconds: 268.12 ops/second Waiting for MDT destroys to complete - open/close 1186 (time 1746134900.17 total 10.00 last 118.54) - open/close 2433 (time 1746134910.17 total 20.01 last 124.70) - open/close 3663 (time 1746134920.19 total 30.02 last 122.83) - open/close 4993 (time 1746134930.19 total 40.02 last 132.96) - open/close 6231 (time 1746134940.21 total 50.04 last 123.53) - open/close 7632 (time 1746134950.21 total 60.05 last 140.07) - open/close 9391 (time 1746134960.22 total 70.05 last 175.84) - open/close 10000 (time 1746134964.16 total 73.99 last 154.43) - open/close 11709 (time 1746134974.16 total 84.00 last 170.83) - open/close 13431 (time 1746134984.17 total 94.00 last 172.14) - open/close 14853 (time 1746134994.17 total 104.00 last 142.17) - open/close 16597 (time 1746135004.17 total 114.01 last 174.33) - open/close 18176 (time 1746135014.18 total 124.01 last 157.80) - open/close 20000 (time 1746135021.84 total 131.67 last 238.15) - open/close 21844 (time 1746135031.84 total 141.67 last 184.39) - open/close 23365 (time 1746135041.84 total 151.68 last 152.06) - open/close 25338 (time 1746135051.84 total 161.68 last 197.28) - open/close 27237 (time 1746135061.85 total 171.68 last 189.79) - open/close 28899 (time 1746135071.85 total 181.69 last 166.14) - open/close 30000 (time 1746135078.46 total 188.29 last 166.65) - open/close 31756 (time 1746135088.46 total 198.30 last 175.52) - open/close 34135 (time 1746135098.46 total 208.30 last 237.86) - open/close 35722 (time 1746135108.47 total 218.31 last 158.59) - open/close 37218 (time 1746135118.48 total 228.31 last 149.48) - open/close 38733 (time 1746135128.49 total 238.32 last 151.40) - open/close 40000 (time 1746135135.60 total 245.43 last 178.15) - open/close 41869 (time 1746135145.60 total 255.43 last 186.88) - open/close 43735 (time 1746135155.60 total 265.43 last 186.59) - open/close 46052 (time 1746135165.60 total 275.43 last 231.70) - open/close 48335 (time 1746135175.60 total 285.44 last 228.21) - open/close 50000 (time 1746135182.27 total 292.11 last 249.68) - open/close 51784 (time 1746135192.27 total 302.11 last 178.39) - open/close 53651 (time 1746135202.27 total 312.11 last 186.68) - open/close 55199 (time 1746135212.28 total 322.11 last 154.74) - open/close 56923 (time 1746135222.28 total 332.12 last 172.31) - open/close 58407 (time 1746135232.28 total 342.12 last 148.38) - open/close 60000 (time 1746135241.08 total 350.91 last 181.14) - open/close 61659 (time 1746135251.08 total 360.92 last 165.81) - open/close 63430 (time 1746135261.09 total 370.92 last 177.07) total: 64768 open/close in 378.35 seconds: 171.19 ops/second - /unlink 4005 (time 1746135279.78 total 10.00 last 400.48) - /unlink 7880 (time 1746135289.78 total 20.00 last 387.49) - /unlink 10000 (time 1746135295.99 total 26.20 last 341.77) - /unlink 13354 (time 1746135305.99 total 36.20 last 335.39) - /unlink 17081 (time 1746135315.99 total 46.21 last 372.64) - /unlink 20000 (time 1746135322.85 total 53.07 last 425.17) - /unlink 22792 (time 1746135332.86 total 63.07 last 279.12) - /unlink 26094 (time 1746135342.86 total 73.08 last 330.16) - /unlink 29953 (time 1746135352.86 total 83.08 last 385.88) - /unlink 32442 (time 1746135362.86 total 93.08 last 248.89) - /unlink 35035 (time 1746135372.86 total 103.08 last 259.26) - /unlink 38118 (time 1746135382.86 total 113.08 last 308.30) - /unlink 40000 (time 1746135390.27 total 120.49 last 253.97) - /unlink 42888 (time 1746135400.28 total 130.49 last 288.71) - /unlink 45935 (time 1746135410.28 total 140.49 last 304.62) - /unlink 48577 (time 1746135420.28 total 150.50 last 264.11) - /unlink 50000 (time 1746135426.21 total 156.42 last 240.11) - /unlink 52645 (time 1746135436.21 total 166.43 last 264.38) - /unlink 56584 (time 1746135446.21 total 176.43 last 393.89) - /unlink 59220 (time 1746135456.21 total 186.43 last 263.59) - /unlink 60000 (time 1746135459.27 total 189.48 last 255.47) - /unlink 62861 (time 1746135469.27 total 199.48 last 286.05) total: 64768 /unlink in 206.82 seconds: 313.17 ops/second Waiting for MDT destroys to complete - open/close 1560 (time 1746135495.71 total 10.00 last 155.97) - open/close 3202 (time 1746135505.71 total 20.00 last 164.18) - open/close 5048 (time 1746135515.71 total 30.01 last 184.55) - open/close 6664 (time 1746135525.72 total 40.01 last 161.50) - open/close 8221 (time 1746135535.72 total 50.02 last 155.65) - open/close 10000 (time 1746135544.68 total 58.98 last 198.45) - open/close 11699 (time 1746135554.69 total 68.98 last 169.85) - open/close 13359 (time 1746135564.70 total 78.99 last 165.85) - open/close 15224 (time 1746135574.70 total 89.00 last 186.43) - open/close 17201 (time 1746135584.70 total 99.00 last 197.70) - open/close 19186 (time 1746135594.70 total 109.00 last 198.41) - open/close 20000 (time 1746135598.78 total 113.08 last 199.68) - open/close 21741 (time 1746135608.78 total 123.08 last 174.05) - open/close 23572 (time 1746135618.79 total 133.09 last 182.99) - open/close 25390 (time 1746135628.79 total 143.09 last 181.76) - open/close 27696 (time 1746135638.79 total 153.09 last 230.54) - open/close 29859 (time 1746135648.80 total 163.09 last 216.21) - open/close 31492 (time 1746135658.80 total 173.09 last 163.29) - open/close 32956 (time 1746135668.81 total 183.10 last 146.27) - open/close 34405 (time 1746135678.81 total 193.11 last 144.87) - open/close 35704 (time 1746135688.81 total 203.11 last 129.87) - open/close 37391 (time 1746135698.81 total 213.11 last 168.67) - open/close 39174 (time 1746135708.82 total 223.11 last 178.29) - open/close 40000 (time 1746135714.19 total 228.48 last 153.72) - open/close 41788 (time 1746135724.19 total 238.49 last 178.77) - open/close 43455 (time 1746135734.20 total 248.50 last 166.52) - open/close 45251 (time 1746135744.20 total 258.50 last 179.57) - open/close 47150 (time 1746135754.20 total 268.50 last 189.87) - open/close 48596 (time 1746135764.21 total 278.51 last 144.51) - open/close 50000 (time 1746135772.59 total 286.88 last 167.56) - open/close 51679 (time 1746135782.60 total 296.89 last 167.80) - open/close 53067 (time 1746135792.60 total 306.89 last 138.80) - open/close 54463 (time 1746135802.60 total 316.89 last 139.59) - open/close 56060 (time 1746135812.60 total 326.89 last 159.69) - open/close 57681 (time 1746135822.60 total 336.89 last 162.08) - open/close 59221 (time 1746135832.60 total 346.90 last 153.90) - open/close 60000 (time 1746135838.91 total 353.21 last 123.50) - open/close 61482 (time 1746135848.91 total 363.21 last 148.19) - open/close 63029 (time 1746135858.92 total 373.22 last 154.54) - open/close 64402 (time 1746135868.93 total 383.22 last 137.25) total: 64768 open/close in 386.08 seconds: 167.76 ops/second - /unlink 2434 (time 1746135883.56 total 10.00 last 243.34) - /unlink 4608 (time 1746135893.56 total 20.01 last 217.30) - /unlink 7002 (time 1746135903.56 total 30.01 last 239.34) - /unlink 9501 (time 1746135913.57 total 40.01 last 249.85) - /unlink 10000 (time 1746135915.92 total 42.37 last 212.00) - /unlink 11988 (time 1746135925.92 total 52.37 last 198.71) - /unlink 14282 (time 1746135935.93 total 62.37 last 229.36) - /unlink 16782 (time 1746135945.93 total 72.37 last 249.97) - /unlink 19465 (time 1746135955.93 total 82.37 last 268.28) - /unlink 20000 (time 1746135958.32 total 84.77 last 223.70) - /unlink 22657 (time 1746135968.32 total 94.77 last 265.57) - /unlink 25177 (time 1746135978.33 total 104.77 last 251.93) - /unlink 27607 (time 1746135988.33 total 114.78 last 242.96) - /unlink 30000 (time 1746135996.88 total 123.33 last 279.89) - /unlink 32516 (time 1746136006.88 total 133.33 last 251.59) - /unlink 34740 (time 1746136016.88 total 143.33 last 222.35) - /unlink 37342 (time 1746136026.89 total 153.33 last 260.03) - /unlink 39888 (time 1746136036.89 total 163.34 last 254.57) - /unlink 42299 (time 1746136046.90 total 173.34 last 240.93) - /unlink 44611 (time 1746136056.90 total 183.35 last 231.04) - /unlink 47289 (time 1746136066.91 total 193.35 last 267.72) - /unlink 49935 (time 1746136076.91 total 203.36 last 264.44) - /unlink 52246 (time 1746136086.91 total 213.36 last 231.06) - /unlink 54562 (time 1746136096.92 total 223.36 last 231.54) - /unlink 57211 (time 1746136106.92 total 233.36 last 264.86) - /unlink 59800 (time 1746136116.92 total 243.37 last 258.76) - /unlink 61858 (time 1746136126.93 total 253.38 last 205.63) - /unlink 64161 (time 1746136136.93 total 263.38 last 230.24) total: 64768 /unlink in 265.72 seconds: 243.75 ops/second Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg441-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg441-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg441-server stop mds service on oleg441-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg441-server stop mds service on oleg441-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg441-server unloading modules via unload_modules_local on: 'oleg441-server' oleg441-server: oleg441-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg441-client: oleg441-client: ssh exited with exit code 2 pdsh@oleg441-client: oleg441-server: ssh exited with exit code 2 pdsh@oleg441-client: oleg441-client: ssh exited with exit code 2 pdsh@oleg441-client: oleg441-server: ssh exited with exit code 2