== replay-dual test 26: dbench and tar with mds failover ========================================================== 17:31:58 (1788816718) Starting client oleg214-client.virtnet: -o user_xattr,flock 192.168.202.114@tcp:/lustre /mnt/lustre Started clients oleg214-client.virtnet: 192.168.202.114@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) Started tar loop with pid 65597 Started dbench loop with 65598 striped dir -i0 -c2 -H all_char /mnt/lustre2/d26.replay-dual/run_dbench striped dir -i0 -c2 -H crush /mnt/lustre/d26.replay-dual/run_tar looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100' on /mnt/lustre at Mon Sep 7 17:32:02 EDT 2026 waiting for dbench pid 65647 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 100 seconds with load 'client.txt' and minimum warmup 20 secs failed to create barrier semaphore 0 of 1 processes prepared for launch 0 sec 1 of 1 processes prepared for launch 0 sec releasing clients UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 1944 1269360 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1756 1269548 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 1564 3605456 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 22024 3579720 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 23588 7185176 1% /mnt/lustre 1 39 1.37 MB/sec warmup 1 sec latency 56.400 ms 1 122 2.75 MB/sec warmup 2 sec latency 98.404 ms 1 178 2.40 MB/sec warmup 3 sec latency 86.597 ms 1 222 2.15 MB/sec warmup 4 sec latency 211.358 ms 1 288 2.14 MB/sec warmup 5 sec latency 63.517 ms 1 348 2.08 MB/sec warmup 6 sec latency 96.067 ms test_26 fail mds1 1 times 1 413 2.08 MB/sec warmup 7 sec latency 76.102 ms 1 477 2.08 MB/sec warmup 8 sec latency 121.106 ms Failing mds1 on oleg214-server Stopping /mnt/lustre-mds1 (opts:) on oleg214-server 1 522 2.00 MB/sec warmup 9 sec latency 135.436 ms 1 600 2.09 MB/sec warmup 10 sec latency 452.051 ms 1 600 1.90 MB/sec warmup 11 sec latency 1452.240 ms 17:32:13 (1788816733) shut down facet: mds1 facet_host: oleg214-server facet_failover_host: oleg214-server 1 600 1.75 MB/sec warmup 12 sec latency 2454.463 ms 1 600 1.61 MB/sec warmup 13 sec latency 3457.069 ms 1 600 1.50 MB/sec warmup 14 sec latency 4457.617 ms 1 600 1.40 MB/sec warmup 15 sec latency 5459.375 ms 1 600 1.31 MB/sec warmup 16 sec latency 6459.742 ms 1 600 1.23 MB/sec warmup 17 sec latency 7459.998 ms 1 600 1.16 MB/sec warmup 18 sec latency 8460.370 ms 1 600 1.10 MB/sec warmup 19 sec latency 9460.677 ms 1 600 0.00 MB/sec execute 1 sec latency 11461.132 ms Failover mds1 to oleg214-server mount facets: mds1 1 600 0.00 MB/sec execute 2 sec latency 12461.289 ms 1 600 0.00 MB/sec execute 3 sec latency 13465.890 ms 1 600 0.00 MB/sec execute 4 sec latency 14466.182 ms 1 600 0.00 MB/sec execute 5 sec latency 15466.365 ms 1 600 0.00 MB/sec execute 6 sec latency 16466.689 ms 1 600 0.00 MB/sec execute 7 sec latency 17466.859 ms Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 1 600 0.00 MB/sec execute 8 sec latency 18467.356 ms 1 600 0.00 MB/sec execute 9 sec latency 19467.848 ms 1 600 0.00 MB/sec execute 10 sec latency 20471.104 ms 1 600 0.00 MB/sec execute 11 sec latency 21471.304 ms 1 600 0.00 MB/sec execute 12 sec latency 22472.259 ms 1 600 0.00 MB/sec execute 13 sec latency 23472.561 ms 1 600 0.00 MB/sec execute 14 sec latency 24472.870 ms 1 600 0.00 MB/sec execute 15 sec latency 25473.923 ms 1 600 0.00 MB/sec execute 16 sec latency 26474.057 ms 1 600 0.00 MB/sec execute 17 sec latency 27474.283 ms 1 600 0.00 MB/sec execute 18 sec latency 28474.702 ms 1 600 0.00 MB/sec execute 19 sec latency 29477.677 ms 1 600 0.00 MB/sec execute 20 sec latency 30477.944 ms 1 600 0.00 MB/sec execute 21 sec latency 31479.891 ms oleg214-server: oleg214-server.virtnet: executing set_default_debug -1 all 1 600 0.00 MB/sec execute 22 sec latency 32480.328 ms pdsh@oleg214-client: oleg214-server: ssh exited with exit code 1 Started lustre-MDT0000 17:32:45 (1788816765) targets are mounted 17:32:45 (1788816765) facet_failover done 1 600 0.00 MB/sec execute 23 sec latency 33480.565 ms 1 600 0.00 MB/sec execute 24 sec latency 34480.712 ms 1 600 0.00 MB/sec execute 25 sec latency 35483.427 ms 1 642 0.00 MB/sec execute 26 sec latency 35942.848 ms 1 704 0.01 MB/sec execute 27 sec latency 84.730 ms 1 721 0.01 MB/sec execute 28 sec latency 498.285 ms oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 1 769 0.01 MB/sec execute 29 sec latency 86.581 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 808 0.02 MB/sec execute 30 sec latency 96.725 ms 1 868 0.02 MB/sec execute 31 sec latency 88.775 ms 1 903 0.02 MB/sec execute 32 sec latency 92.884 ms 1 959 0.03 MB/sec execute 33 sec latency 82.709 ms 1 1000 0.03 MB/sec execute 34 sec latency 109.640 ms 1 1082 0.06 MB/sec execute 35 sec latency 121.912 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2152 1269152 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1944 1269360 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 21172 3585344 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 37776 3568992 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 58948 7154336 1% /mnt/lustre 1 1157 0.06 MB/sec execute 36 sec latency 56.743 ms 1 1224 0.06 MB/sec execute 37 sec latency 64.205 ms 1 1280 0.06 MB/sec execute 38 sec latency 68.867 ms 1 1410 0.13 MB/sec execute 39 sec latency 351.399 ms 1 1491 0.13 MB/sec execute 40 sec latency 62.464 ms 1 1578 0.13 MB/sec execute 41 sec latency 72.368 ms 1 1655 0.12 MB/sec execute 42 sec latency 82.087 ms test_26 fail mds2 2 times 1 1721 0.12 MB/sec execute 43 sec latency 98.297 ms Failing mds2 on oleg214-server 1 1813 0.12 MB/sec execute 44 sec latency 67.553 ms Stopping /mnt/lustre-mds2 (opts:) on oleg214-server 1 1871 0.12 MB/sec execute 45 sec latency 263.423 ms 1 1871 0.12 MB/sec execute 46 sec latency 1264.189 ms 17:33:08 (1788816788) shut down facet: mds2 facet_host: oleg214-server facet_failover_host: oleg214-server 1 1871 0.11 MB/sec execute 47 sec latency 2264.368 ms 1 1871 0.11 MB/sec execute 48 sec latency 3264.534 ms 1 1871 0.11 MB/sec execute 49 sec latency 4264.702 ms 1 1871 0.11 MB/sec execute 50 sec latency 5264.928 ms 1 1871 0.11 MB/sec execute 51 sec latency 6265.115 ms 1 1871 0.10 MB/sec execute 52 sec latency 7266.649 ms 1 1871 0.10 MB/sec execute 53 sec latency 8267.086 ms 1 1871 0.10 MB/sec execute 54 sec latency 9267.300 ms 1 1871 0.10 MB/sec execute 55 sec latency 10267.467 ms 1 1871 0.10 MB/sec execute 56 sec latency 11267.655 ms Failover mds2 to oleg214-server mount facets: mds2 1 1871 0.09 MB/sec execute 57 sec latency 12270.013 ms 1 1871 0.09 MB/sec execute 58 sec latency 13270.197 ms 1 1871 0.09 MB/sec execute 59 sec latency 14270.407 ms 1 1871 0.09 MB/sec execute 60 sec latency 15270.532 ms 1 1871 0.09 MB/sec execute 61 sec latency 16270.698 ms 1 1871 0.09 MB/sec execute 62 sec latency 17271.232 ms Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 1 1871 0.09 MB/sec execute 63 sec latency 18273.981 ms 1 1871 0.08 MB/sec execute 64 sec latency 19274.134 ms 1 1871 0.08 MB/sec execute 65 sec latency 20274.333 ms oleg214-server: oleg214-server.virtnet: executing set_default_debug -1 all 1 1871 0.08 MB/sec execute 66 sec latency 21274.579 ms pdsh@oleg214-client: oleg214-server: ssh exited with exit code 1 1 1871 0.08 MB/sec execute 67 sec latency 22274.774 ms Started lustre-MDT0001 17:33:29 (1788816809) targets are mounted 17:33:29 (1788816809) facet_failover done 1 1871 0.08 MB/sec execute 68 sec latency 23275.206 ms 1 1871 0.08 MB/sec execute 69 sec latency 24275.456 ms 1 1909 0.08 MB/sec execute 70 sec latency 24769.423 ms 1 2006 0.08 MB/sec execute 71 sec latency 67.053 ms 1 2088 0.08 MB/sec execute 72 sec latency 68.166 ms oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 1 2164 0.08 MB/sec execute 73 sec latency 63.416 ms mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec 1 2280 0.08 MB/sec execute 74 sec latency 70.600 ms 1 2386 0.09 MB/sec execute 75 sec latency 43.515 ms 1 2455 0.09 MB/sec execute 76 sec latency 61.526 ms 1 2577 0.10 MB/sec execute 77 sec latency 89.990 ms 1 2770 0.14 MB/sec execute 78 sec latency 53.682 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2520 1268784 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2308 1268996 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 31572 3572520 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 37944 3566592 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 69516 7139112 1% /mnt/lustre 1 2932 0.16 MB/sec execute 79 sec latency 64.261 ms 1 3026 0.16 MB/sec execute 80 sec latency 55.703 ms 1 3197 0.16 MB/sec execute 81 sec latency 765.525 ms 1 3320 0.18 MB/sec execute 82 sec latency 985.766 ms 1 3519 0.21 MB/sec execute 83 sec latency 40.648 ms test_26 fail mds1 3 times 1 3632 0.21 MB/sec execute 84 sec latency 41.232 ms Failing mds1 on oleg214-server 1 3715 0.21 MB/sec execute 85 sec latency 49.290 ms Stopping /mnt/lustre-mds1 (opts:) on oleg214-server 1 3792 0.22 MB/sec execute 86 sec latency 217.214 ms 1 3792 0.22 MB/sec execute 87 sec latency 1217.406 ms 1 3792 0.21 MB/sec execute 88 sec latency 2219.530 ms 1 3792 0.21 MB/sec execute 89 sec latency 3220.738 ms 1 3792 0.21 MB/sec execute 90 sec latency 4220.960 ms 1 3792 0.21 MB/sec execute 91 sec latency 5221.435 ms 1 3792 0.20 MB/sec execute 92 sec latency 6223.518 ms 17:33:55 (1788816835) shut down facet: mds1 facet_host: oleg214-server facet_failover_host: oleg214-server 1 3792 0.20 MB/sec execute 93 sec latency 7228.352 ms 1 3792 0.20 MB/sec execute 94 sec latency 8230.227 ms 1 3792 0.20 MB/sec execute 95 sec latency 9230.414 ms 1 3792 0.20 MB/sec execute 96 sec latency 10231.519 ms 1 3792 0.19 MB/sec execute 97 sec latency 11234.453 ms 1 3792 0.19 MB/sec execute 98 sec latency 12235.062 ms 1 3792 0.19 MB/sec execute 99 sec latency 13238.224 ms 1 cleanup 100 sec 1 cleanup 101 sec 1 cleanup 102 sec 1 cleanup 103 sec Failover mds1 to oleg214-server mount facets: mds1 1 cleanup 104 sec 1 cleanup 105 sec 1 cleanup 106 sec 1 cleanup 107 sec 1 cleanup 108 sec 1 cleanup 109 sec Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 1 cleanup 110 sec 1 cleanup 111 sec 1 cleanup 112 sec 1 cleanup 113 sec oleg214-server: oleg214-server.virtnet: executing set_default_debug -1 all 1 cleanup 114 sec pdsh@oleg214-client: oleg214-server: ssh exited with exit code 1 Started lustre-MDT0000 17:34:17 (1788816857) targets are mounted 17:34:17 (1788816857) facet_failover done 1 cleanup 115 sec 1 cleanup 116 sec 1 cleanup 117 sec 1 cleanup 118 sec 1 cleanup 119 sec 1 cleanup 120 sec 0 cleanup 120 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 488 147.207 31785.661 Close 405 96.186 35942.835 Rename 26 56.997 146.247 Unlink 65 15.344 39.955 Qpathinfo 479 10.233 64.194 Qfileinfo 96 1.029 11.301 Qfsinfo 94 1.636 25.892 Sfileinfo 50 39.295 93.974 Find 193 6.633 79.179 WriteX 297 7.603 50.136 ReadX 953 0.804 121.898 LockX 2 6.635 8.692 UnlockX 2 7.422 7.957 Flush 43 57.573 985.751 Throughput 0.189648 MB/sec 1 clients 1 procs max_latency=35942.848 ms stopping dbench on /mnt/lustre at Mon Sep 7 17:34:22 EDT 2026 with return code 0 clean dbench files on /mnt/lustre /mnt/lustre /mnt/lustre oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 removed 'client.txt' /mnt/lustre dbench successfully finished mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec striped dir -i0 -c2 -H crush /mnt/lustre2/d26.replay-dual/run_dbench looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100' on /mnt/lustre at Mon Sep 7 17:34:27 EDT 2026 waiting for dbench pid 68857 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 100 seconds with load 'client.txt' and minimum warmup 20 secs 0 of 1 processes prepared for launch 0 sec 1 of 1 processes prepared for launch 0 sec releasing clients 1 71 3.15 MB/sec warmup 1 sec latency 44.801 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2916 1268388 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2684 1268620 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 21868 3584600 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 27132 3577736 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 49000 7162336 1% /mnt/lustre 1 129 2.87 MB/sec warmup 2 sec latency 148.565 ms 1 166 2.26 MB/sec warmup 3 sec latency 143.003 ms 1 199 1.96 MB/sec warmup 4 sec latency 321.624 ms 1 255 1.93 MB/sec warmup 5 sec latency 96.239 ms 1 313 1.91 MB/sec warmup 6 sec latency 101.514 ms 1 342 1.75 MB/sec warmup 7 sec latency 458.393 ms 1 408 1.79 MB/sec warmup 8 sec latency 73.975 ms 1 470 1.82 MB/sec warmup 9 sec latency 86.090 ms test_26 fail mds2 4 times 1 518 1.78 MB/sec warmup 10 sec latency 105.515 ms Failing mds2 on oleg214-server 1 606 1.90 MB/sec warmup 11 sec latency 113.957 ms Stopping /mnt/lustre-mds2 (opts:) on oleg214-server 1 663 1.76 MB/sec warmup 12 sec latency 119.482 ms 1 718 1.63 MB/sec warmup 13 sec latency 70.486 ms 1 794 1.53 MB/sec warmup 14 sec latency 53.767 ms 1 882 1.44 MB/sec warmup 15 sec latency 56.084 ms 1 902 1.35 MB/sec warmup 16 sec latency 653.635 ms 1 977 1.28 MB/sec warmup 17 sec latency 63.949 ms 1 995 1.21 MB/sec warmup 18 sec latency 718.645 ms 1 995 1.15 MB/sec warmup 19 sec latency 1718.870 ms 17:34:48 (1788816888) shut down facet: mds2 facet_host: oleg214-server facet_failover_host: oleg214-server 1 995 0.00 MB/sec execute 1 sec latency 3722.036 ms 1 995 0.00 MB/sec execute 2 sec latency 4723.144 ms 1 995 0.00 MB/sec execute 3 sec latency 5723.334 ms 1 995 0.00 MB/sec execute 4 sec latency 6726.804 ms 1 995 0.00 MB/sec execute 5 sec latency 7727.038 ms 1 995 0.00 MB/sec execute 6 sec latency 8727.280 ms 1 995 0.00 MB/sec execute 7 sec latency 9728.492 ms 1 995 0.00 MB/sec execute 8 sec latency 10729.604 ms 1 995 0.00 MB/sec execute 9 sec latency 11730.011 ms 1 995 0.00 MB/sec execute 10 sec latency 12730.429 ms Failover mds2 to oleg214-server mount facets: mds2 1 995 0.00 MB/sec execute 11 sec latency 13730.679 ms 1 995 0.00 MB/sec execute 12 sec latency 14730.825 ms 1 995 0.00 MB/sec execute 13 sec latency 15730.941 ms 1 995 0.00 MB/sec execute 14 sec latency 16731.111 ms 1 995 0.00 MB/sec execute 15 sec latency 17731.330 ms Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 1 995 0.00 MB/sec execute 16 sec latency 18731.582 ms 1 995 0.00 MB/sec execute 17 sec latency 19732.183 ms 1 995 0.00 MB/sec execute 18 sec latency 20733.766 ms oleg214-server: oleg214-server.virtnet: executing set_default_debug -1 all 1 995 0.00 MB/sec execute 19 sec latency 21734.475 ms pdsh@oleg214-client: oleg214-server: ssh exited with exit code 1 1 995 0.00 MB/sec execute 20 sec latency 22734.830 ms Started lustre-MDT0001 17:35:08 (1788816908) targets are mounted 17:35:08 (1788816908) facet_failover done 1 995 0.00 MB/sec execute 21 sec latency 23735.027 ms 1 995 0.00 MB/sec execute 22 sec latency 24735.356 ms 1 1011 0.02 MB/sec execute 23 sec latency 25348.280 ms 1 1124 0.04 MB/sec execute 24 sec latency 87.168 ms 1 1180 0.04 MB/sec execute 25 sec latency 91.750 ms oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 1 1244 0.05 MB/sec execute 26 sec latency 58.520 ms mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec 1 1399 0.15 MB/sec execute 27 sec latency 73.794 ms 1 1467 0.15 MB/sec execute 28 sec latency 87.686 ms 1 1556 0.14 MB/sec execute 29 sec latency 153.187 ms 1 1656 0.14 MB/sec execute 30 sec latency 55.964 ms 1 1770 0.14 MB/sec execute 31 sec latency 73.827 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 3280 1268024 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 3152 1268152 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 32740 3573964 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 40584 3565188 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 73324 7139152 2% /mnt/lustre 1 1845 0.14 MB/sec execute 32 sec latency 66.492 ms 1 1954 0.14 MB/sec execute 33 sec latency 57.901 ms 1 2024 0.14 MB/sec execute 34 sec latency 70.011 ms 1 2137 0.14 MB/sec execute 35 sec latency 78.738 ms 1 2193 0.13 MB/sec execute 36 sec latency 75.478 ms 1 2327 0.16 MB/sec execute 37 sec latency 48.844 ms test_26 fail mds1 5 times 1 2388 0.16 MB/sec execute 38 sec latency 60.754 ms Failing mds1 on oleg214-server 1 2441 0.16 MB/sec execute 39 sec latency 73.184 ms Stopping /mnt/lustre-mds1 (opts:) on oleg214-server 1 2499 0.17 MB/sec execute 40 sec latency 77.381 ms 1 2602 0.17 MB/sec execute 41 sec latency 75.778 ms 17:35:30 (1788816930) shut down 1 2776 0.24 MB/sec execute 42 sec latency 56.043 ms facet: mds1 facet_host: oleg214-server facet_failover_host: oleg214-server 1 2934 0.27 MB/sec execute 43 sec latency 57.682 ms 1 3025 0.26 MB/sec execute 44 sec latency 78.491 ms 1 3299 0.30 MB/sec execute 45 sec latency 54.537 ms 1 3432 0.32 MB/sec execute 46 sec latency 48.861 ms 1 3590 0.35 MB/sec execute 47 sec latency 46.199 ms 1 3701 0.35 MB/sec execute 48 sec latency 59.570 ms 1 3790 0.36 MB/sec execute 49 sec latency 61.084 ms 1 3868 0.36 MB/sec execute 50 sec latency 58.829 ms 1 3944 0.36 MB/sec execute 51 sec latency 51.116 ms 1 4021 0.35 MB/sec execute 52 sec latency 54.230 ms Failover mds1 to oleg214-server mount facets: mds1 1 4073 0.34 MB/sec execute 53 sec latency 73.758 ms 1 4083 0.34 MB/sec execute 54 sec latency 815.144 ms 1 4083 0.33 MB/sec execute 55 sec latency 1816.171 ms 1 4083 0.32 MB/sec execute 56 sec latency 2816.413 ms 1 4083 0.32 MB/sec execute 57 sec latency 3817.430 ms 1 4083 0.31 MB/sec execute 58 sec latency 4817.581 ms 1 4083 0.31 MB/sec execute 59 sec latency 5819.537 ms Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 1 4083 0.30 MB/sec execute 60 sec latency 6819.962 ms 1 4083 0.30 MB/sec execute 61 sec latency 7820.187 ms 1 4083 0.29 MB/sec execute 62 sec latency 8822.844 ms 1 4083 0.29 MB/sec execute 63 sec latency 9824.941 ms 1 4083 0.28 MB/sec execute 64 sec latency 10825.202 ms 1 4083 0.28 MB/sec execute 65 sec latency 11826.232 ms 1 4083 0.28 MB/sec execute 66 sec latency 12827.691 ms 1 4083 0.27 MB/sec execute 67 sec latency 13827.980 ms 1 4083 0.27 MB/sec execute 68 sec latency 14828.205 ms 1 4083 0.26 MB/sec execute 69 sec latency 15828.424 ms 1 4083 0.26 MB/sec execute 70 sec latency 16828.863 ms 1 4083 0.26 MB/sec execute 71 sec latency 17829.939 ms oleg214-server: oleg214-server.virtnet: executing set_default_debug -1 all 1 4083 0.25 MB/sec execute 72 sec latency 18830.117 ms pdsh@oleg214-client: oleg214-server: ssh exited with exit code 1 1 4083 0.25 MB/sec execute 73 sec latency 19830.746 ms Started lustre-MDT0000 17:36:01 (1788816961) targets are mounted 17:36:01 (1788816961) facet_failover done 1 4083 0.25 MB/sec execute 74 sec latency 20831.192 ms 1 4083 0.24 MB/sec execute 75 sec latency 21832.136 ms 1 4113 0.24 MB/sec execute 76 sec latency 22397.767 ms 1 4184 0.24 MB/sec execute 77 sec latency 53.686 ms 1 4257 0.24 MB/sec execute 78 sec latency 88.047 ms oleg214-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 1 4302 0.23 MB/sec execute 79 sec latency 104.524 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 4349 0.23 MB/sec execute 80 sec latency 148.162 ms 1 4395 0.23 MB/sec execute 81 sec latency 80.608 ms 1 4472 0.23 MB/sec execute 82 sec latency 74.268 ms kill: sending signal to 65627 failed: No such process 1 4538 0.23 MB/sec execute 83 sec latency 73.789 ms 1 4609 0.24 MB/sec execute 84 sec latency 70.349 ms 1 4695 0.24 MB/sec execute 85 sec latency 49.341 ms 1 4765 0.24 MB/sec execute 86 sec latency 75.955 ms 1 4862 0.24 MB/sec execute 87 sec latency 45.006 ms 1 4992 0.26 MB/sec execute 88 sec latency 80.131 ms 1 5065 0.26 MB/sec execute 89 sec latency 100.811 ms 1 5137 0.26 MB/sec execute 90 sec latency 114.217 ms 1 5216 0.26 MB/sec execute 91 sec latency 134.050 ms 1 5278 0.25 MB/sec execute 92 sec latency 63.499 ms 1 5344 0.25 MB/sec execute 93 sec latency 74.321 ms 1 5397 0.25 MB/sec execute 94 sec latency 74.426 ms 1 5473 0.25 MB/sec execute 95 sec latency 94.575 ms 1 5551 0.25 MB/sec execute 96 sec latency 74.249 ms 1 5596 0.25 MB/sec execute 97 sec latency 83.285 ms 1 5695 0.24 MB/sec execute 98 sec latency 72.160 ms 1 5760 0.24 MB/sec execute 99 sec latency 71.051 ms dbench killed by signal 15 stopping dbench on /mnt/lustre at Mon Sep 7 17:36:27 EDT 2026 with return code 0 68857 pts/0 S+ 0:00 dbench -c client.txt 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100 killed dbench main pid 68857 clean dbench files on /mnt/lustre /mnt/lustre /mnt/lustre removed 'client.txt' /mnt/lustre dbench successfully finished