== replay-dual test 26: dbench and tar with mds failover ========================================================== 23:22:30 (1786936950) Starting client oleg336-client.virtnet: -o user_xattr,flock 192.168.203.136@tcp:/lustre /mnt/lustre Started clients oleg336-client.virtnet: 192.168.203.136@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) Started tar loop with pid 65450 Started dbench loop with 65451 striped dir -i0 -c2 -H crush2 /mnt/lustre2/d26.replay-dual/run_dbench striped dir -i0 -c2 -H all_char /mnt/lustre/d26.replay-dual/run_tar looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100' on /mnt/lustre at Sun Aug 16 23:22:32 EDT 2026 waiting for dbench pid 65494 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 100 seconds with load 'client.txt' and minimum warmup 20 secs failed to create barrier semaphore 0 of 1 processes prepared for launch 0 sec 1 of 1 processes prepared for launch 0 sec releasing clients UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 1996 1269308 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1784 1269520 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 1564 3605456 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 26120 3579572 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 27684 7185028 1% /mnt/lustre 1 97 4.39 MB/sec warmup 1 sec latency 34.806 ms 1 184 3.69 MB/sec warmup 2 sec latency 108.324 ms 1 282 3.48 MB/sec warmup 3 sec latency 215.694 ms 1 389 3.46 MB/sec warmup 4 sec latency 42.286 ms 1 498 3.47 MB/sec warmup 5 sec latency 79.166 ms test_26 fail mds1 1 times Failing mds1 on oleg336-server Stopping /mnt/lustre-mds1 (opts:) on oleg336-server 1 640 3.51 MB/sec warmup 6 sec latency 35.479 ms 1 660 3.02 MB/sec warmup 7 sec latency 762.725 ms 23:22:39 (1786936959) shut down facet: mds1 facet_host: oleg336-server facet_failover_host: oleg336-server 1 660 2.64 MB/sec warmup 8 sec latency 1762.841 ms 1 660 2.35 MB/sec warmup 9 sec latency 2762.970 ms 1 660 2.11 MB/sec warmup 10 sec latency 3763.099 ms 1 660 1.92 MB/sec warmup 11 sec latency 4763.226 ms 1 660 1.76 MB/sec warmup 12 sec latency 5763.366 ms 1 660 1.63 MB/sec warmup 13 sec latency 6763.521 ms 1 660 1.51 MB/sec warmup 14 sec latency 7763.693 ms 1 660 1.41 MB/sec warmup 15 sec latency 8763.875 ms 1 660 1.32 MB/sec warmup 16 sec latency 9764.030 ms 1 660 1.24 MB/sec warmup 17 sec latency 10764.168 ms Failover mds1 to oleg336-server mount facets: mds1 1 660 1.17 MB/sec warmup 18 sec latency 11764.276 ms 1 660 1.11 MB/sec warmup 19 sec latency 12764.383 ms Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 1 660 0.00 MB/sec execute 1 sec latency 14764.690 ms oleg336-server: oleg336-server.virtnet: executing set_default_debug -1 all pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1 1 660 0.00 MB/sec execute 2 sec latency 15764.797 ms Started lustre-MDT0000 23:22:54 (1786936974) targets are mounted 23:22:54 (1786936974) facet_failover done 1 660 0.00 MB/sec execute 3 sec latency 16764.923 ms 1 660 0.00 MB/sec execute 4 sec latency 17765.046 ms 1 660 0.00 MB/sec execute 5 sec latency 18765.185 ms 1 660 0.00 MB/sec execute 6 sec latency 19765.316 ms 1 735 0.02 MB/sec execute 7 sec latency 20186.341 ms oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 1 884 0.07 MB/sec execute 8 sec latency 76.169 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 916 0.07 MB/sec execute 9 sec latency 625.957 ms 1 1064 0.17 MB/sec execute 10 sec latency 28.828 ms 1 1201 0.17 MB/sec execute 11 sec latency 44.084 ms 1 1389 0.40 MB/sec execute 12 sec latency 57.839 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2372 1268932 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2180 1269124 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 6132 3592784 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 38940 3558396 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 45072 7151180 1% /mnt/lustre 1 1467 0.38 MB/sec execute 13 sec latency 103.166 ms 1 1530 0.35 MB/sec execute 14 sec latency 476.030 ms 1 1603 0.33 MB/sec execute 15 sec latency 80.530 ms 1 1769 0.32 MB/sec execute 16 sec latency 38.988 ms test_26 fail mds2 2 times 1 1930 0.31 MB/sec execute 17 sec latency 34.308 ms Failing mds2 on oleg336-server Stopping /mnt/lustre-mds2 (opts:) on oleg336-server 1 2069 0.30 MB/sec execute 18 sec latency 71.946 ms 1 2259 0.30 MB/sec execute 19 sec latency 337.963 ms 1 2259 0.28 MB/sec execute 20 sec latency 1338.087 ms 1 2259 0.27 MB/sec execute 21 sec latency 2338.211 ms 1 2259 0.26 MB/sec execute 22 sec latency 3338.330 ms 1 2259 0.25 MB/sec execute 23 sec latency 4338.462 ms 1 2259 0.23 MB/sec execute 24 sec latency 5338.603 ms 23:23:17 (1786936997) shut down facet: mds2 facet_host: oleg336-server facet_failover_host: oleg336-server 1 2259 0.23 MB/sec execute 25 sec latency 6338.721 ms 1 2259 0.22 MB/sec execute 26 sec latency 7338.839 ms 1 2259 0.21 MB/sec execute 27 sec latency 8338.962 ms 1 2259 0.20 MB/sec execute 28 sec latency 9339.088 ms 1 2259 0.19 MB/sec execute 29 sec latency 10339.217 ms 1 2259 0.19 MB/sec execute 30 sec latency 11339.354 ms 1 2259 0.18 MB/sec execute 31 sec latency 12339.493 ms 1 2259 0.18 MB/sec execute 32 sec latency 13339.636 ms 1 2259 0.17 MB/sec execute 33 sec latency 14339.766 ms 1 2259 0.17 MB/sec execute 34 sec latency 15339.901 ms Failover mds2 to oleg336-server mount facets: mds2 1 2259 0.16 MB/sec execute 35 sec latency 16340.043 ms 1 2259 0.16 MB/sec execute 36 sec latency 17340.163 ms Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 1 2259 0.15 MB/sec execute 37 sec latency 18340.280 ms 1 2259 0.15 MB/sec execute 38 sec latency 19340.400 ms oleg336-server: oleg336-server.virtnet: executing set_default_debug -1 all pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1 1 2259 0.14 MB/sec execute 39 sec latency 20340.535 ms Started lustre-MDT0001 23:23:31 (1786937011) targets are mounted 23:23:31 (1786937011) facet_failover done 1 2259 0.14 MB/sec execute 40 sec latency 21340.690 ms 1 2259 0.14 MB/sec execute 41 sec latency 22340.849 ms 1 2259 0.13 MB/sec execute 42 sec latency 23340.996 ms 1 2259 0.13 MB/sec execute 43 sec latency 24341.110 ms 1 2418 0.16 MB/sec execute 44 sec latency 24448.641 ms oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec 1 2596 0.17 MB/sec execute 45 sec latency 32.950 ms 1 2912 0.26 MB/sec execute 46 sec latency 43.322 ms 1 3209 0.28 MB/sec execute 47 sec latency 31.249 ms 1 3516 0.36 MB/sec execute 48 sec latency 48.796 ms 1 3696 0.36 MB/sec execute 49 sec latency 37.208 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2864 1268440 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2680 1268624 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 22472 3581820 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 47584 3556504 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 70056 7138324 1% /mnt/lustre 1 3844 0.37 MB/sec execute 50 sec latency 51.834 ms 1 3970 0.37 MB/sec execute 51 sec latency 33.106 ms 1 4076 0.36 MB/sec execute 52 sec latency 58.366 ms 1 4232 0.36 MB/sec execute 53 sec latency 40.665 ms test_26 fail mds1 3 times Failing mds1 on oleg336-server 1 4362 0.36 MB/sec execute 54 sec latency 43.815 ms Stopping /mnt/lustre-mds1 (opts:) on oleg336-server 1 4419 0.36 MB/sec execute 55 sec latency 674.662 ms 23:23:48 (1786937028) shut down facet: mds1 facet_host: oleg336-server facet_failover_host: oleg336-server 1 4419 0.35 MB/sec execute 56 sec latency 1674.805 ms 1 4419 0.34 MB/sec execute 57 sec latency 2674.948 ms 1 4419 0.34 MB/sec execute 58 sec latency 3675.076 ms 1 4419 0.33 MB/sec execute 59 sec latency 4675.209 ms 1 4419 0.33 MB/sec execute 60 sec latency 5675.330 ms 1 4419 0.32 MB/sec execute 61 sec latency 6675.478 ms 1 4419 0.32 MB/sec execute 62 sec latency 7675.618 ms 1 4419 0.31 MB/sec execute 63 sec latency 8675.733 ms 1 4419 0.31 MB/sec execute 64 sec latency 9675.896 ms 1 4419 0.30 MB/sec execute 65 sec latency 10676.050 ms Failover mds1 to oleg336-server mount facets: mds1 1 4419 0.30 MB/sec execute 66 sec latency 11676.197 ms 1 4419 0.29 MB/sec execute 67 sec latency 12676.328 ms 1 4419 0.29 MB/sec execute 68 sec latency 13676.491 ms Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 1 4419 0.28 MB/sec execute 69 sec latency 14676.629 ms oleg336-server: oleg336-server.virtnet: executing set_default_debug -1 all 1 4419 0.28 MB/sec execute 70 sec latency 15676.782 ms pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1 Started lustre-MDT0000 23:24:03 (1786937043) targets are mounted 23:24:03 (1786937043) facet_failover done 1 4419 0.28 MB/sec execute 71 sec latency 16676.940 ms 1 4419 0.27 MB/sec execute 72 sec latency 17677.176 ms 1 4419 0.27 MB/sec execute 73 sec latency 18677.343 ms 1 4419 0.26 MB/sec execute 74 sec latency 19677.499 ms 1 4497 0.26 MB/sec execute 75 sec latency 19955.209 ms oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 1 4652 0.27 MB/sec execute 76 sec latency 50.472 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 4798 0.27 MB/sec execute 77 sec latency 33.495 ms 1 5012 0.31 MB/sec execute 78 sec latency 34.503 ms 1 5164 0.31 MB/sec execute 79 sec latency 30.165 ms 1 5338 0.30 MB/sec execute 80 sec latency 34.707 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 3412 1267892 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 3184 1268120 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 24596 3581744 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 48484 3557936 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 73080 7139680 2% /mnt/lustre 1 5519 0.30 MB/sec execute 81 sec latency 31.143 ms 1 5710 0.30 MB/sec execute 82 sec latency 26.216 ms 1 5916 0.31 MB/sec execute 83 sec latency 39.227 ms 1 6044 0.32 MB/sec execute 84 sec latency 29.230 ms test_26 fail mds2 4 times 1 6331 0.35 MB/sec execute 85 sec latency 33.147 ms Failing mds2 on oleg336-server Stopping /mnt/lustre-mds2 (opts:) on oleg336-server 1 6537 0.37 MB/sec execute 86 sec latency 362.844 ms 1 6537 0.36 MB/sec execute 87 sec latency 1363.012 ms 1 6537 0.36 MB/sec execute 88 sec latency 2363.148 ms 1 6537 0.35 MB/sec execute 89 sec latency 3363.309 ms 1 6537 0.35 MB/sec execute 90 sec latency 4363.479 ms 1 6537 0.35 MB/sec execute 91 sec latency 5363.627 ms 1 6537 0.34 MB/sec execute 92 sec latency 6363.797 ms 23:24:25 (1786937065) shut down facet: mds2 facet_host: oleg336-server facet_failover_host: oleg336-server 1 6537 0.34 MB/sec execute 93 sec latency 7363.961 ms 1 6537 0.33 MB/sec execute 94 sec latency 8364.098 ms 1 6537 0.33 MB/sec execute 95 sec latency 9364.229 ms 1 6537 0.33 MB/sec execute 96 sec latency 10364.381 ms 1 6537 0.32 MB/sec execute 97 sec latency 11364.506 ms 1 6537 0.32 MB/sec execute 98 sec latency 12364.642 ms 1 6537 0.32 MB/sec execute 99 sec latency 13364.773 ms 1 cleanup 100 sec 1 cleanup 101 sec 1 cleanup 102 sec Failover mds2 to oleg336-server mount facets: mds2 1 cleanup 103 sec 1 cleanup 104 sec 1 cleanup 105 sec Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 1 cleanup 106 sec oleg336-server: oleg336-server.virtnet: executing set_default_debug -1 all 1 cleanup 107 sec pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1 Started lustre-MDT0001 23:24:40 (1786937080) targets are mounted 23:24:40 (1786937080) facet_failover done 1 cleanup 108 sec 1 cleanup 109 sec 1 cleanup 110 sec 1 cleanup 111 sec 1 cleanup 112 sec 1 cleanup 113 sec 0 cleanup 113 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 994 40.873 24448.626 Close 768 28.961 20186.320 Rename 45 26.598 47.094 Unlink 168 10.010 33.218 Qpathinfo 944 5.199 46.780 Qfileinfo 178 1.216 5.127 Qfsinfo 151 0.600 5.466 Sfileinfo 98 219.909 19955.181 Find 340 5.110 18.804 WriteX 531 4.601 26.131 ReadX 1573 0.308 26.019 LockX 4 2.388 2.610 UnlockX 4 2.551 3.182 Flush 80 340.844 26199.858 Throughput 0.318028 MB/sec 1 clients 1 procs max_latency=24448.641 ms stopping dbench on /mnt/lustre at Sun Aug 16 23:24:45 EDT 2026 with return code 0 oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 clean dbench files on /mnt/lustre /mnt/lustre /mnt/lustre mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec removed 'client.txt' /mnt/lustre dbench successfully finished striped dir -i0 -c2 -H crush2 /mnt/lustre2/d26.replay-dual/run_dbench looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100' on /mnt/lustre at Sun Aug 16 23:24:48 EDT 2026 waiting for dbench pid 69748 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 100 seconds with load 'client.txt' and minimum warmup 20 secs 0 of 1 processes prepared for launch 0 sec 1 of 1 processes prepared for launch 0 sec releasing clients 1 117 5.31 MB/sec warmup 1 sec latency 39.964 ms 1 204 3.99 MB/sec warmup 2 sec latency 220.175 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 3900 1267404 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 3728 1267576 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 13816 3592560 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 38096 3567788 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 51912 7160348 1% /mnt/lustre 1 313 3.83 MB/sec warmup 3 sec latency 142.630 ms 1 420 3.72 MB/sec warmup 4 sec latency 157.389 ms 1 525 3.61 MB/sec warmup 5 sec latency 301.130 ms 1 712 3.53 MB/sec warmup 6 sec latency 43.355 ms test_26 fail mds1 5 times 1 825 3.07 MB/sec warmup 7 sec latency 60.200 ms Failing mds1 on oleg336-server Stopping /mnt/lustre-mds1 (opts:) on oleg336-server 1 885 2.71 MB/sec warmup 8 sec latency 530.433 ms 23:24:57 (1786937097) shut down facet: mds1 facet_host: oleg336-server facet_failover_host: oleg336-server 1 885 2.41 MB/sec warmup 9 sec latency 1092.827 ms 1 885 2.17 MB/sec warmup 10 sec latency 2093.003 ms 1 885 1.97 MB/sec warmup 11 sec latency 3093.170 ms 1 885 1.81 MB/sec warmup 12 sec latency 4093.341 ms 1 885 1.67 MB/sec warmup 13 sec latency 5093.495 ms 1 885 1.55 MB/sec warmup 14 sec latency 6093.649 ms 1 885 1.44 MB/sec warmup 15 sec latency 7093.867 ms 1 885 1.35 MB/sec warmup 16 sec latency 8094.081 ms 1 885 1.27 MB/sec warmup 17 sec latency 9094.230 ms 1 885 1.20 MB/sec warmup 18 sec latency 10094.471 ms Failover mds1 to oleg336-server mount facets: mds1 1 885 1.14 MB/sec warmup 19 sec latency 11094.715 ms 1 885 0.00 MB/sec execute 1 sec latency 13095.123 ms 1 885 0.00 MB/sec execute 2 sec latency 14095.366 ms Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 1 885 0.00 MB/sec execute 3 sec latency 15095.557 ms 1 885 0.00 MB/sec execute 4 sec latency 16095.661 ms 1 885 0.00 MB/sec execute 5 sec latency 17095.813 ms oleg336-server: oleg336-server.virtnet: executing set_default_debug -1 all pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1 Started lustre-MDT0000 1 885 0.00 MB/sec execute 6 sec latency 18095.981 ms 23:25:14 (1786937114) targets are mounted 23:25:14 (1786937114) facet_failover done 1 885 0.00 MB/sec execute 7 sec latency 19096.163 ms 1 885 0.00 MB/sec execute 8 sec latency 20096.357 ms 1 885 0.00 MB/sec execute 9 sec latency 21096.498 ms 1 885 0.00 MB/sec execute 10 sec latency 22096.651 ms 1 967 0.01 MB/sec execute 11 sec latency 22307.955 ms oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 1 1078 0.10 MB/sec execute 12 sec latency 89.486 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 1214 0.11 MB/sec execute 13 sec latency 39.008 ms 1 1380 0.31 MB/sec execute 14 sec latency 46.623 ms 1 1467 0.29 MB/sec execute 15 sec latency 60.208 ms 1 1534 0.28 MB/sec execute 16 sec latency 167.156 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 3360 1267944 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 3476 1267828 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 16428 3583268 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 40932 3555676 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 57360 7138944 1% /mnt/lustre 1 1574 0.26 MB/sec execute 17 sec latency 138.729 ms 1 1656 0.25 MB/sec execute 18 sec latency 132.808 ms 1 1775 0.24 MB/sec execute 19 sec latency 51.461 ms 1 1884 0.23 MB/sec execute 20 sec latency 45.948 ms 1 2010 0.23 MB/sec execute 21 sec latency 58.955 ms 1 2119 0.23 MB/sec execute 22 sec latency 40.353 ms test_26 fail mds2 6 times Failing mds2 on oleg336-server 1 2250 0.22 MB/sec execute 23 sec latency 45.787 ms Stopping /mnt/lustre-mds2 (opts:) on oleg336-server 1 2349 0.26 MB/sec execute 24 sec latency 389.405 ms 23:25:33 (1786937133) shut down facet: mds2 facet_host: oleg336-server facet_failover_host: oleg336-server 1 2349 0.25 MB/sec execute 25 sec latency 1389.607 ms 1 2349 0.24 MB/sec execute 26 sec latency 2389.840 ms 1 2349 0.23 MB/sec execute 27 sec latency 3389.986 ms 1 2349 0.22 MB/sec execute 28 sec latency 4390.085 ms 1 2349 0.21 MB/sec execute 29 sec latency 5390.175 ms 1 2349 0.21 MB/sec execute 30 sec latency 6390.266 ms 1 2349 0.20 MB/sec execute 31 sec latency 7390.367 ms 1 2349 0.19 MB/sec execute 32 sec latency 8390.494 ms 1 2349 0.19 MB/sec execute 33 sec latency 9390.655 ms 1 2349 0.18 MB/sec execute 34 sec latency 10390.793 ms Failover mds2 to oleg336-server mount facets: mds2 1 2349 0.18 MB/sec execute 35 sec latency 11390.905 ms 1 2349 0.17 MB/sec execute 36 sec latency 12391.011 ms 1 2349 0.17 MB/sec execute 37 sec latency 13391.177 ms 1 2349 0.16 MB/sec execute 38 sec latency 14391.353 ms Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 1 2349 0.16 MB/sec execute 39 sec latency 15391.608 ms 1 2349 0.16 MB/sec execute 40 sec latency 16391.866 ms 1 2349 0.15 MB/sec execute 41 sec latency 17392.059 ms oleg336-server: oleg336-server.virtnet: executing set_default_debug -1 all pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1 1 2349 0.15 MB/sec execute 42 sec latency 18392.267 ms Started lustre-MDT0001 23:25:50 (1786937150) targets are mounted 23:25:50 (1786937150) facet_failover done 1 2349 0.14 MB/sec execute 43 sec latency 19392.469 ms 1 2349 0.14 MB/sec execute 44 sec latency 20392.664 ms 1 2349 0.14 MB/sec execute 45 sec latency 21392.865 ms 1 2389 0.14 MB/sec execute 46 sec latency 21776.840 ms 1 2467 0.15 MB/sec execute 47 sec latency 54.627 ms oleg336-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec 1 2603 0.15 MB/sec execute 48 sec latency 65.496 ms 1 2902 0.23 MB/sec execute 49 sec latency 82.957 ms 1 3002 0.24 MB/sec execute 50 sec latency 80.684 ms 1 3266 0.27 MB/sec execute 51 sec latency 101.192 ms 1 3530 0.32 MB/sec execute 52 sec latency 26.946 ms 1 3696 0.32 MB/sec execute 53 sec latency 29.483 ms 1 3844 0.34 MB/sec execute 54 sec latency 40.534 ms 1 3979 0.33 MB/sec execute 55 sec latency 32.702 ms 1 4052 0.33 MB/sec execute 56 sec latency 379.082 ms 1 4230 0.32 MB/sec execute 57 sec latency 89.048 ms dbench killed by signal 15 stopping dbench on /mnt/lustre at Sun Aug 16 23:26:06 EDT 2026 with return code 0 69748 pts/0 S+ 0:00 dbench -c client.txt 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100 killed dbench main pid 69748 clean dbench files on /mnt/lustre /mnt/lustre /mnt/lustre removed 'client.txt' /mnt/lustre dbench successfully finished