== replay-dual test 26: dbench and tar with mds failover ========================================================== 22:48:35 (1789354115) Starting client oleg347-client.virtnet: -o user_xattr,flock 192.168.203.147@tcp:/lustre /mnt/lustre Started clients oleg347-client.virtnet: 192.168.203.147@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) Started tar loop with pid 65500 Started dbench loop with 65501 striped dir -i0 -c2 -H all_char /mnt/lustre2/d26.replay-dual/run_dbench striped dir -i0 -c2 -H all_char /mnt/lustre/d26.replay-dual/run_tar looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100' on /mnt/lustre at Sun Sep 13 22:48:37 EDT 2026 waiting for dbench pid 65544 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 100 seconds with load 'client.txt' and minimum warmup 20 secs failed to create barrier semaphore 0 of 1 processes prepared for launch 0 sec 1 of 1 processes prepared for launch 0 sec releasing clients 1 121 5.50 MB/sec warmup 1 sec latency 40.420 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 1992 1269312 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1792 1269512 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 1564 3605456 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 26120 3579620 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 27684 7185076 1% /mnt/lustre 1 207 4.05 MB/sec warmup 2 sec latency 150.927 ms 1 315 3.83 MB/sec warmup 3 sec latency 41.751 ms 1 413 3.65 MB/sec warmup 4 sec latency 72.361 ms 1 539 3.79 MB/sec warmup 5 sec latency 72.351 ms test_26 fail mds1 1 times 1 713 3.53 MB/sec warmup 6 sec latency 43.828 ms Failing mds1 on oleg347-server Stopping /mnt/lustre-mds1 (opts:) on oleg347-server 1 796 3.07 MB/sec warmup 7 sec latency 65.356 ms 1 878 2.71 MB/sec warmup 8 sec latency 244.622 ms 22:48:45 (1789354125) shut down facet: mds1 facet_host: oleg347-server facet_failover_host: oleg347-server 1 921 2.42 MB/sec warmup 9 sec latency 579.047 ms 1 995 2.19 MB/sec warmup 10 sec latency 558.783 ms 1 995 1.99 MB/sec warmup 11 sec latency 1558.978 ms 1 995 1.83 MB/sec warmup 12 sec latency 2559.144 ms 1 995 1.69 MB/sec warmup 13 sec latency 3559.316 ms 1 995 1.56 MB/sec warmup 14 sec latency 4559.520 ms 1 995 1.46 MB/sec warmup 15 sec latency 5559.746 ms 1 995 1.37 MB/sec warmup 16 sec latency 6559.936 ms 1 995 1.29 MB/sec warmup 17 sec latency 7560.056 ms 1 995 1.22 MB/sec warmup 18 sec latency 8560.243 ms Failover mds1 to oleg347-server mount facets: mds1 1 995 1.15 MB/sec warmup 19 sec latency 9560.383 ms 1 995 0.00 MB/sec execute 1 sec latency 11560.592 ms Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 1 995 0.00 MB/sec execute 2 sec latency 12560.758 ms 1 995 0.00 MB/sec execute 3 sec latency 13560.907 ms oleg347-server: oleg347-server.virtnet: executing set_default_debug -1 all pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1 1 995 0.00 MB/sec execute 4 sec latency 14561.055 ms Started lustre-MDT0000 22:49:01 (1789354141) targets are mounted 22:49:01 (1789354141) facet_failover done 1 995 0.00 MB/sec execute 5 sec latency 15561.202 ms 1 995 0.00 MB/sec execute 6 sec latency 16561.365 ms 1 995 0.00 MB/sec execute 7 sec latency 17561.482 ms 1 999 0.01 MB/sec execute 8 sec latency 18517.306 ms 1 1132 0.12 MB/sec execute 9 sec latency 78.844 ms oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 1 1259 0.12 MB/sec execute 10 sec latency 33.731 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 1467 0.37 MB/sec execute 11 sec latency 48.823 ms 1 1597 0.35 MB/sec execute 12 sec latency 166.158 ms 1 1723 0.33 MB/sec execute 13 sec latency 52.744 ms 1 1868 0.32 MB/sec execute 14 sec latency 42.986 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2376 1268928 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2176 1269128 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 20688 3584100 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 38032 3567012 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 58720 7151112 1% /mnt/lustre 1 1998 0.31 MB/sec execute 15 sec latency 48.597 ms 1 2137 0.30 MB/sec execute 16 sec latency 46.841 ms 1 2296 0.34 MB/sec execute 17 sec latency 36.649 ms 1 2428 0.34 MB/sec execute 18 sec latency 33.884 ms 1 2583 0.37 MB/sec execute 19 sec latency 82.893 ms test_26 fail mds2 2 times Failing mds2 on oleg347-server Stopping /mnt/lustre-mds2 (opts:) on oleg347-server 1 2902 0.55 MB/sec execute 20 sec latency 33.265 ms 1 3232 0.59 MB/sec execute 21 sec latency 35.721 ms 1 3549 0.75 MB/sec execute 22 sec latency 19.786 ms 1 3767 0.78 MB/sec execute 23 sec latency 18.451 ms 1 3948 0.76 MB/sec execute 24 sec latency 21.925 ms 1 4083 0.73 MB/sec execute 25 sec latency 263.912 ms 1 4083 0.70 MB/sec execute 26 sec latency 1265.276 ms 22:49:24 (1789354164) shut down facet: mds2 facet_host: oleg347-server facet_failover_host: oleg347-server 1 4083 0.67 MB/sec execute 27 sec latency 2265.449 ms 1 4083 0.65 MB/sec execute 28 sec latency 3265.642 ms 1 4083 0.63 MB/sec execute 29 sec latency 4265.811 ms 1 4083 0.61 MB/sec execute 30 sec latency 5265.995 ms 1 4083 0.59 MB/sec execute 31 sec latency 6266.229 ms 1 4083 0.57 MB/sec execute 32 sec latency 7266.397 ms 1 4083 0.55 MB/sec execute 33 sec latency 8266.576 ms 1 4083 0.53 MB/sec execute 34 sec latency 9266.728 ms 1 4083 0.52 MB/sec execute 35 sec latency 10266.857 ms 1 4083 0.50 MB/sec execute 36 sec latency 11267.027 ms Failover mds2 to oleg347-server mount facets: mds2 1 4083 0.49 MB/sec execute 37 sec latency 12267.220 ms 1 4083 0.48 MB/sec execute 38 sec latency 13267.425 ms 1 4083 0.47 MB/sec execute 39 sec latency 14267.576 ms Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 1 4083 0.45 MB/sec execute 40 sec latency 15267.730 ms 1 4083 0.44 MB/sec execute 41 sec latency 16267.918 ms oleg347-server: oleg347-server.virtnet: executing set_default_debug -1 all 1 4083 0.43 MB/sec execute 42 sec latency 17268.104 ms pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1 Started lustre-MDT0001 22:49:40 (1789354180) targets are mounted 22:49:40 (1789354180) facet_failover done 1 4083 0.42 MB/sec execute 43 sec latency 18268.302 ms 1 4083 0.41 MB/sec execute 44 sec latency 19268.500 ms 1 4083 0.40 MB/sec execute 45 sec latency 20268.797 ms 1 4083 0.40 MB/sec execute 46 sec latency 21269.295 ms 1 4179 0.39 MB/sec execute 47 sec latency 21516.024 ms oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 1 4306 0.39 MB/sec execute 48 sec latency 56.910 ms mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec 1 4434 0.38 MB/sec execute 49 sec latency 50.431 ms 1 4541 0.38 MB/sec execute 50 sec latency 50.863 ms 1 4712 0.40 MB/sec execute 51 sec latency 30.057 ms 1 4840 0.39 MB/sec execute 52 sec latency 43.276 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2908 1268396 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2700 1268604 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 22324 3583964 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 48044 3556348 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 70368 7140312 1% /mnt/lustre 1 5042 0.44 MB/sec execute 53 sec latency 44.028 ms 1 5174 0.43 MB/sec execute 54 sec latency 30.232 ms 1 5300 0.43 MB/sec execute 55 sec latency 42.754 ms 1 5426 0.42 MB/sec execute 56 sec latency 37.003 ms 1 5579 0.42 MB/sec execute 57 sec latency 37.496 ms test_26 fail mds1 3 times Failing mds1 on oleg347-server 1 5722 0.41 MB/sec execute 58 sec latency 34.981 ms Stopping /mnt/lustre-mds1 (opts:) on oleg347-server 1 5820 0.41 MB/sec execute 59 sec latency 387.296 ms 1 5820 0.40 MB/sec execute 60 sec latency 1387.838 ms 1 5820 0.39 MB/sec execute 61 sec latency 2388.031 ms 1 5820 0.39 MB/sec execute 62 sec latency 3388.184 ms 1 5820 0.38 MB/sec execute 63 sec latency 4388.347 ms 1 5820 0.37 MB/sec execute 64 sec latency 5388.526 ms 1 5820 0.37 MB/sec execute 65 sec latency 6388.687 ms 22:50:03 (1789354203) shut down facet: mds1 facet_host: oleg347-server facet_failover_host: oleg347-server 1 5820 0.36 MB/sec execute 66 sec latency 7388.851 ms 1 5820 0.36 MB/sec execute 67 sec latency 8389.034 ms 1 5820 0.35 MB/sec execute 68 sec latency 9389.300 ms 1 5820 0.35 MB/sec execute 69 sec latency 10389.641 ms 1 5820 0.34 MB/sec execute 70 sec latency 11389.841 ms 1 5820 0.34 MB/sec execute 71 sec latency 12390.006 ms 1 5820 0.33 MB/sec execute 72 sec latency 13390.185 ms 1 5820 0.33 MB/sec execute 73 sec latency 14390.351 ms 1 5820 0.32 MB/sec execute 74 sec latency 15390.509 ms 1 5820 0.32 MB/sec execute 75 sec latency 16390.681 ms Failover mds1 to oleg347-server mount facets: mds1 1 5820 0.32 MB/sec execute 76 sec latency 17390.868 ms 1 5820 0.31 MB/sec execute 77 sec latency 18391.149 ms 1 5820 0.31 MB/sec execute 78 sec latency 19391.322 ms Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 1 5820 0.30 MB/sec execute 79 sec latency 20391.471 ms 1 5820 0.30 MB/sec execute 80 sec latency 21391.569 ms oleg347-server: oleg347-server.virtnet: executing set_default_debug -1 all 1 5820 0.30 MB/sec execute 81 sec latency 22392.693 ms pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1 Started lustre-MDT0000 22:50:19 (1789354219) targets are mounted 22:50:19 (1789354219) facet_failover done 1 5820 0.29 MB/sec execute 82 sec latency 23392.854 ms 1 5820 0.29 MB/sec execute 83 sec latency 24393.036 ms 1 5820 0.29 MB/sec execute 84 sec latency 25393.213 ms 1 5820 0.28 MB/sec execute 85 sec latency 26393.358 ms 1 5937 0.29 MB/sec execute 86 sec latency 26700.581 ms oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 1 6043 0.30 MB/sec execute 87 sec latency 61.703 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 6311 0.33 MB/sec execute 88 sec latency 45.438 ms 1 6555 0.35 MB/sec execute 89 sec latency 41.230 ms 1 6897 0.36 MB/sec execute 90 sec latency 32.618 ms 1 7164 0.39 MB/sec execute 91 sec latency 37.746 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 3508 1267796 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 3216 1268088 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 23564 3581008 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 49720 3554360 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 73284 7135368 2% /mnt/lustre 1 7312 0.40 MB/sec execute 92 sec latency 44.156 ms 1 7427 0.40 MB/sec execute 93 sec latency 46.687 ms 1 7536 0.40 MB/sec execute 94 sec latency 53.737 ms 1 7649 0.39 MB/sec execute 95 sec latency 57.086 ms 1 7808 0.39 MB/sec execute 96 sec latency 29.536 ms test_26 fail mds2 4 times 1 7933 0.39 MB/sec execute 97 sec latency 55.712 ms Failing mds2 on oleg347-server Stopping /mnt/lustre-mds2 (opts:) on oleg347-server 1 8045 0.39 MB/sec execute 98 sec latency 146.050 ms 22:50:36 (1789354236) shut down facet: mds2 facet_host: oleg347-server facet_failover_host: oleg347-server 1 8045 0.38 MB/sec execute 99 sec latency 1146.342 ms 1 cleanup 100 sec 1 cleanup 101 sec 1 cleanup 102 sec 1 cleanup 103 sec 1 cleanup 104 sec 1 cleanup 105 sec 1 cleanup 106 sec 1 cleanup 107 sec 1 cleanup 108 sec Failover mds2 to oleg347-server mount facets: mds2 1 cleanup 109 sec 1 cleanup 110 sec 1 cleanup 111 sec Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 1 cleanup 112 sec 1 cleanup 113 sec oleg347-server: oleg347-server.virtnet: executing set_default_debug -1 all pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1 1 cleanup 114 sec Started lustre-MDT0001 22:50:52 (1789354252) targets are mounted 22:50:52 (1789354252) facet_failover done 1 cleanup 115 sec 1 cleanup 116 sec 1 cleanup 117 sec 1 cleanup 118 sec 1 cleanup 119 sec 0 cleanup 119 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 1213 53.878 26700.568 Close 888 3.222 11.187 Rename 51 28.033 82.868 Unlink 246 9.812 33.154 Qpathinfo 1128 42.237 21516.003 Qfileinfo 193 1.132 4.486 Qfsinfo 203 0.557 5.114 Sfileinfo 97 19.157 61.685 Find 427 4.837 29.515 WriteX 600 4.494 28.868 ReadX 1913 0.271 74.326 LockX 4 3.320 5.635 UnlockX 4 3.570 4.653 Flush 84 15.810 55.701 Throughput 0.384813 MB/sec 1 clients 1 procs max_latency=26700.581 ms stopping dbench on /mnt/lustre at Sun Sep 13 22:50:57 EDT 2026 with return code 0 clean dbench files on /mnt/lustre /mnt/lustre /mnt/lustre oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 removed 'client.txt' /mnt/lustre dbench successfully finished mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec striped dir -i0 -c2 -H crush /mnt/lustre2/d26.replay-dual/run_dbench looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100' on /mnt/lustre at Sun Sep 13 22:50:59 EDT 2026 waiting for dbench pid 69748 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 100 seconds with load 'client.txt' and minimum warmup 20 secs 0 of 1 processes prepared for launch 0 sec 1 of 1 processes prepared for launch 0 sec releasing clients 1 113 5.24 MB/sec warmup 1 sec latency 41.780 ms 1 200 3.92 MB/sec warmup 2 sec latency 170.855 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 3928 1267376 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 3708 1267596 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 37608 3568360 2% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 16376 3587148 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 53984 7155508 1% /mnt/lustre 1 315 3.83 MB/sec warmup 3 sec latency 123.763 ms 1 445 3.91 MB/sec warmup 4 sec latency 32.808 ms 1 514 3.55 MB/sec warmup 5 sec latency 374.377 ms 1 712 3.53 MB/sec warmup 6 sec latency 31.405 ms 1 845 3.08 MB/sec warmup 7 sec latency 69.767 ms test_26 fail mds1 5 times Failing mds1 on oleg347-server 1 885 2.71 MB/sec warmup 8 sec latency 656.408 ms Stopping /mnt/lustre-mds1 (opts:) on oleg347-server 1 933 2.42 MB/sec warmup 9 sec latency 517.967 ms 22:51:09 (1789354269) shut down facet: mds1 facet_host: oleg347-server facet_failover_host: oleg347-server 1 933 2.18 MB/sec warmup 10 sec latency 1518.182 ms 1 933 1.98 MB/sec warmup 11 sec latency 2518.360 ms 1 933 1.81 MB/sec warmup 12 sec latency 3518.465 ms 1 933 1.67 MB/sec warmup 13 sec latency 4518.624 ms 1 933 1.56 MB/sec warmup 14 sec latency 5518.797 ms 1 933 1.45 MB/sec warmup 15 sec latency 6518.973 ms 1 933 1.36 MB/sec warmup 16 sec latency 7519.166 ms 1 933 1.28 MB/sec warmup 17 sec latency 8519.332 ms 1 933 1.21 MB/sec warmup 18 sec latency 9519.492 ms 1 933 1.15 MB/sec warmup 19 sec latency 10519.664 ms Failover mds1 to oleg347-server mount facets: mds1 1 933 0.00 MB/sec execute 1 sec latency 12519.886 ms 1 933 0.00 MB/sec execute 2 sec latency 13520.048 ms Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 1 933 0.00 MB/sec execute 3 sec latency 14520.184 ms 1 933 0.00 MB/sec execute 4 sec latency 15520.330 ms oleg347-server: oleg347-server.virtnet: executing set_default_debug -1 all pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1 Started lustre-MDT0000 22:51:24 (1789354284) targets are mounted 22:51:24 (1789354284) facet_failover done 1 933 0.00 MB/sec execute 5 sec latency 16520.449 ms 1 933 0.00 MB/sec execute 6 sec latency 17520.568 ms 1 933 0.00 MB/sec execute 7 sec latency 18520.717 ms 1 933 0.00 MB/sec execute 8 sec latency 19520.841 ms 1 933 0.00 MB/sec execute 9 sec latency 20520.968 ms 1 1051 0.11 MB/sec execute 10 sec latency 20625.809 ms oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 1 1196 0.12 MB/sec execute 11 sec latency 34.721 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 1391 0.35 MB/sec execute 12 sec latency 40.007 ms 1 1486 0.33 MB/sec execute 13 sec latency 149.660 ms 1 1581 0.31 MB/sec execute 14 sec latency 84.990 ms 1 1709 0.30 MB/sec execute 15 sec latency 115.213 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 3320 1267984 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 3384 1267920 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 45040 3558920 2% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 21248 3583908 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 66288 7142828 1% /mnt/lustre 1 1857 0.29 MB/sec execute 16 sec latency 89.978 ms 1 2012 0.28 MB/sec execute 17 sec latency 36.916 ms 1 2160 0.27 MB/sec execute 18 sec latency 31.556 ms 1 2346 0.32 MB/sec execute 19 sec latency 55.105 ms 1 2463 0.34 MB/sec execute 20 sec latency 30.815 ms test_26 fail mds2 6 times 1 2719 0.48 MB/sec execute 21 sec latency 38.570 ms Failing mds2 on oleg347-server Stopping /mnt/lustre-mds2 (opts:) on oleg347-server 1 2931 0.53 MB/sec execute 22 sec latency 87.438 ms 22:51:42 (1789354302) shut down 1 2931 0.51 MB/sec execute 23 sec latency 1076.158 ms facet: mds2 facet_host: oleg347-server facet_failover_host: oleg347-server 1 2931 0.49 MB/sec execute 24 sec latency 2076.293 ms 1 2931 0.47 MB/sec execute 25 sec latency 3076.516 ms 1 2931 0.45 MB/sec execute 26 sec latency 4076.701 ms 1 2931 0.43 MB/sec execute 27 sec latency 5076.846 ms 1 2931 0.42 MB/sec execute 28 sec latency 6076.974 ms 1 2931 0.40 MB/sec execute 29 sec latency 7077.114 ms 1 2931 0.39 MB/sec execute 30 sec latency 8077.256 ms 1 2931 0.38 MB/sec execute 31 sec latency 9077.392 ms 1 2931 0.37 MB/sec execute 32 sec latency 10077.546 ms 1 2931 0.35 MB/sec execute 33 sec latency 11077.721 ms Failover mds2 to oleg347-server mount facets: mds2 1 2931 0.34 MB/sec execute 34 sec latency 12077.846 ms 1 2931 0.33 MB/sec execute 35 sec latency 13077.978 ms Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 1 2931 0.33 MB/sec execute 36 sec latency 14078.102 ms 1 2931 0.32 MB/sec execute 37 sec latency 15078.264 ms oleg347-server: oleg347-server.virtnet: executing set_default_debug -1 all 1 2931 0.31 MB/sec execute 38 sec latency 16078.442 ms pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1 Started lustre-MDT0001 22:51:58 (1789354318) targets are mounted 22:51:58 (1789354318) facet_failover done 1 2931 0.30 MB/sec execute 39 sec latency 17078.686 ms 1 2931 0.29 MB/sec execute 40 sec latency 18078.888 ms 1 2931 0.29 MB/sec execute 41 sec latency 19079.073 ms 1 2931 0.28 MB/sec execute 42 sec latency 20079.250 ms 1 3015 0.27 MB/sec execute 43 sec latency 20518.733 ms 1 3342 0.31 MB/sec execute 44 sec latency 35.062 ms oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec 1 3582 0.37 MB/sec execute 45 sec latency 29.792 ms 1 3773 0.39 MB/sec execute 46 sec latency 24.960 ms 1 3907 0.39 MB/sec execute 47 sec latency 28.486 ms 1 4044 0.38 MB/sec execute 48 sec latency 29.818 ms 1 4133 0.37 MB/sec execute 49 sec latency 88.368 ms 1 4289 0.37 MB/sec execute 50 sec latency 51.143 ms dbench killed by signal 15 stopping dbench on /mnt/lustre at Sun Sep 13 22:52:10 EDT 2026 with return code 0 69748 pts/0 S+ 0:00 dbench -c client.txt 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100 killed dbench main pid 69748 clean dbench files on /mnt/lustre /mnt/lustre /mnt/lustre removed 'client.txt' /mnt/lustre dbench successfully finished