== replay-dual test 26: dbench and tar with mds failover ========================================================== 02:06:08 (1789452368) Starting client oleg652-client.virtnet: -o user_xattr,flock 192.168.206.152@tcp:/lustre /mnt/lustre Started clients oleg652-client.virtnet: 192.168.206.152@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) Started tar loop with pid 65597 Started dbench loop with 65598 striped dir -i0 -c2 -H crush /mnt/lustre2/d26.replay-dual/run_dbench striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d26.replay-dual/run_tar looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100' on /mnt/lustre at Tue Sep 15 02:06:13 EDT 2026 waiting for dbench pid 65646 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 100 seconds with load 'client.txt' and minimum warmup 20 secs failed to create barrier semaphore 0 of 1 processes prepared for launch 0 sec 1 of 1 processes prepared for launch 0 sec releasing clients UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 1944 1269360 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1756 1269548 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 1564 3605456 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 13832 3579672 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 15396 7185128 1% /mnt/lustre 1 25 0.71 MB/sec warmup 1 sec latency 21.411 ms 1 82 1.82 MB/sec warmup 2 sec latency 118.238 ms 1 129 1.91 MB/sec warmup 3 sec latency 173.443 ms 1 161 1.64 MB/sec warmup 4 sec latency 142.807 ms 1 185 1.47 MB/sec warmup 5 sec latency 442.016 ms 1 199 1.30 MB/sec warmup 6 sec latency 704.660 ms 1 240 1.30 MB/sec warmup 7 sec latency 136.307 ms 1 287 1.33 MB/sec warmup 8 sec latency 110.869 ms 1 318 1.27 MB/sec warmup 9 sec latency 136.689 ms test_26 fail mds1 1 times 1 350 1.25 MB/sec warmup 10 sec latency 139.939 ms Failing mds1 on oleg652-server 1 397 1.28 MB/sec warmup 11 sec latency 138.683 ms Stopping /mnt/lustre-mds1 (opts:) on oleg652-server 1 436 1.28 MB/sec warmup 12 sec latency 132.010 ms 1 468 1.26 MB/sec warmup 13 sec latency 204.862 ms 1 468 1.17 MB/sec warmup 14 sec latency 1205.596 ms 1 468 1.09 MB/sec warmup 15 sec latency 2205.803 ms 02:06:28 (1789452388) shut down facet: mds1 facet_host: oleg652-server facet_failover_host: oleg652-server 1 468 1.02 MB/sec warmup 16 sec latency 3205.998 ms 1 468 0.96 MB/sec warmup 17 sec latency 4206.294 ms 1 468 0.91 MB/sec warmup 18 sec latency 5209.319 ms 1 468 0.86 MB/sec warmup 19 sec latency 6209.693 ms 1 468 0.00 MB/sec execute 1 sec latency 8210.051 ms 1 468 0.00 MB/sec execute 2 sec latency 9210.287 ms 1 468 0.00 MB/sec execute 3 sec latency 10214.332 ms 1 468 0.00 MB/sec execute 4 sec latency 11214.547 ms 1 468 0.00 MB/sec execute 5 sec latency 12216.431 ms Failover mds1 to oleg652-server mount facets: mds1 1 468 0.00 MB/sec execute 6 sec latency 13219.999 ms 1 468 0.00 MB/sec execute 7 sec latency 14220.265 ms 1 468 0.00 MB/sec execute 8 sec latency 15221.399 ms 1 468 0.00 MB/sec execute 9 sec latency 16221.558 ms 1 468 0.00 MB/sec execute 10 sec latency 17221.875 ms 1 468 0.00 MB/sec execute 11 sec latency 18223.856 ms 1 468 0.00 MB/sec execute 12 sec latency 19226.599 ms 1 468 0.00 MB/sec execute 13 sec latency 20226.963 ms 1 468 0.00 MB/sec execute 14 sec latency 21227.192 ms 1 468 0.00 MB/sec execute 15 sec latency 22228.032 ms 1 468 0.00 MB/sec execute 16 sec latency 23230.351 ms Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 1 468 0.00 MB/sec execute 17 sec latency 24234.286 ms 1 468 0.00 MB/sec execute 18 sec latency 25234.511 ms 1 468 0.00 MB/sec execute 19 sec latency 26236.538 ms 1 468 0.00 MB/sec execute 20 sec latency 27236.714 ms 1 468 0.00 MB/sec execute 21 sec latency 28239.610 ms 1 468 0.00 MB/sec execute 22 sec latency 29239.948 ms 1 468 0.00 MB/sec execute 23 sec latency 30240.967 ms 1 468 0.00 MB/sec execute 24 sec latency 31246.038 ms 1 468 0.00 MB/sec execute 25 sec latency 32246.429 ms 1 468 0.00 MB/sec execute 26 sec latency 33247.041 ms 1 468 0.00 MB/sec execute 27 sec latency 34247.954 ms 1 468 0.00 MB/sec execute 28 sec latency 35252.827 ms oleg652-server: oleg652-server.virtnet: executing set_default_debug -1 all 1 468 0.00 MB/sec execute 29 sec latency 36254.938 ms pdsh@oleg652-client: oleg652-server: ssh exited with exit code 1 1 468 0.00 MB/sec execute 30 sec latency 37255.144 ms Started lustre-MDT0000 02:07:03 (1789452423) targets are mounted 02:07:03 (1789452423) facet_failover done 1 468 0.00 MB/sec execute 31 sec latency 38255.583 ms 1 503 0.03 MB/sec execute 32 sec latency 38366.146 ms 1 552 0.08 MB/sec execute 33 sec latency 215.864 ms 1 612 0.13 MB/sec execute 34 sec latency 556.447 ms 1 671 0.14 MB/sec execute 35 sec latency 101.955 ms 1 706 0.13 MB/sec execute 36 sec latency 127.307 ms oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 1 736 0.13 MB/sec execute 37 sec latency 110.857 ms 1 776 0.13 MB/sec execute 38 sec latency 106.935 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 806 0.13 MB/sec execute 39 sec latency 140.681 ms 1 851 0.13 MB/sec execute 40 sec latency 103.056 ms 1 892 0.13 MB/sec execute 41 sec latency 95.573 ms 1 923 0.13 MB/sec execute 42 sec latency 98.505 ms 1 967 0.13 MB/sec execute 43 sec latency 101.029 ms 1 1007 0.13 MB/sec execute 44 sec latency 93.854 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2144 1269160 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 1948 1269356 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 10480 3594428 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 44136 3558840 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 54616 7153268 1% /mnt/lustre 1 1074 0.14 MB/sec execute 45 sec latency 81.323 ms 1 1132 0.14 MB/sec execute 46 sec latency 86.026 ms 1 1172 0.14 MB/sec execute 47 sec latency 232.746 ms 1 1177 0.14 MB/sec execute 48 sec latency 1007.267 ms 1 1214 0.14 MB/sec execute 49 sec latency 104.961 ms 1 1244 0.13 MB/sec execute 50 sec latency 146.934 ms 1 1274 0.13 MB/sec execute 51 sec latency 115.237 ms 1 1410 0.18 MB/sec execute 52 sec latency 158.932 ms 1 1468 0.18 MB/sec execute 53 sec latency 91.213 ms test_26 fail mds2 2 times 1 1511 0.18 MB/sec execute 54 sec latency 149.188 ms 1 1560 0.18 MB/sec execute 55 sec latency 85.006 ms Failing mds2 on oleg652-server 1 1597 0.17 MB/sec execute 56 sec latency 66.826 ms Stopping /mnt/lustre-mds2 (opts:) on oleg652-server 1 1647 0.17 MB/sec execute 57 sec latency 77.742 ms 1 1686 0.17 MB/sec execute 58 sec latency 115.085 ms 1 1770 0.17 MB/sec execute 59 sec latency 109.792 ms 02:07:33 (1789452453) shut down facet: mds2 facet_host: oleg652-server facet_failover_host: oleg652-server 1 1836 0.17 MB/sec execute 60 sec latency 105.676 ms 1 1924 0.16 MB/sec execute 61 sec latency 75.550 ms 1 2008 0.16 MB/sec execute 62 sec latency 105.126 ms 1 2108 0.16 MB/sec execute 63 sec latency 99.971 ms 1 2184 0.16 MB/sec execute 64 sec latency 63.697 ms 1 2259 0.16 MB/sec execute 65 sec latency 265.340 ms 1 2259 0.16 MB/sec execute 66 sec latency 1266.438 ms 1 2259 0.15 MB/sec execute 67 sec latency 2269.599 ms 1 2259 0.15 MB/sec execute 68 sec latency 3269.762 ms 1 2259 0.15 MB/sec execute 69 sec latency 4270.125 ms Failover mds2 to oleg652-server 1 2259 0.15 MB/sec execute 70 sec latency 5270.314 ms mount facets: mds2 1 2259 0.15 MB/sec execute 71 sec latency 6270.947 ms 1 2259 0.14 MB/sec execute 72 sec latency 7271.206 ms 1 2259 0.14 MB/sec execute 73 sec latency 8271.364 ms 1 2259 0.14 MB/sec execute 74 sec latency 9271.600 ms 1 2259 0.14 MB/sec execute 75 sec latency 10271.771 ms 1 2259 0.14 MB/sec execute 76 sec latency 11273.352 ms 1 2259 0.13 MB/sec execute 77 sec latency 12273.571 ms 1 2259 0.13 MB/sec execute 78 sec latency 13273.835 ms Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 1 2259 0.13 MB/sec execute 79 sec latency 14274.139 ms 1 2259 0.13 MB/sec execute 80 sec latency 15274.955 ms 1 2259 0.13 MB/sec execute 81 sec latency 16275.411 ms 1 2259 0.13 MB/sec execute 82 sec latency 17276.458 ms 1 2259 0.12 MB/sec execute 83 sec latency 18277.652 ms 1 2259 0.12 MB/sec execute 84 sec latency 19277.885 ms 1 2259 0.12 MB/sec execute 85 sec latency 20278.106 ms oleg652-server: oleg652-server.virtnet: executing set_default_debug -1 all 1 2259 0.12 MB/sec execute 86 sec latency 21279.494 ms pdsh@oleg652-client: oleg652-server: ssh exited with exit code 1 1 2259 0.12 MB/sec execute 87 sec latency 22279.847 ms Started lustre-MDT0001 02:08:01 (1789452481) targets are mounted 02:08:01 (1789452481) facet_failover done 1 2305 0.13 MB/sec execute 88 sec latency 22715.206 ms 1 2356 0.13 MB/sec execute 89 sec latency 170.674 ms 1 2393 0.13 MB/sec execute 90 sec latency 118.673 ms 1 2428 0.13 MB/sec execute 91 sec latency 101.831 ms 1 2455 0.13 MB/sec execute 92 sec latency 189.404 ms 1 2489 0.13 MB/sec execute 93 sec latency 139.546 ms 1 2566 0.13 MB/sec execute 94 sec latency 149.029 ms oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 1 2597 0.13 MB/sec execute 95 sec latency 130.532 ms 1 2758 0.16 MB/sec execute 96 sec latency 112.428 ms mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec 1 2879 0.16 MB/sec execute 97 sec latency 114.839 ms 1 2923 0.17 MB/sec execute 98 sec latency 101.186 ms 1 2990 0.17 MB/sec execute 99 sec latency 97.825 ms 1 cleanup 100 sec 1 cleanup 101 sec 1 cleanup 102 sec 1 cleanup 103 sec UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2344 1268960 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2144 1269160 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 12536 3594036 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 55632 3548500 2% /mnt/lustre[OST:1] filesystem_summary: 7666232 68168 7142536 1% /mnt/lustre 0 cleanup 104 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 409 102.038 22715.190 Close 342 122.678 38366.116 Rename 21 88.878 189.383 Unlink 47 25.699 57.151 Qpathinfo 397 15.955 97.660 Qfileinfo 105 1.989 24.484 Qfsinfo 58 3.398 24.178 Sfileinfo 50 65.182 136.810 Find 134 11.215 42.020 WriteX 328 12.666 62.220 ReadX 769 1.359 59.355 LockX 2 8.958 11.544 UnlockX 2 20.921 33.040 Flush 38 57.809 170.661 Throughput 0.172442 MB/sec 1 clients 1 procs max_latency=38366.146 ms stopping dbench on /mnt/lustre at Tue Sep 15 02:08:17 EDT 2026 with return code 0 clean dbench files on /mnt/lustre /mnt/lustre /mnt/lustre removed 'client.txt' /mnt/lustre dbench successfully finished striped dir -i0 -c2 -H crush /mnt/lustre2/d26.replay-dual/run_dbench looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100' on /mnt/lustre at Tue Sep 15 02:08:26 EDT 2026 waiting for dbench pid 67890 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 100 seconds with load 'client.txt' and minimum warmup 20 secs 0 of 1 processes prepared for launch 0 sec 1 of 1 processes prepared for launch 0 sec releasing clients test_26 fail mds1 3 times 1 29 0.98 MB/sec warmup 1 sec latency 15.745 ms Failing mds1 on oleg652-server 1 74 1.57 MB/sec warmup 2 sec latency 134.492 ms Stopping /mnt/lustre-mds1 (opts:) on oleg652-server 1 128 1.92 MB/sec warmup 3 sec latency 138.713 ms 1 155 1.64 MB/sec warmup 4 sec latency 339.037 ms 1 155 1.31 MB/sec warmup 5 sec latency 1339.297 ms 1 155 1.09 MB/sec warmup 6 sec latency 2339.621 ms 1 155 0.93 MB/sec warmup 7 sec latency 3339.825 ms 1 155 0.82 MB/sec warmup 8 sec latency 4340.080 ms 1 155 0.73 MB/sec warmup 9 sec latency 5340.317 ms 1 155 0.65 MB/sec warmup 10 sec latency 6340.530 ms 1 155 0.59 MB/sec warmup 11 sec latency 7340.784 ms 02:08:37 (1789452517) shut down facet: mds1 facet_host: oleg652-server facet_failover_host: oleg652-server 1 155 0.55 MB/sec warmup 12 sec latency 8341.348 ms 1 155 0.50 MB/sec warmup 13 sec latency 9341.841 ms 1 155 0.47 MB/sec warmup 14 sec latency 10343.851 ms 1 155 0.44 MB/sec warmup 15 sec latency 11344.365 ms 1 155 0.41 MB/sec warmup 16 sec latency 12344.575 ms 1 155 0.38 MB/sec warmup 17 sec latency 13344.767 ms 1 155 0.36 MB/sec warmup 18 sec latency 14344.939 ms 1 155 0.34 MB/sec warmup 19 sec latency 15345.466 ms 1 155 0.00 MB/sec execute 1 sec latency 17347.257 ms Failover mds1 to oleg652-server mount facets: mds1 1 155 0.00 MB/sec execute 2 sec latency 18349.043 ms 1 155 0.00 MB/sec execute 3 sec latency 19368.948 ms 1 155 0.00 MB/sec execute 4 sec latency 20369.761 ms 1 155 0.00 MB/sec execute 5 sec latency 21392.993 ms 1 155 0.00 MB/sec execute 6 sec latency 22393.375 ms 1 155 0.00 MB/sec execute 7 sec latency 23395.297 ms 1 155 0.00 MB/sec execute 8 sec latency 24410.110 ms 1 155 0.00 MB/sec execute 9 sec latency 25410.492 ms 1 155 0.00 MB/sec execute 10 sec latency 26410.691 ms 1 155 0.00 MB/sec execute 11 sec latency 27411.103 ms 1 155 0.00 MB/sec execute 12 sec latency 28412.040 ms Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 1 155 0.00 MB/sec execute 13 sec latency 29412.190 ms 1 155 0.00 MB/sec execute 14 sec latency 30412.423 ms 1 155 0.00 MB/sec execute 15 sec latency 31414.940 ms 1 155 0.00 MB/sec execute 16 sec latency 32415.784 ms 1 155 0.00 MB/sec execute 17 sec latency 33416.035 ms 1 155 0.00 MB/sec execute 18 sec latency 34416.216 ms 1 155 0.00 MB/sec execute 19 sec latency 35427.829 ms 1 155 0.00 MB/sec execute 20 sec latency 36429.154 ms 1 155 0.00 MB/sec execute 21 sec latency 37430.686 ms 1 155 0.00 MB/sec execute 22 sec latency 38431.119 ms 1 155 0.00 MB/sec execute 23 sec latency 39432.256 ms 1 155 0.00 MB/sec execute 24 sec latency 40433.784 ms 1 155 0.00 MB/sec execute 25 sec latency 41434.015 ms oleg652-server: oleg652-server.virtnet: executing set_default_debug -1 all 1 155 0.00 MB/sec execute 26 sec latency 42434.389 ms pdsh@oleg652-client: oleg652-server: ssh exited with exit code 1 1 155 0.00 MB/sec execute 27 sec latency 43436.024 ms Started lustre-MDT0000 02:09:13 (1789452553) targets are mounted 02:09:13 (1789452553) facet_failover done 1 155 0.00 MB/sec execute 28 sec latency 44436.348 ms 1 184 0.03 MB/sec execute 29 sec latency 44643.684 ms 1 224 0.07 MB/sec execute 30 sec latency 351.584 ms 1 279 0.13 MB/sec execute 31 sec latency 88.615 ms 1 329 0.17 MB/sec execute 32 sec latency 110.839 ms oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 1 373 0.20 MB/sec execute 33 sec latency 112.469 ms 1 413 0.24 MB/sec execute 34 sec latency 120.805 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 447 0.26 MB/sec execute 35 sec latency 153.673 ms 1 493 0.30 MB/sec execute 36 sec latency 104.308 ms 1 531 0.32 MB/sec execute 37 sec latency 139.090 ms 1 639 0.38 MB/sec execute 38 sec latency 109.163 ms 1 689 0.37 MB/sec execute 39 sec latency 89.824 ms 1 720 0.37 MB/sec execute 40 sec latency 126.749 ms UUID 1K-blocks Used Available Use% Mounted on 1 762 0.36 MB/sec execute 41 sec latency 111.297 ms lustre-MDT0000_UUID 1414116 2856 1268448 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2624 1268680 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 30692 3568148 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 25520 3579892 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 56212 7148040 1% /mnt/lustre 1 794 0.35 MB/sec execute 42 sec latency 106.617 ms 1 825 0.35 MB/sec execute 43 sec latency 106.615 ms 1 884 0.34 MB/sec execute 44 sec latency 81.902 ms 1 884 0.34 MB/sec execute 45 sec latency 1049.656 ms 1 913 0.33 MB/sec execute 46 sec latency 1104.868 ms 1 954 0.32 MB/sec execute 47 sec latency 181.602 ms 1 990 0.32 MB/sec execute 48 sec latency 92.008 ms 1 1060 0.33 MB/sec execute 49 sec latency 87.222 ms test_26 fail mds2 4 times 1 1118 0.33 MB/sec execute 50 sec latency 90.805 ms 1 1155 0.32 MB/sec execute 51 sec latency 135.981 ms Failing mds2 on oleg652-server 1 1193 0.32 MB/sec execute 52 sec latency 116.785 ms Stopping /mnt/lustre-mds2 (opts:) on oleg652-server 1 1228 0.31 MB/sec execute 53 sec latency 96.288 ms 1 1270 0.31 MB/sec execute 54 sec latency 106.823 ms 1 1278 0.30 MB/sec execute 55 sec latency 825.060 ms 02:09:42 (1789452582) shut down facet: mds2 facet_host: oleg652-server facet_failover_host: oleg652-server 1 1278 0.30 MB/sec execute 56 sec latency 1826.913 ms 1 1278 0.29 MB/sec execute 57 sec latency 2827.118 ms 1 1278 0.29 MB/sec execute 58 sec latency 3827.369 ms 1 1278 0.28 MB/sec execute 59 sec latency 4827.614 ms 1 1278 0.28 MB/sec execute 60 sec latency 5828.357 ms 1 1278 0.27 MB/sec execute 61 sec latency 6828.674 ms 1 1278 0.27 MB/sec execute 62 sec latency 7828.929 ms 1 1278 0.26 MB/sec execute 63 sec latency 8829.831 ms 1 1278 0.26 MB/sec execute 64 sec latency 9831.603 ms 1 1278 0.25 MB/sec execute 65 sec latency 10831.773 ms 1 1278 0.25 MB/sec execute 66 sec latency 11832.292 ms Failover mds2 to oleg652-server mount facets: mds2 1 1278 0.25 MB/sec execute 67 sec latency 12832.535 ms 1 1278 0.24 MB/sec execute 68 sec latency 13832.749 ms 1 1278 0.24 MB/sec execute 69 sec latency 14834.271 ms 1 1278 0.24 MB/sec execute 70 sec latency 15836.918 ms 1 1278 0.23 MB/sec execute 71 sec latency 16837.164 ms 1 1278 0.23 MB/sec execute 72 sec latency 17838.295 ms 1 1278 0.23 MB/sec execute 73 sec latency 18839.428 ms 1 1278 0.22 MB/sec execute 74 sec latency 19839.775 ms Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 1 1278 0.22 MB/sec execute 75 sec latency 20841.144 ms 1 1278 0.22 MB/sec execute 76 sec latency 21844.390 ms 1 1278 0.22 MB/sec execute 77 sec latency 22845.103 ms 1 1278 0.21 MB/sec execute 78 sec latency 23846.020 ms 1 1278 0.21 MB/sec execute 79 sec latency 24846.233 ms oleg652-server: oleg652-server.virtnet: executing set_default_debug -1 all 1 1278 0.21 MB/sec execute 80 sec latency 25852.338 ms pdsh@oleg652-client: oleg652-server: ssh exited with exit code 1 1 1278 0.20 MB/sec execute 81 sec latency 26853.726 ms Started lustre-MDT0001 02:10:08 (1789452608) targets are mounted 02:10:08 (1789452608) facet_failover done 1 1278 0.20 MB/sec execute 82 sec latency 27854.960 ms 1 1394 0.23 MB/sec execute 83 sec latency 28308.144 ms 1 1425 0.23 MB/sec execute 84 sec latency 113.150 ms 1 1470 0.23 MB/sec execute 85 sec latency 305.365 ms 1 1519 0.23 MB/sec execute 86 sec latency 90.721 ms 1 1591 0.22 MB/sec execute 87 sec latency 67.403 ms oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 1 1646 0.22 MB/sec execute 88 sec latency 87.591 ms 1 1689 0.22 MB/sec execute 89 sec latency 145.334 ms mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec 1 1759 0.22 MB/sec execute 90 sec latency 105.779 ms 1 1814 0.22 MB/sec execute 91 sec latency 109.984 ms 1 1858 0.21 MB/sec execute 92 sec latency 96.710 ms 1 1911 0.21 MB/sec execute 93 sec latency 105.174 ms 1 1988 0.21 MB/sec execute 94 sec latency 70.782 ms 1 2024 0.21 MB/sec execute 95 sec latency 109.156 ms tar: Unexpected EOF in archive tar: Unexpected EOF in archive 1 2115 0.21 MB/sec execute 96 sec latency 78.901 ms 1 2162 0.21 MB/sec execute 97 sec latency 102.630 ms 1 2221 0.21 MB/sec execute 98 sec latency 82.749 ms 1 2327 0.21 MB/sec execute 99 sec latency 88.414 ms 1 cleanup 100 sec 1 cleanup 101 sec 1 cleanup 102 sec 1 cleanup 103 sec 0 cleanup 103 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 375 173.309 44643.668 Close 331 11.443 68.921 Rename 12 88.257 145.309 Unlink 31 27.010 50.594 Qpathinfo 311 14.662 106.799 Qfileinfo 113 1.650 13.562 Qfsinfo 38 4.285 38.434 Sfileinfo 38 54.978 116.760 Find 104 282.816 28308.131 WriteX 450 13.008 88.211 ReadX 379 0.671 71.520 Flush 31 50.972 105.212 Throughput 0.214585 MB/sec 1 clients 1 procs max_latency=44643.684 ms stopping dbench on /mnt/lustre at Tue Sep 15 02:10:29 EDT 2026 with return code 0 clean dbench files on /mnt/lustre /mnt/lustre /mnt/lustre removed 'client.txt' /mnt/lustre dbench successfully finished striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre2/d26.replay-dual/run_dbench tar: Error is not recoverable: exiting now