== replay-dual test 26: dbench and tar with mds failover ========================================================== 11:12:19 (1782227539) Starting client oleg234-client.virtnet: -o user_xattr,flock 192.168.202.134@tcp:/lustre /mnt/lustre Started clients oleg234-client.virtnet: 192.168.202.134@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Started tar loop with pid 44968 Started dbench loop with 44969 looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210560 2048 2206464 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 14336 3738624 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 2048 3762176 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 16384 7500800 1% /mnt/lustre '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100' on /mnt/lustre at Tue Jun 23 11:12:25 EDT 2026 waiting for dbench pid 45037 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 100 seconds with load 'client.txt' and minimum warmup 20 secs failed to create barrier semaphore 0 of 1 processes prepared for launch 0 sec 1 of 1 processes prepared for launch 0 sec releasing clients 1 14 0.00 MB/sec warmup 1 sec latency 222.971 ms 1 72 1.57 MB/sec warmup 2 sec latency 251.825 ms 1 113 1.74 MB/sec warmup 3 sec latency 206.754 ms 1 146 1.57 MB/sec warmup 4 sec latency 181.283 ms 1 181 1.46 MB/sec warmup 5 sec latency 133.837 ms test_26 fail mds1 1 times 1 207 1.35 MB/sec warmup 6 sec latency 176.561 ms 1 235 1.27 MB/sec warmup 7 sec latency 206.396 ms Failing mds1 on oleg234-server 1 263 1.24 MB/sec warmup 8 sec latency 151.864 ms Stopping /mnt/lustre-mds1 (opts:) on oleg234-server 1 290 1.19 MB/sec warmup 9 sec latency 169.729 ms 1 322 1.17 MB/sec warmup 10 sec latency 134.439 ms 1 322 1.07 MB/sec warmup 11 sec latency 1093.626 ms 1 322 0.98 MB/sec warmup 12 sec latency 2094.290 ms 1 322 0.90 MB/sec warmup 13 sec latency 3094.660 ms 1 322 0.84 MB/sec warmup 14 sec latency 4094.894 ms 11:12:40 (1782227560) shut down facet: mds1 facet_host: oleg234-server facet_failover_host: oleg234-server 1 322 0.78 MB/sec warmup 15 sec latency 5097.804 ms 1 322 0.73 MB/sec warmup 16 sec latency 6098.028 ms 1 322 0.69 MB/sec warmup 17 sec latency 7098.302 ms 1 322 0.65 MB/sec warmup 18 sec latency 8099.259 ms 1 322 0.62 MB/sec warmup 19 sec latency 9099.575 ms 1 322 0.00 MB/sec execute 1 sec latency 11103.548 ms 1 322 0.00 MB/sec execute 2 sec latency 12106.641 ms 1 322 0.00 MB/sec execute 3 sec latency 13106.869 ms 1 322 0.00 MB/sec execute 4 sec latency 14108.702 ms 1 322 0.00 MB/sec execute 5 sec latency 15120.568 ms Failover mds1 to oleg234-server mount facets: mds1 1 322 0.00 MB/sec execute 6 sec latency 16123.840 ms 1 322 0.00 MB/sec execute 7 sec latency 17124.244 ms 1 322 0.00 MB/sec execute 8 sec latency 18124.833 ms Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 1 322 0.00 MB/sec execute 9 sec latency 19125.082 ms 1 322 0.00 MB/sec execute 10 sec latency 20125.480 ms 1 322 0.00 MB/sec execute 11 sec latency 21125.827 ms 1 322 0.00 MB/sec execute 12 sec latency 22126.088 ms 1 322 0.00 MB/sec execute 13 sec latency 23127.867 ms 1 322 0.00 MB/sec execute 14 sec latency 24128.697 ms 1 322 0.00 MB/sec execute 15 sec latency 25129.012 ms 1 322 0.00 MB/sec execute 16 sec latency 26129.365 ms oleg234-server: oleg234-server.virtnet: executing set_default_debug -1 all 1 322 0.00 MB/sec execute 17 sec latency 27132.353 ms pdsh@oleg234-client: oleg234-server: ssh exited with exit code 1 1 322 0.00 MB/sec execute 18 sec latency 28133.453 ms Started lustre-MDT0000 11:13:04 (1782227584) targets are mounted 11:13:04 (1782227584) facet_failover done 1 322 0.00 MB/sec execute 19 sec latency 29134.443 ms 1 322 0.00 MB/sec execute 20 sec latency 30136.365 ms 1 322 0.00 MB/sec execute 21 sec latency 31136.887 ms 1 322 0.00 MB/sec execute 22 sec latency 32137.132 ms 1 322 0.00 MB/sec execute 23 sec latency 33137.429 ms 1 322 0.00 MB/sec execute 24 sec latency 34137.676 ms 1 322 0.00 MB/sec execute 25 sec latency 35140.286 ms 1 322 0.00 MB/sec execute 26 sec latency 36140.523 ms 1 322 0.00 MB/sec execute 27 sec latency 37140.733 ms 1 322 0.00 MB/sec execute 28 sec latency 38140.931 ms 1 322 0.00 MB/sec execute 29 sec latency 39141.925 ms 1 323 0.00 MB/sec execute 30 sec latency 39919.155 ms 1 360 0.03 MB/sec execute 31 sec latency 325.590 ms 1 401 0.08 MB/sec execute 32 sec latency 140.553 ms 1 437 0.11 MB/sec execute 33 sec latency 122.076 ms 1 472 0.14 MB/sec execute 34 sec latency 193.014 ms 1 503 0.16 MB/sec execute 35 sec latency 140.101 ms oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1 534 0.19 MB/sec execute 36 sec latency 194.484 ms 1 611 0.25 MB/sec execute 37 sec latency 141.592 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 659 0.25 MB/sec execute 38 sec latency 109.282 ms 1 676 0.24 MB/sec execute 39 sec latency 237.626 ms 1 710 0.24 MB/sec execute 40 sec latency 136.217 ms 1 730 0.23 MB/sec execute 41 sec latency 181.789 ms 1 762 0.23 MB/sec execute 42 sec latency 144.001 ms 1 790 0.23 MB/sec execute 43 sec latency 236.444 ms 1 808 0.22 MB/sec execute 44 sec latency 193.175 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210560 2304 2206208 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 35840 3704832 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 15360 3718144 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 51200 7422976 1% /mnt/lustre 1 851 0.22 MB/sec execute 45 sec latency 157.922 ms 1 877 0.22 MB/sec execute 46 sec latency 221.057 ms 1 902 0.21 MB/sec execute 47 sec latency 147.271 ms 1 916 0.21 MB/sec execute 48 sec latency 237.109 ms 1 962 0.21 MB/sec execute 49 sec latency 85.683 ms 1 983 0.20 MB/sec execute 50 sec latency 327.333 ms test_26 fail mds1 2 times 1 1055 0.22 MB/sec execute 51 sec latency 152.096 ms 1 1099 0.22 MB/sec execute 52 sec latency 102.533 ms Failing mds1 on oleg234-server 1 1138 0.21 MB/sec execute 53 sec latency 99.177 ms Stopping /mnt/lustre-mds1 (opts:) on oleg234-server 1 1176 0.21 MB/sec execute 54 sec latency 111.825 ms 1 1196 0.21 MB/sec execute 55 sec latency 425.628 ms 1 1196 0.20 MB/sec execute 56 sec latency 1425.782 ms 1 1196 0.20 MB/sec execute 57 sec latency 2426.186 ms 11:13:43 (1782227623) shut down facet: mds1 facet_host: oleg234-server facet_failover_host: oleg234-server 1 1196 0.19 MB/sec execute 58 sec latency 3426.538 ms 1 1196 0.19 MB/sec execute 59 sec latency 4426.930 ms 1 1196 0.19 MB/sec execute 60 sec latency 5427.253 ms 1 1196 0.18 MB/sec execute 61 sec latency 6427.795 ms 1 1196 0.18 MB/sec execute 62 sec latency 7431.360 ms 1 1196 0.18 MB/sec execute 63 sec latency 8431.551 ms 1 1196 0.18 MB/sec execute 64 sec latency 9431.799 ms 1 1196 0.17 MB/sec execute 65 sec latency 10432.157 ms 1 1196 0.17 MB/sec execute 66 sec latency 11432.352 ms 1 1196 0.17 MB/sec execute 67 sec latency 12432.625 ms Failover mds1 to oleg234-server mount facets: mds1 1 1196 0.17 MB/sec execute 68 sec latency 13434.224 ms 1 1196 0.16 MB/sec execute 69 sec latency 14435.242 ms 1 1196 0.16 MB/sec execute 70 sec latency 15435.606 ms 1 1196 0.16 MB/sec execute 71 sec latency 16435.907 ms Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 1 1196 0.16 MB/sec execute 72 sec latency 17436.181 ms 1 1196 0.15 MB/sec execute 73 sec latency 18436.777 ms 1 1196 0.15 MB/sec execute 74 sec latency 19438.314 ms 1 1196 0.15 MB/sec execute 75 sec latency 20441.575 ms 1 1196 0.15 MB/sec execute 76 sec latency 21442.903 ms 1 1196 0.15 MB/sec execute 77 sec latency 22444.250 ms 1 1196 0.14 MB/sec execute 78 sec latency 23444.813 ms 1 1196 0.14 MB/sec execute 79 sec latency 24445.233 ms 1 1196 0.14 MB/sec execute 80 sec latency 25446.107 ms 1 1196 0.14 MB/sec execute 81 sec latency 26447.108 ms 1 1196 0.14 MB/sec execute 82 sec latency 27448.871 ms 1 1196 0.14 MB/sec execute 83 sec latency 28452.606 ms 1 1196 0.13 MB/sec execute 84 sec latency 29452.934 ms 1 1196 0.13 MB/sec execute 85 sec latency 30453.166 ms 1 1196 0.13 MB/sec execute 86 sec latency 31454.417 ms 1 1196 0.13 MB/sec execute 87 sec latency 32454.830 ms 1 1196 0.13 MB/sec execute 88 sec latency 33457.838 ms 1 1196 0.13 MB/sec execute 89 sec latency 34458.385 ms oleg234-server: oleg234-server.virtnet: executing set_default_debug -1 all 1 1196 0.13 MB/sec execute 90 sec latency 35458.794 ms 1 1196 0.12 MB/sec execute 91 sec latency 36459.115 ms pdsh@oleg234-client: oleg234-server: ssh exited with exit code 1 1 1196 0.12 MB/sec execute 92 sec latency 37459.380 ms Started lustre-MDT0000 11:14:18 (1782227658) targets are mounted 11:14:18 (1782227658) facet_failover done 1 1196 0.12 MB/sec execute 93 sec latency 38459.697 ms 1 1196 0.12 MB/sec execute 94 sec latency 39459.849 ms 1 1196 0.12 MB/sec execute 95 sec latency 40460.091 ms 1 1200 0.12 MB/sec execute 96 sec latency 41161.863 ms 1 1229 0.12 MB/sec execute 97 sec latency 131.214 ms 1 1253 0.12 MB/sec execute 98 sec latency 150.309 ms 1 1384 0.14 MB/sec execute 99 sec latency 105.832 ms 1 cleanup 100 sec 1 cleanup 101 sec oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1 cleanup 102 sec 1 cleanup 103 sec mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 0 cleanup 104 sec 0 cleanup 104 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 150 80.551 325.570 Close 134 310.600 39919.007 Rename 10 112.910 237.079 Unlink 17 30.544 63.397 Qpathinfo 99 441.828 41161.827 Qfileinfo 58 2.606 13.290 Qfsinfo 33 3.778 20.984 Sfileinfo 34 66.346 136.986 Find 48 7.946 75.744 WriteX 266 14.409 97.242 ReadX 224 3.176 77.803 Flush 24 152.024 327.310 Throughput 0.143765 MB/sec 1 clients 1 procs max_latency=41161.863 ms stopping dbench on /mnt/lustre at Tue Jun 23 11:14:30 EDT 2026 with return code 0 clean dbench files on /mnt/lustre /mnt/lustre /mnt/lustre removed 'client.txt' /mnt/lustre dbench successfully finished looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210560 2560 2205952 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 26624 3720192 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 24576 3724288 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 51200 7444480 1% /mnt/lustre '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100' on /mnt/lustre at Tue Jun 23 11:14:36 EDT 2026 waiting for dbench pid 47043 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 100 seconds with load 'client.txt' and minimum warmup 20 secs 0 of 1 processes prepared for launch 0 sec 1 of 1 processes prepared for launch 0 sec releasing clients 1 31 1.05 MB/sec warmup 1 sec latency 23.052 ms 1 84 1.90 MB/sec warmup 2 sec latency 126.671 ms 1 121 1.83 MB/sec warmup 3 sec latency 202.078 ms 1 155 1.63 MB/sec warmup 4 sec latency 137.591 ms 1 184 1.48 MB/sec warmup 5 sec latency 168.868 ms 1 207 1.35 MB/sec warmup 6 sec latency 185.448 ms test_26 fail mds1 3 times 1 239 1.30 MB/sec warmup 7 sec latency 153.671 ms 1 271 1.27 MB/sec warmup 8 sec latency 153.072 ms Failing mds1 on oleg234-server 1 305 1.24 MB/sec warmup 9 sec latency 138.961 ms Stopping /mnt/lustre-mds1 (opts:) on oleg234-server 1 339 1.22 MB/sec warmup 10 sec latency 182.870 ms 1 357 1.16 MB/sec warmup 11 sec latency 390.705 ms 1 357 1.06 MB/sec warmup 12 sec latency 1391.537 ms 1 357 0.98 MB/sec warmup 13 sec latency 2391.813 ms 11:14:50 (1782227690) shut down facet: mds1 facet_host: oleg234-server facet_failover_host: oleg234-server 1 357 0.91 MB/sec warmup 14 sec latency 3392.001 ms 1 357 0.85 MB/sec warmup 15 sec latency 4392.166 ms 1 357 0.80 MB/sec warmup 16 sec latency 5393.508 ms 1 357 0.75 MB/sec warmup 17 sec latency 6393.865 ms 1 357 0.71 MB/sec warmup 18 sec latency 7394.064 ms 1 357 0.67 MB/sec warmup 19 sec latency 8394.482 ms 1 357 0.00 MB/sec execute 1 sec latency 10399.989 ms 1 357 0.00 MB/sec execute 2 sec latency 11401.011 ms 1 357 0.00 MB/sec execute 3 sec latency 12401.542 ms 1 357 0.00 MB/sec execute 4 sec latency 13402.093 ms Failover mds1 to oleg234-server mount facets: mds1 1 357 0.00 MB/sec execute 5 sec latency 14402.288 ms 1 357 0.00 MB/sec execute 6 sec latency 15402.786 ms 1 357 0.00 MB/sec execute 7 sec latency 16403.112 ms 1 357 0.00 MB/sec execute 8 sec latency 17407.104 ms Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 1 357 0.00 MB/sec execute 9 sec latency 18407.386 ms 1 357 0.00 MB/sec execute 10 sec latency 19407.658 ms 1 357 0.00 MB/sec execute 11 sec latency 20407.939 ms 1 357 0.00 MB/sec execute 12 sec latency 21409.827 ms 1 357 0.00 MB/sec execute 13 sec latency 22414.103 ms 1 357 0.00 MB/sec execute 14 sec latency 23414.544 ms oleg234-server: oleg234-server.virtnet: executing set_default_debug -1 all 1 357 0.00 MB/sec execute 15 sec latency 24414.736 ms pdsh@oleg234-client: oleg234-server: ssh exited with exit code 1 1 357 0.00 MB/sec execute 16 sec latency 25415.022 ms Started lustre-MDT0000 11:15:13 (1782227713) targets are mounted 11:15:13 (1782227713) facet_failover done 1 357 0.00 MB/sec execute 17 sec latency 26415.675 ms 1 357 0.00 MB/sec execute 18 sec latency 27416.897 ms 1 357 0.00 MB/sec execute 19 sec latency 28417.239 ms 1 357 0.00 MB/sec execute 20 sec latency 29417.721 ms 1 357 0.00 MB/sec execute 21 sec latency 30419.454 ms 1 357 0.00 MB/sec execute 22 sec latency 31419.718 ms 1 357 0.00 MB/sec execute 23 sec latency 32420.917 ms 1 357 0.00 MB/sec execute 24 sec latency 33421.382 ms 1 357 0.00 MB/sec execute 25 sec latency 34422.321 ms 1 357 0.00 MB/sec execute 26 sec latency 35423.661 ms 1 357 0.00 MB/sec execute 27 sec latency 36427.784 ms 1 357 0.00 MB/sec execute 28 sec latency 37430.509 ms 1 357 0.00 MB/sec execute 29 sec latency 38431.024 ms 1 357 0.00 MB/sec execute 30 sec latency 39434.427 ms 1 381 0.03 MB/sec execute 31 sec latency 39615.914 ms 1 421 0.06 MB/sec execute 32 sec latency 148.374 ms 1 461 0.10 MB/sec execute 33 sec latency 142.513 ms 1 493 0.13 MB/sec execute 34 sec latency 124.065 ms 1 522 0.15 MB/sec execute 35 sec latency 135.441 ms oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1 606 0.23 MB/sec execute 36 sec latency 166.999 ms 1 655 0.22 MB/sec execute 37 sec latency 106.782 ms mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 676 0.22 MB/sec execute 38 sec latency 314.072 ms 1 708 0.22 MB/sec execute 39 sec latency 130.440 ms 1 733 0.21 MB/sec execute 40 sec latency 228.108 ms 1 772 0.21 MB/sec execute 41 sec latency 166.769 ms 1 799 0.21 MB/sec execute 42 sec latency 222.513 ms 1 845 0.20 MB/sec execute 43 sec latency 124.211 ms UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210560 2944 2205568 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 36864 3705856 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3722240 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 68608 7428096 1% /mnt/lustre 1 874 0.20 MB/sec execute 44 sec latency 324.818 ms 1 900 0.20 MB/sec execute 45 sec latency 144.423 ms 1 921 0.20 MB/sec execute 46 sec latency 184.444 ms 1 969 0.19 MB/sec execute 47 sec latency 88.803 ms 1 988 0.19 MB/sec execute 48 sec latency 204.436 ms 1 1059 0.21 MB/sec execute 49 sec latency 98.277 ms test_26 fail mds1 4 times 1 1116 0.20 MB/sec execute 50 sec latency 75.600 ms 1 1146 0.20 MB/sec execute 51 sec latency 123.718 ms Failing mds1 on oleg234-server 1 1190 0.20 MB/sec execute 52 sec latency 162.042 ms Stopping /mnt/lustre-mds1 (opts:) on oleg234-server 1 1225 0.19 MB/sec execute 53 sec latency 105.129 ms 1 1236 0.19 MB/sec execute 54 sec latency 555.187 ms 1 1236 0.19 MB/sec execute 55 sec latency 1555.404 ms 1 1236 0.18 MB/sec execute 56 sec latency 2555.668 ms 11:15:53 (1782227753) shut down facet: mds1 facet_host: oleg234-server facet_failover_host: oleg234-server 1 1236 0.18 MB/sec execute 57 sec latency 3555.975 ms 1 1236 0.18 MB/sec execute 58 sec latency 4556.280 ms 1 1236 0.17 MB/sec execute 59 sec latency 5556.769 ms 1 1236 0.17 MB/sec execute 60 sec latency 6556.977 ms 1 1236 0.17 MB/sec execute 61 sec latency 7557.615 ms 1 1236 0.17 MB/sec execute 62 sec latency 8557.822 ms 1 1236 0.16 MB/sec execute 63 sec latency 9558.118 ms 1 1236 0.16 MB/sec execute 64 sec latency 10558.652 ms 1 1236 0.16 MB/sec execute 65 sec latency 11559.624 ms 1 1236 0.16 MB/sec execute 66 sec latency 12560.508 ms Failover mds1 to oleg234-server mount facets: mds1 1 1236 0.15 MB/sec execute 67 sec latency 13561.016 ms 1 1236 0.15 MB/sec execute 68 sec latency 14561.281 ms 1 1236 0.15 MB/sec execute 69 sec latency 15563.159 ms 1 1236 0.15 MB/sec execute 70 sec latency 16563.348 ms 1 1236 0.14 MB/sec execute 71 sec latency 17563.583 ms Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 1 1236 0.14 MB/sec execute 72 sec latency 18563.853 ms 1 1236 0.14 MB/sec execute 73 sec latency 19564.459 ms 1 1236 0.14 MB/sec execute 74 sec latency 20565.167 ms 1 1236 0.14 MB/sec execute 75 sec latency 21569.000 ms 1 1236 0.14 MB/sec execute 76 sec latency 22569.464 ms 1 1236 0.13 MB/sec execute 77 sec latency 23570.132 ms 1 1236 0.13 MB/sec execute 78 sec latency 24570.399 ms 1 1236 0.13 MB/sec execute 79 sec latency 25571.730 ms 1 1236 0.13 MB/sec execute 80 sec latency 26572.888 ms 1 1236 0.13 MB/sec execute 81 sec latency 27573.189 ms 1 1236 0.13 MB/sec execute 82 sec latency 28573.453 ms 1 1236 0.12 MB/sec execute 83 sec latency 29577.314 ms 1 1236 0.12 MB/sec execute 84 sec latency 30577.558 ms 1 1236 0.12 MB/sec execute 85 sec latency 31577.833 ms 1 1236 0.12 MB/sec execute 86 sec latency 32578.943 ms oleg234-server: oleg234-server.virtnet: executing set_default_debug -1 all 1 1236 0.12 MB/sec execute 87 sec latency 33580.372 ms pdsh@oleg234-client: oleg234-server: ssh exited with exit code 1 1 1236 0.12 MB/sec execute 88 sec latency 34581.981 ms Started lustre-MDT0000 11:16:25 (1782227785) targets are mounted 11:16:25 (1782227785) facet_failover done 1 1236 0.12 MB/sec execute 89 sec latency 35583.067 ms 1 1236 0.11 MB/sec execute 90 sec latency 36583.293 ms 1 1236 0.11 MB/sec execute 91 sec latency 37584.443 ms 1 1236 0.11 MB/sec execute 92 sec latency 38585.974 ms 1 1236 0.11 MB/sec execute 93 sec latency 39586.792 ms 1 1236 0.11 MB/sec execute 94 sec latency 40587.061 ms 1 1236 0.11 MB/sec execute 95 sec latency 41588.280 ms 1 1251 0.11 MB/sec execute 96 sec latency 42051.128 ms 1 1318 0.12 MB/sec execute 97 sec latency 101.418 ms 1 1387 0.13 MB/sec execute 98 sec latency 485.562 ms 1 1419 0.13 MB/sec execute 99 sec latency 118.597 ms 1 cleanup 100 sec oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1 cleanup 101 sec 1 cleanup 102 sec mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 1 cleanup 103 sec 0 cleanup 104 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 154 329.074 39615.898 Close 141 14.127 40.808 Rename 10 101.182 162.020 Unlink 19 34.083 55.495 Qpathinfo 108 24.257 198.949 Qfileinfo 55 3.009 26.160 Qfsinfo 33 3.129 18.273 Sfileinfo 34 1297.568 42051.032 Find 49 7.073 66.848 WriteX 248 13.511 66.517 ReadX 230 2.568 61.916 Flush 24 137.070 324.753 Throughput 0.133461 MB/sec 1 clients 1 procs max_latency=42051.128 ms stopping dbench on /mnt/lustre at Tue Jun 23 11:16:41 EDT 2026 with return code 0 clean dbench files on /mnt/lustre /mnt/lustre /mnt/lustre removed 'client.txt' /mnt/lustre dbench successfully finished looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 1 -D /mnt/lustre2/d26.replay-dual/run_dbench -t 100' on /mnt/lustre at Tue Jun 23 11:16:46 EDT 2026 waiting for dbench pid 49025 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 100 seconds with load 'client.txt' and minimum warmup 20 secs 0 of 1 processes prepared for launch 0 sec 1 of 1 processes prepared for launch 0 sec releasing clients 1 39 1.37 MB/sec warmup 1 sec latency 118.569 ms 1 94 2.09 MB/sec warmup 2 sec latency 131.135 ms 1 137 2.01 MB/sec warmup 3 sec latency 159.942 ms 1 173 1.76 MB/sec warmup 4 sec latency 126.649 ms 1 214 1.67 MB/sec warmup 5 sec latency 142.602 ms tar: Unexpected EOF in archive tar: Unexpected EOF in archive 1 255 1.61 MB/sec warmup 6 sec latency 143.483 ms 1 289 1.53 MB/sec warmup 7 sec latency 206.639 ms