== sanity test 398b: DIO and buffer IO race ============== 13:59:14 (1721411954)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0704051 s, 179 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=28123: Fri Jul 19 13:59:18 2024
   read: IOPS=157, BW=629KiB/s (644kB/s)(1480KiB/2354msec)
    clat (usec): min=4, max=37868, avg=1588.36, stdev=3591.37
     lat (usec): min=4, max=37869, avg=1589.17, stdev=3591.36
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    8], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   14], 40.00th=[   21], 50.00th=[   58], 60.00th=[ 1385],
     | 70.00th=[ 1811], 80.00th=[ 2376], 90.00th=[ 3032], 95.00th=[ 3982],
     | 99.00th=[20579], 99.50th=[24773], 99.90th=[38011], 99.95th=[38011],
     | 99.99th=[38011]
   bw (  KiB/s): min=  408, max=  782, per=23.73%, avg=603.50, stdev=153.31, samples=4
   iops        : min=  102, max=  195, avg=150.75, stdev=38.13, samples=4
  write: IOPS=169, BW=676KiB/s (693kB/s)(1592KiB/2354msec)
    clat (usec): min=95, max=34013, avg=4416.28, stdev=6225.05
     lat (usec): min=96, max=34014, avg=4417.00, stdev=6225.06
    clat percentiles (usec):
     |  1.00th=[  102],  5.00th=[  120], 10.00th=[  137], 20.00th=[  172],
     | 30.00th=[  269], 40.00th=[  947], 50.00th=[ 1565], 60.00th=[ 2311],
     | 70.00th=[ 4293], 80.00th=[ 8717], 90.00th=[13042], 95.00th=[18482],
     | 99.00th=[27132], 99.50th=[33817], 99.90th=[33817], 99.95th=[33817],
     | 99.99th=[33817]
   bw (  KiB/s): min=  456, max=  878, per=24.64%, avg=659.50, stdev=222.44, samples=4
   iops        : min=  114, max=  219, avg=164.75, stdev=55.45, samples=4
  lat (usec)   : 10=8.07%, 20=10.81%, 50=4.95%, 100=1.17%, 250=15.23%
  lat (usec)   : 500=2.21%, 750=1.04%, 1000=4.30%
  lat (msec)   : 2=17.19%, 4=17.06%, 10=7.42%, 20=8.20%, 50=2.34%
  cpu          : usr=0.17%, sys=8.20%, ctx=1020, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=28124: Fri Jul 19 13:59:18 2024
   read: IOPS=153, BW=616KiB/s (631kB/s)(1436KiB/2332msec)
    clat (usec): min=3, max=36385, avg=1442.83, stdev=3739.26
     lat (usec): min=3, max=36386, avg=1443.40, stdev=3739.28
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    6], 10.00th=[    8], 20.00th=[    9],
     | 30.00th=[   12], 40.00th=[   17], 50.00th=[   31], 60.00th=[ 1106],
     | 70.00th=[ 1647], 80.00th=[ 2147], 90.00th=[ 3097], 95.00th=[ 3785],
     | 99.00th=[25035], 99.50th=[30278], 99.90th=[36439], 99.95th=[36439],
     | 99.99th=[36439]
   bw (  KiB/s): min=  448, max=  726, per=23.18%, avg=589.50, stdev=134.36, samples=4
   iops        : min=  112, max=  181, avg=147.25, stdev=33.42, samples=4
  write: IOPS=175, BW=702KiB/s (718kB/s)(1636KiB/2332msec)
    clat (usec): min=95, max=36972, avg=4413.61, stdev=6286.22
     lat (usec): min=95, max=36973, avg=4414.29, stdev=6286.22
    clat percentiles (usec):
     |  1.00th=[  102],  5.00th=[  118], 10.00th=[  126], 20.00th=[  165],
     | 30.00th=[  217], 40.00th=[  840], 50.00th=[ 1287], 60.00th=[ 2180],
     | 70.00th=[ 5145], 80.00th=[ 9241], 90.00th=[13042], 95.00th=[17433],
     | 99.00th=[26608], 99.50th=[33817], 99.90th=[36963], 99.95th=[36963],
     | 99.99th=[36963]
   bw (  KiB/s): min=  528, max=  726, per=23.97%, avg=641.50, stdev=84.84, samples=4
   iops        : min=  132, max=  181, avg=160.25, stdev=21.05, samples=4
  lat (usec)   : 4=0.26%, 10=10.81%, 20=9.64%, 50=4.43%, 100=1.17%
  lat (usec)   : 250=17.19%, 500=1.30%, 750=1.43%, 1000=5.21%
  lat (msec)   : 2=15.89%, 4=13.80%, 10=8.07%, 20=8.59%, 50=2.21%
  cpu          : usr=0.26%, sys=7.46%, ctx=966, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=28125: Fri Jul 19 13:59:18 2024
   read: IOPS=177, BW=709KiB/s (726kB/s)(1564KiB/2206msec)
    clat (usec): min=3, max=21242, avg=1202.46, stdev=2328.82
     lat (usec): min=3, max=21242, avg=1203.08, stdev=2328.82
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[   16], 50.00th=[   30], 60.00th=[ 1004],
     | 70.00th=[ 1516], 80.00th=[ 2024], 90.00th=[ 2900], 95.00th=[ 3752],
     | 99.00th=[14222], 99.50th=[16188], 99.90th=[21365], 99.95th=[21365],
     | 99.99th=[21365]
   bw (  KiB/s): min=  616, max=  776, per=27.27%, avg=693.50, stdev=81.21, samples=4
   iops        : min=  154, max=  194, avg=173.25, stdev=20.19, samples=4
  write: IOPS=170, BW=684KiB/s (700kB/s)(1508KiB/2206msec)
    clat (usec): min=98, max=30032, avg=4581.37, stdev=6609.26
     lat (usec): min=99, max=30033, avg=4582.42, stdev=6609.15
    clat percentiles (usec):
     |  1.00th=[  103],  5.00th=[  119], 10.00th=[  135], 20.00th=[  174],
     | 30.00th=[  273], 40.00th=[  988], 50.00th=[ 1500], 60.00th=[ 2278],
     | 70.00th=[ 3490], 80.00th=[ 9372], 90.00th=[14091], 95.00th=[20055],
     | 99.00th=[27657], 99.50th=[29230], 99.90th=[30016], 99.95th=[30016],
     | 99.99th=[30016]
   bw (  KiB/s): min=  456, max=  752, per=24.94%, avg=667.50, stdev=141.41, samples=4
   iops        : min=  114, max=  188, avg=166.75, stdev=35.28, samples=4
  lat (usec)   : 4=0.78%, 10=11.07%, 20=11.59%, 50=2.99%, 100=1.30%
  lat (usec)   : 250=14.58%, 500=1.69%, 750=1.17%, 1000=4.95%
  lat (msec)   : 2=18.10%, 4=15.10%, 10=6.51%, 20=7.68%, 50=2.47%
  cpu          : usr=0.27%, sys=7.53%, ctx=950, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=28126: Fri Jul 19 13:59:18 2024
   read: IOPS=168, BW=675KiB/s (691kB/s)(1508KiB/2235msec)
    clat (usec): min=4, max=27593, avg=1239.50, stdev=2524.32
     lat (usec): min=4, max=27594, avg=1240.16, stdev=2524.30
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    8], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   13], 40.00th=[   20], 50.00th=[   35], 60.00th=[ 1139],
     | 70.00th=[ 1680], 80.00th=[ 2180], 90.00th=[ 2704], 95.00th=[ 3556],
     | 99.00th=[14615], 99.50th=[21627], 99.90th=[27657], 99.95th=[27657],
     | 99.99th=[27657]
   bw (  KiB/s): min=  576, max=  696, per=25.56%, avg=650.00, stdev=52.00, samples=4
   iops        : min=  144, max=  174, avg=162.50, stdev=13.00, samples=4
  write: IOPS=174, BW=700KiB/s (717kB/s)(1564KiB/2235msec)
    clat (usec): min=98, max=34457, avg=4498.20, stdev=6007.54
     lat (usec): min=98, max=34458, avg=4499.56, stdev=6007.16
    clat percentiles (usec):
     |  1.00th=[  104],  5.00th=[  118], 10.00th=[  139], 20.00th=[  174],
     | 30.00th=[  297], 40.00th=[ 1037], 50.00th=[ 1713], 60.00th=[ 2507],
     | 70.00th=[ 4686], 80.00th=[ 9634], 90.00th=[12649], 95.00th=[17433],
     | 99.00th=[24249], 99.50th=[29230], 99.90th=[34341], 99.95th=[34341],
     | 99.99th=[34341]
   bw (  KiB/s): min=  592, max=  720, per=25.26%, avg=676.00, stdev=60.40, samples=4
   iops        : min=  148, max=  180, avg=169.00, stdev=15.10, samples=4
  lat (usec)   : 10=8.98%, 20=10.81%, 50=5.34%, 100=0.91%, 250=13.93%
  lat (usec)   : 500=2.34%, 750=1.30%, 1000=3.52%
  lat (msec)   : 2=16.93%, 4=18.36%, 10=6.90%, 20=8.85%, 50=1.82%
  cpu          : usr=0.22%, sys=7.52%, ctx=991, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=2544KiB/s (2605kB/s), 616KiB/s-709KiB/s (631kB/s-726kB/s), io=5988KiB (6132kB), run=2206-2354msec
  WRITE: bw=2676KiB/s (2741kB/s), 676KiB/s-702KiB/s (693kB/s-718kB/s), io=6300KiB (6451kB), run=2206-2354msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=28119: Fri Jul 19 13:59:24 2024
   read: IOPS=45, BW=183KiB/s (187kB/s)(1480KiB/8108msec)
    clat (usec): min=642, max=26194, avg=1734.87, stdev=2425.88
     lat (usec): min=643, max=26195, avg=1735.67, stdev=2425.89
    clat percentiles (usec):
     |  1.00th=[  693],  5.00th=[  766], 10.00th=[  816], 20.00th=[  881],
     | 30.00th=[  930], 40.00th=[  996], 50.00th=[ 1074], 60.00th=[ 1237],
     | 70.00th=[ 1369], 80.00th=[ 1696], 90.00th=[ 2245], 95.00th=[ 4686],
     | 99.00th=[12649], 99.50th=[19006], 99.90th=[26084], 99.95th=[26084],
     | 99.99th=[26084]
   bw (  KiB/s): min=  104, max=  296, per=25.74%, avg=181.44, stdev=53.74, samples=16
   iops        : min=   26, max=   74, avg=45.31, stdev=13.48, samples=16
  write: IOPS=49, BW=196KiB/s (201kB/s)(1592KiB/8108msec)
    clat (usec): min=9944, max=50188, avg=18732.79, stdev=5865.11
     lat (usec): min=9945, max=50190, avg=18733.70, stdev=5865.15
    clat percentiles (usec):
     |  1.00th=[10552],  5.00th=[11207], 10.00th=[12125], 20.00th=[13435],
     | 30.00th=[14877], 40.00th=[17171], 50.00th=[18220], 60.00th=[19530],
     | 70.00th=[20841], 80.00th=[22414], 90.00th=[25297], 95.00th=[30016],
     | 99.00th=[37487], 99.50th=[43254], 99.90th=[50070], 99.95th=[50070],
     | 99.99th=[50070]
   bw (  KiB/s): min=  144, max=  240, per=26.41%, avg=195.94, stdev=31.67, samples=16
   iops        : min=   36, max=   60, avg=48.94, stdev= 7.98, samples=16
  lat (usec)   : 750=1.95%, 1000=17.71%
  lat (msec)   : 2=21.61%, 4=4.17%, 10=1.30%, 20=33.72%, 50=19.40%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.12%, sys=1.39%, ctx=805, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=28120: Fri Jul 19 13:59:24 2024
   read: IOPS=42, BW=169KiB/s (173kB/s)(1436KiB/8486msec)
    clat (usec): min=655, max=22444, avg=1879.11, stdev=2683.43
     lat (usec): min=655, max=22445, avg=1879.86, stdev=2683.47
    clat percentiles (usec):
     |  1.00th=[  709],  5.00th=[  775], 10.00th=[  816], 20.00th=[  873],
     | 30.00th=[  938], 40.00th=[ 1004], 50.00th=[ 1123], 60.00th=[ 1237],
     | 70.00th=[ 1434], 80.00th=[ 1713], 90.00th=[ 2638], 95.00th=[ 8586],
     | 99.00th=[14877], 99.50th=[21627], 99.90th=[22414], 99.95th=[22414],
     | 99.99th=[22414]
   bw (  KiB/s): min=   80, max=  280, per=23.75%, avg=167.44, stdev=53.53, samples=16
   iops        : min=   20, max=   70, avg=41.81, stdev=13.35, samples=16
  write: IOPS=48, BW=193KiB/s (197kB/s)(1636KiB/8486msec)
    clat (usec): min=9080, max=56163, avg=19064.44, stdev=6780.73
     lat (usec): min=9081, max=56164, avg=19065.52, stdev=6780.88
    clat percentiles (usec):
     |  1.00th=[ 9503],  5.00th=[10683], 10.00th=[11600], 20.00th=[13304],
     | 30.00th=[15270], 40.00th=[17171], 50.00th=[18744], 60.00th=[19792],
     | 70.00th=[21103], 80.00th=[22676], 90.00th=[25822], 95.00th=[31065],
     | 99.00th=[46924], 99.50th=[47973], 99.90th=[56361], 99.95th=[56361],
     | 99.99th=[56361]
   bw (  KiB/s): min=  136, max=  232, per=25.26%, avg=187.44, stdev=33.41, samples=16
   iops        : min=   34, max=   58, avg=46.81, stdev= 8.34, samples=16
  lat (usec)   : 750=1.04%, 1000=17.45%
  lat (msec)   : 2=20.83%, 4=4.43%, 10=2.21%, 20=33.20%, 50=20.57%
  lat (msec)   : 100=0.26%
  cpu          : usr=0.15%, sys=1.33%, ctx=829, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=28121: Fri Jul 19 13:59:24 2024
   read: IOPS=47, BW=191KiB/s (195kB/s)(1564KiB/8194msec)
    clat (usec): min=643, max=29409, avg=1845.42, stdev=2764.82
     lat (usec): min=644, max=29410, avg=1846.17, stdev=2764.94
    clat percentiles (usec):
     |  1.00th=[  709],  5.00th=[  775], 10.00th=[  807], 20.00th=[  873],
     | 30.00th=[  922], 40.00th=[  988], 50.00th=[ 1037], 60.00th=[ 1156],
     | 70.00th=[ 1319], 80.00th=[ 1663], 90.00th=[ 2704], 95.00th=[ 6456],
     | 99.00th=[12780], 99.50th=[25035], 99.90th=[29492], 99.95th=[29492],
     | 99.99th=[29492]
   bw (  KiB/s): min=   87, max=  360, per=27.51%, avg=193.94, stdev=83.20, samples=16
   iops        : min=   21, max=   90, avg=48.44, stdev=20.86, samples=16
  write: IOPS=46, BW=184KiB/s (188kB/s)(1508KiB/8194msec)
    clat (usec): min=9689, max=52510, avg=19796.07, stdev=6489.82
     lat (usec): min=9690, max=52511, avg=19796.96, stdev=6489.85
    clat percentiles (usec):
     |  1.00th=[10552],  5.00th=[11731], 10.00th=[12387], 20.00th=[13960],
     | 30.00th=[15926], 40.00th=[17957], 50.00th=[19268], 60.00th=[20579],
     | 70.00th=[21890], 80.00th=[23462], 90.00th=[27657], 95.00th=[33817],
     | 99.00th=[41681], 99.50th=[43254], 99.90th=[52691], 99.95th=[52691],
     | 99.99th=[52691]
   bw (  KiB/s): min=  128, max=  224, per=24.72%, avg=183.44, stdev=31.76, samples=16
   iops        : min=   32, max=   56, avg=45.81, stdev= 8.02, samples=16
  lat (usec)   : 750=1.69%, 1000=19.53%
  lat (msec)   : 2=22.01%, 4=4.04%, 10=1.95%, 20=29.56%, 50=21.09%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.07%, sys=1.44%, ctx=826, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=28122: Fri Jul 19 13:59:24 2024
   read: IOPS=45, BW=183KiB/s (188kB/s)(1508KiB/8235msec)
    clat (usec): min=645, max=35905, avg=2063.75, stdev=3685.23
     lat (usec): min=646, max=35906, avg=2064.54, stdev=3685.24
    clat percentiles (usec):
     |  1.00th=[  734],  5.00th=[  791], 10.00th=[  832], 20.00th=[  898],
     | 30.00th=[  955], 40.00th=[ 1004], 50.00th=[ 1106], 60.00th=[ 1205],
     | 70.00th=[ 1352], 80.00th=[ 1680], 90.00th=[ 2671], 95.00th=[ 8225],
     | 99.00th=[21103], 99.50th=[26870], 99.90th=[35914], 99.95th=[35914],
     | 99.99th=[35914]
   bw (  KiB/s): min=  112, max=  264, per=25.60%, avg=180.50, stdev=42.63, samples=16
   iops        : min=   28, max=   66, avg=45.12, stdev=10.66, samples=16
  write: IOPS=47, BW=190KiB/s (194kB/s)(1564KiB/8235msec)
    clat (usec): min=10562, max=51969, avg=19039.03, stdev=6139.56
     lat (usec): min=10563, max=51971, avg=19039.93, stdev=6139.58
    clat percentiles (usec):
     |  1.00th=[10683],  5.00th=[11600], 10.00th=[12387], 20.00th=[13435],
     | 30.00th=[15401], 40.00th=[17433], 50.00th=[18744], 60.00th=[20055],
     | 70.00th=[21103], 80.00th=[22414], 90.00th=[24249], 95.00th=[28967],
     | 99.00th=[46400], 99.50th=[47973], 99.90th=[52167], 99.95th=[52167],
     | 99.99th=[52167]
   bw (  KiB/s): min=  128, max=  232, per=25.54%, avg=189.50, stdev=31.56, samples=16
   iops        : min=   32, max=   58, avg=47.38, stdev= 7.89, samples=16
  lat (usec)   : 750=0.91%, 1000=18.23%
  lat (msec)   : 2=22.53%, 4=4.04%, 10=1.69%, 20=32.03%, 50=20.44%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.11%, sys=1.45%, ctx=852, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=706KiB/s (723kB/s), 169KiB/s-191KiB/s (173kB/s-195kB/s), io=5988KiB (6132kB), run=8108-8486msec
  WRITE: bw=742KiB/s (760kB/s), 184KiB/s-196KiB/s (188kB/s-201kB/s), io=6300KiB (6451kB), run=8108-8486msec