== sanity test 398b: DIO and buffer IO race ============== 09:27:53 (1721222873)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.112401 s, 112 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=24766: Wed Jul 17 09:27:56 2024
   read: IOPS=178, BW=716KiB/s (733kB/s)(1480KiB/2068msec)
    clat (usec): min=3, max=38614, avg=1543.74, stdev=3630.67
     lat (usec): min=4, max=38615, avg=1544.52, stdev=3630.64
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[   17], 50.00th=[   46], 60.00th=[ 1254],
     | 70.00th=[ 1631], 80.00th=[ 2180], 90.00th=[ 2835], 95.00th=[ 5145],
     | 99.00th=[19006], 99.50th=[28181], 99.90th=[38536], 99.95th=[38536],
     | 99.99th=[38536]
   bw (  KiB/s): min=  560, max=  848, per=25.04%, avg=704.00, stdev=128.66, samples=4
   iops        : min=  140, max=  212, avg=176.00, stdev=32.17, samples=4
  write: IOPS=192, BW=770KiB/s (788kB/s)(1592KiB/2068msec)
    clat (usec): min=95, max=26351, avg=3732.78, stdev=5277.10
     lat (usec): min=95, max=26351, avg=3733.44, stdev=5277.08
    clat percentiles (usec):
     |  1.00th=[  100],  5.00th=[  115], 10.00th=[  121], 20.00th=[  141],
     | 30.00th=[  194], 40.00th=[  840], 50.00th=[ 1352], 60.00th=[ 2147],
     | 70.00th=[ 3294], 80.00th=[ 7635], 90.00th=[12125], 95.00th=[15008],
     | 99.00th=[23200], 99.50th=[25822], 99.90th=[26346], 99.95th=[26346],
     | 99.99th=[26346]
   bw (  KiB/s): min=  584, max=  872, per=26.04%, avg=770.00, stdev=132.16, samples=4
   iops        : min=  146, max=  218, avg=192.50, stdev=33.04, samples=4
  lat (usec)   : 4=0.13%, 10=12.11%, 20=8.07%, 50=4.04%, 100=1.69%
  lat (usec)   : 250=17.32%, 500=1.69%, 750=1.04%, 1000=3.65%
  lat (msec)   : 2=17.32%, 4=15.76%, 10=8.07%, 20=7.94%, 50=1.17%
  cpu          : usr=0.15%, sys=8.18%, ctx=941, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24767: Wed Jul 17 09:27:56 2024
   read: IOPS=175, BW=702KiB/s (719kB/s)(1436KiB/2046msec)
    clat (usec): min=4, max=22130, avg=1228.42, stdev=2510.58
     lat (usec): min=4, max=22130, avg=1228.98, stdev=2510.62
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[   17], 50.00th=[   32], 60.00th=[ 1020],
     | 70.00th=[ 1565], 80.00th=[ 1942], 90.00th=[ 2704], 95.00th=[ 3490],
     | 99.00th=[16712], 99.50th=[17171], 99.90th=[22152], 99.95th=[22152],
     | 99.99th=[22152]
   bw (  KiB/s): min=  656, max=  744, per=24.90%, avg=700.00, stdev=37.24, samples=4
   iops        : min=  164, max=  186, avg=175.00, stdev= 9.31, samples=4
  write: IOPS=199, BW=800KiB/s (819kB/s)(1636KiB/2046msec)
    clat (usec): min=91, max=29583, avg=3896.57, stdev=5824.09
     lat (usec): min=92, max=29584, avg=3897.23, stdev=5824.08
    clat percentiles (usec):
     |  1.00th=[   96],  5.00th=[  114], 10.00th=[  123], 20.00th=[  149],
     | 30.00th=[  235], 40.00th=[  668], 50.00th=[ 1221], 60.00th=[ 1893],
     | 70.00th=[ 3064], 80.00th=[ 7308], 90.00th=[12518], 95.00th=[18482],
     | 99.00th=[23462], 99.50th=[25035], 99.90th=[29492], 99.95th=[29492],
     | 99.99th=[29492]
   bw (  KiB/s): min=  616, max=  888, per=26.78%, avg=792.00, stdev=121.85, samples=4
   iops        : min=  154, max=  222, avg=198.00, stdev=30.46, samples=4
  lat (usec)   : 10=11.07%, 20=10.16%, 50=2.99%, 100=1.43%, 250=16.54%
  lat (usec)   : 500=3.39%, 750=2.08%, 1000=4.30%
  lat (msec)   : 2=18.62%, 4=13.67%, 10=6.51%, 20=7.42%, 50=1.82%
  cpu          : usr=0.10%, sys=8.16%, ctx=931, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24768: Wed Jul 17 09:27:56 2024
   read: IOPS=183, BW=734KiB/s (752kB/s)(1564KiB/2130msec)
    clat (usec): min=3, max=24517, avg=1498.27, stdev=3064.67
     lat (usec): min=3, max=24517, avg=1498.89, stdev=3064.68
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[   17], 50.00th=[   41], 60.00th=[ 1237],
     | 70.00th=[ 1663], 80.00th=[ 2212], 90.00th=[ 3097], 95.00th=[ 5407],
     | 99.00th=[19530], 99.50th=[23725], 99.90th=[24511], 99.95th=[24511],
     | 99.99th=[24511]
   bw (  KiB/s): min=  648, max=  990, per=27.02%, avg=759.50, stdev=155.64, samples=4
   iops        : min=  162, max=  247, avg=189.75, stdev=38.66, samples=4
  write: IOPS=176, BW=708KiB/s (725kB/s)(1508KiB/2130msec)
    clat (usec): min=101, max=24765, avg=4069.88, stdev=5738.47
     lat (usec): min=101, max=24766, avg=4070.54, stdev=5738.47
    clat percentiles (usec):
     |  1.00th=[  102],  5.00th=[  121], 10.00th=[  129], 20.00th=[  167],
     | 30.00th=[  239], 40.00th=[  791], 50.00th=[ 1270], 60.00th=[ 2040],
     | 70.00th=[ 3523], 80.00th=[ 8848], 90.00th=[12911], 95.00th=[18482],
     | 99.00th=[22152], 99.50th=[22676], 99.90th=[24773], 99.95th=[24773],
     | 99.99th=[24773]
   bw (  KiB/s): min=  536, max=  870, per=24.13%, avg=713.50, stdev=153.11, samples=4
   iops        : min=  134, max=  217, avg=178.25, stdev=38.11, samples=4
  lat (usec)   : 4=0.91%, 10=13.93%, 20=6.77%, 50=4.30%, 100=0.52%
  lat (usec)   : 250=15.36%, 500=2.21%, 750=1.82%, 1000=4.95%
  lat (msec)   : 2=17.06%, 4=15.36%, 10=7.03%, 20=8.07%, 50=1.69%
  cpu          : usr=0.33%, sys=8.64%, ctx=935, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24769: Wed Jul 17 09:27:56 2024
   read: IOPS=179, BW=719KiB/s (736kB/s)(1508KiB/2097msec)
    clat (usec): min=3, max=24332, avg=1492.80, stdev=3234.30
     lat (usec): min=3, max=24333, avg=1494.31, stdev=3234.32
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[   16], 50.00th=[   30], 60.00th=[ 1090],
     | 70.00th=[ 1663], 80.00th=[ 2147], 90.00th=[ 3032], 95.00th=[ 5538],
     | 99.00th=[20579], 99.50th=[22938], 99.90th=[24249], 99.95th=[24249],
     | 99.99th=[24249]
   bw (  KiB/s): min=  640, max=  822, per=25.52%, avg=717.50, stdev=88.62, samples=4
   iops        : min=  160, max=  205, avg=179.25, stdev=21.96, samples=4
  write: IOPS=186, BW=746KiB/s (764kB/s)(1564KiB/2097msec)
    clat (usec): min=94, max=26986, avg=3902.39, stdev=5688.69
     lat (usec): min=94, max=26987, avg=3903.25, stdev=5688.55
    clat percentiles (usec):
     |  1.00th=[   99],  5.00th=[  115], 10.00th=[  125], 20.00th=[  143],
     | 30.00th=[  192], 40.00th=[  570], 50.00th=[ 1074], 60.00th=[ 1893],
     | 70.00th=[ 3195], 80.00th=[ 8848], 90.00th=[12649], 95.00th=[16712],
     | 99.00th=[23200], 99.50th=[24249], 99.90th=[26870], 99.95th=[26870],
     | 99.99th=[26870]
   bw (  KiB/s): min=  576, max=  886, per=25.35%, avg=749.50, stdev=130.98, samples=4
   iops        : min=  144, max=  221, avg=187.25, stdev=32.57, samples=4
  lat (usec)   : 4=0.91%, 10=11.85%, 20=8.72%, 50=4.17%, 100=1.82%
  lat (usec)   : 250=17.32%, 500=2.21%, 750=2.21%, 1000=3.78%
  lat (msec)   : 2=16.28%, 4=13.80%, 10=6.90%, 20=8.20%, 50=1.82%
  cpu          : usr=0.14%, sys=7.68%, ctx=921, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=2811KiB/s (2879kB/s), 702KiB/s-734KiB/s (719kB/s-752kB/s), io=5988KiB (6132kB), run=2046-2130msec
  WRITE: bw=2958KiB/s (3029kB/s), 708KiB/s-800KiB/s (725kB/s-819kB/s), io=6300KiB (6451kB), run=2046-2130msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=24771: Wed Jul 17 09:28:03 2024
   read: IOPS=45, BW=180KiB/s (185kB/s)(1480KiB/8207msec)
    clat (usec): min=563, max=35213, avg=2000.86, stdev=3354.13
     lat (usec): min=564, max=35214, avg=2001.54, stdev=3354.16
    clat percentiles (usec):
     |  1.00th=[  619],  5.00th=[  725], 10.00th=[  783], 20.00th=[  857],
     | 30.00th=[  938], 40.00th=[ 1004], 50.00th=[ 1123], 60.00th=[ 1319],
     | 70.00th=[ 1565], 80.00th=[ 1909], 90.00th=[ 2900], 95.00th=[ 4424],
     | 99.00th=[20317], 99.50th=[23987], 99.90th=[35390], 99.95th=[35390],
     | 99.99th=[35390]
   bw (  KiB/s): min=  103, max=  288, per=24.56%, avg=175.88, stdev=50.74, samples=16
   iops        : min=   25, max=   72, avg=43.88, stdev=12.74, samples=16
  write: IOPS=48, BW=194KiB/s (199kB/s)(1592KiB/8207msec)
    clat (usec): min=10280, max=63726, avg=18732.68, stdev=6319.47
     lat (usec): min=10281, max=63729, avg=18733.49, stdev=6319.48
    clat percentiles (usec):
     |  1.00th=[10421],  5.00th=[11600], 10.00th=[11994], 20.00th=[13042],
     | 30.00th=[14877], 40.00th=[16712], 50.00th=[18482], 60.00th=[20055],
     | 70.00th=[20841], 80.00th=[21890], 90.00th=[25035], 95.00th=[28705],
     | 99.00th=[48497], 99.50th=[51119], 99.90th=[63701], 99.95th=[63701],
     | 99.99th=[63701]
   bw (  KiB/s): min=  104, max=  232, per=25.55%, avg=192.38, stdev=30.91, samples=16
   iops        : min=   26, max=   58, avg=48.00, stdev= 7.69, samples=16
  lat (usec)   : 750=3.39%, 1000=15.89%
  lat (msec)   : 2=20.57%, 4=5.21%, 10=1.56%, 20=31.90%, 50=21.09%
  lat (msec)   : 100=0.39%
  cpu          : usr=0.15%, sys=1.32%, ctx=891, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24772: Wed Jul 17 09:28:03 2024
   read: IOPS=42, BW=172KiB/s (176kB/s)(1436KiB/8363msec)
    clat (usec): min=537, max=34854, avg=2003.30, stdev=3544.11
     lat (usec): min=537, max=34854, avg=2003.93, stdev=3544.11
    clat percentiles (usec):
     |  1.00th=[  693],  5.00th=[  758], 10.00th=[  824], 20.00th=[  881],
     | 30.00th=[  979], 40.00th=[ 1057], 50.00th=[ 1172], 60.00th=[ 1336],
     | 70.00th=[ 1565], 80.00th=[ 1909], 90.00th=[ 2900], 95.00th=[ 3589],
     | 99.00th=[23200], 99.50th=[34341], 99.90th=[34866], 99.95th=[34866],
     | 99.99th=[34866]
   bw (  KiB/s): min=   80, max=  287, per=24.07%, avg=172.38, stdev=56.25, samples=16
   iops        : min=   20, max=   71, avg=43.00, stdev=13.94, samples=16
  write: IOPS=48, BW=196KiB/s (200kB/s)(1636KiB/8363msec)
    clat (usec): min=9904, max=78392, avg=18665.32, stdev=5985.52
     lat (usec): min=9905, max=78393, avg=18666.20, stdev=5985.41
    clat percentiles (usec):
     |  1.00th=[10683],  5.00th=[11600], 10.00th=[12125], 20.00th=[13173],
     | 30.00th=[15533], 40.00th=[17171], 50.00th=[18482], 60.00th=[19792],
     | 70.00th=[20841], 80.00th=[22152], 90.00th=[23725], 95.00th=[27395],
     | 99.00th=[36439], 99.50th=[40109], 99.90th=[78119], 99.95th=[78119],
     | 99.99th=[78119]
   bw (  KiB/s): min=  144, max=  232, per=25.61%, avg=192.88, stdev=28.70, samples=16
   iops        : min=   36, max=   58, avg=48.12, stdev= 7.14, samples=16
  lat (usec)   : 750=2.08%, 1000=13.54%
  lat (msec)   : 2=22.27%, 4=6.64%, 10=0.91%, 20=35.03%, 50=19.40%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.07%, sys=1.39%, ctx=874, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24773: Wed Jul 17 09:28:03 2024
   read: IOPS=49, BW=197KiB/s (201kB/s)(1564KiB/7953msec)
    clat (usec): min=587, max=23315, avg=2009.39, stdev=3000.10
     lat (usec): min=587, max=23316, avg=2010.05, stdev=3000.11
    clat percentiles (usec):
     |  1.00th=[  693],  5.00th=[  775], 10.00th=[  816], 20.00th=[  881],
     | 30.00th=[  963], 40.00th=[ 1012], 50.00th=[ 1123], 60.00th=[ 1254],
     | 70.00th=[ 1450], 80.00th=[ 1958], 90.00th=[ 2933], 95.00th=[ 7046],
     | 99.00th=[19530], 99.50th=[22676], 99.90th=[23200], 99.95th=[23200],
     | 99.99th=[23200]
   bw (  KiB/s): min=   96, max=  384, per=28.15%, avg=201.53, stdev=88.23, samples=15
   iops        : min=   24, max=   96, avg=50.33, stdev=22.07, samples=15
  write: IOPS=47, BW=190KiB/s (194kB/s)(1508KiB/7953msec)
    clat (usec): min=10163, max=90613, avg=18987.03, stdev=7372.90
     lat (usec): min=10164, max=90614, avg=18987.83, stdev=7373.05
    clat percentiles (usec):
     |  1.00th=[10814],  5.00th=[11600], 10.00th=[12125], 20.00th=[12911],
     | 30.00th=[14353], 40.00th=[16712], 50.00th=[18220], 60.00th=[19530],
     | 70.00th=[21103], 80.00th=[22414], 90.00th=[25297], 95.00th=[32637],
     | 99.00th=[46400], 99.50th=[47973], 99.90th=[90702], 99.95th=[90702],
     | 99.99th=[90702]
   bw (  KiB/s): min=  120, max=  224, per=24.99%, avg=188.20, stdev=32.24, samples=15
   iops        : min=   30, max=   56, avg=47.00, stdev= 8.04, samples=15
  lat (usec)   : 750=1.56%, 1000=18.49%
  lat (msec)   : 2=21.35%, 4=6.12%, 10=1.30%, 20=31.77%, 50=19.27%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.18%, sys=1.40%, ctx=884, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24774: Wed Jul 17 09:28:03 2024
   read: IOPS=45, BW=184KiB/s (188kB/s)(1508KiB/8210msec)
    clat (usec): min=598, max=37494, avg=2132.00, stdev=3993.78
     lat (usec): min=598, max=37494, avg=2132.62, stdev=3993.77
    clat percentiles (usec):
     |  1.00th=[  627],  5.00th=[  709], 10.00th=[  783], 20.00th=[  848],
     | 30.00th=[  914], 40.00th=[  996], 50.00th=[ 1090], 60.00th=[ 1254],
     | 70.00th=[ 1582], 80.00th=[ 2073], 90.00th=[ 2999], 95.00th=[ 4752],
     | 99.00th=[24249], 99.50th=[36963], 99.90th=[37487], 99.95th=[37487],
     | 99.99th=[37487]
   bw (  KiB/s): min=   79, max=  264, per=25.19%, avg=180.38, stdev=62.82, samples=16
   iops        : min=   19, max=   66, avg=45.00, stdev=15.74, samples=16
  write: IOPS=47, BW=190KiB/s (195kB/s)(1564KiB/8210msec)
    clat (usec): min=10131, max=93255, avg=18920.00, stdev=6772.39
     lat (usec): min=10132, max=93258, avg=18920.77, stdev=6772.46
    clat percentiles (usec):
     |  1.00th=[10814],  5.00th=[11469], 10.00th=[12256], 20.00th=[13698],
     | 30.00th=[15664], 40.00th=[17695], 50.00th=[18744], 60.00th=[19792],
     | 70.00th=[20579], 80.00th=[21627], 90.00th=[23725], 95.00th=[27395],
     | 99.00th=[44303], 99.50th=[49546], 99.90th=[92799], 99.95th=[92799],
     | 99.99th=[92799]
   bw (  KiB/s): min=  104, max=  240, per=25.15%, avg=189.38, stdev=36.50, samples=16
   iops        : min=   26, max=   60, avg=47.25, stdev= 9.17, samples=16
  lat (usec)   : 750=3.39%, 1000=16.28%
  lat (msec)   : 2=19.27%, 4=7.29%, 10=1.17%, 20=33.85%, 50=18.62%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.16%, sys=1.27%, ctx=887, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=716KiB/s (733kB/s), 172KiB/s-197KiB/s (176kB/s-201kB/s), io=5988KiB (6132kB), run=7953-8363msec
  WRITE: bw=753KiB/s (771kB/s), 190KiB/s-196KiB/s (194kB/s-200kB/s), io=6300KiB (6451kB), run=7953-8363msec