== sanity test 398b: DIO and buffer IO race ============== 20:56:53 (1718931413)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0680715 s, 185 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=19339: Thu Jun 20 20:56:56 2024
   read: IOPS=164, BW=656KiB/s (672kB/s)(1480KiB/2255msec)
    clat (usec): min=3, max=27842, avg=1717.72, stdev=3505.79
     lat (usec): min=4, max=27843, avg=1718.34, stdev=3505.77
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   13], 40.00th=[   21], 50.00th=[   57], 60.00th=[ 1532],
     | 70.00th=[ 2057], 80.00th=[ 2540], 90.00th=[ 3589], 95.00th=[ 4555],
     | 99.00th=[23200], 99.50th=[27657], 99.90th=[27919], 99.95th=[27919],
     | 99.99th=[27919]
   bw (  KiB/s): min=  544, max=  736, per=23.73%, avg=630.00, stdev=86.99, samples=4
   iops        : min=  136, max=  184, avg=157.50, stdev=21.75, samples=4
  write: IOPS=176, BW=706KiB/s (723kB/s)(1592KiB/2255msec)
    clat (usec): min=92, max=40611, avg=4049.09, stdev=6087.95
     lat (usec): min=92, max=40612, avg=4049.80, stdev=6087.97
    clat percentiles (usec):
     |  1.00th=[   97],  5.00th=[  116], 10.00th=[  131], 20.00th=[  165],
     | 30.00th=[  196], 40.00th=[  783], 50.00th=[ 1156], 60.00th=[ 1975],
     | 70.00th=[ 3032], 80.00th=[ 8717], 90.00th=[12780], 95.00th=[17957],
     | 99.00th=[23987], 99.50th=[25822], 99.90th=[40633], 99.95th=[40633],
     | 99.99th=[40633]
   bw (  KiB/s): min=  568, max=  760, per=24.42%, avg=682.00, stdev=84.25, samples=4
   iops        : min=  142, max=  190, avg=170.50, stdev=21.06, samples=4
  lat (usec)   : 4=0.26%, 10=9.90%, 20=8.98%, 50=4.56%, 100=1.04%
  lat (usec)   : 250=17.06%, 500=1.95%, 750=0.91%, 1000=4.69%
  lat (msec)   : 2=15.10%, 4=17.71%, 10=7.81%, 20=7.55%, 50=2.47%
  cpu          : usr=0.44%, sys=8.07%, ctx=933, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19340: Thu Jun 20 20:56:56 2024
   read: IOPS=162, BW=650KiB/s (665kB/s)(1436KiB/2210msec)
    clat (usec): min=3, max=25788, avg=1367.60, stdev=2434.87
     lat (usec): min=3, max=25789, avg=1368.21, stdev=2434.88
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   12],
     | 30.00th=[   18], 40.00th=[   26], 50.00th=[  198], 60.00th=[ 1385],
     | 70.00th=[ 2008], 80.00th=[ 2507], 90.00th=[ 2999], 95.00th=[ 3621],
     | 99.00th=[15664], 99.50th=[17695], 99.90th=[25822], 99.95th=[25822],
     | 99.99th=[25822]
   bw (  KiB/s): min=  544, max=  752, per=23.43%, avg=622.00, stdev=90.60, samples=4
   iops        : min=  136, max=  188, avg=155.50, stdev=22.65, samples=4
  write: IOPS=185, BW=740KiB/s (758kB/s)(1636KiB/2210msec)
    clat (usec): min=96, max=30007, avg=4182.38, stdev=5782.91
     lat (usec): min=97, max=30008, avg=4183.26, stdev=5782.89
    clat percentiles (usec):
     |  1.00th=[  108],  5.00th=[  124], 10.00th=[  139], 20.00th=[  172],
     | 30.00th=[  227], 40.00th=[  857], 50.00th=[ 1287], 60.00th=[ 2147],
     | 70.00th=[ 4015], 80.00th=[ 8848], 90.00th=[13042], 95.00th=[15926],
     | 99.00th=[23725], 99.50th=[24773], 99.90th=[30016], 99.95th=[30016],
     | 99.99th=[30016]
   bw (  KiB/s): min=  616, max=  832, per=25.13%, avg=702.00, stdev=92.00, samples=4
   iops        : min=  154, max=  208, avg=175.50, stdev=23.00, samples=4
  lat (usec)   : 4=0.26%, 10=7.29%, 20=7.81%, 50=6.51%, 100=1.30%
  lat (usec)   : 250=17.58%, 500=2.34%, 750=1.17%, 1000=4.43%
  lat (msec)   : 2=15.62%, 4=17.84%, 10=8.72%, 20=7.68%, 50=1.43%
  cpu          : usr=0.23%, sys=7.79%, ctx=949, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19341: Thu Jun 20 20:56:56 2024
   read: IOPS=180, BW=723KiB/s (740kB/s)(1564KiB/2164msec)
    clat (usec): min=3, max=25315, avg=1588.81, stdev=3127.33
     lat (usec): min=3, max=25315, avg=1589.38, stdev=3127.33
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    6], 10.00th=[    8], 20.00th=[   11],
     | 30.00th=[   13], 40.00th=[   20], 50.00th=[  208], 60.00th=[ 1385],
     | 70.00th=[ 2024], 80.00th=[ 2376], 90.00th=[ 3195], 95.00th=[ 4015],
     | 99.00th=[19530], 99.50th=[23200], 99.90th=[25297], 99.95th=[25297],
     | 99.99th=[25297]
   bw (  KiB/s): min=  576, max=  752, per=25.61%, avg=680.00, stdev=74.19, samples=4
   iops        : min=  144, max=  188, avg=170.00, stdev=18.55, samples=4
  write: IOPS=174, BW=697KiB/s (714kB/s)(1508KiB/2164msec)
    clat (usec): min=95, max=29932, avg=4072.11, stdev=5776.06
     lat (usec): min=96, max=29933, avg=4072.87, stdev=5776.06
    clat percentiles (usec):
     |  1.00th=[  103],  5.00th=[  117], 10.00th=[  135], 20.00th=[  167],
     | 30.00th=[  210], 40.00th=[  848], 50.00th=[ 1401], 60.00th=[ 2212],
     | 70.00th=[ 3949], 80.00th=[ 7832], 90.00th=[12125], 95.00th=[16581],
     | 99.00th=[25560], 99.50th=[26084], 99.90th=[30016], 99.95th=[30016],
     | 99.99th=[30016]
   bw (  KiB/s): min=  592, max=  704, per=23.85%, avg=666.00, stdev=50.33, samples=4
   iops        : min=  148, max=  176, avg=166.50, stdev=12.58, samples=4
  lat (usec)   : 4=0.39%, 10=9.77%, 20=10.29%, 50=4.30%, 100=0.91%
  lat (usec)   : 250=16.15%, 500=1.95%, 750=0.52%, 1000=4.56%
  lat (msec)   : 2=14.58%, 4=19.40%, 10=7.94%, 20=7.16%, 50=2.08%
  cpu          : usr=0.23%, sys=8.00%, ctx=968, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19342: Thu Jun 20 20:56:56 2024
   read: IOPS=177, BW=709KiB/s (726kB/s)(1508KiB/2126msec)
    clat (usec): min=3, max=18521, avg=1198.23, stdev=2017.97
     lat (usec): min=4, max=18522, avg=1198.99, stdev=2017.86
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   14], 40.00th=[   20], 50.00th=[   46], 60.00th=[ 1205],
     | 70.00th=[ 1811], 80.00th=[ 2245], 90.00th=[ 2900], 95.00th=[ 3556],
     | 99.00th=[13960], 99.50th=[17171], 99.90th=[18482], 99.95th=[18482],
     | 99.99th=[18482]
   bw (  KiB/s): min=  576, max=  784, per=25.54%, avg=678.00, stdev=92.23, samples=4
   iops        : min=  144, max=  196, avg=169.50, stdev=23.06, samples=4
  write: IOPS=183, BW=736KiB/s (753kB/s)(1564KiB/2126msec)
    clat (usec): min=98, max=31993, avg=4261.01, stdev=6024.36
     lat (usec): min=98, max=31994, avg=4261.98, stdev=6024.24
    clat percentiles (usec):
     |  1.00th=[  104],  5.00th=[  128], 10.00th=[  155], 20.00th=[  180],
     | 30.00th=[  273], 40.00th=[  881], 50.00th=[ 1303], 60.00th=[ 2114],
     | 70.00th=[ 3982], 80.00th=[ 8848], 90.00th=[12911], 95.00th=[16909],
     | 99.00th=[25297], 99.50th=[26346], 99.90th=[32113], 99.95th=[32113],
     | 99.99th=[32113]
   bw (  KiB/s): min=  608, max=  816, per=25.42%, avg=710.00, stdev=117.82, samples=4
   iops        : min=  152, max=  204, avg=177.50, stdev=29.46, samples=4
  lat (usec)   : 4=0.26%, 10=8.72%, 20=10.68%, 50=5.08%, 100=0.52%
  lat (usec)   : 250=14.84%, 500=2.73%, 750=0.78%, 1000=5.99%
  lat (msec)   : 2=16.93%, 4=16.93%, 10=7.16%, 20=7.16%, 50=2.21%
  cpu          : usr=0.33%, sys=7.76%, ctx=949, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=2655KiB/s (2719kB/s), 650KiB/s-723KiB/s (665kB/s-740kB/s), io=5988KiB (6132kB), run=2126-2255msec
  WRITE: bw=2794KiB/s (2861kB/s), 697KiB/s-740KiB/s (714kB/s-758kB/s), io=6300KiB (6451kB), run=2126-2255msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=19334: Thu Jun 20 20:57:03 2024
   read: IOPS=44, BW=176KiB/s (180kB/s)(1480KiB/8398msec)
    clat (usec): min=666, max=26614, avg=1900.33, stdev=2746.50
     lat (usec): min=666, max=26615, avg=1901.05, stdev=2746.49
    clat percentiles (usec):
     |  1.00th=[  701],  5.00th=[  766], 10.00th=[  816], 20.00th=[  881],
     | 30.00th=[  938], 40.00th=[ 1020], 50.00th=[ 1172], 60.00th=[ 1385],
     | 70.00th=[ 1631], 80.00th=[ 1991], 90.00th=[ 2573], 95.00th=[ 5211],
     | 99.00th=[18220], 99.50th=[23200], 99.90th=[26608], 99.95th=[26608],
     | 99.99th=[26608]
   bw (  KiB/s): min=  112, max=  296, per=25.35%, avg=174.38, stdev=47.67, samples=16
   iops        : min=   28, max=   74, avg=43.50, stdev=11.93, samples=16
  write: IOPS=47, BW=190KiB/s (194kB/s)(1592KiB/8398msec)
    clat (usec): min=10092, max=47003, avg=19302.64, stdev=5694.03
     lat (usec): min=10093, max=47004, avg=19303.48, stdev=5694.05
    clat percentiles (usec):
     |  1.00th=[10421],  5.00th=[11731], 10.00th=[12780], 20.00th=[13829],
     | 30.00th=[15533], 40.00th=[17433], 50.00th=[19006], 60.00th=[20579],
     | 70.00th=[21890], 80.00th=[23200], 90.00th=[25560], 95.00th=[27657],
     | 99.00th=[41681], 99.50th=[42730], 99.90th=[46924], 99.95th=[46924],
     | 99.99th=[46924]
   bw (  KiB/s): min=  136, max=  232, per=25.95%, avg=187.88, stdev=27.75, samples=16
   iops        : min=   34, max=   58, avg=46.88, stdev= 7.09, samples=16
  lat (usec)   : 750=1.69%, 1000=16.67%
  lat (msec)   : 2=20.31%, 4=6.77%, 10=1.56%, 20=29.56%, 50=23.44%
  cpu          : usr=0.11%, sys=1.38%, ctx=871, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19335: Thu Jun 20 20:57:03 2024
   read: IOPS=41, BW=165KiB/s (169kB/s)(1436KiB/8691msec)
    clat (usec): min=654, max=21191, avg=2060.88, stdev=2994.29
     lat (usec): min=655, max=21192, avg=2061.65, stdev=2994.29
    clat percentiles (usec):
     |  1.00th=[  725],  5.00th=[  799], 10.00th=[  840], 20.00th=[  906],
     | 30.00th=[  963], 40.00th=[ 1057], 50.00th=[ 1172], 60.00th=[ 1336],
     | 70.00th=[ 1696], 80.00th=[ 2008], 90.00th=[ 2999], 95.00th=[ 6718],
     | 99.00th=[18482], 99.50th=[20055], 99.90th=[21103], 99.95th=[21103],
     | 99.99th=[21103]
   bw (  KiB/s): min=   80, max=  240, per=23.80%, avg=163.76, stdev=46.31, samples=17
   iops        : min=   20, max=   60, avg=40.94, stdev=11.58, samples=17
  write: IOPS=47, BW=188KiB/s (193kB/s)(1636KiB/8691msec)
    clat (usec): min=9802, max=44194, avg=19408.56, stdev=5941.16
     lat (usec): min=9803, max=44195, avg=19409.51, stdev=5941.08
    clat percentiles (usec):
     |  1.00th=[10814],  5.00th=[11469], 10.00th=[12387], 20.00th=[13566],
     | 30.00th=[15401], 40.00th=[17695], 50.00th=[19268], 60.00th=[20841],
     | 70.00th=[21890], 80.00th=[23200], 90.00th=[26346], 95.00th=[30540],
     | 99.00th=[39584], 99.50th=[41157], 99.90th=[44303], 99.95th=[44303],
     | 99.99th=[44303]
   bw (  KiB/s): min=  136, max=  250, per=25.76%, avg=186.47, stdev=30.16, samples=17
   iops        : min=   34, max=   62, avg=46.59, stdev= 7.48, samples=17
  lat (usec)   : 750=0.91%, 1000=15.76%
  lat (msec)   : 2=20.57%, 4=6.25%, 10=1.56%, 20=30.86%, 50=24.09%
  cpu          : usr=0.15%, sys=1.32%, ctx=856, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19336: Thu Jun 20 20:57:03 2024
   read: IOPS=48, BW=193KiB/s (197kB/s)(1564KiB/8113msec)
    clat (usec): min=602, max=33173, avg=2012.06, stdev=2993.39
     lat (usec): min=603, max=33174, avg=2012.77, stdev=2993.39
    clat percentiles (usec):
     |  1.00th=[  685],  5.00th=[  791], 10.00th=[  832], 20.00th=[  898],
     | 30.00th=[  963], 40.00th=[ 1057], 50.00th=[ 1139], 60.00th=[ 1303],
     | 70.00th=[ 1631], 80.00th=[ 1926], 90.00th=[ 2966], 95.00th=[ 8160],
     | 99.00th=[15926], 99.50th=[20841], 99.90th=[33162], 99.95th=[33162],
     | 99.99th=[33162]
   bw (  KiB/s): min=  104, max=  328, per=28.32%, avg=194.88, stdev=71.43, samples=16
   iops        : min=   26, max=   82, avg=48.62, stdev=17.86, samples=16
  write: IOPS=46, BW=186KiB/s (190kB/s)(1508KiB/8113msec)
    clat (usec): min=10199, max=50576, avg=19405.15, stdev=5244.72
     lat (usec): min=10200, max=50577, avg=19406.03, stdev=5244.75
    clat percentiles (usec):
     |  1.00th=[10945],  5.00th=[12518], 10.00th=[13042], 20.00th=[14615],
     | 30.00th=[16450], 40.00th=[17957], 50.00th=[19006], 60.00th=[20317],
     | 70.00th=[21365], 80.00th=[22938], 90.00th=[25822], 95.00th=[27657],
     | 99.00th=[37487], 99.50th=[40633], 99.90th=[50594], 99.95th=[50594],
     | 99.99th=[50594]
   bw (  KiB/s): min=  144, max=  223, per=25.60%, avg=185.37, stdev=24.36, samples=16
   iops        : min=   36, max=   55, avg=46.25, stdev= 6.01, samples=16
  lat (usec)   : 750=1.43%, 1000=16.28%
  lat (msec)   : 2=23.57%, 4=6.25%, 10=1.56%, 20=29.95%, 50=20.83%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.07%, sys=1.48%, ctx=861, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19337: Thu Jun 20 20:57:03 2024
   read: IOPS=45, BW=181KiB/s (185kB/s)(1508KiB/8353msec)
    clat (usec): min=583, max=20688, avg=1970.93, stdev=2895.40
     lat (usec): min=584, max=20688, avg=1971.68, stdev=2895.41
    clat percentiles (usec):
     |  1.00th=[  717],  5.00th=[  799], 10.00th=[  840], 20.00th=[  914],
     | 30.00th=[  971], 40.00th=[ 1057], 50.00th=[ 1172], 60.00th=[ 1369],
     | 70.00th=[ 1680], 80.00th=[ 2008], 90.00th=[ 2606], 95.00th=[ 4293],
     | 99.00th=[18482], 99.50th=[20579], 99.90th=[20579], 99.95th=[20579],
     | 99.99th=[20579]
   bw (  KiB/s): min=  104, max=  232, per=25.85%, avg=177.88, stdev=40.92, samples=16
   iops        : min=   26, max=   58, avg=44.38, stdev=10.13, samples=16
  write: IOPS=46, BW=187KiB/s (192kB/s)(1564KiB/8353msec)
    clat (usec): min=10023, max=50802, avg=19435.90, stdev=5880.55
     lat (usec): min=10024, max=50802, avg=19436.77, stdev=5880.56
    clat percentiles (usec):
     |  1.00th=[10552],  5.00th=[12125], 10.00th=[12911], 20.00th=[14222],
     | 30.00th=[15795], 40.00th=[17695], 50.00th=[19268], 60.00th=[20579],
     | 70.00th=[21365], 80.00th=[22938], 90.00th=[25822], 95.00th=[30540],
     | 99.00th=[39584], 99.50th=[49021], 99.90th=[50594], 99.95th=[50594],
     | 99.99th=[50594]
   bw (  KiB/s): min=  136, max=  223, per=25.60%, avg=185.37, stdev=26.50, samples=16
   iops        : min=   34, max=   55, avg=46.25, stdev= 6.52, samples=16
  lat (usec)   : 750=1.04%, 1000=15.36%
  lat (msec)   : 2=22.66%, 4=7.55%, 10=0.91%, 20=29.95%, 50=22.40%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.11%, sys=1.47%, ctx=878, majf=0, minf=31
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=689KiB/s (706kB/s), 165KiB/s-193KiB/s (169kB/s-197kB/s), io=5988KiB (6132kB), run=8113-8691msec
  WRITE: bw=725KiB/s (742kB/s), 186KiB/s-190KiB/s (190kB/s-194kB/s), io=6300KiB (6451kB), run=8113-8691msec