== sanity test 398b: DIO and buffer IO race ============== 20:00:28 (1736989228)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.100732 s, 125 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=23828: Wed Jan 15 20:00:31 2025
   read: IOPS=196, BW=785KiB/s (804kB/s)(1480KiB/1886msec)
    clat (usec): min=3, max=18801, avg=1396.85, stdev=2698.76
     lat (usec): min=3, max=18801, avg=1397.86, stdev=2699.15
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   13], 40.00th=[   22], 50.00th=[   52], 60.00th=[ 1004],
     | 70.00th=[ 1713], 80.00th=[ 2311], 90.00th=[ 3261], 95.00th=[ 4424],
     | 99.00th=[16712], 99.50th=[18220], 99.90th=[18744], 99.95th=[18744],
     | 99.99th=[18744]
   bw (  KiB/s): min=  712, max=  816, per=23.69%, avg=752.00, stdev=56.00, samples=3
   iops        : min=  178, max=  204, avg=188.00, stdev=14.00, samples=3
  write: IOPS=211, BW=844KiB/s (864kB/s)(1592KiB/1886msec)
    clat (usec): min=84, max=36426, avg=3403.95, stdev=4639.53
     lat (usec): min=84, max=36426, avg=3404.89, stdev=4639.54
    clat percentiles (usec):
     |  1.00th=[  100],  5.00th=[  115], 10.00th=[  133], 20.00th=[  169],
     | 30.00th=[  212], 40.00th=[  848], 50.00th=[ 1352], 60.00th=[ 1909],
     | 70.00th=[ 3359], 80.00th=[ 6783], 90.00th=[ 9896], 95.00th=[12911],
     | 99.00th=[17957], 99.50th=[19792], 99.90th=[36439], 99.95th=[36439],
     | 99.99th=[36439]
   bw (  KiB/s): min=  736, max=  928, per=24.11%, avg=805.33, stdev=106.53, samples=3
   iops        : min=  184, max=  232, avg=201.33, stdev=26.63, samples=3
  lat (usec)   : 4=0.13%, 10=10.03%, 20=8.20%, 50=5.60%, 100=2.08%
  lat (usec)   : 250=16.41%, 500=1.69%, 750=1.43%, 1000=6.12%
  lat (msec)   : 2=16.15%, 4=14.71%, 10=11.07%, 20=6.25%, 50=0.13%
  cpu          : usr=0.27%, sys=8.59%, ctx=1004, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23829: Wed Jan 15 20:00:31 2025
   read: IOPS=194, BW=777KiB/s (795kB/s)(1436KiB/1849msec)
    clat (usec): min=3, max=23485, avg=1455.09, stdev=2902.83
     lat (usec): min=4, max=23486, avg=1455.83, stdev=2902.79
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   11],
     | 30.00th=[   13], 40.00th=[   23], 50.00th=[   41], 60.00th=[ 1172],
     | 70.00th=[ 1778], 80.00th=[ 2311], 90.00th=[ 3064], 95.00th=[ 4752],
     | 99.00th=[18220], 99.50th=[21103], 99.90th=[23462], 99.95th=[23462],
     | 99.99th=[23462]
   bw (  KiB/s): min=  712, max=  912, per=25.04%, avg=794.67, stdev=104.41, samples=3
   iops        : min=  178, max=  228, avg=198.67, stdev=26.10, samples=3
  write: IOPS=221, BW=885KiB/s (906kB/s)(1636KiB/1849msec)
    clat (usec): min=90, max=28028, avg=3215.92, stdev=4569.81
     lat (usec): min=90, max=28028, avg=3217.52, stdev=4569.75
    clat percentiles (usec):
     |  1.00th=[   98],  5.00th=[  112], 10.00th=[  128], 20.00th=[  163],
     | 30.00th=[  200], 40.00th=[  668], 50.00th=[ 1156], 60.00th=[ 1860],
     | 70.00th=[ 2835], 80.00th=[ 6718], 90.00th=[10290], 95.00th=[12780],
     | 99.00th=[18482], 99.50th=[22938], 99.90th=[27919], 99.95th=[27919],
     | 99.99th=[27919]
   bw (  KiB/s): min=  792, max=  944, per=26.43%, avg=882.67, stdev=80.13, samples=3
   iops        : min=  198, max=  236, avg=220.67, stdev=20.03, samples=3
  lat (usec)   : 4=0.26%, 10=8.07%, 20=9.24%, 50=6.25%, 100=1.43%
  lat (usec)   : 250=16.54%, 500=3.26%, 750=2.08%, 1000=4.30%
  lat (msec)   : 2=16.02%, 4=16.15%, 10=9.64%, 20=6.12%, 50=0.65%
  cpu          : usr=0.38%, sys=9.58%, ctx=950, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23830: Wed Jan 15 20:00:31 2025
   read: IOPS=222, BW=888KiB/s (909kB/s)(1564KiB/1761msec)
    clat (usec): min=3, max=19776, avg=1182.52, stdev=2329.43
     lat (usec): min=3, max=19778, avg=1183.29, stdev=2329.49
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[   15], 50.00th=[   24], 60.00th=[ 1004],
     | 70.00th=[ 1565], 80.00th=[ 2008], 90.00th=[ 2671], 95.00th=[ 3785],
     | 99.00th=[14746], 99.50th=[18744], 99.90th=[19792], 99.95th=[19792],
     | 99.99th=[19792]
   bw (  KiB/s): min=  704, max=  968, per=26.55%, avg=842.67, stdev=132.50, samples=3
   iops        : min=  176, max=  242, avg=210.67, stdev=33.13, samples=3
  write: IOPS=214, BW=856KiB/s (877kB/s)(1508KiB/1761msec)
    clat (usec): min=92, max=24225, avg=3401.88, stdev=4748.92
     lat (usec): min=92, max=24226, avg=3403.20, stdev=4749.32
    clat percentiles (usec):
     |  1.00th=[   98],  5.00th=[  109], 10.00th=[  123], 20.00th=[  165],
     | 30.00th=[  204], 40.00th=[  758], 50.00th=[ 1287], 60.00th=[ 2057],
     | 70.00th=[ 3228], 80.00th=[ 6259], 90.00th=[10290], 95.00th=[14746],
     | 99.00th=[20317], 99.50th=[23200], 99.90th=[24249], 99.95th=[24249],
     | 99.99th=[24249]
   bw (  KiB/s): min=  776, max=  928, per=24.91%, avg=832.00, stdev=83.52, samples=3
   iops        : min=  194, max=  232, avg=208.00, stdev=20.88, samples=3
  lat (usec)   : 4=1.56%, 10=11.46%, 20=10.55%, 50=3.91%, 100=1.04%
  lat (usec)   : 250=15.76%, 500=1.69%, 750=1.69%, 1000=4.69%
  lat (msec)   : 2=17.19%, 4=15.62%, 10=8.98%, 20=5.34%, 50=0.52%
  cpu          : usr=0.11%, sys=8.69%, ctx=919, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23831: Wed Jan 15 20:00:31 2025
   read: IOPS=209, BW=838KiB/s (858kB/s)(1508KiB/1800msec)
    clat (usec): min=3, max=15132, avg=1134.52, stdev=2154.20
     lat (usec): min=3, max=15132, avg=1135.15, stdev=2154.25
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   12], 40.00th=[   15], 50.00th=[   24], 60.00th=[  906],
     | 70.00th=[ 1565], 80.00th=[ 1909], 90.00th=[ 2802], 95.00th=[ 4047],
     | 99.00th=[12911], 99.50th=[13435], 99.90th=[15139], 99.95th=[15139],
     | 99.99th=[15139]
   bw (  KiB/s): min=  672, max=  888, per=24.03%, avg=762.67, stdev=112.10, samples=3
   iops        : min=  168, max=  222, avg=190.67, stdev=28.02, samples=3
  write: IOPS=217, BW=869KiB/s (890kB/s)(1564KiB/1800msec)
    clat (usec): min=88, max=23290, avg=3469.74, stdev=4791.90
     lat (usec): min=88, max=23290, avg=3470.95, stdev=4791.56
    clat percentiles (usec):
     |  1.00th=[   94],  5.00th=[  109], 10.00th=[  123], 20.00th=[  159],
     | 30.00th=[  202], 40.00th=[  832], 50.00th=[ 1369], 60.00th=[ 1909],
     | 70.00th=[ 2999], 80.00th=[ 7373], 90.00th=[10421], 95.00th=[14746],
     | 99.00th=[19530], 99.50th=[22414], 99.90th=[23200], 99.95th=[23200],
     | 99.99th=[23200]
   bw (  KiB/s): min=  760, max=  864, per=24.75%, avg=826.67, stdev=57.87, samples=3
   iops        : min=  190, max=  216, avg=206.67, stdev=14.47, samples=3
  lat (usec)   : 4=1.04%, 10=12.37%, 20=9.37%, 50=4.56%, 100=2.21%
  lat (usec)   : 250=14.45%, 500=2.08%, 750=1.69%, 1000=5.21%
  lat (msec)   : 2=19.14%, 4=11.33%, 10=10.16%, 20=5.99%, 50=0.39%
  cpu          : usr=0.17%, sys=8.84%, ctx=856, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3175KiB/s (3251kB/s), 777KiB/s-888KiB/s (795kB/s-909kB/s), io=5988KiB (6132kB), run=1761-1886msec
  WRITE: bw=3340KiB/s (3421kB/s), 844KiB/s-885KiB/s (864kB/s-906kB/s), io=6300KiB (6451kB), run=1761-1886msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=23833: Wed Jan 15 20:00:36 2025
   read: IOPS=56, BW=227KiB/s (233kB/s)(1480KiB/6509msec)
    clat (usec): min=625, max=28911, avg=2020.02, stdev=3485.23
     lat (usec): min=626, max=28914, avg=2020.74, stdev=3485.29
    clat percentiles (usec):
     |  1.00th=[  635],  5.00th=[  709], 10.00th=[  758], 20.00th=[  824],
     | 30.00th=[  889], 40.00th=[  963], 50.00th=[ 1057], 60.00th=[ 1172],
     | 70.00th=[ 1352], 80.00th=[ 1729], 90.00th=[ 2704], 95.00th=[ 7242],
     | 99.00th=[20579], 99.50th=[23987], 99.90th=[28967], 99.95th=[28967],
     | 99.99th=[28967]
   bw (  KiB/s): min=  136, max=  335, per=25.15%, avg=227.38, stdev=62.18, samples=13
   iops        : min=   34, max=   83, avg=56.69, stdev=15.36, samples=13
  write: IOPS=61, BW=245KiB/s (250kB/s)(1592KiB/6509msec)
    clat (usec): min=6639, max=56031, avg=14455.05, stdev=5332.84
     lat (usec): min=6640, max=56032, avg=14455.86, stdev=5332.87
    clat percentiles (usec):
     |  1.00th=[ 7242],  5.00th=[ 8455], 10.00th=[ 9110], 20.00th=[10290],
     | 30.00th=[11863], 40.00th=[13042], 50.00th=[14222], 60.00th=[14877],
     | 70.00th=[15926], 80.00th=[16909], 90.00th=[19530], 95.00th=[21890],
     | 99.00th=[35914], 99.50th=[49546], 99.90th=[55837], 99.95th=[55837],
     | 99.99th=[55837]
   bw (  KiB/s): min=  152, max=  312, per=25.66%, avg=244.00, stdev=51.06, samples=13
   iops        : min=   38, max=   78, avg=60.85, stdev=12.69, samples=13
  lat (usec)   : 750=4.30%, 1000=16.67%
  lat (msec)   : 2=19.92%, 4=3.65%, 10=11.07%, 20=39.32%, 50=4.95%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.18%, sys=1.69%, ctx=821, majf=0, minf=31
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23834: Wed Jan 15 20:00:36 2025
   read: IOPS=54, BW=217KiB/s (222kB/s)(1436KiB/6620msec)
    clat (usec): min=625, max=28924, avg=1772.13, stdev=2848.71
     lat (usec): min=625, max=28926, avg=1772.85, stdev=2848.74
    clat percentiles (usec):
     |  1.00th=[  660],  5.00th=[  717], 10.00th=[  750], 20.00th=[  816],
     | 30.00th=[  865], 40.00th=[  947], 50.00th=[ 1037], 60.00th=[ 1139],
     | 70.00th=[ 1287], 80.00th=[ 1647], 90.00th=[ 2409], 95.00th=[ 5080],
     | 99.00th=[17695], 99.50th=[20055], 99.90th=[28967], 99.95th=[28967],
     | 99.99th=[28967]
   bw (  KiB/s): min=   96, max=  352, per=23.81%, avg=215.23, stdev=70.91, samples=13
   iops        : min=   24, max=   88, avg=53.69, stdev=17.73, samples=13
  write: IOPS=61, BW=247KiB/s (253kB/s)(1636KiB/6620msec)
    clat (usec): min=6820, max=57133, avg=14612.96, stdev=5513.05
     lat (usec): min=6821, max=57135, avg=14613.77, stdev=5513.08
    clat percentiles (usec):
     |  1.00th=[ 7570],  5.00th=[ 8586], 10.00th=[ 9372], 20.00th=[10290],
     | 30.00th=[11600], 40.00th=[12780], 50.00th=[13960], 60.00th=[14877],
     | 70.00th=[15926], 80.00th=[17433], 90.00th=[20579], 95.00th=[23987],
     | 99.00th=[33162], 99.50th=[39584], 99.90th=[56886], 99.95th=[56886],
     | 99.99th=[56886]
   bw (  KiB/s): min=  160, max=  296, per=25.67%, avg=244.08, stdev=48.27, samples=13
   iops        : min=   40, max=   74, avg=60.92, stdev=12.01, samples=13
  lat (usec)   : 750=4.82%, 1000=17.06%
  lat (msec)   : 2=18.62%, 4=3.39%, 10=10.29%, 20=39.97%, 50=5.73%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.15%, sys=1.65%, ctx=833, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23835: Wed Jan 15 20:00:36 2025
   read: IOPS=63, BW=253KiB/s (259kB/s)(1564KiB/6178msec)
    clat (usec): min=627, max=20018, avg=1468.61, stdev=1756.04
     lat (usec): min=628, max=20019, avg=1469.30, stdev=1756.08
    clat percentiles (usec):
     |  1.00th=[  635],  5.00th=[  709], 10.00th=[  750], 20.00th=[  791],
     | 30.00th=[  857], 40.00th=[  914], 50.00th=[  979], 60.00th=[ 1090],
     | 70.00th=[ 1237], 80.00th=[ 1385], 90.00th=[ 2311], 95.00th=[ 3982],
     | 99.00th=[10552], 99.50th=[12387], 99.90th=[20055], 99.95th=[20055],
     | 99.99th=[20055]
   bw (  KiB/s): min=  112, max=  440, per=28.37%, avg=256.50, stdev=93.82, samples=12
   iops        : min=   28, max=  110, avg=64.00, stdev=23.41, samples=12
  write: IOPS=61, BW=244KiB/s (250kB/s)(1508KiB/6178msec)
    clat (usec): min=7105, max=61807, avg=14840.54, stdev=5641.66
     lat (usec): min=7106, max=61810, avg=14841.41, stdev=5641.73
    clat percentiles (usec):
     |  1.00th=[ 7701],  5.00th=[ 8455], 10.00th=[ 9241], 20.00th=[10290],
     | 30.00th=[11863], 40.00th=[13173], 50.00th=[14353], 60.00th=[15270],
     | 70.00th=[16188], 80.00th=[17433], 90.00th=[20055], 95.00th=[23462],
     | 99.00th=[36963], 99.50th=[47973], 99.90th=[61604], 99.95th=[61604],
     | 99.99th=[61604]
   bw (  KiB/s): min=  144, max=  312, per=25.64%, avg=243.83, stdev=49.21, samples=12
   iops        : min=   36, max=   78, avg=60.83, stdev=12.28, samples=12
  lat (usec)   : 750=4.95%, 1000=20.83%
  lat (msec)   : 2=19.40%, 4=3.26%, 10=10.42%, 20=36.20%, 50=4.82%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.16%, sys=1.73%, ctx=805, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23836: Wed Jan 15 20:00:36 2025
   read: IOPS=57, BW=231KiB/s (236kB/s)(1508KiB/6535msec)
    clat (usec): min=577, max=24446, avg=1685.48, stdev=2783.72
     lat (usec): min=577, max=24447, avg=1686.17, stdev=2783.73
    clat percentiles (usec):
     |  1.00th=[  644],  5.00th=[  685], 10.00th=[  742], 20.00th=[  799],
     | 30.00th=[  865], 40.00th=[  922], 50.00th=[  979], 60.00th=[ 1074],
     | 70.00th=[ 1221], 80.00th=[ 1401], 90.00th=[ 2245], 95.00th=[ 4293],
     | 99.00th=[16909], 99.50th=[23987], 99.90th=[24511], 99.95th=[24511],
     | 99.99th=[24511]
   bw (  KiB/s): min=  104, max=  360, per=25.24%, avg=228.15, stdev=72.63, samples=13
   iops        : min=   26, max=   90, avg=56.92, stdev=18.21, samples=13
  write: IOPS=59, BW=239KiB/s (245kB/s)(1564KiB/6535msec)
    clat (usec): min=6621, max=75650, avg=15070.11, stdev=7106.63
     lat (usec): min=6622, max=75652, avg=15071.03, stdev=7106.61
    clat percentiles (usec):
     |  1.00th=[ 7439],  5.00th=[ 8291], 10.00th=[ 8979], 20.00th=[10159],
     | 30.00th=[11338], 40.00th=[13173], 50.00th=[14353], 60.00th=[15533],
     | 70.00th=[16319], 80.00th=[17433], 90.00th=[19530], 95.00th=[25035],
     | 99.00th=[55837], 99.50th=[61604], 99.90th=[76022], 99.95th=[76022],
     | 99.99th=[76022]
   bw (  KiB/s): min=  120, max=  320, per=25.09%, avg=238.62, stdev=60.46, samples=13
   iops        : min=   30, max=   80, avg=59.54, stdev=15.11, samples=13
  lat (usec)   : 750=5.60%, 1000=20.18%
  lat (msec)   : 2=17.45%, 4=3.39%, 10=10.68%, 20=37.37%, 50=4.82%
  lat (msec)   : 100=0.52%
  cpu          : usr=0.15%, sys=1.67%, ctx=828, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=905KiB/s (926kB/s), 217KiB/s-253KiB/s (222kB/s-259kB/s), io=5988KiB (6132kB), run=6178-6620msec
  WRITE: bw=952KiB/s (975kB/s), 239KiB/s-247KiB/s (245kB/s-253kB/s), io=6300KiB (6451kB), run=6178-6620msec