== sanity test 398b: DIO and buffer IO race ============== 13:38:52 (1721410732)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0779396 s, 161 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=24287: Fri Jul 19 13:38:55 2024
   read: IOPS=180, BW=721KiB/s (738kB/s)(1480KiB/2054msec)
    clat (usec): min=4, max=28995, avg=1555.26, stdev=2894.02
     lat (usec): min=4, max=28996, avg=1555.88, stdev=2894.08
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    8], 10.00th=[   10], 20.00th=[   12],
     | 30.00th=[   18], 40.00th=[   27], 50.00th=[  799], 60.00th=[ 1532],
     | 70.00th=[ 2057], 80.00th=[ 2573], 90.00th=[ 3097], 95.00th=[ 4146],
     | 99.00th=[14615], 99.50th=[25297], 99.90th=[28967], 99.95th=[28967],
     | 99.99th=[28967]
   bw (  KiB/s): min=  592, max=  864, per=26.00%, avg=716.00, stdev=112.10, samples=4
   iops        : min=  148, max=  216, avg=179.00, stdev=28.02, samples=4
  write: IOPS=193, BW=775KiB/s (794kB/s)(1592KiB/2054msec)
    clat (usec): min=88, max=33404, avg=3682.05, stdev=5429.67
     lat (usec): min=88, max=33405, avg=3682.78, stdev=5429.67
    clat percentiles (usec):
     |  1.00th=[  106],  5.00th=[  124], 10.00th=[  139], 20.00th=[  169],
     | 30.00th=[  247], 40.00th=[  938], 50.00th=[ 1385], 60.00th=[ 1958],
     | 70.00th=[ 2900], 80.00th=[ 7635], 90.00th=[11731], 95.00th=[15795],
     | 99.00th=[23462], 99.50th=[26346], 99.90th=[33424], 99.95th=[33424],
     | 99.99th=[33424]
   bw (  KiB/s): min=  648, max=  888, per=26.79%, avg=776.00, stdev=105.93, samples=4
   iops        : min=  162, max=  222, avg=194.00, stdev=26.48, samples=4
  lat (usec)   : 10=6.64%, 20=10.29%, 50=5.47%, 100=0.91%, 250=15.89%
  lat (usec)   : 500=2.34%, 750=0.91%, 1000=4.30%
  lat (msec)   : 2=17.84%, 4=20.18%, 10=6.90%, 20=6.64%, 50=1.69%
  cpu          : usr=0.10%, sys=10.18%, ctx=966, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24288: Fri Jul 19 13:38:55 2024
   read: IOPS=165, BW=661KiB/s (676kB/s)(1436KiB/2174msec)
    clat (usec): min=3, max=23565, avg=1041.08, stdev=1859.81
     lat (usec): min=3, max=23566, avg=1041.95, stdev=1859.68
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    6], 10.00th=[    9], 20.00th=[   10],
     | 30.00th=[   13], 40.00th=[   17], 50.00th=[   31], 60.00th=[ 1172],
     | 70.00th=[ 1680], 80.00th=[ 2180], 90.00th=[ 2638], 95.00th=[ 2999],
     | 99.00th=[ 4883], 99.50th=[13960], 99.90th=[23462], 99.95th=[23462],
     | 99.99th=[23462]
   bw (  KiB/s): min=  264, max=  840, per=22.59%, avg=622.00, stdev=251.24, samples=4
   iops        : min=   66, max=  210, avg=155.50, stdev=62.81, samples=4
  write: IOPS=188, BW=753KiB/s (771kB/s)(1636KiB/2174msec)
    clat (usec): min=99, max=83231, avg=4372.57, stdev=7332.41
     lat (usec): min=100, max=83233, avg=4373.35, stdev=7332.40
    clat percentiles (usec):
     |  1.00th=[  108],  5.00th=[  126], 10.00th=[  139], 20.00th=[  165],
     | 30.00th=[  265], 40.00th=[  947], 50.00th=[ 1467], 60.00th=[ 2147],
     | 70.00th=[ 4146], 80.00th=[ 7767], 90.00th=[11731], 95.00th=[19006],
     | 99.00th=[30540], 99.50th=[31065], 99.90th=[83362], 99.95th=[83362],
     | 99.99th=[83362]
   bw (  KiB/s): min=  336, max=  888, per=24.37%, avg=706.00, stdev=258.52, samples=4
   iops        : min=   84, max=  222, avg=176.50, stdev=64.63, samples=4
  lat (usec)   : 4=0.13%, 10=9.37%, 20=10.68%, 50=4.95%, 100=0.65%
  lat (usec)   : 250=16.41%, 500=1.82%, 750=1.69%, 1000=3.91%
  lat (msec)   : 2=16.93%, 4=16.67%, 10=9.37%, 20=5.21%, 50=2.08%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.23%, sys=7.78%, ctx=919, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24289: Fri Jul 19 13:38:55 2024
   read: IOPS=190, BW=761KiB/s (779kB/s)(1564KiB/2056msec)
    clat (usec): min=4, max=28684, avg=1546.31, stdev=3029.23
     lat (usec): min=4, max=28685, avg=1546.94, stdev=3029.23
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    6], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   14], 40.00th=[   25], 50.00th=[  938], 60.00th=[ 1598],
     | 70.00th=[ 1958], 80.00th=[ 2507], 90.00th=[ 3195], 95.00th=[ 3785],
     | 99.00th=[20579], 99.50th=[24773], 99.90th=[28705], 99.95th=[28705],
     | 99.99th=[28705]
   bw (  KiB/s): min=  648, max=  912, per=27.65%, avg=761.50, stdev=114.40, samples=4
   iops        : min=  162, max=  228, avg=190.25, stdev=28.57, samples=4
  write: IOPS=183, BW=733KiB/s (751kB/s)(1508KiB/2056msec)
    clat (usec): min=101, max=67340, avg=3821.90, stdev=6514.43
     lat (usec): min=102, max=67342, avg=3822.70, stdev=6514.65
    clat percentiles (usec):
     |  1.00th=[  106],  5.00th=[  127], 10.00th=[  137], 20.00th=[  176],
     | 30.00th=[  229], 40.00th=[  881], 50.00th=[ 1172], 60.00th=[ 2040],
     | 70.00th=[ 3097], 80.00th=[ 6063], 90.00th=[11207], 95.00th=[17957],
     | 99.00th=[28181], 99.50th=[33817], 99.90th=[67634], 99.95th=[67634],
     | 99.99th=[67634]
   bw (  KiB/s): min=  560, max=  792, per=24.91%, avg=721.50, stdev=108.56, samples=4
   iops        : min=  140, max=  198, avg=180.25, stdev=27.08, samples=4
  lat (usec)   : 10=10.42%, 20=9.11%, 50=4.82%, 100=0.52%, 250=15.49%
  lat (usec)   : 500=2.21%, 750=1.17%, 1000=3.52%
  lat (msec)   : 2=18.23%, 4=19.40%, 10=7.68%, 20=5.60%, 50=1.69%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.24%, sys=9.48%, ctx=975, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24290: Fri Jul 19 13:38:55 2024
   read: IOPS=176, BW=707KiB/s (724kB/s)(1508KiB/2132msec)
    clat (usec): min=3, max=25504, avg=1204.36, stdev=2776.84
     lat (usec): min=3, max=25504, avg=1206.60, stdev=2777.45
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    6], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[   16], 50.00th=[   24], 60.00th=[   99],
     | 70.00th=[ 1483], 80.00th=[ 2057], 90.00th=[ 2802], 95.00th=[ 3818],
     | 99.00th=[19268], 99.50th=[24511], 99.90th=[25560], 99.95th=[25560],
     | 99.99th=[25560]
   bw (  KiB/s): min=  232, max=  944, per=23.89%, avg=658.00, stdev=307.38, samples=4
   iops        : min=   58, max=  236, avg=164.50, stdev=76.84, samples=4
  write: IOPS=183, BW=734KiB/s (751kB/s)(1564KiB/2132msec)
    clat (usec): min=92, max=80525, avg=4262.77, stdev=7196.25
     lat (usec): min=93, max=80527, avg=4263.44, stdev=7196.30
    clat percentiles (usec):
     |  1.00th=[  108],  5.00th=[  122], 10.00th=[  137], 20.00th=[  184],
     | 30.00th=[  241], 40.00th=[  914], 50.00th=[ 1450], 60.00th=[ 2089],
     | 70.00th=[ 3523], 80.00th=[ 7701], 90.00th=[11600], 95.00th=[18482],
     | 99.00th=[32375], 99.50th=[33424], 99.90th=[80217], 99.95th=[80217],
     | 99.99th=[80217]
   bw (  KiB/s): min=  288, max=  904, per=23.68%, avg=686.00, stdev=278.39, samples=4
   iops        : min=   72, max=  226, avg=171.50, stdev=69.60, samples=4
  lat (usec)   : 4=0.13%, 10=10.94%, 20=12.11%, 50=5.47%, 100=1.17%
  lat (usec)   : 250=15.76%, 500=2.73%, 750=0.91%, 1000=2.73%
  lat (msec)   : 2=16.80%, 4=14.45%, 10=7.94%, 20=6.77%, 50=1.95%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.38%, sys=7.79%, ctx=870, majf=0, minf=36
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=2754KiB/s (2820kB/s), 661KiB/s-761KiB/s (676kB/s-779kB/s), io=5988KiB (6132kB), run=2054-2174msec
  WRITE: bw=2898KiB/s (2967kB/s), 733KiB/s-775KiB/s (751kB/s-794kB/s), io=6300KiB (6451kB), run=2054-2174msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=24283: Fri Jul 19 13:39:02 2024
   read: IOPS=43, BW=173KiB/s (177kB/s)(1480KiB/8566msec)
    clat (usec): min=685, max=19998, avg=2059.38, stdev=3045.68
     lat (usec): min=686, max=19999, avg=2060.07, stdev=3045.70
    clat percentiles (usec):
     |  1.00th=[  734],  5.00th=[  816], 10.00th=[  857], 20.00th=[  930],
     | 30.00th=[  996], 40.00th=[ 1057], 50.00th=[ 1188], 60.00th=[ 1336],
     | 70.00th=[ 1532], 80.00th=[ 1942], 90.00th=[ 2900], 95.00th=[ 6587],
     | 99.00th=[19006], 99.50th=[20055], 99.90th=[20055], 99.95th=[20055],
     | 99.99th=[20055]
   bw (  KiB/s): min=  111, max=  272, per=24.47%, avg=168.88, stdev=48.84, samples=16
   iops        : min=   27, max=   68, avg=42.12, stdev=12.26, samples=16
  write: IOPS=46, BW=186KiB/s (190kB/s)(1592KiB/8566msec)
    clat (usec): min=10230, max=83597, avg=19557.34, stdev=6377.50
     lat (usec): min=10231, max=83599, avg=19558.16, stdev=6377.53
    clat percentiles (usec):
     |  1.00th=[10814],  5.00th=[11994], 10.00th=[12780], 20.00th=[14091],
     | 30.00th=[15795], 40.00th=[18220], 50.00th=[19530], 60.00th=[20841],
     | 70.00th=[21627], 80.00th=[22938], 90.00th=[24511], 95.00th=[27919],
     | 99.00th=[40109], 99.50th=[49546], 99.90th=[83362], 99.95th=[83362],
     | 99.99th=[83362]
   bw (  KiB/s): min=  136, max=  208, per=25.19%, avg=182.88, stdev=23.80, samples=16
   iops        : min=   34, max=   52, avg=45.62, stdev= 5.89, samples=16
  lat (usec)   : 750=0.52%, 1000=14.58%
  lat (msec)   : 2=23.96%, 4=6.12%, 10=1.04%, 20=29.43%, 50=24.22%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.14%, sys=1.39%, ctx=818, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24284: Fri Jul 19 13:39:02 2024
   read: IOPS=41, BW=166KiB/s (170kB/s)(1436KiB/8666msec)
    clat (usec): min=651, max=35040, avg=1956.86, stdev=3282.77
     lat (usec): min=651, max=35041, avg=1957.55, stdev=3282.78
    clat percentiles (usec):
     |  1.00th=[  701],  5.00th=[  799], 10.00th=[  865], 20.00th=[  930],
     | 30.00th=[ 1004], 40.00th=[ 1074], 50.00th=[ 1237], 60.00th=[ 1352],
     | 70.00th=[ 1500], 80.00th=[ 1762], 90.00th=[ 2311], 95.00th=[ 6128],
     | 99.00th=[20579], 99.50th=[28967], 99.90th=[34866], 99.95th=[34866],
     | 99.99th=[34866]
   bw (  KiB/s): min=   80, max=  256, per=23.87%, avg=164.71, stdev=43.35, samples=17
   iops        : min=   20, max=   64, avg=41.06, stdev=10.84, samples=17
  write: IOPS=47, BW=189KiB/s (193kB/s)(1636KiB/8666msec)
    clat (usec): min=9960, max=82936, avg=19440.22, stdev=6617.50
     lat (usec): min=9961, max=82938, avg=19441.12, stdev=6617.55
    clat percentiles (usec):
     |  1.00th=[10814],  5.00th=[11731], 10.00th=[12387], 20.00th=[13960],
     | 30.00th=[15664], 40.00th=[17695], 50.00th=[19268], 60.00th=[20579],
     | 70.00th=[21890], 80.00th=[22938], 90.00th=[24773], 95.00th=[29492],
     | 99.00th=[41157], 99.50th=[46924], 99.90th=[83362], 99.95th=[83362],
     | 99.99th=[83362]
   bw (  KiB/s): min=  136, max=  241, per=25.79%, avg=187.24, stdev=29.65, samples=17
   iops        : min=   34, max=   60, avg=46.71, stdev= 7.39, samples=17
  lat (usec)   : 750=1.30%, 1000=12.63%
  lat (msec)   : 2=25.91%, 4=4.17%, 10=1.30%, 20=30.47%, 50=24.09%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.10%, sys=1.44%, ctx=810, majf=0, minf=31
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24285: Fri Jul 19 13:39:02 2024
   read: IOPS=47, BW=191KiB/s (196kB/s)(1564KiB/8191msec)
    clat (usec): min=607, max=35546, avg=2116.12, stdev=3637.63
     lat (usec): min=607, max=35548, avg=2116.85, stdev=3637.72
    clat percentiles (usec):
     |  1.00th=[  725],  5.00th=[  816], 10.00th=[  848], 20.00th=[  914],
     | 30.00th=[  963], 40.00th=[ 1012], 50.00th=[ 1106], 60.00th=[ 1270],
     | 70.00th=[ 1450], 80.00th=[ 1713], 90.00th=[ 2278], 95.00th=[10552],
     | 99.00th=[19792], 99.50th=[23462], 99.90th=[35390], 99.95th=[35390],
     | 99.99th=[35390]
   bw (  KiB/s): min=  104, max=  328, per=28.19%, avg=194.50, stdev=72.07, samples=16
   iops        : min=   26, max=   82, avg=48.62, stdev=18.02, samples=16
  write: IOPS=46, BW=184KiB/s (189kB/s)(1508KiB/8191msec)
    clat (usec): min=11198, max=63107, avg=19475.44, stdev=6022.33
     lat (usec): min=11199, max=63109, avg=19476.32, stdev=6022.50
    clat percentiles (usec):
     |  1.00th=[11469],  5.00th=[12387], 10.00th=[12911], 20.00th=[14484],
     | 30.00th=[15795], 40.00th=[17695], 50.00th=[19268], 60.00th=[20579],
     | 70.00th=[21627], 80.00th=[22676], 90.00th=[24511], 95.00th=[29230],
     | 99.00th=[40109], 99.50th=[53740], 99.90th=[63177], 99.95th=[63177],
     | 99.99th=[63177]
   bw (  KiB/s): min=  120, max=  216, per=25.34%, avg=184.00, stdev=26.29, samples=16
   iops        : min=   30, max=   54, avg=46.00, stdev= 6.57, samples=16
  lat (usec)   : 750=0.65%, 1000=18.23%
  lat (msec)   : 2=24.87%, 4=3.91%, 10=0.52%, 20=29.82%, 50=21.74%
  lat (msec)   : 100=0.26%
  cpu          : usr=0.12%, sys=1.49%, ctx=821, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=24286: Fri Jul 19 13:39:02 2024
   read: IOPS=45, BW=182KiB/s (186kB/s)(1508KiB/8299msec)
    clat (usec): min=734, max=26879, avg=1979.36, stdev=3058.23
     lat (usec): min=734, max=26880, avg=1980.12, stdev=3058.24
    clat percentiles (usec):
     |  1.00th=[  758],  5.00th=[  840], 10.00th=[  889], 20.00th=[  938],
     | 30.00th=[  996], 40.00th=[ 1090], 50.00th=[ 1188], 60.00th=[ 1303],
     | 70.00th=[ 1467], 80.00th=[ 1795], 90.00th=[ 2278], 95.00th=[ 6849],
     | 99.00th=[20055], 99.50th=[24773], 99.90th=[26870], 99.95th=[26870],
     | 99.99th=[26870]
   bw (  KiB/s): min=  112, max=  232, per=26.01%, avg=179.44, stdev=36.68, samples=16
   iops        : min=   28, max=   58, avg=44.81, stdev= 9.14, samples=16
  write: IOPS=47, BW=188KiB/s (193kB/s)(1564KiB/8299msec)
    clat (usec): min=10360, max=90183, avg=19284.60, stdev=6514.56
     lat (usec): min=10361, max=90186, avg=19285.45, stdev=6514.62
    clat percentiles (usec):
     |  1.00th=[11076],  5.00th=[11863], 10.00th=[12387], 20.00th=[13829],
     | 30.00th=[15533], 40.00th=[17695], 50.00th=[19268], 60.00th=[20579],
     | 70.00th=[21627], 80.00th=[22938], 90.00th=[24773], 95.00th=[27132],
     | 99.00th=[44827], 99.50th=[45876], 99.90th=[90702], 99.95th=[90702],
     | 99.99th=[90702]
   bw (  KiB/s): min=  128, max=  216, per=25.82%, avg=187.44, stdev=24.12, samples=16
   iops        : min=   32, max=   54, avg=46.81, stdev= 6.06, samples=16
  lat (usec)   : 750=0.26%, 1000=14.84%
  lat (msec)   : 2=26.69%, 4=4.30%, 10=0.91%, 20=29.95%, 50=22.92%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.13%, sys=1.42%, ctx=814, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=691KiB/s (708kB/s), 166KiB/s-191KiB/s (170kB/s-196kB/s), io=5988KiB (6132kB), run=8191-8666msec
  WRITE: bw=727KiB/s (744kB/s), 184KiB/s-189KiB/s (189kB/s-193kB/s), io=6300KiB (6451kB), run=8191-8666msec