== sanity test 398b: DIO and buffer IO race ============== 21:35:00 (1736994900)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0698094 s, 180 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=23574: Wed Jan 15 21:35:04 2025
   read: IOPS=163, BW=653KiB/s (669kB/s)(1480KiB/2265msec)
    clat (usec): min=3, max=24306, avg=1485.30, stdev=2700.21
     lat (usec): min=4, max=24307, avg=1486.00, stdev=2700.20
    clat percentiles (usec):
     |  1.00th=[    6],  5.00th=[    8], 10.00th=[   10], 20.00th=[   12],
     | 30.00th=[   15], 40.00th=[   21], 50.00th=[   31], 60.00th=[ 1369],
     | 70.00th=[ 2114], 80.00th=[ 2671], 90.00th=[ 3621], 95.00th=[ 4490],
     | 99.00th=[15139], 99.50th=[23462], 99.90th=[24249], 99.95th=[24249],
     | 99.99th=[24249]
   bw (  KiB/s): min=  431, max=  824, per=23.83%, avg=629.75, stdev=173.25, samples=4
   iops        : min=  107, max=  206, avg=157.25, stdev=43.60, samples=4
  write: IOPS=175, BW=703KiB/s (720kB/s)(1592KiB/2265msec)
    clat (usec): min=83, max=54799, avg=4289.10, stdev=6252.35
     lat (usec): min=83, max=54799, avg=4290.29, stdev=6252.55
    clat percentiles (usec):
     |  1.00th=[   91],  5.00th=[  143], 10.00th=[  151], 20.00th=[  180],
     | 30.00th=[  249], 40.00th=[ 1074], 50.00th=[ 1565], 60.00th=[ 2540],
     | 70.00th=[ 4424], 80.00th=[ 8291], 90.00th=[12387], 95.00th=[16450],
     | 99.00th=[27919], 99.50th=[36439], 99.90th=[54789], 99.95th=[54789],
     | 99.99th=[54789]
   bw (  KiB/s): min=  463, max=  800, per=24.59%, avg=683.75, stdev=150.79, samples=4
   iops        : min=  115, max=  200, avg=170.75, stdev=38.06, samples=4
  lat (usec)   : 4=0.13%, 10=5.73%, 20=13.15%, 50=7.03%, 100=0.78%
  lat (usec)   : 250=15.36%, 500=2.21%, 750=0.65%, 1000=2.21%
  lat (msec)   : 2=13.80%, 4=19.66%, 10=11.07%, 20=6.25%, 50=1.82%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.57%, sys=8.52%, ctx=967, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23575: Wed Jan 15 21:35:04 2025
   read: IOPS=160, BW=640KiB/s (656kB/s)(1436KiB/2242msec)
    clat (usec): min=4, max=22142, avg=1393.14, stdev=2178.84
     lat (usec): min=4, max=22143, avg=1393.96, stdev=2178.76
    clat percentiles (usec):
     |  1.00th=[    6],  5.00th=[    8], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   15], 40.00th=[   24], 50.00th=[  619], 60.00th=[ 1549],
     | 70.00th=[ 2008], 80.00th=[ 2671], 90.00th=[ 3425], 95.00th=[ 4146],
     | 99.00th=[10552], 99.50th=[18220], 99.90th=[22152], 99.95th=[22152],
     | 99.99th=[22152]
   bw (  KiB/s): min=  518, max=  720, per=24.12%, avg=637.50, stdev=96.68, samples=4
   iops        : min=  129, max=  180, avg=159.25, stdev=24.38, samples=4
  write: IOPS=182, BW=730KiB/s (747kB/s)(1636KiB/2242msec)
    clat (usec): min=91, max=58548, avg=4237.26, stdev=6313.85
     lat (usec): min=91, max=58549, avg=4238.40, stdev=6313.73
    clat percentiles (usec):
     |  1.00th=[  125],  5.00th=[  145], 10.00th=[  155], 20.00th=[  184],
     | 30.00th=[  260], 40.00th=[ 1045], 50.00th=[ 1500], 60.00th=[ 2409],
     | 70.00th=[ 4080], 80.00th=[ 7570], 90.00th=[13173], 95.00th=[16581],
     | 99.00th=[23987], 99.50th=[32900], 99.90th=[58459], 99.95th=[58459],
     | 99.99th=[58459]
   bw (  KiB/s): min=  598, max=  792, per=25.80%, avg=717.50, stdev=88.38, samples=4
   iops        : min=  149, max=  198, avg=179.25, stdev=22.32, samples=4
  lat (usec)   : 10=6.64%, 20=10.81%, 50=5.08%, 100=0.65%, 250=16.02%
  lat (usec)   : 500=2.47%, 750=0.39%, 1000=2.47%
  lat (msec)   : 2=17.45%, 4=19.40%, 10=9.77%, 20=7.29%, 50=1.43%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.27%, sys=9.10%, ctx=977, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23576: Wed Jan 15 21:35:04 2025
   read: IOPS=175, BW=703KiB/s (720kB/s)(1564KiB/2224msec)
    clat (usec): min=3, max=31120, avg=1621.09, stdev=3277.73
     lat (usec): min=3, max=31120, avg=1621.76, stdev=3277.73
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    8], 10.00th=[   10], 20.00th=[   13],
     | 30.00th=[   17], 40.00th=[   26], 50.00th=[   81], 60.00th=[ 1500],
     | 70.00th=[ 1991], 80.00th=[ 2573], 90.00th=[ 3523], 95.00th=[ 4752],
     | 99.00th=[16712], 99.50th=[29492], 99.90th=[31065], 99.95th=[31065],
     | 99.99th=[31065]
   bw (  KiB/s): min=  600, max=  768, per=25.63%, avg=677.50, stdev=73.49, samples=4
   iops        : min=  150, max=  192, avg=169.25, stdev=18.32, samples=4
  write: IOPS=169, BW=678KiB/s (694kB/s)(1508KiB/2224msec)
    clat (usec): min=101, max=39481, avg=4195.37, stdev=5759.12
     lat (usec): min=102, max=39482, avg=4196.72, stdev=5759.44
    clat percentiles (usec):
     |  1.00th=[  115],  5.00th=[  141], 10.00th=[  155], 20.00th=[  186],
     | 30.00th=[  388], 40.00th=[ 1074], 50.00th=[ 1795], 60.00th=[ 2573],
     | 70.00th=[ 4293], 80.00th=[ 8029], 90.00th=[11863], 95.00th=[16450],
     | 99.00th=[26084], 99.50th=[32113], 99.90th=[39584], 99.95th=[39584],
     | 99.99th=[39584]
   bw (  KiB/s): min=  584, max=  790, per=23.86%, avg=663.50, stdev=96.67, samples=4
   iops        : min=  146, max=  197, avg=165.75, stdev=23.95, samples=4
  lat (usec)   : 4=0.26%, 10=5.34%, 20=12.50%, 50=6.77%, 100=1.04%
  lat (usec)   : 250=13.28%, 500=2.86%, 750=0.39%, 1000=2.99%
  lat (msec)   : 2=16.28%, 4=19.66%, 10=10.55%, 20=6.64%, 50=1.43%
  cpu          : usr=0.31%, sys=9.22%, ctx=1038, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23577: Wed Jan 15 21:35:04 2025
   read: IOPS=176, BW=707KiB/s (724kB/s)(1508KiB/2132msec)
    clat (usec): min=4, max=21668, avg=1366.09, stdev=2283.92
     lat (usec): min=4, max=21668, avg=1366.88, stdev=2283.94
    clat percentiles (usec):
     |  1.00th=[    6],  5.00th=[    7], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   15], 40.00th=[   21], 50.00th=[   34], 60.00th=[ 1270],
     | 70.00th=[ 2024], 80.00th=[ 2737], 90.00th=[ 3392], 95.00th=[ 4293],
     | 99.00th=[13173], 99.50th=[15664], 99.90th=[21627], 99.95th=[21627],
     | 99.99th=[21627]
   bw (  KiB/s): min=  670, max=  736, per=26.16%, avg=691.50, stdev=30.04, samples=4
   iops        : min=  167, max=  184, avg=172.75, stdev= 7.63, samples=4
  write: IOPS=183, BW=734KiB/s (751kB/s)(1564KiB/2132msec)
    clat (usec): min=83, max=41723, avg=4114.89, stdev=5689.57
     lat (usec): min=84, max=41725, avg=4116.01, stdev=5689.41
    clat percentiles (usec):
     |  1.00th=[  101],  5.00th=[  139], 10.00th=[  155], 20.00th=[  176],
     | 30.00th=[  343], 40.00th=[ 1123], 50.00th=[ 1680], 60.00th=[ 2474],
     | 70.00th=[ 4113], 80.00th=[ 7898], 90.00th=[11469], 95.00th=[15008],
     | 99.00th=[25560], 99.50th=[30802], 99.90th=[41681], 99.95th=[41681],
     | 99.99th=[41681]
   bw (  KiB/s): min=  678, max=  848, per=26.16%, avg=727.50, stdev=80.67, samples=4
   iops        : min=  169, max=  212, avg=181.75, stdev=20.27, samples=4
  lat (usec)   : 10=7.94%, 20=11.46%, 50=6.51%, 100=0.91%, 250=14.45%
  lat (usec)   : 500=2.47%, 750=1.17%, 1000=2.34%
  lat (msec)   : 2=15.10%, 4=18.62%, 10=11.20%, 20=6.51%, 50=1.30%
  cpu          : usr=0.42%, sys=9.01%, ctx=996, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=2644KiB/s (2707kB/s), 640KiB/s-707KiB/s (656kB/s-724kB/s), io=5988KiB (6132kB), run=2132-2265msec
  WRITE: bw=2781KiB/s (2848kB/s), 678KiB/s-734KiB/s (694kB/s-751kB/s), io=6300KiB (6451kB), run=2132-2265msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=23570: Wed Jan 15 21:35:10 2025
   read: IOPS=44, BW=179KiB/s (183kB/s)(1480KiB/8269msec)
    clat (usec): min=744, max=43230, avg=2306.37, stdev=3846.56
     lat (usec): min=744, max=43231, avg=2307.59, stdev=3846.72
    clat percentiles (usec):
     |  1.00th=[  766],  5.00th=[  857], 10.00th=[  938], 20.00th=[ 1057],
     | 30.00th=[ 1156], 40.00th=[ 1270], 50.00th=[ 1401], 60.00th=[ 1516],
     | 70.00th=[ 1647], 80.00th=[ 2008], 90.00th=[ 2868], 95.00th=[ 7504],
     | 99.00th=[21103], 99.50th=[28443], 99.90th=[43254], 99.95th=[43254],
     | 99.99th=[43254]
   bw (  KiB/s): min=  111, max=  304, per=24.45%, avg=174.31, stdev=52.76, samples=16
   iops        : min=   27, max=   76, avg=43.44, stdev=13.25, samples=16
  write: IOPS=48, BW=193KiB/s (197kB/s)(1592KiB/8269msec)
    clat (usec): min=7961, max=63364, avg=18589.11, stdev=6734.01
     lat (usec): min=7964, max=63365, avg=18590.53, stdev=6733.99
    clat percentiles (usec):
     |  1.00th=[ 8979],  5.00th=[10552], 10.00th=[11600], 20.00th=[13042],
     | 30.00th=[14746], 40.00th=[16581], 50.00th=[18220], 60.00th=[19530],
     | 70.00th=[20317], 80.00th=[22152], 90.00th=[25035], 95.00th=[29492],
     | 99.00th=[47449], 99.50th=[55837], 99.90th=[63177], 99.95th=[63177],
     | 99.99th=[63177]
   bw (  KiB/s): min=  127, max=  232, per=25.34%, avg=190.31, stdev=34.32, samples=16
   iops        : min=   31, max=   58, avg=47.44, stdev= 8.66, samples=16
  lat (usec)   : 750=0.26%, 1000=6.51%
  lat (msec)   : 2=31.64%, 4=6.12%, 10=3.26%, 20=34.64%, 50=17.19%
  lat (msec)   : 100=0.39%
  cpu          : usr=0.41%, sys=1.73%, ctx=845, majf=0, minf=31
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23571: Wed Jan 15 21:35:10 2025
   read: IOPS=42, BW=171KiB/s (175kB/s)(1436KiB/8387msec)
    clat (usec): min=704, max=36409, avg=2314.72, stdev=3558.65
     lat (usec): min=705, max=36410, avg=2315.86, stdev=3558.64
    clat percentiles (usec):
     |  1.00th=[  766],  5.00th=[  857], 10.00th=[  930], 20.00th=[ 1029],
     | 30.00th=[ 1139], 40.00th=[ 1237], 50.00th=[ 1369], 60.00th=[ 1500],
     | 70.00th=[ 1729], 80.00th=[ 2180], 90.00th=[ 3523], 95.00th=[ 8717],
     | 99.00th=[16581], 99.50th=[28443], 99.90th=[36439], 99.95th=[36439],
     | 99.99th=[36439]
   bw (  KiB/s): min=   79, max=  248, per=23.89%, avg=170.31, stdev=56.23, samples=16
   iops        : min=   19, max=   62, avg=42.44, stdev=14.05, samples=16
  write: IOPS=48, BW=195KiB/s (200kB/s)(1636KiB/8387msec)
    clat (usec): min=8832, max=67121, avg=18422.00, stdev=6643.72
     lat (usec): min=8833, max=67122, avg=18423.78, stdev=6644.08
    clat percentiles (usec):
     |  1.00th=[ 9634],  5.00th=[10421], 10.00th=[11338], 20.00th=[12911],
     | 30.00th=[14484], 40.00th=[16188], 50.00th=[17957], 60.00th=[19268],
     | 70.00th=[20317], 80.00th=[22414], 90.00th=[26346], 95.00th=[29492],
     | 99.00th=[42730], 99.50th=[47449], 99.90th=[67634], 99.95th=[67634],
     | 99.99th=[67634]
   bw (  KiB/s): min=  135, max=  240, per=25.41%, avg=190.81, stdev=35.37, samples=16
   iops        : min=   33, max=   60, avg=47.56, stdev= 8.82, samples=16
  lat (usec)   : 750=0.39%, 1000=7.16%
  lat (msec)   : 2=28.78%, 4=6.77%, 10=3.26%, 20=35.42%, 50=17.97%
  lat (msec)   : 100=0.26%
  cpu          : usr=0.13%, sys=1.97%, ctx=831, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23572: Wed Jan 15 21:35:10 2025
   read: IOPS=49, BW=196KiB/s (201kB/s)(1564KiB/7967msec)
    clat (usec): min=735, max=35422, avg=2292.43, stdev=3449.21
     lat (usec): min=736, max=35423, avg=2293.50, stdev=3449.21
    clat percentiles (usec):
     |  1.00th=[  783],  5.00th=[  898], 10.00th=[  938], 20.00th=[ 1037],
     | 30.00th=[ 1139], 40.00th=[ 1237], 50.00th=[ 1385], 60.00th=[ 1549],
     | 70.00th=[ 1745], 80.00th=[ 2040], 90.00th=[ 2966], 95.00th=[ 7767],
     | 99.00th=[21890], 99.50th=[24249], 99.90th=[35390], 99.95th=[35390],
     | 99.99th=[35390]
   bw (  KiB/s): min=  104, max=  384, per=28.02%, avg=199.80, stdev=89.17, samples=15
   iops        : min=   26, max=   96, avg=49.80, stdev=22.29, samples=15
  write: IOPS=47, BW=189KiB/s (194kB/s)(1508KiB/7967msec)
    clat (usec): min=8534, max=47550, avg=18702.45, stdev=6207.73
     lat (usec): min=8535, max=47552, avg=18703.85, stdev=6207.80
    clat percentiles (usec):
     |  1.00th=[ 9241],  5.00th=[10552], 10.00th=[11600], 20.00th=[13173],
     | 30.00th=[15139], 40.00th=[16909], 50.00th=[18220], 60.00th=[19530],
     | 70.00th=[20841], 80.00th=[22414], 90.00th=[25822], 95.00th=[29492],
     | 99.00th=[43254], 99.50th=[45876], 99.90th=[47449], 99.95th=[47449],
     | 99.99th=[47449]
   bw (  KiB/s): min=  128, max=  240, per=24.90%, avg=187.00, stdev=34.47, samples=15
   iops        : min=   32, max=   60, avg=46.60, stdev= 8.64, samples=15
  lat (usec)   : 750=0.13%, 1000=7.81%
  lat (msec)   : 2=31.77%, 4=7.16%, 10=3.26%, 20=30.73%, 50=19.14%
  cpu          : usr=0.34%, sys=1.98%, ctx=839, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23573: Wed Jan 15 21:35:10 2025
   read: IOPS=46, BW=185KiB/s (189kB/s)(1508KiB/8169msec)
    clat (usec): min=730, max=24006, avg=2369.72, stdev=3377.05
     lat (usec): min=731, max=24007, avg=2370.83, stdev=3377.14
    clat percentiles (usec):
     |  1.00th=[  775],  5.00th=[  873], 10.00th=[  922], 20.00th=[ 1045],
     | 30.00th=[ 1172], 40.00th=[ 1270], 50.00th=[ 1401], 60.00th=[ 1532],
     | 70.00th=[ 1762], 80.00th=[ 2180], 90.00th=[ 3261], 95.00th=[10683],
     | 99.00th=[19006], 99.50th=[23725], 99.90th=[23987], 99.95th=[23987],
     | 99.99th=[23987]
   bw (  KiB/s): min=   95, max=  264, per=25.50%, avg=181.81, stdev=52.77, samples=16
   iops        : min=   23, max=   66, avg=45.31, stdev=13.37, samples=16
  write: IOPS=47, BW=191KiB/s (196kB/s)(1564KiB/8169msec)
    clat (usec): min=9082, max=45567, avg=18556.01, stdev=5975.29
     lat (usec): min=9085, max=45569, avg=18557.40, stdev=5975.40
    clat percentiles (usec):
     |  1.00th=[ 9634],  5.00th=[10683], 10.00th=[11469], 20.00th=[13173],
     | 30.00th=[15008], 40.00th=[16581], 50.00th=[18482], 60.00th=[19530],
     | 70.00th=[21103], 80.00th=[22676], 90.00th=[25297], 95.00th=[28967],
     | 99.00th=[39584], 99.50th=[43779], 99.90th=[45351], 99.95th=[45351],
     | 99.99th=[45351]
   bw (  KiB/s): min=  135, max=  256, per=25.34%, avg=190.31, stdev=35.41, samples=16
   iops        : min=   33, max=   64, avg=47.44, stdev= 8.83, samples=16
  lat (usec)   : 750=0.39%, 1000=7.03%
  lat (msec)   : 2=29.95%, 4=7.68%, 10=2.99%, 20=33.72%, 50=18.23%
  cpu          : usr=0.26%, sys=1.90%, ctx=831, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=714KiB/s (731kB/s), 171KiB/s-196KiB/s (175kB/s-201kB/s), io=5988KiB (6132kB), run=7967-8387msec
  WRITE: bw=751KiB/s (769kB/s), 189KiB/s-195KiB/s (194kB/s-200kB/s), io=6300KiB (6451kB), run=7967-8387msec