== sanity test 398b: DIO and buffer IO race ============== 19:31:00 (1736987460)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0706846 s, 178 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=23459: Wed Jan 15 19:31:03 2025
   read: IOPS=202, BW=810KiB/s (830kB/s)(1480KiB/1827msec)
    clat (usec): min=4, max=24246, avg=1203.15, stdev=2518.66
     lat (usec): min=4, max=24247, avg=1203.96, stdev=2518.60
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   13], 40.00th=[   18], 50.00th=[   27], 60.00th=[  938],
     | 70.00th=[ 1565], 80.00th=[ 2180], 90.00th=[ 2835], 95.00th=[ 3523],
     | 99.00th=[15270], 99.50th=[23462], 99.90th=[24249], 99.95th=[24249],
     | 99.99th=[24249]
   bw (  KiB/s): min=  688, max=  848, per=24.09%, avg=789.33, stdev=88.12, samples=3
   iops        : min=  172, max=  212, avg=197.33, stdev=22.03, samples=3
  write: IOPS=217, BW=871KiB/s (892kB/s)(1592KiB/1827msec)
    clat (usec): min=88, max=45499, avg=3439.77, stdev=5184.51
     lat (usec): min=88, max=45500, avg=3440.53, stdev=5184.48
    clat percentiles (usec):
     |  1.00th=[   93],  5.00th=[  114], 10.00th=[  135], 20.00th=[  184],
     | 30.00th=[  562], 40.00th=[  979], 50.00th=[ 1254], 60.00th=[ 2057],
     | 70.00th=[ 3163], 80.00th=[ 5735], 90.00th=[10290], 95.00th=[13960],
     | 99.00th=[23200], 99.50th=[26608], 99.90th=[45351], 99.95th=[45351],
     | 99.99th=[45351]
   bw (  KiB/s): min=  696, max=  984, per=24.90%, avg=858.67, stdev=147.59, samples=3
   iops        : min=  174, max=  246, avg=214.67, stdev=36.90, samples=3
  lat (usec)   : 10=10.29%, 20=10.55%, 50=5.86%, 100=1.69%, 250=12.89%
  lat (usec)   : 500=1.82%, 750=1.69%, 1000=6.38%
  lat (msec)   : 2=16.80%, 4=16.93%, 10=8.72%, 20=5.34%, 50=1.04%
  cpu          : usr=0.27%, sys=9.75%, ctx=950, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23461: Wed Jan 15 19:31:03 2025
   read: IOPS=197, BW=791KiB/s (810kB/s)(1436KiB/1815msec)
    clat (usec): min=3, max=24437, avg=1177.24, stdev=2385.30
     lat (usec): min=3, max=24438, avg=1178.41, stdev=2385.25
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    8], 20.00th=[   11],
     | 30.00th=[   13], 40.00th=[   17], 50.00th=[   29], 60.00th=[ 1106],
     | 70.00th=[ 1631], 80.00th=[ 2114], 90.00th=[ 2638], 95.00th=[ 3326],
     | 99.00th=[12911], 99.50th=[21103], 99.90th=[24511], 99.95th=[24511],
     | 99.99th=[24511]
   bw (  KiB/s): min=  448, max= 1136, per=23.92%, avg=784.00, stdev=344.28, samples=3
   iops        : min=  112, max=  284, avg=196.00, stdev=86.07, samples=3
  write: IOPS=225, BW=901KiB/s (923kB/s)(1636KiB/1815msec)
    clat (usec): min=96, max=27734, avg=3370.09, stdev=4762.59
     lat (usec): min=97, max=27735, avg=3370.88, stdev=4762.72
    clat percentiles (usec):
     |  1.00th=[  100],  5.00th=[  116], 10.00th=[  141], 20.00th=[  180],
     | 30.00th=[  269], 40.00th=[  881], 50.00th=[ 1352], 60.00th=[ 2008],
     | 70.00th=[ 3294], 80.00th=[ 6587], 90.00th=[ 9110], 95.00th=[12387],
     | 99.00th=[21365], 99.50th=[25560], 99.90th=[27657], 99.95th=[27657],
     | 99.99th=[27657]
   bw (  KiB/s): min=  536, max= 1056, per=24.59%, avg=848.00, stdev=275.16, samples=3
   iops        : min=  134, max=  264, avg=212.00, stdev=68.79, samples=3
  lat (usec)   : 4=0.78%, 10=8.46%, 20=11.46%, 50=3.78%, 100=1.17%
  lat (usec)   : 250=15.62%, 500=1.95%, 750=2.08%, 1000=4.56%
  lat (msec)   : 2=18.10%, 4=15.89%, 10=11.20%, 20=3.91%, 50=1.04%
  cpu          : usr=0.22%, sys=10.20%, ctx=977, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23462: Wed Jan 15 19:31:03 2025
   read: IOPS=221, BW=886KiB/s (907kB/s)(1564KiB/1766msec)
    clat (usec): min=3, max=16945, avg=1405.92, stdev=2251.23
     lat (usec): min=3, max=16946, avg=1406.50, stdev=2251.25
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   13], 40.00th=[   21], 50.00th=[  865], 60.00th=[ 1434],
     | 70.00th=[ 1844], 80.00th=[ 2376], 90.00th=[ 3097], 95.00th=[ 4228],
     | 99.00th=[13435], 99.50th=[16450], 99.90th=[16909], 99.95th=[16909],
     | 99.99th=[16909]
   bw (  KiB/s): min=  568, max= 1000, per=25.88%, avg=848.00, stdev=242.78, samples=3
   iops        : min=  142, max=  250, avg=212.00, stdev=60.70, samples=3
  write: IOPS=213, BW=854KiB/s (874kB/s)(1508KiB/1766msec)
    clat (usec): min=95, max=49224, avg=3200.30, stdev=5142.68
     lat (usec): min=95, max=49225, avg=3201.02, stdev=5142.69
    clat percentiles (usec):
     |  1.00th=[  104],  5.00th=[  120], 10.00th=[  139], 20.00th=[  186],
     | 30.00th=[  429], 40.00th=[  955], 50.00th=[ 1532], 60.00th=[ 1844],
     | 70.00th=[ 2573], 80.00th=[ 4621], 90.00th=[ 8586], 95.00th=[13960],
     | 99.00th=[23462], 99.50th=[28181], 99.90th=[49021], 99.95th=[49021],
     | 99.99th=[49021]
   bw (  KiB/s): min=  672, max=  976, per=24.52%, avg=845.33, stdev=156.43, samples=3
   iops        : min=  168, max=  244, avg=211.33, stdev=39.11, samples=3
  lat (usec)   : 4=0.65%, 10=9.90%, 20=8.98%, 50=4.17%, 100=0.78%
  lat (usec)   : 250=13.02%, 500=2.34%, 750=1.69%, 1000=5.47%
  lat (msec)   : 2=21.74%, 4=17.45%, 10=9.24%, 20=3.78%, 50=0.78%
  cpu          : usr=0.34%, sys=9.74%, ctx=1025, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23463: Wed Jan 15 19:31:03 2025
   read: IOPS=219, BW=879KiB/s (900kB/s)(1508KiB/1715msec)
    clat (usec): min=3, max=28711, avg=1240.75, stdev=2617.19
     lat (usec): min=3, max=28712, avg=1242.15, stdev=2621.94
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   12], 40.00th=[   15], 50.00th=[   26], 60.00th=[ 1045],
     | 70.00th=[ 1614], 80.00th=[ 2147], 90.00th=[ 2966], 95.00th=[ 3752],
     | 99.00th=[15533], 99.50th=[21365], 99.90th=[28705], 99.95th=[28705],
     | 99.99th=[28705]
   bw (  KiB/s): min=  528, max=  950, per=23.17%, avg=759.33, stdev=213.92, samples=3
   iops        : min=  132, max=  237, avg=189.67, stdev=53.26, samples=3
  write: IOPS=227, BW=912KiB/s (934kB/s)(1564KiB/1715msec)
    clat (usec): min=89, max=48912, avg=3154.92, stdev=5348.68
     lat (usec): min=90, max=48913, avg=3155.72, stdev=5348.67
    clat percentiles (usec):
     |  1.00th=[   95],  5.00th=[  111], 10.00th=[  124], 20.00th=[  161],
     | 30.00th=[  190], 40.00th=[  515], 50.00th=[ 1172], 60.00th=[ 1729],
     | 70.00th=[ 2638], 80.00th=[ 4883], 90.00th=[ 8586], 95.00th=[13304],
     | 99.00th=[27132], 99.50th=[34866], 99.90th=[49021], 99.95th=[49021],
     | 99.99th=[49021]
   bw (  KiB/s): min=  568, max= 1000, per=23.72%, avg=818.00, stdev=223.88, samples=3
   iops        : min=  142, max=  250, avg=204.33, stdev=55.90, samples=3
  lat (usec)   : 4=0.91%, 10=11.46%, 20=10.03%, 50=4.69%, 100=0.78%
  lat (usec)   : 250=16.80%, 500=2.73%, 750=1.69%, 1000=4.69%
  lat (msec)   : 2=15.76%, 4=16.80%, 10=8.85%, 20=3.65%, 50=1.17%
  cpu          : usr=0.35%, sys=8.63%, ctx=928, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3278KiB/s (3356kB/s), 791KiB/s-886KiB/s (810kB/s-907kB/s), io=5988KiB (6132kB), run=1715-1827msec
  WRITE: bw=3448KiB/s (3531kB/s), 854KiB/s-912KiB/s (874kB/s-934kB/s), io=6300KiB (6451kB), run=1715-1827msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=23456: Wed Jan 15 19:31:09 2025
   read: IOPS=50, BW=200KiB/s (205kB/s)(1480KiB/7400msec)
    clat (usec): min=589, max=28280, avg=2124.36, stdev=2997.00
     lat (usec): min=590, max=28281, avg=2125.23, stdev=2997.20
    clat percentiles (usec):
     |  1.00th=[  717],  5.00th=[  799], 10.00th=[  865], 20.00th=[  963],
     | 30.00th=[ 1037], 40.00th=[ 1139], 50.00th=[ 1270], 60.00th=[ 1418],
     | 70.00th=[ 1762], 80.00th=[ 2278], 90.00th=[ 3326], 95.00th=[ 5538],
     | 99.00th=[19268], 99.50th=[20841], 99.90th=[28181], 99.95th=[28181],
     | 99.99th=[28181]
   bw (  KiB/s): min=  112, max=  328, per=25.32%, avg=199.29, stdev=64.28, samples=14
   iops        : min=   28, max=   82, avg=49.71, stdev=16.08, samples=14
  write: IOPS=53, BW=215KiB/s (220kB/s)(1592KiB/7400msec)
    clat (usec): min=8178, max=50003, avg=16590.90, stdev=5361.11
     lat (usec): min=8179, max=50004, avg=16591.96, stdev=5361.11
    clat percentiles (usec):
     |  1.00th=[ 9503],  5.00th=[10421], 10.00th=[11076], 20.00th=[12125],
     | 30.00th=[13304], 40.00th=[14484], 50.00th=[15795], 60.00th=[17171],
     | 70.00th=[18744], 80.00th=[20055], 90.00th=[21890], 95.00th=[26346],
     | 99.00th=[36439], 99.50th=[42730], 99.90th=[50070], 99.95th=[50070],
     | 99.99th=[50070]
   bw (  KiB/s): min=  128, max=  248, per=25.86%, avg=214.14, stdev=34.63, samples=14
   iops        : min=   32, max=   62, avg=53.43, stdev= 8.62, samples=14
  lat (usec)   : 750=1.30%, 1000=10.94%
  lat (msec)   : 2=24.09%, 4=8.72%, 10=3.12%, 20=41.15%, 50=10.55%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.16%, sys=1.81%, ctx=903, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23457: Wed Jan 15 19:31:09 2025
   read: IOPS=47, BW=189KiB/s (193kB/s)(1436KiB/7601msec)
    clat (usec): min=672, max=28450, avg=2073.51, stdev=2684.58
     lat (usec): min=672, max=28450, avg=2074.36, stdev=2684.56
    clat percentiles (usec):
     |  1.00th=[  693],  5.00th=[  807], 10.00th=[  848], 20.00th=[  930],
     | 30.00th=[ 1037], 40.00th=[ 1123], 50.00th=[ 1254], 60.00th=[ 1500],
     | 70.00th=[ 1745], 80.00th=[ 2638], 90.00th=[ 3326], 95.00th=[ 4948],
     | 99.00th=[15401], 99.50th=[21365], 99.90th=[28443], 99.95th=[28443],
     | 99.99th=[28443]
   bw (  KiB/s): min=   96, max=  296, per=23.84%, avg=187.60, stdev=54.78, samples=15
   iops        : min=   24, max=   74, avg=46.80, stdev=13.72, samples=15
  write: IOPS=53, BW=215KiB/s (220kB/s)(1636KiB/7601msec)
    clat (usec): min=8612, max=99429, avg=16730.50, stdev=6246.84
     lat (usec): min=8613, max=99430, avg=16731.67, stdev=6246.96
    clat percentiles (usec):
     |  1.00th=[ 9372],  5.00th=[10290], 10.00th=[10945], 20.00th=[12125],
     | 30.00th=[13435], 40.00th=[15139], 50.00th=[16450], 60.00th=[17695],
     | 70.00th=[18744], 80.00th=[20055], 90.00th=[21890], 95.00th=[23462],
     | 99.00th=[34341], 99.50th=[38011], 99.90th=[99091], 99.95th=[99091],
     | 99.99th=[99091]
   bw (  KiB/s): min=  160, max=  256, per=25.81%, avg=213.73, stdev=24.99, samples=15
   iops        : min=   40, max=   64, avg=53.33, stdev= 6.23, samples=15
  lat (usec)   : 750=1.04%, 1000=11.85%
  lat (msec)   : 2=21.09%, 4=9.77%, 10=3.52%, 20=41.80%, 50=10.81%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.16%, sys=1.78%, ctx=920, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23458: Wed Jan 15 19:31:09 2025
   read: IOPS=54, BW=217KiB/s (222kB/s)(1564KiB/7203msec)
    clat (usec): min=687, max=30212, avg=1981.43, stdev=2651.58
     lat (usec): min=688, max=30213, avg=1982.27, stdev=2651.60
    clat percentiles (usec):
     |  1.00th=[  734],  5.00th=[  799], 10.00th=[  857], 20.00th=[  930],
     | 30.00th=[  996], 40.00th=[ 1106], 50.00th=[ 1221], 60.00th=[ 1369],
     | 70.00th=[ 1713], 80.00th=[ 2409], 90.00th=[ 3064], 95.00th=[ 4293],
     | 99.00th=[13042], 99.50th=[21627], 99.90th=[30278], 99.95th=[30278],
     | 99.99th=[30278]
   bw (  KiB/s): min=  104, max=  344, per=27.94%, avg=219.86, stdev=80.15, samples=14
   iops        : min=   26, max=   86, avg=54.86, stdev=20.11, samples=14
  write: IOPS=52, BW=209KiB/s (214kB/s)(1508KiB/7203msec)
    clat (usec): min=7872, max=81989, avg=17008.53, stdev=6396.98
     lat (usec): min=7873, max=81990, avg=17009.57, stdev=6397.03
    clat percentiles (usec):
     |  1.00th=[ 8356],  5.00th=[10159], 10.00th=[10945], 20.00th=[12387],
     | 30.00th=[13698], 40.00th=[15139], 50.00th=[16319], 60.00th=[17433],
     | 70.00th=[18482], 80.00th=[20055], 90.00th=[22676], 95.00th=[27132],
     | 99.00th=[41157], 99.50th=[49021], 99.90th=[82314], 99.95th=[82314],
     | 99.99th=[82314]
   bw (  KiB/s): min=  120, max=  256, per=25.31%, avg=209.57, stdev=36.19, samples=14
   iops        : min=   30, max=   64, avg=52.29, stdev= 9.03, samples=14
  lat (usec)   : 750=0.78%, 1000=14.71%
  lat (msec)   : 2=22.79%, 4=9.77%, 10=3.52%, 20=38.41%, 50=9.90%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.12%, sys=1.92%, ctx=909, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23460: Wed Jan 15 19:31:09 2025
   read: IOPS=51, BW=205KiB/s (210kB/s)(1508KiB/7347msec)
    clat (usec): min=695, max=28263, avg=2196.42, stdev=3123.32
     lat (usec): min=696, max=28264, avg=2197.30, stdev=3123.32
    clat percentiles (usec):
     |  1.00th=[  758],  5.00th=[  832], 10.00th=[  873], 20.00th=[  955],
     | 30.00th=[ 1057], 40.00th=[ 1156], 50.00th=[ 1287], 60.00th=[ 1467],
     | 70.00th=[ 1827], 80.00th=[ 2540], 90.00th=[ 3425], 95.00th=[ 4883],
     | 99.00th=[20317], 99.50th=[22938], 99.90th=[28181], 99.95th=[28181],
     | 99.99th=[28181]
   bw (  KiB/s): min=  112, max=  296, per=25.85%, avg=203.43, stdev=56.89, samples=14
   iops        : min=   28, max=   74, avg=50.86, stdev=14.22, samples=14
  write: IOPS=53, BW=213KiB/s (218kB/s)(1564KiB/7347msec)
    clat (usec): min=7615, max=51746, avg=16627.74, stdev=5276.36
     lat (usec): min=7616, max=51747, avg=16628.80, stdev=5276.38
    clat percentiles (usec):
     |  1.00th=[ 8717],  5.00th=[10421], 10.00th=[11076], 20.00th=[12125],
     | 30.00th=[13566], 40.00th=[14877], 50.00th=[16319], 60.00th=[17433],
     | 70.00th=[18220], 80.00th=[19530], 90.00th=[21627], 95.00th=[26084],
     | 99.00th=[40109], 99.50th=[43779], 99.90th=[51643], 99.95th=[51643],
     | 99.99th=[51643]
   bw (  KiB/s): min=  144, max=  256, per=25.60%, avg=212.00, stdev=34.34, samples=14
   iops        : min=   36, max=   64, avg=53.00, stdev= 8.58, samples=14
  lat (usec)   : 750=0.39%, 1000=11.98%
  lat (msec)   : 2=24.22%, 4=8.98%, 10=3.65%, 20=41.54%, 50=9.11%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.20%, sys=1.85%, ctx=926, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=788KiB/s (807kB/s), 189KiB/s-217KiB/s (193kB/s-222kB/s), io=5988KiB (6132kB), run=7203-7601msec
  WRITE: bw=829KiB/s (849kB/s), 209KiB/s-215KiB/s (214kB/s-220kB/s), io=6300KiB (6451kB), run=7203-7601msec