== sanity test 398b: DIO and buffer IO race ============== 08:18:41 (1717157921)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.094532 s, 133 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=19576: Fri May 31 08:18:44 2024
   read: IOPS=187, BW=750KiB/s (768kB/s)(1480KiB/1973msec)
    clat (usec): min=3, max=48805, avg=1345.52, stdev=3761.33
     lat (usec): min=3, max=48806, avg=1346.07, stdev=3761.35
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    8],
     | 30.00th=[   10], 40.00th=[   13], 50.00th=[   23], 60.00th=[  865],
     | 70.00th=[ 1401], 80.00th=[ 1893], 90.00th=[ 2769], 95.00th=[ 3621],
     | 99.00th=[21365], 99.50th=[23725], 99.90th=[49021], 99.95th=[49021],
     | 99.99th=[49021]
   bw (  KiB/s): min=  624, max=  832, per=24.78%, avg=736.00, stdev=104.92, samples=3
   iops        : min=  156, max=  208, avg=184.00, stdev=26.23, samples=3
  write: IOPS=201, BW=807KiB/s (826kB/s)(1592KiB/1973msec)
    clat (usec): min=90, max=31435, avg=3676.26, stdev=5729.72
     lat (usec): min=90, max=31435, avg=3676.84, stdev=5729.71
    clat percentiles (usec):
     |  1.00th=[   91],  5.00th=[  103], 10.00th=[  109], 20.00th=[  121],
     | 30.00th=[  143], 40.00th=[  570], 50.00th=[  930], 60.00th=[ 1631],
     | 70.00th=[ 2409], 80.00th=[ 7898], 90.00th=[12125], 95.00th=[17695],
     | 99.00th=[22676], 99.50th=[27395], 99.90th=[31327], 99.95th=[31327],
     | 99.99th=[31327]
   bw (  KiB/s): min=  632, max=  912, per=25.00%, avg=781.33, stdev=140.93, samples=3
   iops        : min=  158, max=  228, avg=195.33, stdev=35.23, samples=3
  lat (usec)   : 4=1.30%, 10=14.32%, 20=6.77%, 50=4.04%, 100=3.26%
  lat (usec)   : 250=16.80%, 500=1.43%, 750=2.21%, 1000=7.03%
  lat (msec)   : 2=16.28%, 4=11.33%, 10=6.38%, 20=7.16%, 50=1.69%
  cpu          : usr=0.25%, sys=7.20%, ctx=850, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19577: Fri May 31 08:18:44 2024
   read: IOPS=178, BW=712KiB/s (729kB/s)(1436KiB/2016msec)
    clat (usec): min=3, max=28225, avg=1281.91, stdev=2933.01
     lat (usec): min=3, max=28225, avg=1282.45, stdev=2933.02
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    6], 20.00th=[    8],
     | 30.00th=[    9], 40.00th=[   12], 50.00th=[   21], 60.00th=[  947],
     | 70.00th=[ 1516], 80.00th=[ 2040], 90.00th=[ 2704], 95.00th=[ 3851],
     | 99.00th=[17695], 99.50th=[21890], 99.90th=[28181], 99.95th=[28181],
     | 99.99th=[28181]
   bw (  KiB/s): min=  624, max=  792, per=23.91%, avg=710.00, stdev=73.43, samples=4
   iops        : min=  156, max=  198, avg=177.50, stdev=18.36, samples=4
  write: IOPS=202, BW=812KiB/s (831kB/s)(1636KiB/2016msec)
    clat (usec): min=84, max=39002, avg=3773.48, stdev=6059.48
     lat (usec): min=85, max=39003, avg=3774.33, stdev=6059.67
    clat percentiles (usec):
     |  1.00th=[   96],  5.00th=[  104], 10.00th=[  113], 20.00th=[  126],
     | 30.00th=[  163], 40.00th=[  660], 50.00th=[ 1012], 60.00th=[ 1582],
     | 70.00th=[ 2409], 80.00th=[ 7504], 90.00th=[12387], 95.00th=[16450],
     | 99.00th=[23725], 99.50th=[26870], 99.90th=[39060], 99.95th=[39060],
     | 99.99th=[39060]
   bw (  KiB/s): min=  744, max=  856, per=25.98%, avg=812.00, stdev=51.64, samples=4
   iops        : min=  186, max=  214, avg=203.00, stdev=12.91, samples=4
  lat (usec)   : 4=1.30%, 10=15.49%, 20=6.51%, 50=2.60%, 100=2.08%
  lat (usec)   : 250=17.58%, 500=1.43%, 750=2.21%, 1000=5.73%
  lat (msec)   : 2=17.32%, 4=13.28%, 10=4.82%, 20=7.16%, 50=2.47%
  cpu          : usr=0.25%, sys=7.30%, ctx=873, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19578: Fri May 31 08:18:44 2024
   read: IOPS=204, BW=819KiB/s (839kB/s)(1564KiB/1909msec)
    clat (usec): min=3, max=41709, avg=1473.76, stdev=3878.04
     lat (usec): min=3, max=41710, avg=1474.25, stdev=3878.07
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    6], 20.00th=[    8],
     | 30.00th=[    9], 40.00th=[   11], 50.00th=[   20], 60.00th=[  840],
     | 70.00th=[ 1516], 80.00th=[ 2073], 90.00th=[ 2868], 95.00th=[ 4359],
     | 99.00th=[24773], 99.50th=[28181], 99.90th=[41681], 99.95th=[41681],
     | 99.99th=[41681]
   bw (  KiB/s): min=  584, max= 1040, per=26.67%, avg=792.00, stdev=230.62, samples=3
   iops        : min=  146, max=  260, avg=198.00, stdev=57.65, samples=3
  write: IOPS=197, BW=790KiB/s (809kB/s)(1508KiB/1909msec)
    clat (usec): min=83, max=32879, avg=3517.23, stdev=5657.86
     lat (usec): min=83, max=32880, avg=3517.90, stdev=5657.80
    clat percentiles (usec):
     |  1.00th=[   89],  5.00th=[   98], 10.00th=[  109], 20.00th=[  120],
     | 30.00th=[  141], 40.00th=[  235], 50.00th=[  922], 60.00th=[ 1483],
     | 70.00th=[ 2573], 80.00th=[ 7046], 90.00th=[11469], 95.00th=[17433],
     | 99.00th=[23200], 99.50th=[26608], 99.90th=[32900], 99.95th=[32900],
     | 99.99th=[32900]
   bw (  KiB/s): min=  640, max= 1064, per=25.51%, avg=797.33, stdev=232.18, samples=3
   iops        : min=  160, max=  266, avg=199.33, stdev=58.05, samples=3
  lat (usec)   : 4=2.34%, 10=17.19%, 20=5.99%, 50=2.86%, 100=3.91%
  lat (usec)   : 250=17.45%, 500=1.43%, 750=1.43%, 1000=4.56%
  lat (msec)   : 2=15.23%, 4=12.63%, 10=7.29%, 20=5.60%, 50=2.08%
  cpu          : usr=0.26%, sys=7.55%, ctx=880, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19579: Fri May 31 08:18:44 2024
   read: IOPS=191, BW=766KiB/s (784kB/s)(1508KiB/1969msec)
    clat (usec): min=3, max=31688, avg=1116.91, stdev=2496.64
     lat (usec): min=3, max=31690, avg=1117.45, stdev=2496.69
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    8],
     | 30.00th=[   10], 40.00th=[   12], 50.00th=[   20], 60.00th=[  873],
     | 70.00th=[ 1450], 80.00th=[ 1942], 90.00th=[ 2671], 95.00th=[ 3425],
     | 99.00th=[13435], 99.50th=[15270], 99.90th=[31589], 99.95th=[31589],
     | 99.99th=[31589]
   bw (  KiB/s): min=  688, max=  808, per=25.59%, avg=760.00, stdev=63.50, samples=3
   iops        : min=  172, max=  202, avg=190.00, stdev=15.87, samples=3
  write: IOPS=198, BW=794KiB/s (813kB/s)(1564KiB/1969msec)
    clat (usec): min=87, max=39052, avg=3927.60, stdev=6329.13
     lat (usec): min=88, max=39053, avg=3928.33, stdev=6329.12
    clat percentiles (usec):
     |  1.00th=[   95],  5.00th=[  104], 10.00th=[  110], 20.00th=[  122],
     | 30.00th=[  161], 40.00th=[  449], 50.00th=[  963], 60.00th=[ 1582],
     | 70.00th=[ 2671], 80.00th=[ 8848], 90.00th=[12387], 95.00th=[15795],
     | 99.00th=[33162], 99.50th=[35390], 99.90th=[39060], 99.95th=[39060],
     | 99.99th=[39060]
   bw (  KiB/s): min=  696, max= 1016, per=26.37%, avg=824.00, stdev=169.33, samples=3
   iops        : min=  174, max=  254, avg=206.00, stdev=42.33, samples=3
  lat (usec)   : 4=1.17%, 10=15.36%, 20=8.72%, 50=2.47%, 100=2.08%
  lat (usec)   : 250=17.58%, 500=1.56%, 750=2.34%, 1000=4.69%
  lat (msec)   : 2=17.97%, 4=10.68%, 10=6.12%, 20=7.42%, 50=1.82%
  cpu          : usr=0.30%, sys=7.62%, ctx=931, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=2970KiB/s (3042kB/s), 712KiB/s-819KiB/s (729kB/s-839kB/s), io=5988KiB (6132kB), run=1909-2016msec
  WRITE: bw=3125KiB/s (3200kB/s), 790KiB/s-812KiB/s (809kB/s-831kB/s), io=6300KiB (6451kB), run=1909-2016msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=19580: Fri May 31 08:18:50 2024
   read: IOPS=44, BW=179KiB/s (183kB/s)(1480KiB/8271msec)
    clat (usec): min=617, max=54716, avg=2313.62, stdev=4753.27
     lat (usec): min=618, max=54718, avg=2314.29, stdev=4753.31
    clat percentiles (usec):
     |  1.00th=[  701],  5.00th=[  783], 10.00th=[  816], 20.00th=[  865],
     | 30.00th=[  922], 40.00th=[  955], 50.00th=[ 1045], 60.00th=[ 1205],
     | 70.00th=[ 1401], 80.00th=[ 1696], 90.00th=[ 3228], 95.00th=[ 9896],
     | 99.00th=[25822], 99.50th=[33424], 99.90th=[54789], 99.95th=[54789],
     | 99.99th=[54789]
   bw (  KiB/s): min=   96, max=  280, per=24.65%, avg=175.50, stdev=50.97, samples=16
   iops        : min=   24, max=   70, avg=43.88, stdev=12.74, samples=16
  write: IOPS=48, BW=192KiB/s (197kB/s)(1592KiB/8271msec)
    clat (usec): min=10290, max=56442, avg=18609.14, stdev=5956.34
     lat (usec): min=10291, max=56443, avg=18609.94, stdev=5956.30
    clat percentiles (usec):
     |  1.00th=[11076],  5.00th=[11731], 10.00th=[12256], 20.00th=[13435],
     | 30.00th=[14353], 40.00th=[16188], 50.00th=[18482], 60.00th=[19792],
     | 70.00th=[20841], 80.00th=[22152], 90.00th=[24249], 95.00th=[29230],
     | 99.00th=[41157], 99.50th=[49021], 99.90th=[56361], 99.95th=[56361],
     | 99.99th=[56361]
   bw (  KiB/s): min=  120, max=  232, per=25.57%, avg=191.50, stdev=37.35, samples=16
   iops        : min=   30, max=   58, avg=47.87, stdev= 9.34, samples=16
  lat (usec)   : 750=2.08%, 1000=19.79%
  lat (msec)   : 2=19.66%, 4=2.60%, 10=1.69%, 20=33.07%, 50=20.83%
  lat (msec)   : 100=0.26%
  cpu          : usr=0.13%, sys=1.28%, ctx=834, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19581: Fri May 31 08:18:50 2024
   read: IOPS=42, BW=171KiB/s (175kB/s)(1436KiB/8402msec)
    clat (usec): min=556, max=54656, avg=1926.04, stdev=3868.82
     lat (usec): min=556, max=54657, avg=1926.70, stdev=3868.84
    clat percentiles (usec):
     |  1.00th=[  668],  5.00th=[  717], 10.00th=[  758], 20.00th=[  832],
     | 30.00th=[  898], 40.00th=[  963], 50.00th=[ 1074], 60.00th=[ 1205],
     | 70.00th=[ 1352], 80.00th=[ 1663], 90.00th=[ 2278], 95.00th=[ 5866],
     | 99.00th=[18220], 99.50th=[21890], 99.90th=[54789], 99.95th=[54789],
     | 99.99th=[54789]
   bw (  KiB/s): min=   80, max=  264, per=24.23%, avg=172.50, stdev=46.74, samples=16
   iops        : min=   20, max=   66, avg=43.12, stdev=11.68, samples=16
  write: IOPS=48, BW=195KiB/s (199kB/s)(1636KiB/8402msec)
    clat (usec): min=10321, max=49695, avg=18831.31, stdev=5383.75
     lat (usec): min=10322, max=49696, avg=18832.11, stdev=5383.77
    clat percentiles (usec):
     |  1.00th=[10683],  5.00th=[11600], 10.00th=[12256], 20.00th=[13566],
     | 30.00th=[15139], 40.00th=[17695], 50.00th=[19268], 60.00th=[20317],
     | 70.00th=[21103], 80.00th=[22414], 90.00th=[24511], 95.00th=[26870],
     | 99.00th=[34341], 99.50th=[41681], 99.90th=[49546], 99.95th=[49546],
     | 99.99th=[49546]
   bw (  KiB/s): min=  144, max=  232, per=25.70%, avg=192.50, stdev=24.52, samples=16
   iops        : min=   36, max=   58, avg=48.12, stdev= 6.13, samples=16
  lat (usec)   : 750=3.65%, 1000=17.06%
  lat (msec)   : 2=19.92%, 4=3.26%, 10=1.30%, 20=32.03%, 50=22.66%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.07%, sys=1.29%, ctx=814, majf=0, minf=31
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19582: Fri May 31 08:18:50 2024
   read: IOPS=47, BW=192KiB/s (196kB/s)(1564KiB/8167msec)
    clat (usec): min=590, max=33331, avg=1840.23, stdev=3389.44
     lat (usec): min=591, max=33331, avg=1840.86, stdev=3389.45
    clat percentiles (usec):
     |  1.00th=[  644],  5.00th=[  709], 10.00th=[  750], 20.00th=[  816],
     | 30.00th=[  873], 40.00th=[  922], 50.00th=[  988], 60.00th=[ 1123],
     | 70.00th=[ 1270], 80.00th=[ 1549], 90.00th=[ 1975], 95.00th=[ 6456],
     | 99.00th=[20579], 99.50th=[24511], 99.90th=[33424], 99.95th=[33424],
     | 99.99th=[33424]
   bw (  KiB/s): min=   96, max=  312, per=27.32%, avg=194.50, stdev=68.05, samples=16
   iops        : min=   24, max=   78, avg=48.63, stdev=17.01, samples=16
  write: IOPS=46, BW=185KiB/s (189kB/s)(1508KiB/8167msec)
    clat (usec): min=10657, max=63367, avg=19731.13, stdev=6519.06
     lat (usec): min=10658, max=63369, avg=19731.90, stdev=6519.06
    clat percentiles (usec):
     |  1.00th=[11207],  5.00th=[12256], 10.00th=[12780], 20.00th=[14222],
     | 30.00th=[16319], 40.00th=[17957], 50.00th=[19530], 60.00th=[20579],
     | 70.00th=[21627], 80.00th=[23200], 90.00th=[25297], 95.00th=[31327],
     | 99.00th=[46924], 99.50th=[54789], 99.90th=[63177], 99.95th=[63177],
     | 99.99th=[63177]
   bw (  KiB/s): min=  128, max=  216, per=24.57%, avg=184.00, stdev=31.73, samples=16
   iops        : min=   32, max=   54, avg=46.00, stdev= 7.93, samples=16
  lat (usec)   : 750=5.08%, 1000=21.22%
  lat (msec)   : 2=19.66%, 4=2.21%, 10=0.91%, 20=28.78%, 50=21.74%
  lat (msec)   : 100=0.39%
  cpu          : usr=0.07%, sys=1.31%, ctx=843, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19583: Fri May 31 08:18:50 2024
   read: IOPS=45, BW=182KiB/s (186kB/s)(1508KiB/8297msec)
    clat (usec): min=619, max=36302, avg=1874.06, stdev=3375.07
     lat (usec): min=620, max=36303, avg=1874.87, stdev=3375.02
    clat percentiles (usec):
     |  1.00th=[  635],  5.00th=[  717], 10.00th=[  766], 20.00th=[  824],
     | 30.00th=[  873], 40.00th=[  955], 50.00th=[ 1037], 60.00th=[ 1221],
     | 70.00th=[ 1385], 80.00th=[ 1565], 90.00th=[ 2180], 95.00th=[ 6128],
     | 99.00th=[21103], 99.50th=[23200], 99.90th=[36439], 99.95th=[36439],
     | 99.99th=[36439]
   bw (  KiB/s): min=   80, max=  248, per=24.97%, avg=177.81, stdev=51.78, samples=16
   iops        : min=   20, max=   62, avg=44.31, stdev=13.00, samples=16
  write: IOPS=47, BW=189KiB/s (193kB/s)(1564KiB/8297msec)
    clat (usec): min=10372, max=61002, avg=19389.99, stdev=6690.36
     lat (usec): min=10373, max=61004, avg=19390.83, stdev=6690.45
    clat percentiles (usec):
     |  1.00th=[11076],  5.00th=[11994], 10.00th=[12518], 20.00th=[13698],
     | 30.00th=[15401], 40.00th=[17171], 50.00th=[19006], 60.00th=[20317],
     | 70.00th=[21365], 80.00th=[22676], 90.00th=[24773], 95.00th=[30540],
     | 99.00th=[54264], 99.50th=[56886], 99.90th=[61080], 99.95th=[61080],
     | 99.99th=[61080]
   bw (  KiB/s): min=  120, max=  224, per=24.87%, avg=186.31, stdev=28.13, samples=16
   iops        : min=   30, max=   56, avg=46.44, stdev= 7.04, samples=16
  lat (usec)   : 750=4.17%, 1000=18.88%
  lat (msec)   : 2=20.31%, 4=3.12%, 10=0.65%, 20=29.95%, 50=22.40%
  lat (msec)   : 100=0.52%
  cpu          : usr=0.11%, sys=1.23%, ctx=813, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=713KiB/s (730kB/s), 171KiB/s-192KiB/s (175kB/s-196kB/s), io=5988KiB (6132kB), run=8167-8402msec
  WRITE: bw=750KiB/s (768kB/s), 185KiB/s-195KiB/s (189kB/s-199kB/s), io=6300KiB (6451kB), run=8167-8402msec