== sanity test 398b: DIO and buffer IO race ============== 04:53:01 (1722243181)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.100244 s, 126 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=18912: Mon Jul 29 04:53:05 2024
   read: IOPS=163, BW=655KiB/s (670kB/s)(1480KiB/2261msec)
    clat (usec): min=4, max=28558, avg=3374.61, stdev=4290.34
     lat (usec): min=5, max=28559, avg=3375.31, stdev=4290.38
    clat percentiles (usec):
     |  1.00th=[    6],  5.00th=[    8], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   14], 40.00th=[   16], 50.00th=[ 2114], 60.00th=[ 3818],
     | 70.00th=[ 5211], 80.00th=[ 6259], 90.00th=[ 8094], 95.00th=[10683],
     | 99.00th=[18482], 99.50th=[27395], 99.90th=[28443], 99.95th=[28443],
     | 99.99th=[28443]
   bw (  KiB/s): min=  336, max=  816, per=23.54%, avg=602.00, stdev=198.27, samples=4
   iops        : min=   84, max=  204, avg=150.50, stdev=49.57, samples=4
  write: IOPS=176, BW=704KiB/s (721kB/s)(1592KiB/2261msec)
    clat (usec): min=90, max=43882, avg=2481.96, stdev=3811.28
     lat (usec): min=90, max=43882, avg=2482.74, stdev=3811.26
    clat percentiles (usec):
     |  1.00th=[  114],  5.00th=[  135], 10.00th=[  143], 20.00th=[  155],
     | 30.00th=[  180], 40.00th=[  253], 50.00th=[  627], 60.00th=[ 1942],
     | 70.00th=[ 2966], 80.00th=[ 4752], 90.00th=[ 6849], 95.00th=[ 8979],
     | 99.00th=[15401], 99.50th=[20317], 99.90th=[43779], 99.95th=[43779],
     | 99.99th=[43779]
   bw (  KiB/s): min=  408, max=  928, per=24.45%, avg=658.00, stdev=239.86, samples=4
   iops        : min=  102, max=  232, avg=164.50, stdev=59.96, samples=4
  lat (usec)   : 10=7.16%, 20=13.54%, 50=1.43%, 100=0.39%, 250=20.96%
  lat (usec)   : 500=4.17%, 750=1.69%, 1000=1.04%
  lat (msec)   : 2=5.34%, 4=13.80%, 10=25.39%, 20=4.43%, 50=0.65%
  cpu          : usr=2.52%, sys=9.51%, ctx=715, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=18913: Mon Jul 29 04:53:05 2024
   read: IOPS=153, BW=613KiB/s (628kB/s)(1436KiB/2341msec)
    clat (usec): min=4, max=46925, avg=3668.94, stdev=4834.96
     lat (usec): min=4, max=46925, avg=3669.71, stdev=4834.94
    clat percentiles (usec):
     |  1.00th=[    6],  5.00th=[    7], 10.00th=[    9], 20.00th=[   12],
     | 30.00th=[   15], 40.00th=[   22], 50.00th=[ 3097], 60.00th=[ 4293],
     | 70.00th=[ 5342], 80.00th=[ 6521], 90.00th=[ 7963], 95.00th=[10552],
     | 99.00th=[18744], 99.50th=[35390], 99.90th=[46924], 99.95th=[46924],
     | 99.99th=[46924]
   bw (  KiB/s): min=  376, max=  832, per=21.90%, avg=560.00, stdev=193.00, samples=4
   iops        : min=   94, max=  208, avg=140.00, stdev=48.25, samples=4
  write: IOPS=174, BW=699KiB/s (716kB/s)(1636KiB/2341msec)
    clat (usec): min=92, max=29371, avg=2458.67, stdev=3645.88
     lat (usec): min=92, max=29372, avg=2459.50, stdev=3645.91
    clat percentiles (usec):
     |  1.00th=[  102],  5.00th=[  131], 10.00th=[  145], 20.00th=[  157],
     | 30.00th=[  178], 40.00th=[  247], 50.00th=[  619], 60.00th=[ 1811],
     | 70.00th=[ 2638], 80.00th=[ 4359], 90.00th=[ 6980], 95.00th=[ 9896],
     | 99.00th=[14615], 99.50th=[17695], 99.90th=[29492], 99.95th=[29492],
     | 99.99th=[29492]
   bw (  KiB/s): min=  392, max=  744, per=22.37%, avg=602.00, stdev=149.29, samples=4
   iops        : min=   98, max=  186, avg=150.50, stdev=37.32, samples=4
  lat (usec)   : 10=7.42%, 20=10.94%, 50=1.43%, 100=0.39%, 250=21.22%
  lat (usec)   : 500=4.17%, 750=1.82%, 1000=0.78%
  lat (msec)   : 2=6.12%, 4=14.45%, 10=25.78%, 20=4.82%, 50=0.65%
  cpu          : usr=0.00%, sys=12.14%, ctx=770, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=18914: Mon Jul 29 04:53:05 2024
   read: IOPS=170, BW=683KiB/s (700kB/s)(1564KiB/2289msec)
    clat (usec): min=4, max=32634, avg=3567.37, stdev=4636.04
     lat (usec): min=4, max=32635, avg=3568.90, stdev=4635.45
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   11],
     | 30.00th=[   13], 40.00th=[   16], 50.00th=[ 2278], 60.00th=[ 4178],
     | 70.00th=[ 5407], 80.00th=[ 6652], 90.00th=[ 9110], 95.00th=[10552],
     | 99.00th=[21103], 99.50th=[31327], 99.90th=[32637], 99.95th=[32637],
     | 99.99th=[32637]
   bw (  KiB/s): min=  384, max=  840, per=23.31%, avg=596.00, stdev=189.37, samples=4
   iops        : min=   96, max=  210, avg=149.00, stdev=47.34, samples=4
  write: IOPS=164, BW=659KiB/s (675kB/s)(1508KiB/2289msec)
    clat (usec): min=94, max=26376, avg=2338.12, stdev=3127.44
     lat (usec): min=94, max=26376, avg=2338.96, stdev=3127.53
    clat percentiles (usec):
     |  1.00th=[  117],  5.00th=[  133], 10.00th=[  143], 20.00th=[  157],
     | 30.00th=[  180], 40.00th=[  285], 50.00th=[  758], 60.00th=[ 2057],
     | 70.00th=[ 2835], 80.00th=[ 4424], 90.00th=[ 6652], 95.00th=[ 8094],
     | 99.00th=[12911], 99.50th=[17171], 99.90th=[26346], 99.95th=[26346],
     | 99.99th=[26346]
   bw (  KiB/s): min=  480, max=  832, per=22.37%, avg=602.00, stdev=159.38, samples=4
   iops        : min=  120, max=  208, avg=150.50, stdev=39.85, samples=4
  lat (usec)   : 10=8.98%, 20=13.15%, 50=0.91%, 100=0.13%, 250=18.62%
  lat (usec)   : 500=4.95%, 750=1.56%, 1000=1.04%
  lat (msec)   : 2=4.95%, 4=13.80%, 10=27.08%, 20=4.17%, 50=0.65%
  cpu          : usr=0.96%, sys=10.14%, ctx=717, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=18915: Mon Jul 29 04:53:05 2024
   read: IOPS=168, BW=676KiB/s (692kB/s)(1508KiB/2232msec)
    clat (usec): min=3, max=34924, avg=3315.19, stdev=4580.99
     lat (usec): min=3, max=34925, avg=3315.85, stdev=4581.03
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   13], 40.00th=[   16], 50.00th=[ 1450], 60.00th=[ 3916],
     | 70.00th=[ 5014], 80.00th=[ 6259], 90.00th=[ 8160], 95.00th=[ 9634],
     | 99.00th=[21627], 99.50th=[33424], 99.90th=[34866], 99.95th=[34866],
     | 99.99th=[34866]
   bw (  KiB/s): min=  376, max=  832, per=25.19%, avg=644.00, stdev=197.96, samples=4
   iops        : min=   94, max=  208, avg=161.00, stdev=49.49, samples=4
  write: IOPS=175, BW=701KiB/s (718kB/s)(1564KiB/2232msec)
    clat (usec): min=80, max=19651, avg=2446.14, stdev=3316.66
     lat (usec): min=81, max=19652, avg=2446.94, stdev=3316.67
    clat percentiles (usec):
     |  1.00th=[   94],  5.00th=[  127], 10.00th=[  139], 20.00th=[  155],
     | 30.00th=[  172], 40.00th=[  221], 50.00th=[  996], 60.00th=[ 2024],
     | 70.00th=[ 2802], 80.00th=[ 4228], 90.00th=[ 7177], 95.00th=[ 9503],
     | 99.00th=[14877], 99.50th=[17957], 99.90th=[19530], 99.95th=[19530],
     | 99.99th=[19530]
   bw (  KiB/s): min=  432, max=  896, per=24.82%, avg=668.00, stdev=202.65, samples=4
   iops        : min=  108, max=  224, avg=167.00, stdev=50.66, samples=4
  lat (usec)   : 4=0.13%, 10=8.72%, 20=13.41%, 50=1.17%, 100=0.78%
  lat (usec)   : 250=20.57%, 500=3.78%, 750=0.39%, 1000=0.39%
  lat (msec)   : 2=5.99%, 4=14.71%, 10=25.26%, 20=4.17%, 50=0.52%
  cpu          : usr=0.00%, sys=12.33%, ctx=774, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=2558KiB/s (2619kB/s), 613KiB/s-683KiB/s (628kB/s-700kB/s), io=5988KiB (6132kB), run=2232-2341msec
  WRITE: bw=2691KiB/s (2756kB/s), 659KiB/s-704KiB/s (675kB/s-721kB/s), io=6300KiB (6451kB), run=2232-2341msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=18917: Mon Jul 29 04:53:07 2024
   read: IOPS=86, BW=345KiB/s (354kB/s)(1480KiB/4285msec)
    clat (usec): min=1098, max=49192, avg=3872.06, stdev=4389.88
     lat (usec): min=1098, max=49194, avg=3873.19, stdev=4389.82
    clat percentiles (usec):
     |  1.00th=[ 1188],  5.00th=[ 1401], 10.00th=[ 1500], 20.00th=[ 1680],
     | 30.00th=[ 1909], 40.00th=[ 2114], 50.00th=[ 2376], 60.00th=[ 2769],
     | 70.00th=[ 3326], 80.00th=[ 4686], 90.00th=[ 8029], 95.00th=[12780],
     | 99.00th=[19268], 99.50th=[28705], 99.90th=[49021], 99.95th=[49021],
     | 99.99th=[49021]
   bw (  KiB/s): min=  159, max=  520, per=23.87%, avg=333.50, stdev=128.09, samples=8
   iops        : min=   39, max=  130, avg=83.00, stdev=32.27, samples=8
  write: IOPS=92, BW=372KiB/s (380kB/s)(1592KiB/4285msec)
    clat (usec): min=2485, max=37259, avg=7132.49, stdev=3844.96
     lat (usec): min=2487, max=37260, avg=7133.58, stdev=3844.99
    clat percentiles (usec):
     |  1.00th=[ 2802],  5.00th=[ 3458], 10.00th=[ 3785], 20.00th=[ 4555],
     | 30.00th=[ 5080], 40.00th=[ 5604], 50.00th=[ 6063], 60.00th=[ 6587],
     | 70.00th=[ 7373], 80.00th=[ 8848], 90.00th=[12125], 95.00th=[15008],
     | 99.00th=[22414], 99.50th=[23725], 99.90th=[37487], 99.95th=[37487],
     | 99.99th=[37487]
   bw (  KiB/s): min=  175, max=  568, per=24.57%, avg=361.25, stdev=162.16, samples=8
   iops        : min=   43, max=  142, avg=90.00, stdev=40.67, samples=8
  lat (msec)   : 2=16.54%, 4=26.04%, 10=45.70%, 20=10.55%, 50=1.17%
  cpu          : usr=0.00%, sys=5.07%, ctx=830, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=18918: Mon Jul 29 04:53:07 2024
   read: IOPS=84, BW=337KiB/s (345kB/s)(1436KiB/4265msec)
    clat (usec): min=1071, max=49448, avg=3366.54, stdev=3579.68
     lat (usec): min=1072, max=49449, avg=3368.14, stdev=3579.51
    clat percentiles (usec):
     |  1.00th=[ 1188],  5.00th=[ 1336], 10.00th=[ 1500], 20.00th=[ 1663],
     | 30.00th=[ 1909], 40.00th=[ 2114], 50.00th=[ 2343], 60.00th=[ 2638],
     | 70.00th=[ 3130], 80.00th=[ 3851], 90.00th=[ 6128], 95.00th=[10290],
     | 99.00th=[15533], 99.50th=[16319], 99.90th=[49546], 99.95th=[49546],
     | 99.99th=[49546]
   bw (  KiB/s): min=  160, max=  526, per=23.74%, avg=331.62, stdev=152.59, samples=8
   iops        : min=   40, max=  131, avg=82.75, stdev=38.12, samples=8
  write: IOPS=95, BW=384KiB/s (393kB/s)(1636KiB/4265msec)
    clat (usec): min=2303, max=31504, avg=7437.59, stdev=4190.26
     lat (usec): min=2305, max=31506, avg=7441.06, stdev=4191.81
    clat percentiles (usec):
     |  1.00th=[ 2802],  5.00th=[ 3458], 10.00th=[ 3949], 20.00th=[ 4555],
     | 30.00th=[ 5080], 40.00th=[ 5538], 50.00th=[ 6063], 60.00th=[ 6718],
     | 70.00th=[ 7701], 80.00th=[ 9503], 90.00th=[13173], 95.00th=[16581],
     | 99.00th=[21627], 99.50th=[22676], 99.90th=[31589], 99.95th=[31589],
     | 99.99th=[31589]
   bw (  KiB/s): min=  216, max=  576, per=25.20%, avg=370.50, stdev=142.00, samples=8
   iops        : min=   54, max=  144, avg=92.50, stdev=35.42, samples=8
  lat (msec)   : 2=16.02%, 4=27.73%, 10=44.01%, 20=10.68%, 50=1.56%
  cpu          : usr=0.00%, sys=5.02%, ctx=815, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=18919: Mon Jul 29 04:53:07 2024
   read: IOPS=92, BW=371KiB/s (380kB/s)(1564KiB/4214msec)
    clat (usec): min=955, max=35684, avg=3633.87, stdev=3564.23
     lat (usec): min=956, max=35684, avg=3634.74, stdev=3564.24
    clat percentiles (usec):
     |  1.00th=[ 1074],  5.00th=[ 1319], 10.00th=[ 1467], 20.00th=[ 1663],
     | 30.00th=[ 1893], 40.00th=[ 2114], 50.00th=[ 2343], 60.00th=[ 2769],
     | 70.00th=[ 3326], 80.00th=[ 4228], 90.00th=[ 7570], 95.00th=[11338],
     | 99.00th=[16581], 99.50th=[21627], 99.90th=[35914], 99.95th=[35914],
     | 99.99th=[35914]
   bw (  KiB/s): min=  176, max=  656, per=26.54%, avg=370.75, stdev=175.35, samples=8
   iops        : min=   44, max=  164, avg=92.50, stdev=43.94, samples=8
  write: IOPS=89, BW=358KiB/s (366kB/s)(1508KiB/4214msec)
    clat (usec): min=2201, max=48814, avg=7376.13, stdev=4344.32
     lat (usec): min=2202, max=48815, avg=7377.33, stdev=4344.41
    clat percentiles (usec):
     |  1.00th=[ 2671],  5.00th=[ 3359], 10.00th=[ 4113], 20.00th=[ 4752],
     | 30.00th=[ 5145], 40.00th=[ 5538], 50.00th=[ 5997], 60.00th=[ 6652],
     | 70.00th=[ 7570], 80.00th=[ 9503], 90.00th=[12649], 95.00th=[15664],
     | 99.00th=[22676], 99.50th=[22938], 99.90th=[49021], 99.95th=[49021],
     | 99.99th=[49021]
   bw (  KiB/s): min=  207, max=  528, per=23.72%, avg=348.75, stdev=138.33, samples=8
   iops        : min=   51, max=  132, avg=87.00, stdev=34.76, samples=8
  lat (usec)   : 1000=0.13%
  lat (msec)   : 2=17.71%, 4=26.04%, 10=43.75%, 20=11.33%, 50=1.04%
  cpu          : usr=0.00%, sys=5.15%, ctx=803, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=18920: Mon Jul 29 04:53:07 2024
   read: IOPS=89, BW=356KiB/s (365kB/s)(1508KiB/4235msec)
    clat (usec): min=910, max=31319, avg=3734.04, stdev=3765.40
     lat (usec): min=911, max=31320, avg=3734.91, stdev=3765.49
    clat percentiles (usec):
     |  1.00th=[ 1057],  5.00th=[ 1336], 10.00th=[ 1467], 20.00th=[ 1663],
     | 30.00th=[ 1876], 40.00th=[ 2147], 50.00th=[ 2343], 60.00th=[ 2671],
     | 70.00th=[ 3425], 80.00th=[ 4752], 90.00th=[ 8225], 95.00th=[11994],
     | 99.00th=[16450], 99.50th=[27657], 99.90th=[31327], 99.95th=[31327],
     | 99.99th=[31327]
   bw (  KiB/s): min=  176, max=  584, per=24.60%, avg=343.63, stdev=133.18, samples=8
   iops        : min=   44, max=  146, avg=85.88, stdev=33.29, samples=8
  write: IOPS=92, BW=369KiB/s (378kB/s)(1564KiB/4235msec)
    clat (usec): min=1845, max=42862, avg=7194.55, stdev=3981.27
     lat (usec): min=1846, max=42864, avg=7196.72, stdev=3980.52
    clat percentiles (usec):
     |  1.00th=[ 2409],  5.00th=[ 3654], 10.00th=[ 4080], 20.00th=[ 4752],
     | 30.00th=[ 5211], 40.00th=[ 5473], 50.00th=[ 6063], 60.00th=[ 6456],
     | 70.00th=[ 7373], 80.00th=[ 9110], 90.00th=[11863], 95.00th=[15270],
     | 99.00th=[22152], 99.50th=[24773], 99.90th=[42730], 99.95th=[42730],
     | 99.99th=[42730]
   bw (  KiB/s): min=  192, max=  616, per=24.68%, avg=362.75, stdev=155.88, samples=8
   iops        : min=   48, max=  154, avg=90.62, stdev=39.00, samples=8
  lat (usec)   : 1000=0.26%
  lat (msec)   : 2=16.93%, 4=24.09%, 10=47.01%, 20=10.55%, 50=1.17%
  cpu          : usr=0.00%, sys=4.79%, ctx=825, majf=0, minf=31
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=1397KiB/s (1431kB/s), 337KiB/s-371KiB/s (345kB/s-380kB/s), io=5988KiB (6132kB), run=4214-4285msec
  WRITE: bw=1470KiB/s (1506kB/s), 358KiB/s-384KiB/s (366kB/s-393kB/s), io=6300KiB (6451kB), run=4214-4285msec