== sanity test 398b: DIO and buffer IO race ============== 06:49:22 (1721386162)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0590451 s, 213 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=23565: Fri Jul 19 06:49:26 2024
   read: IOPS=157, BW=630KiB/s (645kB/s)(1480KiB/2348msec)
    clat (usec): min=3, max=65529, avg=1900.84, stdev=4744.56
     lat (usec): min=4, max=65530, avg=1901.47, stdev=4744.59
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   15], 40.00th=[   25], 50.00th=[  988], 60.00th=[ 1500],
     | 70.00th=[ 1975], 80.00th=[ 2507], 90.00th=[ 3425], 95.00th=[ 5735],
     | 99.00th=[25560], 99.50th=[26870], 99.90th=[65274], 99.95th=[65274],
     | 99.99th=[65274]
   bw (  KiB/s): min=  456, max=  744, per=23.25%, avg=576.00, stdev=131.62, samples=4
   iops        : min=  114, max=  186, avg=144.00, stdev=32.90, samples=4
  write: IOPS=169, BW=678KiB/s (694kB/s)(1592KiB/2348msec)
    clat (usec): min=90, max=42261, avg=4092.20, stdev=6236.26
     lat (usec): min=90, max=42261, avg=4092.94, stdev=6236.25
    clat percentiles (usec):
     |  1.00th=[  100],  5.00th=[  113], 10.00th=[  131], 20.00th=[  169],
     | 30.00th=[  225], 40.00th=[  881], 50.00th=[ 1287], 60.00th=[ 1991],
     | 70.00th=[ 2933], 80.00th=[ 8094], 90.00th=[12518], 95.00th=[16909],
     | 99.00th=[29492], 99.50th=[34341], 99.90th=[42206], 99.95th=[42206],
     | 99.99th=[42206]
   bw (  KiB/s): min=  472, max= 1024, per=24.10%, avg=628.00, stdev=264.57, samples=4
   iops        : min=  118, max=  256, avg=157.00, stdev=66.14, samples=4
  lat (usec)   : 4=0.13%, 10=7.42%, 20=9.51%, 50=5.08%, 100=0.91%
  lat (usec)   : 250=16.15%, 500=1.17%, 750=1.56%, 1000=5.34%
  lat (msec)   : 2=18.36%, 4=16.41%, 10=8.46%, 20=7.03%, 50=2.34%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.21%, sys=8.73%, ctx=996, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23566: Fri Jul 19 06:49:26 2024
   read: IOPS=148, BW=594KiB/s (608kB/s)(1436KiB/2417msec)
    clat (usec): min=3, max=60284, avg=2176.68, stdev=5289.92
     lat (usec): min=3, max=60284, avg=2177.35, stdev=5289.93
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   16], 40.00th=[   27], 50.00th=[  922], 60.00th=[ 1549],
     | 70.00th=[ 2147], 80.00th=[ 2769], 90.00th=[ 3654], 95.00th=[ 7767],
     | 99.00th=[26346], 99.50th=[34866], 99.90th=[60031], 99.95th=[60031],
     | 99.99th=[60031]
   bw (  KiB/s): min=  408, max=  720, per=23.50%, avg=582.00, stdev=130.54, samples=4
   iops        : min=  102, max=  180, avg=145.50, stdev=32.63, samples=4
  write: IOPS=169, BW=677KiB/s (693kB/s)(1636KiB/2417msec)
    clat (usec): min=93, max=45776, avg=3974.53, stdev=5904.09
     lat (usec): min=93, max=45776, avg=3976.32, stdev=5903.97
    clat percentiles (usec):
     |  1.00th=[  101],  5.00th=[  111], 10.00th=[  129], 20.00th=[  169],
     | 30.00th=[  210], 40.00th=[  865], 50.00th=[ 1205], 60.00th=[ 2147],
     | 70.00th=[ 3687], 80.00th=[ 7373], 90.00th=[12518], 95.00th=[17171],
     | 99.00th=[23200], 99.50th=[25560], 99.90th=[45876], 99.95th=[45876],
     | 99.99th=[45876]
   bw (  KiB/s): min=  504, max=  736, per=24.17%, avg=630.00, stdev=96.31, samples=4
   iops        : min=  126, max=  184, avg=157.50, stdev=24.08, samples=4
  lat (usec)   : 4=0.13%, 10=7.94%, 20=8.33%, 50=5.08%, 100=1.04%
  lat (usec)   : 250=17.32%, 500=1.69%, 750=1.04%, 1000=5.47%
  lat (msec)   : 2=14.58%, 4=17.97%, 10=9.64%, 20=7.03%, 50=2.60%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.17%, sys=8.77%, ctx=1032, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23567: Fri Jul 19 06:49:26 2024
   read: IOPS=167, BW=672KiB/s (688kB/s)(1564KiB/2329msec)
    clat (usec): min=3, max=35532, avg=1859.45, stdev=4091.10
     lat (usec): min=3, max=35532, avg=1860.27, stdev=4091.03
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   15], 40.00th=[   25], 50.00th=[  971], 60.00th=[ 1582],
     | 70.00th=[ 1926], 80.00th=[ 2507], 90.00th=[ 3294], 95.00th=[ 5473],
     | 99.00th=[26870], 99.50th=[28705], 99.90th=[35390], 99.95th=[35390],
     | 99.99th=[35390]
   bw (  KiB/s): min=  424, max=  720, per=24.55%, avg=608.00, stdev=129.82, samples=4
   iops        : min=  106, max=  180, avg=152.00, stdev=32.46, samples=4
  write: IOPS=161, BW=647KiB/s (663kB/s)(1508KiB/2329msec)
    clat (usec): min=93, max=39429, avg=4209.45, stdev=5996.78
     lat (usec): min=94, max=39429, avg=4211.19, stdev=5996.35
    clat percentiles (usec):
     |  1.00th=[   97],  5.00th=[  119], 10.00th=[  145], 20.00th=[  204],
     | 30.00th=[  832], 40.00th=[ 1139], 50.00th=[ 1696], 60.00th=[ 2507],
     | 70.00th=[ 3916], 80.00th=[ 7635], 90.00th=[11994], 95.00th=[16188],
     | 99.00th=[27657], 99.50th=[31589], 99.90th=[39584], 99.95th=[39584],
     | 99.99th=[39584]
   bw (  KiB/s): min=  480, max=  728, per=23.25%, avg=606.00, stdev=108.59, samples=4
   iops        : min=  120, max=  182, avg=151.50, stdev=27.15, samples=4
  lat (usec)   : 4=0.13%, 10=8.33%, 20=10.29%, 50=4.82%, 100=1.43%
  lat (usec)   : 250=10.68%, 500=2.60%, 750=0.65%, 1000=5.21%
  lat (msec)   : 2=18.75%, 4=19.14%, 10=9.37%, 20=5.99%, 50=2.60%
  cpu          : usr=0.13%, sys=9.11%, ctx=1067, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23568: Fri Jul 19 06:49:26 2024
   read: IOPS=156, BW=627KiB/s (642kB/s)(1508KiB/2404msec)
    clat (usec): min=3, max=75693, avg=2244.08, stdev=5698.01
     lat (usec): min=3, max=75694, avg=2244.74, stdev=5698.15
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    9], 20.00th=[   10],
     | 30.00th=[   15], 40.00th=[   28], 50.00th=[ 1004], 60.00th=[ 1647],
     | 70.00th=[ 2212], 80.00th=[ 2704], 90.00th=[ 3785], 95.00th=[ 9110],
     | 99.00th=[24511], 99.50th=[41681], 99.90th=[76022], 99.95th=[76022],
     | 99.99th=[76022]
   bw (  KiB/s): min=  456, max=  760, per=22.12%, avg=548.00, stdev=142.29, samples=4
   iops        : min=  114, max=  190, avg=137.00, stdev=35.57, samples=4
  write: IOPS=162, BW=651KiB/s (666kB/s)(1564KiB/2404msec)
    clat (usec): min=95, max=38813, avg=3945.29, stdev=5835.98
     lat (usec): min=95, max=38814, avg=3946.13, stdev=5836.20
    clat percentiles (usec):
     |  1.00th=[   99],  5.00th=[  113], 10.00th=[  135], 20.00th=[  169],
     | 30.00th=[  233], 40.00th=[  996], 50.00th=[ 1483], 60.00th=[ 1991],
     | 70.00th=[ 2933], 80.00th=[ 7701], 90.00th=[12125], 95.00th=[15926],
     | 99.00th=[26608], 99.50th=[32900], 99.90th=[39060], 99.95th=[39060],
     | 99.99th=[39060]
   bw (  KiB/s): min=  456, max=  776, per=22.87%, avg=596.00, stdev=136.63, samples=4
   iops        : min=  114, max=  194, avg=149.00, stdev=34.16, samples=4
  lat (usec)   : 4=0.52%, 10=9.37%, 20=7.42%, 50=4.04%, 100=1.69%
  lat (usec)   : 250=15.49%, 500=1.30%, 750=0.52%, 1000=4.56%
  lat (msec)   : 2=18.49%, 4=19.01%, 10=8.33%, 20=6.64%, 50=2.47%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.29%, sys=7.07%, ctx=1002, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=2477KiB/s (2537kB/s), 594KiB/s-672KiB/s (608kB/s-688kB/s), io=5988KiB (6132kB), run=2329-2417msec
  WRITE: bw=2607KiB/s (2669kB/s), 647KiB/s-678KiB/s (663kB/s-694kB/s), io=6300KiB (6451kB), run=2329-2417msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=23570: Fri Jul 19 06:49:32 2024
   read: IOPS=43, BW=175KiB/s (179kB/s)(1480KiB/8455msec)
    clat (usec): min=571, max=26993, avg=2214.55, stdev=3772.93
     lat (usec): min=571, max=26993, avg=2215.36, stdev=3772.94
    clat percentiles (usec):
     |  1.00th=[  668],  5.00th=[  766], 10.00th=[  832], 20.00th=[  881],
     | 30.00th=[  955], 40.00th=[ 1020], 50.00th=[ 1123], 60.00th=[ 1237],
     | 70.00th=[ 1434], 80.00th=[ 1778], 90.00th=[ 3556], 95.00th=[ 9372],
     | 99.00th=[20579], 99.50th=[23987], 99.90th=[26870], 99.95th=[26870],
     | 99.99th=[26870]
   bw (  KiB/s): min=  104, max=  303, per=25.35%, avg=174.44, stdev=56.73, samples=16
   iops        : min=   26, max=   75, avg=43.56, stdev=14.07, samples=16
  write: IOPS=47, BW=188KiB/s (193kB/s)(1592KiB/8455msec)
    clat (usec): min=10150, max=62260, avg=19151.95, stdev=6567.68
     lat (usec): min=10151, max=62261, avg=19152.81, stdev=6567.69
    clat percentiles (usec):
     |  1.00th=[10683],  5.00th=[11600], 10.00th=[12256], 20.00th=[13698],
     | 30.00th=[15401], 40.00th=[17171], 50.00th=[19006], 60.00th=[19792],
     | 70.00th=[21103], 80.00th=[22676], 90.00th=[25035], 95.00th=[29230],
     | 99.00th=[51643], 99.50th=[61604], 99.90th=[62129], 99.95th=[62129],
     | 99.99th=[62129]
   bw (  KiB/s): min=  120, max=  216, per=25.75%, avg=186.44, stdev=30.87, samples=16
   iops        : min=   30, max=   54, avg=46.56, stdev= 7.71, samples=16
  lat (usec)   : 750=1.69%, 1000=16.02%
  lat (msec)   : 2=22.53%, 4=3.91%, 10=1.82%, 20=33.46%, 50=20.05%
  lat (msec)   : 100=0.52%
  cpu          : usr=0.14%, sys=1.40%, ctx=821, majf=0, minf=31
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23571: Fri Jul 19 06:49:32 2024
   read: IOPS=41, BW=165KiB/s (169kB/s)(1436KiB/8698msec)
    clat (usec): min=654, max=23764, avg=2131.15, stdev=3371.09
     lat (usec): min=655, max=23764, avg=2131.99, stdev=3371.11
    clat percentiles (usec):
     |  1.00th=[  701],  5.00th=[  791], 10.00th=[  816], 20.00th=[  922],
     | 30.00th=[  988], 40.00th=[ 1057], 50.00th=[ 1139], 60.00th=[ 1254],
     | 70.00th=[ 1467], 80.00th=[ 1762], 90.00th=[ 3359], 95.00th=[ 8848],
     | 99.00th=[21627], 99.50th=[23200], 99.90th=[23725], 99.95th=[23725],
     | 99.99th=[23725]
   bw (  KiB/s): min=   71, max=  256, per=23.48%, avg=161.53, stdev=51.71, samples=17
   iops        : min=   17, max=   64, avg=40.29, stdev=13.01, samples=17
  write: IOPS=47, BW=188KiB/s (193kB/s)(1636KiB/8698msec)
    clat (usec): min=10254, max=86184, avg=19372.35, stdev=7193.00
     lat (usec): min=10254, max=86185, avg=19373.29, stdev=7192.96
    clat percentiles (usec):
     |  1.00th=[10814],  5.00th=[11600], 10.00th=[12125], 20.00th=[13304],
     | 30.00th=[14877], 40.00th=[17433], 50.00th=[19268], 60.00th=[20317],
     | 70.00th=[21365], 80.00th=[22938], 90.00th=[26084], 95.00th=[30802],
     | 99.00th=[42730], 99.50th=[51643], 99.90th=[86508], 99.95th=[86508],
     | 99.99th=[86508]
   bw (  KiB/s): min=  119, max=  232, per=25.37%, avg=183.65, stdev=34.94, samples=17
   iops        : min=   29, max=   58, avg=45.76, stdev= 8.76, samples=17
  lat (usec)   : 750=1.17%, 1000=13.93%
  lat (msec)   : 2=24.35%, 4=3.12%, 10=2.21%, 20=32.29%, 50=22.53%
  lat (msec)   : 100=0.39%
  cpu          : usr=0.10%, sys=1.45%, ctx=859, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23572: Fri Jul 19 06:49:32 2024
   read: IOPS=49, BW=197KiB/s (202kB/s)(1564KiB/7921msec)
    clat (usec): min=659, max=31853, avg=1840.59, stdev=2933.19
     lat (usec): min=660, max=31854, avg=1841.55, stdev=2933.23
    clat percentiles (usec):
     |  1.00th=[  676],  5.00th=[  758], 10.00th=[  799], 20.00th=[  857],
     | 30.00th=[  906], 40.00th=[  971], 50.00th=[ 1029], 60.00th=[ 1188],
     | 70.00th=[ 1303], 80.00th=[ 1532], 90.00th=[ 2474], 95.00th=[ 6915],
     | 99.00th=[16909], 99.50th=[19268], 99.90th=[31851], 99.95th=[31851],
     | 99.99th=[31851]
   bw (  KiB/s): min=   88, max=  384, per=29.28%, avg=201.47, stdev=95.63, samples=15
   iops        : min=   22, max=   96, avg=50.27, stdev=23.98, samples=15
  write: IOPS=47, BW=190KiB/s (195kB/s)(1508KiB/7921msec)
    clat (usec): min=9841, max=94145, avg=19065.76, stdev=7176.88
     lat (usec): min=9842, max=94146, avg=19066.64, stdev=7176.90
    clat percentiles (usec):
     |  1.00th=[10683],  5.00th=[11600], 10.00th=[12256], 20.00th=[13173],
     | 30.00th=[14484], 40.00th=[16450], 50.00th=[18482], 60.00th=[20317],
     | 70.00th=[21365], 80.00th=[22938], 90.00th=[26346], 95.00th=[29754],
     | 99.00th=[41681], 99.50th=[51643], 99.90th=[93848], 99.95th=[93848],
     | 99.99th=[93848]
   bw (  KiB/s): min=  119, max=  240, per=26.06%, avg=188.67, stdev=35.35, samples=15
   iops        : min=   29, max=   60, avg=47.07, stdev= 8.93, samples=15
  lat (usec)   : 750=2.34%, 1000=21.61%
  lat (msec)   : 2=20.44%, 4=3.39%, 10=1.43%, 20=30.73%, 50=19.79%
  lat (msec)   : 100=0.26%
  cpu          : usr=0.04%, sys=1.55%, ctx=837, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=23573: Fri Jul 19 06:49:32 2024
   read: IOPS=45, BW=182KiB/s (186kB/s)(1508KiB/8298msec)
    clat (usec): min=634, max=30096, avg=2209.76, stdev=3743.09
     lat (usec): min=634, max=30097, avg=2210.49, stdev=3743.12
    clat percentiles (usec):
     |  1.00th=[  685],  5.00th=[  758], 10.00th=[  816], 20.00th=[  898],
     | 30.00th=[  955], 40.00th=[ 1057], 50.00th=[ 1172], 60.00th=[ 1287],
     | 70.00th=[ 1418], 80.00th=[ 1729], 90.00th=[ 2999], 95.00th=[ 9896],
     | 99.00th=[20579], 99.50th=[24773], 99.90th=[30016], 99.95th=[30016],
     | 99.99th=[30016]
   bw (  KiB/s): min=   80, max=  256, per=26.16%, avg=180.00, stdev=53.07, samples=16
   iops        : min=   20, max=   64, avg=45.00, stdev=13.27, samples=16
  write: IOPS=47, BW=188KiB/s (193kB/s)(1564KiB/8298msec)
    clat (usec): min=9223, max=68871, avg=19056.19, stdev=6679.49
     lat (usec): min=9224, max=68872, avg=19057.12, stdev=6679.51
    clat percentiles (usec):
     |  1.00th=[10814],  5.00th=[11600], 10.00th=[12125], 20.00th=[13304],
     | 30.00th=[14615], 40.00th=[17171], 50.00th=[19006], 60.00th=[20055],
     | 70.00th=[21365], 80.00th=[22676], 90.00th=[25297], 95.00th=[27657],
     | 99.00th=[45351], 99.50th=[55837], 99.90th=[68682], 99.95th=[68682],
     | 99.99th=[68682]
   bw (  KiB/s): min=  120, max=  224, per=25.97%, avg=188.00, stdev=33.56, samples=16
   iops        : min=   30, max=   56, avg=47.00, stdev= 8.39, samples=16
  lat (usec)   : 750=1.69%, 1000=15.49%
  lat (msec)   : 2=24.48%, 4=3.65%, 10=1.56%, 20=32.03%, 50=20.70%
  lat (msec)   : 100=0.39%
  cpu          : usr=0.11%, sys=1.42%, ctx=836, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=688KiB/s (705kB/s), 165KiB/s-197KiB/s (169kB/s-202kB/s), io=5988KiB (6132kB), run=7921-8698msec
  WRITE: bw=724KiB/s (742kB/s), 188KiB/s-190KiB/s (193kB/s-195kB/s), io=6300KiB (6451kB), run=7921-8698msec