== sanity test 398b: DIO and buffer IO race ============== 18:06:01 (1718402761)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0783859 s, 161 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=14286: Fri Jun 14 18:06:04 2024
   read: IOPS=349, BW=1396KiB/s (1430kB/s)(1480KiB/1060msec)
    clat (usec): min=3, max=19070, avg=1596.23, stdev=2122.65
     lat (usec): min=3, max=19071, avg=1596.84, stdev=2122.70
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    8], 10.00th=[    9], 20.00th=[   12],
     | 30.00th=[   14], 40.00th=[   58], 50.00th=[ 1303], 60.00th=[ 1795],
     | 70.00th=[ 2245], 80.00th=[ 2802], 90.00th=[ 3392], 95.00th=[ 4113],
     | 99.00th=[12649], 99.50th=[13960], 99.90th=[19006], 99.95th=[19006],
     | 99.99th=[19006]
   bw (  KiB/s): min= 1112, max= 1656, per=24.73%, avg=1384.00, stdev=384.67, samples=2
   iops        : min=  278, max=  414, avg=346.00, stdev=96.17, samples=2
  write: IOPS=375, BW=1502KiB/s (1538kB/s)(1592KiB/1060msec)
    clat (usec): min=79, max=7255, avg=1153.83, stdev=1365.64
     lat (usec): min=80, max=7256, avg=1155.35, stdev=1365.55
    clat percentiles (usec):
     |  1.00th=[   90],  5.00th=[  109], 10.00th=[  121], 20.00th=[  159],
     | 30.00th=[  178], 40.00th=[  225], 50.00th=[  750], 60.00th=[ 1020],
     | 70.00th=[ 1270], 80.00th=[ 2180], 90.00th=[ 2933], 95.00th=[ 3884],
     | 99.00th=[ 6718], 99.50th=[ 6849], 99.90th=[ 7242], 99.95th=[ 7242],
     | 99.99th=[ 7242]
   bw (  KiB/s): min= 1048, max= 1928, per=25.28%, avg=1488.00, stdev=622.25, samples=2
   iops        : min=  262, max=  482, avg=372.00, stdev=155.56, samples=2
  lat (usec)   : 4=0.13%, 10=6.77%, 20=11.33%, 50=0.78%, 100=1.82%
  lat (usec)   : 250=21.09%, 500=2.73%, 750=1.82%, 1000=6.25%
  lat (msec)   : 2=18.23%, 4=24.22%, 10=4.17%, 20=0.65%
  cpu          : usr=0.38%, sys=17.28%, ctx=889, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14287: Fri Jun 14 18:06:04 2024
   read: IOPS=338, BW=1353KiB/s (1386kB/s)(1436KiB/1061msec)
    clat (usec): min=4, max=23500, avg=1712.82, stdev=2404.73
     lat (usec): min=4, max=23500, avg=1713.41, stdev=2404.74
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   11],
     | 30.00th=[   14], 40.00th=[  103], 50.00th=[ 1516], 60.00th=[ 2008],
     | 70.00th=[ 2442], 80.00th=[ 2835], 90.00th=[ 3425], 95.00th=[ 4424],
     | 99.00th=[11600], 99.50th=[21627], 99.90th=[23462], 99.95th=[23462],
     | 99.99th=[23462]
   bw (  KiB/s): min=  928, max= 1664, per=23.16%, avg=1296.00, stdev=520.43, samples=2
   iops        : min=  232, max=  416, avg=324.00, stdev=130.11, samples=2
  write: IOPS=385, BW=1542KiB/s (1579kB/s)(1636KiB/1061msec)
    clat (usec): min=91, max=12822, avg=1058.82, stdev=1415.37
     lat (usec): min=92, max=12822, avg=1059.91, stdev=1415.26
    clat percentiles (usec):
     |  1.00th=[  100],  5.00th=[  117], 10.00th=[  130], 20.00th=[  159],
     | 30.00th=[  180], 40.00th=[  239], 50.00th=[  412], 60.00th=[  865],
     | 70.00th=[ 1156], 80.00th=[ 1811], 90.00th=[ 2835], 95.00th=[ 4047],
     | 99.00th=[ 5342], 99.50th=[ 6718], 99.90th=[12780], 99.95th=[12780],
     | 99.99th=[12780]
   bw (  KiB/s): min= 1096, max= 1832, per=24.87%, avg=1464.00, stdev=520.43, samples=2
   iops        : min=  274, max=  458, avg=366.00, stdev=130.11, samples=2
  lat (usec)   : 10=7.42%, 20=9.77%, 50=1.04%, 100=0.91%, 250=22.01%
  lat (usec)   : 500=5.21%, 750=1.69%, 1000=7.55%
  lat (msec)   : 2=16.80%, 4=22.14%, 10=4.56%, 20=0.65%, 50=0.26%
  cpu          : usr=0.66%, sys=19.62%, ctx=915, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14288: Fri Jun 14 18:06:04 2024
   read: IOPS=365, BW=1462KiB/s (1497kB/s)(1564KiB/1070msec)
    clat (usec): min=3, max=25259, avg=1600.95, stdev=2056.57
     lat (usec): min=3, max=25259, avg=1601.96, stdev=2056.44
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    7], 10.00th=[    8], 20.00th=[   11],
     | 30.00th=[   14], 40.00th=[  217], 50.00th=[ 1532], 60.00th=[ 1909],
     | 70.00th=[ 2409], 80.00th=[ 2769], 90.00th=[ 3359], 95.00th=[ 4178],
     | 99.00th=[ 6915], 99.50th=[14353], 99.90th=[25297], 99.95th=[25297],
     | 99.99th=[25297]
   bw (  KiB/s): min= 1008, max= 1704, per=24.23%, avg=1356.00, stdev=492.15, samples=2
   iops        : min=  252, max=  426, avg=339.00, stdev=123.04, samples=2
  write: IOPS=352, BW=1409KiB/s (1443kB/s)(1508KiB/1070msec)
    clat (usec): min=86, max=13378, avg=1134.83, stdev=1482.54
     lat (usec): min=87, max=13380, avg=1136.61, stdev=1483.01
    clat percentiles (usec):
     |  1.00th=[   96],  5.00th=[  112], 10.00th=[  126], 20.00th=[  161],
     | 30.00th=[  186], 40.00th=[  258], 50.00th=[  799], 60.00th=[ 1012],
     | 70.00th=[ 1221], 80.00th=[ 1876], 90.00th=[ 2704], 95.00th=[ 3523],
     | 99.00th=[ 7504], 99.50th=[10552], 99.90th=[13435], 99.95th=[13435],
     | 99.99th=[13435]
   bw (  KiB/s): min= 1024, max= 1632, per=22.56%, avg=1328.00, stdev=429.92, samples=2
   iops        : min=  256, max=  408, avg=332.00, stdev=107.48, samples=2
  lat (usec)   : 4=0.52%, 10=7.55%, 20=10.81%, 50=0.91%, 100=1.17%
  lat (usec)   : 250=18.49%, 500=3.78%, 750=1.17%, 1000=6.77%
  lat (msec)   : 2=20.57%, 4=23.18%, 10=4.43%, 20=0.52%, 50=0.13%
  cpu          : usr=0.75%, sys=19.08%, ctx=983, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14289: Fri Jun 14 18:06:04 2024
   read: IOPS=368, BW=1473KiB/s (1508kB/s)(1508KiB/1024msec)
    clat (usec): min=4, max=18409, avg=1593.79, stdev=2068.80
     lat (usec): min=4, max=18410, avg=1594.49, stdev=2068.78
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   11],
     | 30.00th=[   14], 40.00th=[  453], 50.00th=[ 1385], 60.00th=[ 1844],
     | 70.00th=[ 2245], 80.00th=[ 2769], 90.00th=[ 3392], 95.00th=[ 4146],
     | 99.00th=[ 8455], 99.50th=[17171], 99.90th=[18482], 99.95th=[18482],
     | 99.99th=[18482]
   bw (  KiB/s): min= 1120, max= 1768, per=25.80%, avg=1444.00, stdev=458.21, samples=2
   iops        : min=  280, max=  442, avg=361.00, stdev=114.55, samples=2
  write: IOPS=381, BW=1527KiB/s (1564kB/s)(1564KiB/1024msec)
    clat (usec): min=88, max=12628, avg=1046.47, stdev=1350.20
     lat (usec): min=88, max=12630, avg=1047.26, stdev=1350.29
    clat percentiles (usec):
     |  1.00th=[   95],  5.00th=[  110], 10.00th=[  127], 20.00th=[  161],
     | 30.00th=[  180], 40.00th=[  235], 50.00th=[  635], 60.00th=[  979],
     | 70.00th=[ 1156], 80.00th=[ 1549], 90.00th=[ 2638], 95.00th=[ 3720],
     | 99.00th=[ 6063], 99.50th=[ 6521], 99.90th=[12649], 99.95th=[12649],
     | 99.99th=[12649]
   bw (  KiB/s): min= 1096, max= 1920, per=25.62%, avg=1508.00, stdev=582.66, samples=2
   iops        : min=  274, max=  480, avg=377.00, stdev=145.66, samples=2
  lat (usec)   : 10=8.85%, 20=9.11%, 50=0.65%, 100=1.30%, 250=20.18%
  lat (usec)   : 500=4.43%, 750=1.69%, 1000=6.64%
  lat (msec)   : 2=21.09%, 4=21.35%, 10=4.17%, 20=0.52%
  cpu          : usr=0.78%, sys=17.69%, ctx=945, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=5596KiB/s (5731kB/s), 1353KiB/s-1473KiB/s (1386kB/s-1508kB/s), io=5988KiB (6132kB), run=1024-1070msec
  WRITE: bw=5888KiB/s (6029kB/s), 1409KiB/s-1542KiB/s (1443kB/s-1579kB/s), io=6300KiB (6451kB), run=1024-1070msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=14291: Fri Jun 14 18:06:04 2024
   read: IOPS=219, BW=878KiB/s (899kB/s)(1480KiB/1685msec)
    clat (usec): min=518, max=34111, avg=1588.96, stdev=2207.55
     lat (usec): min=518, max=34112, avg=1590.56, stdev=2207.88
    clat percentiles (usec):
     |  1.00th=[  668],  5.00th=[  717], 10.00th=[  750], 20.00th=[  799],
     | 30.00th=[  840], 40.00th=[  881], 50.00th=[  955], 60.00th=[ 1057],
     | 70.00th=[ 1221], 80.00th=[ 1647], 90.00th=[ 3326], 95.00th=[ 4490],
     | 99.00th=[ 7504], 99.50th=[12780], 99.90th=[34341], 99.95th=[34341],
     | 99.99th=[34341]
   bw (  KiB/s): min=  536, max= 1128, per=22.89%, avg=808.00, stdev=298.90, samples=3
   iops        : min=  134, max=  282, avg=202.00, stdev=74.73, samples=3
  write: IOPS=236, BW=945KiB/s (967kB/s)(1592KiB/1685msec)
    clat (usec): min=1206, max=30649, avg=2733.96, stdev=1920.03
     lat (usec): min=1207, max=30650, avg=2734.75, stdev=1920.02
    clat percentiles (usec):
     |  1.00th=[ 1221],  5.00th=[ 1500], 10.00th=[ 1663], 20.00th=[ 1795],
     | 30.00th=[ 1893], 40.00th=[ 2008], 50.00th=[ 2114], 60.00th=[ 2311],
     | 70.00th=[ 2769], 80.00th=[ 3556], 90.00th=[ 4555], 95.00th=[ 5604],
     | 99.00th=[ 8356], 99.50th=[ 9110], 99.90th=[30540], 99.95th=[30540],
     | 99.99th=[30540]
   bw (  KiB/s): min=  560, max= 1384, per=24.12%, avg=896.00, stdev=432.52, samples=3
   iops        : min=  140, max=  346, avg=224.00, stdev=108.13, samples=3
  lat (usec)   : 750=4.69%, 1000=21.87%
  lat (msec)   : 2=33.85%, 4=28.12%, 10=11.07%, 20=0.13%, 50=0.26%
  cpu          : usr=0.36%, sys=6.24%, ctx=811, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14292: Fri Jun 14 18:06:04 2024
   read: IOPS=211, BW=847KiB/s (867kB/s)(1436KiB/1696msec)
    clat (usec): min=613, max=34738, avg=1538.46, stdev=2234.41
     lat (usec): min=614, max=34740, avg=1539.16, stdev=2234.45
    clat percentiles (usec):
     |  1.00th=[  660],  5.00th=[  717], 10.00th=[  750], 20.00th=[  799],
     | 30.00th=[  848], 40.00th=[  898], 50.00th=[  947], 60.00th=[ 1037],
     | 70.00th=[ 1205], 80.00th=[ 1696], 90.00th=[ 2704], 95.00th=[ 4293],
     | 99.00th=[ 8717], 99.50th=[13960], 99.90th=[34866], 99.95th=[34866],
     | 99.99th=[34866]
   bw (  KiB/s): min=  544, max= 1272, per=22.57%, avg=796.67, stdev=411.92, samples=3
   iops        : min=  136, max=  318, avg=199.00, stdev=103.12, samples=3
  write: IOPS=241, BW=965KiB/s (988kB/s)(1636KiB/1696msec)
    clat (usec): min=1050, max=11974, avg=2774.25, stdev=1607.70
     lat (usec): min=1050, max=11975, avg=2775.48, stdev=1607.66
    clat percentiles (usec):
     |  1.00th=[ 1188],  5.00th=[ 1385], 10.00th=[ 1532], 20.00th=[ 1778],
     | 30.00th=[ 1909], 40.00th=[ 1991], 50.00th=[ 2147], 60.00th=[ 2376],
     | 70.00th=[ 2868], 80.00th=[ 3523], 90.00th=[ 5145], 95.00th=[ 6259],
     | 99.00th=[ 8848], 99.50th=[10421], 99.90th=[11994], 99.95th=[11994],
     | 99.99th=[11994]
   bw (  KiB/s): min=  616, max= 1312, per=24.11%, avg=895.33, stdev=367.76, samples=3
   iops        : min=  154, max=  328, avg=223.67, stdev=92.03, samples=3
  lat (usec)   : 750=4.30%, 1000=22.92%
  lat (msec)   : 2=33.85%, 4=27.73%, 10=10.42%, 20=0.65%, 50=0.13%
  cpu          : usr=0.29%, sys=5.84%, ctx=804, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14293: Fri Jun 14 18:06:04 2024
   read: IOPS=235, BW=944KiB/s (967kB/s)(1564KiB/1657msec)
    clat (usec): min=588, max=7445, avg=1437.15, stdev=1160.59
     lat (usec): min=588, max=7446, avg=1437.81, stdev=1160.60
    clat percentiles (usec):
     |  1.00th=[  627],  5.00th=[  709], 10.00th=[  742], 20.00th=[  791],
     | 30.00th=[  848], 40.00th=[  889], 50.00th=[  938], 60.00th=[  996],
     | 70.00th=[ 1188], 80.00th=[ 1795], 90.00th=[ 3064], 95.00th=[ 4113],
     | 99.00th=[ 5932], 99.50th=[ 6521], 99.90th=[ 7439], 99.95th=[ 7439],
     | 99.99th=[ 7439]
   bw (  KiB/s): min=  560, max= 1312, per=24.93%, avg=880.00, stdev=388.31, samples=3
   iops        : min=  140, max=  328, avg=220.00, stdev=97.08, samples=3
  write: IOPS=227, BW=910KiB/s (932kB/s)(1508KiB/1657msec)
    clat (usec): min=1183, max=40672, avg=2879.27, stdev=2427.37
     lat (usec): min=1184, max=40675, avg=2880.08, stdev=2427.49
    clat percentiles (usec):
     |  1.00th=[ 1221],  5.00th=[ 1401], 10.00th=[ 1614], 20.00th=[ 1811],
     | 30.00th=[ 1893], 40.00th=[ 2024], 50.00th=[ 2212], 60.00th=[ 2507],
     | 70.00th=[ 3032], 80.00th=[ 3720], 90.00th=[ 4752], 95.00th=[ 5800],
     | 99.00th=[ 9372], 99.50th=[ 9765], 99.90th=[40633], 99.95th=[40633],
     | 99.99th=[40633]
   bw (  KiB/s): min=  648, max= 1288, per=23.41%, avg=869.33, stdev=362.77, samples=3
   iops        : min=  162, max=  322, avg=217.33, stdev=90.69, samples=3
  lat (usec)   : 750=5.47%, 1000=25.52%
  lat (msec)   : 2=29.04%, 4=29.30%, 10=10.55%, 50=0.13%
  cpu          : usr=0.72%, sys=5.56%, ctx=798, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14294: Fri Jun 14 18:06:04 2024
   read: IOPS=224, BW=898KiB/s (919kB/s)(1508KiB/1680msec)
    clat (usec): min=547, max=12797, avg=1527.14, stdev=1322.19
     lat (usec): min=547, max=12798, avg=1527.87, stdev=1322.15
    clat percentiles (usec):
     |  1.00th=[  627],  5.00th=[  709], 10.00th=[  742], 20.00th=[  799],
     | 30.00th=[  857], 40.00th=[  898], 50.00th=[  971], 60.00th=[ 1090],
     | 70.00th=[ 1352], 80.00th=[ 1991], 90.00th=[ 3425], 95.00th=[ 4228],
     | 99.00th=[ 5866], 99.50th=[ 9765], 99.90th=[12780], 99.95th=[12780],
     | 99.99th=[12780]
   bw (  KiB/s): min=  568, max= 1160, per=23.40%, avg=826.00, stdev=303.23, samples=3
   iops        : min=  142, max=  290, avg=206.33, stdev=75.87, samples=3
  write: IOPS=232, BW=931KiB/s (953kB/s)(1564KiB/1680msec)
    clat (usec): min=1163, max=26128, avg=2798.93, stdev=1950.35
     lat (usec): min=1164, max=26130, avg=2799.72, stdev=1950.42
    clat percentiles (usec):
     |  1.00th=[ 1221],  5.00th=[ 1385], 10.00th=[ 1565], 20.00th=[ 1778],
     | 30.00th=[ 1860], 40.00th=[ 1975], 50.00th=[ 2114], 60.00th=[ 2343],
     | 70.00th=[ 2868], 80.00th=[ 3654], 90.00th=[ 4817], 95.00th=[ 6325],
     | 99.00th=[ 9241], 99.50th=[13435], 99.90th=[26084], 99.95th=[26084],
     | 99.99th=[26084]
   bw (  KiB/s): min=  608, max= 1336, per=23.53%, avg=874.00, stdev=401.63, samples=3
   iops        : min=  152, max=  334, avg=218.33, stdev=100.53, samples=3
  lat (usec)   : 750=5.21%, 1000=20.18%
  lat (msec)   : 2=36.33%, 4=27.21%, 10=10.68%, 20=0.26%, 50=0.13%
  cpu          : usr=0.60%, sys=5.60%, ctx=828, majf=0, minf=30
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3531KiB/s (3615kB/s), 847KiB/s-944KiB/s (867kB/s-967kB/s), io=5988KiB (6132kB), run=1657-1696msec
  WRITE: bw=3715KiB/s (3804kB/s), 910KiB/s-965KiB/s (932kB/s-988kB/s), io=6300KiB (6451kB), run=1657-1696msec