== sanity test 398b: DIO and buffer IO race ============== 07:14:28 (1747826068)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0818289 s, 154 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=29944: Wed May 21 07:14:30 2025
   read: IOPS=343, BW=1373KiB/s (1406kB/s)(1480KiB/1078msec)
    clat (usec): min=4, max=27572, avg=1628.88, stdev=2732.69
     lat (usec): min=4, max=27573, avg=1629.49, stdev=2732.75
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[   14], 50.00th=[   17], 60.00th=[ 1106],
     | 70.00th=[ 2343], 80.00th=[ 3261], 90.00th=[ 4555], 95.00th=[ 5538],
     | 99.00th=[10814], 99.50th=[17957], 99.90th=[27657], 99.95th=[27657],
     | 99.99th=[27657]
   bw (  KiB/s): min= 1208, max= 1408, per=24.53%, avg=1308.00, stdev=141.42, samples=2
   iops        : min=  302, max=  352, avg=327.00, stdev=35.36, samples=2
  write: IOPS=369, BW=1477KiB/s (1512kB/s)(1592KiB/1078msec)
    clat (usec): min=81, max=17043, avg=1156.57, stdev=1842.61
     lat (usec): min=82, max=17044, avg=1157.27, stdev=1842.61
    clat percentiles (usec):
     |  1.00th=[   92],  5.00th=[  121], 10.00th=[  137], 20.00th=[  149],
     | 30.00th=[  161], 40.00th=[  180], 50.00th=[  198], 60.00th=[  306],
     | 70.00th=[ 1270], 80.00th=[ 2073], 90.00th=[ 3458], 95.00th=[ 5014],
     | 99.00th=[ 8356], 99.50th=[10552], 99.90th=[17171], 99.95th=[17171],
     | 99.99th=[17171]
   bw (  KiB/s): min= 1392, max= 1472, per=25.53%, avg=1432.00, stdev=56.57, samples=2
   iops        : min=  348, max=  368, avg=358.00, stdev=14.14, samples=2
  lat (usec)   : 10=10.68%, 20=14.45%, 50=2.60%, 100=0.65%, 250=29.56%
  lat (usec)   : 500=2.73%, 750=0.39%, 1000=1.43%
  lat (msec)   : 2=10.55%, 4=15.49%, 10=10.42%, 20=0.91%, 50=0.13%
  cpu          : usr=0.09%, sys=16.62%, ctx=678, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=29945: Wed May 21 07:14:30 2025
   read: IOPS=319, BW=1279KiB/s (1309kB/s)(1436KiB/1123msec)
    clat (usec): min=3, max=24572, avg=1786.99, stdev=2980.62
     lat (usec): min=3, max=24573, avg=1788.37, stdev=2981.43
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[   13], 50.00th=[   18], 60.00th=[ 1385],
     | 70.00th=[ 2638], 80.00th=[ 3425], 90.00th=[ 4752], 95.00th=[ 6718],
     | 99.00th=[14222], 99.50th=[23200], 99.90th=[24511], 99.95th=[24511],
     | 99.99th=[24511]
   bw (  KiB/s): min= 1288, max= 1320, per=24.46%, avg=1304.00, stdev=22.63, samples=2
   iops        : min=  322, max=  330, avg=326.00, stdev= 5.66, samples=2
  write: IOPS=364, BW=1457KiB/s (1492kB/s)(1636KiB/1123msec)
    clat (usec): min=73, max=12383, avg=1138.37, stdev=1872.78
     lat (usec): min=73, max=12384, avg=1139.07, stdev=1872.81
    clat percentiles (usec):
     |  1.00th=[   79],  5.00th=[  110], 10.00th=[  125], 20.00th=[  143],
     | 30.00th=[  157], 40.00th=[  172], 50.00th=[  200], 60.00th=[  273],
     | 70.00th=[ 1221], 80.00th=[ 1762], 90.00th=[ 3130], 95.00th=[ 4883],
     | 99.00th=[ 8717], 99.50th=[10552], 99.90th=[12387], 99.95th=[12387],
     | 99.99th=[12387]
   bw (  KiB/s): min= 1336, max= 1600, per=26.17%, avg=1468.00, stdev=186.68, samples=2
   iops        : min=  334, max=  400, avg=367.00, stdev=46.67, samples=2
  lat (usec)   : 4=0.52%, 10=11.85%, 20=11.98%, 50=1.43%, 100=2.08%
  lat (usec)   : 250=29.43%, 500=2.21%, 750=0.91%, 1000=1.30%
  lat (msec)   : 2=12.24%, 4=16.15%, 10=8.72%, 20=0.91%, 50=0.26%
  cpu          : usr=0.45%, sys=15.78%, ctx=639, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=29946: Wed May 21 07:14:30 2025
   read: IOPS=399, BW=1598KiB/s (1636kB/s)(1564KiB/979msec)
    clat (usec): min=3, max=33896, avg=1372.20, stdev=2733.98
     lat (usec): min=3, max=33897, avg=1372.74, stdev=2734.06
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    7], 20.00th=[    8],
     | 30.00th=[    9], 40.00th=[   11], 50.00th=[   13], 60.00th=[   15],
     | 70.00th=[ 1991], 80.00th=[ 2999], 90.00th=[ 4293], 95.00th=[ 5211],
     | 99.00th=[10683], 99.50th=[17695], 99.90th=[33817], 99.95th=[33817],
     | 99.99th=[33817]
   bw (  KiB/s): min= 1688, max= 1688, per=31.66%, avg=1688.00, stdev= 0.00, samples=1
   iops        : min=  422, max=  422, avg=422.00, stdev= 0.00, samples=1
  write: IOPS=385, BW=1540KiB/s (1577kB/s)(1508KiB/979msec)
    clat (usec): min=80, max=27261, avg=1130.70, stdev=2254.53
     lat (usec): min=80, max=27262, avg=1132.25, stdev=2254.60
    clat percentiles (usec):
     |  1.00th=[   85],  5.00th=[  106], 10.00th=[  120], 20.00th=[  139],
     | 30.00th=[  149], 40.00th=[  161], 50.00th=[  182], 60.00th=[  217],
     | 70.00th=[ 1029], 80.00th=[ 1549], 90.00th=[ 3458], 95.00th=[ 4948],
     | 99.00th=[10683], 99.50th=[11863], 99.90th=[27132], 99.95th=[27132],
     | 99.99th=[27132]
   bw (  KiB/s): min= 1744, max= 1744, per=31.09%, avg=1744.00, stdev= 0.00, samples=1
   iops        : min=  436, max=  436, avg=436.00, stdev= 0.00, samples=1
  lat (usec)   : 4=0.91%, 10=18.10%, 20=13.28%, 50=1.04%, 100=2.08%
  lat (usec)   : 250=29.56%, 500=2.08%, 750=0.39%, 1000=0.39%
  lat (msec)   : 2=7.68%, 4=14.71%, 10=8.72%, 20=0.78%, 50=0.26%
  cpu          : usr=0.51%, sys=17.48%, ctx=537, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=29947: Wed May 21 07:14:30 2025
   read: IOPS=368, BW=1476KiB/s (1511kB/s)(1508KiB/1022msec)
    clat (usec): min=3, max=22240, avg=1580.32, stdev=2661.90
     lat (usec): min=3, max=22241, avg=1582.80, stdev=2661.74
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[   12], 50.00th=[   14], 60.00th=[   24],
     | 70.00th=[ 2343], 80.00th=[ 3458], 90.00th=[ 4621], 95.00th=[ 5932],
     | 99.00th=[11863], 99.50th=[18744], 99.90th=[22152], 99.95th=[22152],
     | 99.99th=[22152]
   bw (  KiB/s): min= 1288, max= 1568, per=26.78%, avg=1428.00, stdev=197.99, samples=2
   iops        : min=  322, max=  392, avg=357.00, stdev=49.50, samples=2
  write: IOPS=382, BW=1530KiB/s (1567kB/s)(1564KiB/1022msec)
    clat (usec): min=79, max=14553, avg=1061.78, stdev=1898.36
     lat (usec): min=80, max=14554, avg=1062.64, stdev=1898.38
    clat percentiles (usec):
     |  1.00th=[   89],  5.00th=[  116], 10.00th=[  135], 20.00th=[  147],
     | 30.00th=[  155], 40.00th=[  169], 50.00th=[  188], 60.00th=[  225],
     | 70.00th=[  545], 80.00th=[ 1467], 90.00th=[ 3720], 95.00th=[ 5342],
     | 99.00th=[ 8160], 99.50th=[13042], 99.90th=[14615], 99.95th=[14615],
     | 99.99th=[14615]
   bw (  KiB/s): min= 1424, max= 1560, per=26.60%, avg=1492.00, stdev=96.17, samples=2
   iops        : min=  356, max=  390, avg=373.00, stdev=24.04, samples=2
  lat (usec)   : 4=0.26%, 10=13.67%, 20=14.84%, 50=1.04%, 100=1.04%
  lat (usec)   : 250=30.60%, 500=3.65%, 750=1.30%, 1000=0.91%
  lat (msec)   : 2=8.07%, 4=13.02%, 10=10.68%, 20=0.78%, 50=0.13%
  cpu          : usr=0.39%, sys=15.38%, ctx=596, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=5332KiB/s (5460kB/s), 1279KiB/s-1598KiB/s (1309kB/s-1636kB/s), io=5988KiB (6132kB), run=979-1123msec
  WRITE: bw=5610KiB/s (5745kB/s), 1457KiB/s-1540KiB/s (1492kB/s-1577kB/s), io=6300KiB (6451kB), run=979-1123msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=29949: Wed May 21 07:14:31 2025
   read: IOPS=159, BW=639KiB/s (654kB/s)(1480KiB/2317msec)
    clat (usec): min=698, max=23029, avg=2121.78, stdev=2419.69
     lat (usec): min=698, max=23030, avg=2122.77, stdev=2419.87
    clat percentiles (usec):
     |  1.00th=[  742],  5.00th=[  873], 10.00th=[  914], 20.00th=[ 1012],
     | 30.00th=[ 1090], 40.00th=[ 1188], 50.00th=[ 1369], 60.00th=[ 1532],
     | 70.00th=[ 1762], 80.00th=[ 2245], 90.00th=[ 4293], 95.00th=[ 6325],
     | 99.00th=[14353], 99.50th=[18744], 99.90th=[22938], 99.95th=[22938],
     | 99.99th=[22938]
   bw (  KiB/s): min=  320, max=  888, per=23.82%, avg=615.50, stdev=274.42, samples=4
   iops        : min=   80, max=  222, avg=153.75, stdev=68.49, samples=4
  write: IOPS=171, BW=687KiB/s (704kB/s)(1592KiB/2317msec)
    clat (usec): min=1336, max=15945, avg=3816.70, stdev=2077.86
     lat (usec): min=1337, max=15946, avg=3817.74, stdev=2077.93
    clat percentiles (usec):
     |  1.00th=[ 1549],  5.00th=[ 1942], 10.00th=[ 2114], 20.00th=[ 2442],
     | 30.00th=[ 2638], 40.00th=[ 2868], 50.00th=[ 3130], 60.00th=[ 3490],
     | 70.00th=[ 3982], 80.00th=[ 4948], 90.00th=[ 6521], 95.00th=[ 8356],
     | 99.00th=[12518], 99.50th=[15533], 99.90th=[15926], 99.95th=[15926],
     | 99.99th=[15926]
   bw (  KiB/s): min=  376, max= 1072, per=24.99%, avg=679.50, stdev=336.24, samples=4
   iops        : min=   94, max=  268, avg=169.75, stdev=83.98, samples=4
  lat (usec)   : 750=0.65%, 1000=8.33%
  lat (msec)   : 2=31.25%, 4=39.71%, 10=18.36%, 20=1.56%, 50=0.13%
  cpu          : usr=0.60%, sys=5.57%, ctx=806, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=29950: Wed May 21 07:14:31 2025
   read: IOPS=160, BW=642KiB/s (657kB/s)(1436KiB/2237msec)
    clat (usec): min=691, max=23399, avg=1954.82, stdev=1982.07
     lat (usec): min=692, max=23401, avg=1955.60, stdev=1982.11
    clat percentiles (usec):
     |  1.00th=[  791],  5.00th=[  873], 10.00th=[  914], 20.00th=[ 1004],
     | 30.00th=[ 1090], 40.00th=[ 1188], 50.00th=[ 1287], 60.00th=[ 1434],
     | 70.00th=[ 1729], 80.00th=[ 2278], 90.00th=[ 3752], 95.00th=[ 5407],
     | 99.00th=[10421], 99.50th=[11731], 99.90th=[23462], 99.95th=[23462],
     | 99.99th=[23462]
   bw (  KiB/s): min=  368, max=  952, per=24.69%, avg=638.00, stdev=282.95, samples=4
   iops        : min=   92, max=  238, avg=159.50, stdev=70.74, samples=4
  write: IOPS=182, BW=731KiB/s (749kB/s)(1636KiB/2237msec)
    clat (usec): min=1380, max=17272, avg=3722.13, stdev=2105.49
     lat (usec): min=1381, max=17273, avg=3723.14, stdev=2105.59
    clat percentiles (usec):
     |  1.00th=[ 1598],  5.00th=[ 1926], 10.00th=[ 2180], 20.00th=[ 2442],
     | 30.00th=[ 2671], 40.00th=[ 2868], 50.00th=[ 3064], 60.00th=[ 3326],
     | 70.00th=[ 3687], 80.00th=[ 4424], 90.00th=[ 6456], 95.00th=[ 7832],
     | 99.00th=[12780], 99.50th=[14484], 99.90th=[17171], 99.95th=[17171],
     | 99.99th=[17171]
   bw (  KiB/s): min=  392, max= 1056, per=26.33%, avg=716.00, stdev=314.93, samples=4
   iops        : min=   98, max=  264, avg=179.00, stdev=78.73, samples=4
  lat (usec)   : 750=0.13%, 1000=8.85%
  lat (msec)   : 2=29.95%, 4=42.97%, 10=16.28%, 20=1.69%, 50=0.13%
  cpu          : usr=0.13%, sys=6.22%, ctx=802, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=29951: Wed May 21 07:14:31 2025
   read: IOPS=179, BW=718KiB/s (735kB/s)(1564KiB/2179msec)
    clat (usec): min=692, max=12694, avg=1848.89, stdev=1678.13
     lat (usec): min=693, max=12695, avg=1849.73, stdev=1678.16
    clat percentiles (usec):
     |  1.00th=[  758],  5.00th=[  857], 10.00th=[  914], 20.00th=[ 1020],
     | 30.00th=[ 1074], 40.00th=[ 1156], 50.00th=[ 1254], 60.00th=[ 1418],
     | 70.00th=[ 1614], 80.00th=[ 1926], 90.00th=[ 3458], 95.00th=[ 5866],
     | 99.00th=[ 8979], 99.50th=[11994], 99.90th=[12649], 99.95th=[12649],
     | 99.99th=[12649]
   bw (  KiB/s): min=  360, max= 1120, per=26.99%, avg=697.50, stdev=366.40, samples=4
   iops        : min=   90, max=  280, avg=174.25, stdev=91.51, samples=4
  write: IOPS=173, BW=692KiB/s (709kB/s)(1508KiB/2179msec)
    clat (usec): min=1362, max=30801, avg=3829.15, stdev=2527.89
     lat (usec): min=1363, max=30804, avg=3830.47, stdev=2527.96
    clat percentiles (usec):
     |  1.00th=[ 1500],  5.00th=[ 2040], 10.00th=[ 2147], 20.00th=[ 2442],
     | 30.00th=[ 2573], 40.00th=[ 2868], 50.00th=[ 3163], 60.00th=[ 3392],
     | 70.00th=[ 3752], 80.00th=[ 4555], 90.00th=[ 6456], 95.00th=[ 8586],
     | 99.00th=[12649], 99.50th=[17171], 99.90th=[30802], 99.95th=[30802],
     | 99.99th=[30802]
   bw (  KiB/s): min=  440, max=  960, per=24.77%, avg=673.50, stdev=273.07, samples=4
   iops        : min=  110, max=  240, avg=168.25, stdev=68.16, samples=4
  lat (usec)   : 750=0.39%, 1000=8.33%
  lat (msec)   : 2=34.64%, 4=39.32%, 10=15.49%, 20=1.69%, 50=0.13%
  cpu          : usr=0.64%, sys=5.56%, ctx=792, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=29952: Wed May 21 07:14:31 2025
   read: IOPS=171, BW=686KiB/s (703kB/s)(1508KiB/2198msec)
    clat (usec): min=649, max=13417, avg=1905.26, stdev=1763.68
     lat (usec): min=649, max=13417, avg=1906.13, stdev=1763.69
    clat percentiles (usec):
     |  1.00th=[  725],  5.00th=[  848], 10.00th=[  914], 20.00th=[  988],
     | 30.00th=[ 1074], 40.00th=[ 1156], 50.00th=[ 1237], 60.00th=[ 1401],
     | 70.00th=[ 1631], 80.00th=[ 2180], 90.00th=[ 3884], 95.00th=[ 6128],
     | 99.00th=[ 9241], 99.50th=[12125], 99.90th=[13435], 99.95th=[13435],
     | 99.99th=[13435]
   bw (  KiB/s): min=  368, max=  920, per=25.70%, avg=664.00, stdev=281.18, samples=4
   iops        : min=   92, max=  230, avg=166.00, stdev=70.29, samples=4
  write: IOPS=177, BW=712KiB/s (729kB/s)(1564KiB/2198msec)
    clat (usec): min=1409, max=30761, avg=3757.53, stdev=2401.62
     lat (usec): min=1410, max=30763, avg=3758.60, stdev=2401.65
    clat percentiles (usec):
     |  1.00th=[ 1467],  5.00th=[ 1860], 10.00th=[ 2180], 20.00th=[ 2442],
     | 30.00th=[ 2671], 40.00th=[ 2933], 50.00th=[ 3097], 60.00th=[ 3392],
     | 70.00th=[ 3752], 80.00th=[ 4424], 90.00th=[ 6325], 95.00th=[ 8094],
     | 99.00th=[11731], 99.50th=[18482], 99.90th=[30802], 99.95th=[30802],
     | 99.99th=[30802]
   bw (  KiB/s): min=  424, max= 1040, per=25.82%, avg=702.00, stdev=309.11, samples=4
   iops        : min=  106, max=  260, avg=175.50, stdev=77.28, samples=4
  lat (usec)   : 750=0.65%, 1000=10.16%
  lat (msec)   : 2=31.64%, 4=39.71%, 10=16.54%, 20=1.17%, 50=0.13%
  cpu          : usr=0.46%, sys=5.83%, ctx=803, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=2584KiB/s (2646kB/s), 639KiB/s-718KiB/s (654kB/s-735kB/s), io=5988KiB (6132kB), run=2179-2317msec
  WRITE: bw=2719KiB/s (2784kB/s), 687KiB/s-731KiB/s (704kB/s-749kB/s), io=6300KiB (6451kB), run=2179-2317msec