== sanity test 398b: DIO and buffer IO race ============== 20:09:11 (1736989751)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0758553 s, 166 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=20042: Wed Jan 15 20:09:14 2025
   read: IOPS=280, BW=1122KiB/s (1149kB/s)(1480KiB/1319msec)
    clat (usec): min=3, max=19636, avg=1966.96, stdev=2371.80
     lat (usec): min=4, max=19637, avg=1967.95, stdev=2371.87
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    8], 10.00th=[   10], 20.00th=[   12],
     | 30.00th=[   15], 40.00th=[   81], 50.00th=[ 1844], 60.00th=[ 2442],
     | 70.00th=[ 2835], 80.00th=[ 3359], 90.00th=[ 4228], 95.00th=[ 5080],
     | 99.00th=[10814], 99.50th=[19530], 99.90th=[19530], 99.95th=[19530],
     | 99.99th=[19530]
   bw (  KiB/s): min=  824, max= 1101, per=22.44%, avg=962.50, stdev=195.87, samples=2
   iops        : min=  206, max=  275, avg=240.50, stdev=48.79, samples=2
  write: IOPS=301, BW=1207KiB/s (1236kB/s)(1592KiB/1319msec)
    clat (usec): min=87, max=9680, avg=1449.54, stdev=1671.05
     lat (usec): min=88, max=9681, avg=1450.61, stdev=1670.94
    clat percentiles (usec):
     |  1.00th=[  101],  5.00th=[  129], 10.00th=[  145], 20.00th=[  163],
     | 30.00th=[  188], 40.00th=[  221], 50.00th=[  955], 60.00th=[ 1287],
     | 70.00th=[ 1795], 80.00th=[ 2606], 90.00th=[ 3916], 95.00th=[ 5014],
     | 99.00th=[ 6915], 99.50th=[ 8455], 99.90th=[ 9634], 99.95th=[ 9634],
     | 99.99th=[ 9634]
   bw (  KiB/s): min=  872, max= 1277, per=23.81%, avg=1074.50, stdev=286.38, samples=2
   iops        : min=  218, max=  319, avg=268.50, stdev=71.42, samples=2
  lat (usec)   : 4=0.26%, 10=5.99%, 20=12.37%, 50=0.39%, 100=0.78%
  lat (usec)   : 250=21.74%, 500=1.69%, 750=0.78%, 1000=2.34%
  lat (msec)   : 2=16.28%, 4=26.56%, 10=10.29%, 20=0.52%
  cpu          : usr=0.15%, sys=15.55%, ctx=884, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20043: Wed Jan 15 20:09:14 2025
   read: IOPS=257, BW=1029KiB/s (1053kB/s)(1436KiB/1396msec)
    clat (usec): min=3, max=29424, avg=2285.46, stdev=2787.44
     lat (usec): min=4, max=29425, avg=2286.08, stdev=2787.46
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    8], 10.00th=[   11], 20.00th=[   13],
     | 30.00th=[   17], 40.00th=[ 1123], 50.00th=[ 2057], 60.00th=[ 2671],
     | 70.00th=[ 3359], 80.00th=[ 3785], 90.00th=[ 4621], 95.00th=[ 5932],
     | 99.00th=[14222], 99.50th=[16712], 99.90th=[29492], 99.95th=[29492],
     | 99.99th=[29492]
   bw (  KiB/s): min=  824, max=  992, per=21.17%, avg=908.00, stdev=118.79, samples=2
   iops        : min=  206, max=  248, avg=227.00, stdev=29.70, samples=2
  write: IOPS=292, BW=1172KiB/s (1200kB/s)(1636KiB/1396msec)
    clat (usec): min=88, max=7840, avg=1372.06, stdev=1569.54
     lat (usec): min=88, max=7841, avg=1373.25, stdev=1569.55
    clat percentiles (usec):
     |  1.00th=[  117],  5.00th=[  129], 10.00th=[  145], 20.00th=[  165],
     | 30.00th=[  186], 40.00th=[  231], 50.00th=[  988], 60.00th=[ 1254],
     | 70.00th=[ 1598], 80.00th=[ 2409], 90.00th=[ 3687], 95.00th=[ 5211],
     | 99.00th=[ 6325], 99.50th=[ 6652], 99.90th=[ 7832], 99.95th=[ 7832],
     | 99.99th=[ 7832]
   bw (  KiB/s): min=  920, max= 1144, per=22.87%, avg=1032.00, stdev=158.39, samples=2
   iops        : min=  230, max=  286, avg=258.00, stdev=39.60, samples=2
  lat (usec)   : 4=0.13%, 10=4.56%, 20=11.07%, 50=0.78%, 100=0.26%
  lat (usec)   : 250=21.87%, 500=2.47%, 750=0.65%, 1000=2.99%
  lat (msec)   : 2=19.01%, 4=23.96%, 10=11.59%, 20=0.52%, 50=0.13%
  cpu          : usr=0.29%, sys=17.35%, ctx=959, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20044: Wed Jan 15 20:09:14 2025
   read: IOPS=305, BW=1223KiB/s (1252kB/s)(1564KiB/1279msec)
    clat (usec): min=3, max=20877, avg=1974.85, stdev=2331.09
     lat (usec): min=3, max=20878, avg=1975.89, stdev=2330.97
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   12],
     | 30.00th=[   15], 40.00th=[   32], 50.00th=[ 1680], 60.00th=[ 2343],
     | 70.00th=[ 2900], 80.00th=[ 3589], 90.00th=[ 4424], 95.00th=[ 5342],
     | 99.00th=[12387], 99.50th=[13566], 99.90th=[20841], 99.95th=[20841],
     | 99.99th=[20841]
   bw (  KiB/s): min=  872, max= 1085, per=22.81%, avg=978.50, stdev=150.61, samples=2
   iops        : min=  218, max=  271, avg=244.50, stdev=37.48, samples=2
  write: IOPS=294, BW=1179KiB/s (1207kB/s)(1508KiB/1279msec)
    clat (usec): min=71, max=10071, avg=1303.74, stdev=1599.30
     lat (usec): min=72, max=10073, avg=1304.75, stdev=1599.34
    clat percentiles (usec):
     |  1.00th=[   94],  5.00th=[  128], 10.00th=[  137], 20.00th=[  163],
     | 30.00th=[  186], 40.00th=[  231], 50.00th=[  578], 60.00th=[ 1139],
     | 70.00th=[ 1418], 80.00th=[ 2245], 90.00th=[ 3589], 95.00th=[ 4948],
     | 99.00th=[ 6718], 99.50th=[ 8225], 99.90th=[10028], 99.95th=[10028],
     | 99.99th=[10028]
   bw (  KiB/s): min=  896, max= 1013, per=21.15%, avg=954.50, stdev=82.73, samples=2
   iops        : min=  224, max=  253, avg=238.50, stdev=20.51, samples=2
  lat (usec)   : 4=0.39%, 10=8.20%, 20=11.20%, 50=0.91%, 100=0.91%
  lat (usec)   : 250=20.05%, 500=3.78%, 750=0.39%, 1000=2.47%
  lat (msec)   : 2=16.28%, 4=24.35%, 10=10.42%, 20=0.52%, 50=0.13%
  cpu          : usr=0.31%, sys=15.26%, ctx=865, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20045: Wed Jan 15 20:09:14 2025
   read: IOPS=275, BW=1103KiB/s (1130kB/s)(1508KiB/1367msec)
    clat (usec): min=3, max=31573, avg=2233.42, stdev=3258.11
     lat (usec): min=4, max=31574, avg=2234.10, stdev=3258.13
    clat percentiles (usec):
     |  1.00th=[    6],  5.00th=[    8], 10.00th=[   10], 20.00th=[   13],
     | 30.00th=[   16], 40.00th=[ 1188], 50.00th=[ 1958], 60.00th=[ 2474],
     | 70.00th=[ 3032], 80.00th=[ 3556], 90.00th=[ 4228], 95.00th=[ 5014],
     | 99.00th=[20841], 99.50th=[27657], 99.90th=[31589], 99.95th=[31589],
     | 99.99th=[31589]
   bw (  KiB/s): min=  872, max= 1021, per=22.07%, avg=946.50, stdev=105.36, samples=2
   iops        : min=  218, max=  255, avg=236.50, stdev=26.16, samples=2
  write: IOPS=286, BW=1144KiB/s (1172kB/s)(1564KiB/1367msec)
    clat (usec): min=93, max=8839, avg=1313.54, stdev=1593.87
     lat (usec): min=93, max=8840, avg=1314.31, stdev=1593.87
    clat percentiles (usec):
     |  1.00th=[  104],  5.00th=[  129], 10.00th=[  143], 20.00th=[  165],
     | 30.00th=[  186], 40.00th=[  212], 50.00th=[  725], 60.00th=[ 1172],
     | 70.00th=[ 1532], 80.00th=[ 2311], 90.00th=[ 3556], 95.00th=[ 4752],
     | 99.00th=[ 7701], 99.50th=[ 8455], 99.90th=[ 8848], 99.95th=[ 8848],
     | 99.99th=[ 8848]
   bw (  KiB/s): min=  912, max= 1069, per=21.95%, avg=990.50, stdev=111.02, samples=2
   iops        : min=  228, max=  267, avg=247.50, stdev=27.58, samples=2
  lat (usec)   : 4=0.13%, 10=5.99%, 20=11.33%, 50=1.17%, 100=0.13%
  lat (usec)   : 250=23.05%, 500=1.95%, 750=0.65%, 1000=1.95%
  lat (msec)   : 2=18.36%, 4=25.39%, 10=8.72%, 20=0.65%, 50=0.52%
  cpu          : usr=0.66%, sys=16.03%, ctx=934, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=4289KiB/s (4392kB/s), 1029KiB/s-1223KiB/s (1053kB/s-1252kB/s), io=5988KiB (6132kB), run=1279-1396msec
  WRITE: bw=4513KiB/s (4621kB/s), 1144KiB/s-1207KiB/s (1172kB/s-1236kB/s), io=6300KiB (6451kB), run=1279-1396msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=20046: Wed Jan 15 20:09:15 2025
   read: IOPS=182, BW=731KiB/s (748kB/s)(1480KiB/2025msec)
    clat (usec): min=595, max=24507, avg=1926.04, stdev=2236.76
     lat (usec): min=595, max=24508, avg=1926.78, stdev=2236.90
    clat percentiles (usec):
     |  1.00th=[  652],  5.00th=[  701], 10.00th=[  783], 20.00th=[  832],
     | 30.00th=[  898], 40.00th=[ 1029], 50.00th=[ 1139], 60.00th=[ 1270],
     | 70.00th=[ 1565], 80.00th=[ 2343], 90.00th=[ 4113], 95.00th=[ 5997],
     | 99.00th=[10290], 99.50th=[16581], 99.90th=[24511], 99.95th=[24511],
     | 99.99th=[24511]
   bw (  KiB/s): min=  496, max= 1224, per=24.40%, avg=721.50, stdev=338.02, samples=4
   iops        : min=  124, max=  306, avg=180.25, stdev=84.56, samples=4
  write: IOPS=196, BW=786KiB/s (805kB/s)(1592KiB/2025msec)
    clat (usec): min=1138, max=14437, avg=3280.10, stdev=1971.16
     lat (usec): min=1139, max=14438, avg=3281.14, stdev=1971.32
    clat percentiles (usec):
     |  1.00th=[ 1319],  5.00th=[ 1549], 10.00th=[ 1745], 20.00th=[ 1942],
     | 30.00th=[ 2073], 40.00th=[ 2278], 50.00th=[ 2507], 60.00th=[ 2933],
     | 70.00th=[ 3523], 80.00th=[ 4359], 90.00th=[ 5932], 95.00th=[ 7177],
     | 99.00th=[11731], 99.50th=[11863], 99.90th=[14484], 99.95th=[14484],
     | 99.99th=[14484]
   bw (  KiB/s): min=  464, max= 1328, per=25.12%, avg=781.50, stdev=393.42, samples=4
   iops        : min=  116, max=  332, avg=195.25, stdev=98.34, samples=4
  lat (usec)   : 750=3.39%, 1000=14.45%
  lat (msec)   : 2=30.34%, 4=34.11%, 10=16.41%, 20=1.17%, 50=0.13%
  cpu          : usr=0.54%, sys=5.39%, ctx=811, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20047: Wed Jan 15 20:09:15 2025
   read: IOPS=178, BW=715KiB/s (733kB/s)(1436KiB/2007msec)
    clat (usec): min=601, max=24493, avg=1906.00, stdev=2269.43
     lat (usec): min=602, max=24494, avg=1906.78, stdev=2269.48
    clat percentiles (usec):
     |  1.00th=[  652],  5.00th=[  717], 10.00th=[  766], 20.00th=[  832],
     | 30.00th=[  898], 40.00th=[  963], 50.00th=[ 1057], 60.00th=[ 1221],
     | 70.00th=[ 1549], 80.00th=[ 2114], 90.00th=[ 4621], 95.00th=[ 6063],
     | 99.00th=[ 9634], 99.50th=[15401], 99.90th=[24511], 99.95th=[24511],
     | 99.99th=[24511]
   bw (  KiB/s): min=  440, max= 1213, per=24.19%, avg=715.25, stdev=353.64, samples=4
   iops        : min=  110, max=  303, avg=178.75, stdev=88.29, samples=4
  write: IOPS=203, BW=815KiB/s (835kB/s)(1636KiB/2007msec)
    clat (usec): min=1198, max=12147, avg=3213.44, stdev=1739.52
     lat (usec): min=1199, max=12149, avg=3214.43, stdev=1739.56
    clat percentiles (usec):
     |  1.00th=[ 1303],  5.00th=[ 1598], 10.00th=[ 1762], 20.00th=[ 1926],
     | 30.00th=[ 2114], 40.00th=[ 2245], 50.00th=[ 2474], 60.00th=[ 3032],
     | 70.00th=[ 3589], 80.00th=[ 4359], 90.00th=[ 5669], 95.00th=[ 6915],
     | 99.00th=[ 8979], 99.50th=[ 9503], 99.90th=[12125], 99.95th=[12125],
     | 99.99th=[12125]
   bw (  KiB/s): min=  552, max= 1365, per=26.14%, avg=813.25, stdev=382.19, samples=4
   iops        : min=  138, max=  341, avg=203.25, stdev=95.43, samples=4
  lat (usec)   : 750=3.78%, 1000=17.06%
  lat (msec)   : 2=29.04%, 4=31.38%, 10=18.23%, 20=0.39%, 50=0.13%
  cpu          : usr=0.80%, sys=5.58%, ctx=804, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20048: Wed Jan 15 20:09:15 2025
   read: IOPS=194, BW=779KiB/s (798kB/s)(1564KiB/2008msec)
    clat (usec): min=589, max=21971, avg=1907.87, stdev=2086.21
     lat (usec): min=589, max=21972, avg=1908.85, stdev=2086.52
    clat percentiles (usec):
     |  1.00th=[  652],  5.00th=[  709], 10.00th=[  758], 20.00th=[  816],
     | 30.00th=[  865], 40.00th=[  938], 50.00th=[ 1037], 60.00th=[ 1237],
     | 70.00th=[ 1598], 80.00th=[ 2376], 90.00th=[ 4948], 95.00th=[ 6325],
     | 99.00th=[ 9896], 99.50th=[11469], 99.90th=[21890], 99.95th=[21890],
     | 99.99th=[21890]
   bw (  KiB/s): min=  440, max= 1477, per=26.42%, avg=781.25, stdev=481.01, samples=4
   iops        : min=  110, max=  369, avg=195.25, stdev=120.13, samples=4
  write: IOPS=187, BW=751KiB/s (769kB/s)(1508KiB/2008msec)
    clat (usec): min=1124, max=32969, avg=3328.92, stdev=2415.70
     lat (usec): min=1124, max=32970, avg=3329.75, stdev=2415.79
    clat percentiles (usec):
     |  1.00th=[ 1270],  5.00th=[ 1532], 10.00th=[ 1795], 20.00th=[ 1926],
     | 30.00th=[ 2089], 40.00th=[ 2212], 50.00th=[ 2442], 60.00th=[ 2802],
     | 70.00th=[ 3556], 80.00th=[ 4686], 90.00th=[ 6325], 95.00th=[ 7111],
     | 99.00th=[ 9896], 99.50th=[11994], 99.90th=[32900], 99.95th=[32900],
     | 99.99th=[32900]
   bw (  KiB/s): min=  456, max= 1285, per=24.02%, avg=747.25, stdev=375.42, samples=4
   iops        : min=  114, max=  321, avg=186.75, stdev=93.73, samples=4
  lat (usec)   : 750=4.69%, 1000=19.66%
  lat (msec)   : 2=26.30%, 4=30.86%, 10=17.71%, 20=0.52%, 50=0.26%
  cpu          : usr=0.75%, sys=5.13%, ctx=809, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20049: Wed Jan 15 20:09:15 2025
   read: IOPS=187, BW=750KiB/s (767kB/s)(1508KiB/2012msec)
    clat (usec): min=556, max=23833, avg=1954.07, stdev=2239.58
     lat (usec): min=557, max=23834, avg=1954.91, stdev=2239.55
    clat percentiles (usec):
     |  1.00th=[  660],  5.00th=[  709], 10.00th=[  766], 20.00th=[  824],
     | 30.00th=[  889], 40.00th=[  971], 50.00th=[ 1090], 60.00th=[ 1254],
     | 70.00th=[ 1532], 80.00th=[ 2245], 90.00th=[ 4817], 95.00th=[ 6259],
     | 99.00th=[ 9372], 99.50th=[15533], 99.90th=[23725], 99.95th=[23725],
     | 99.99th=[23725]
   bw (  KiB/s): min=  472, max= 1261, per=25.20%, avg=745.25, stdev=368.36, samples=4
   iops        : min=  118, max=  315, avg=186.25, stdev=91.97, samples=4
  write: IOPS=194, BW=777KiB/s (796kB/s)(1564KiB/2012msec)
    clat (usec): min=1147, max=24108, avg=3242.29, stdev=2132.66
     lat (usec): min=1147, max=24109, avg=3243.22, stdev=2132.74
    clat percentiles (usec):
     |  1.00th=[ 1352],  5.00th=[ 1647], 10.00th=[ 1811], 20.00th=[ 1942],
     | 30.00th=[ 2089], 40.00th=[ 2245], 50.00th=[ 2442], 60.00th=[ 2802],
     | 70.00th=[ 3294], 80.00th=[ 4424], 90.00th=[ 5604], 95.00th=[ 7177],
     | 99.00th=[10290], 99.50th=[12649], 99.90th=[23987], 99.95th=[23987],
     | 99.99th=[23987]
   bw (  KiB/s): min=  488, max= 1349, per=25.05%, avg=779.25, stdev=404.02, samples=4
   iops        : min=  122, max=  337, avg=194.75, stdev=100.89, samples=4
  lat (usec)   : 750=4.56%, 1000=16.41%
  lat (msec)   : 2=28.12%, 4=32.29%, 10=17.58%, 20=0.78%, 50=0.26%
  cpu          : usr=0.45%, sys=5.57%, ctx=801, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=2957KiB/s (3028kB/s), 715KiB/s-779KiB/s (733kB/s-798kB/s), io=5988KiB (6132kB), run=2007-2025msec
  WRITE: bw=3111KiB/s (3186kB/s), 751KiB/s-815KiB/s (769kB/s-835kB/s), io=6300KiB (6451kB), run=2007-2025msec