== sanity test 398b: DIO and buffer IO race ============== 20:02:34 (1736989354)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0676691 s, 186 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=19130: Wed Jan 15 20:02:37 2025
   read: IOPS=378, BW=1515KiB/s (1551kB/s)(1480KiB/977msec)
    clat (usec): min=3, max=22143, avg=1522.56, stdev=2198.42
     lat (usec): min=4, max=22144, avg=1523.15, stdev=2198.49
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    6], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[   24], 50.00th=[ 1270], 60.00th=[ 1762],
     | 70.00th=[ 2212], 80.00th=[ 2671], 90.00th=[ 3130], 95.00th=[ 4080],
     | 99.00th=[ 8979], 99.50th=[19268], 99.90th=[22152], 99.95th=[22152],
     | 99.99th=[22152]
   bw (  KiB/s): min= 1320, max= 1320, per=23.92%, avg=1320.00, stdev= 0.00, samples=1
   iops        : min=  330, max=  330, avg=330.00, stdev= 0.00, samples=1
  write: IOPS=407, BW=1629KiB/s (1669kB/s)(1592KiB/977msec)
    clat (usec): min=88, max=9436, avg=1016.93, stdev=1222.83
     lat (usec): min=88, max=9437, avg=1018.51, stdev=1223.40
    clat percentiles (usec):
     |  1.00th=[   92],  5.00th=[  102], 10.00th=[  114], 20.00th=[  139],
     | 30.00th=[  165], 40.00th=[  194], 50.00th=[  482], 60.00th=[  922],
     | 70.00th=[ 1139], 80.00th=[ 1762], 90.00th=[ 2638], 95.00th=[ 3654],
     | 99.00th=[ 5080], 99.50th=[ 6259], 99.90th=[ 9372], 99.95th=[ 9372],
     | 99.99th=[ 9372]
   bw (  KiB/s): min= 1344, max= 1344, per=23.15%, avg=1344.00, stdev= 0.00, samples=1
   iops        : min=  336, max=  336, avg=336.00, stdev= 0.00, samples=1
  lat (usec)   : 4=0.13%, 10=10.68%, 20=8.33%, 50=0.65%, 100=2.08%
  lat (usec)   : 250=22.40%, 500=2.86%, 750=1.04%, 1000=7.03%
  lat (msec)   : 2=18.23%, 4=22.53%, 10=3.65%, 20=0.26%, 50=0.13%
  cpu          : usr=0.31%, sys=16.39%, ctx=849, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19131: Wed Jan 15 20:02:37 2025
   read: IOPS=330, BW=1324KiB/s (1355kB/s)(1436KiB/1085msec)
    clat (usec): min=3, max=25653, avg=1778.90, stdev=2293.70
     lat (usec): min=3, max=25653, avg=1779.77, stdev=2293.81
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   15], 40.00th=[ 1012], 50.00th=[ 1647], 60.00th=[ 2040],
     | 70.00th=[ 2474], 80.00th=[ 2933], 90.00th=[ 3621], 95.00th=[ 4424],
     | 99.00th=[ 8717], 99.50th=[20841], 99.90th=[25560], 99.95th=[25560],
     | 99.99th=[25560]
   bw (  KiB/s): min= 1112, max= 1528, per=23.92%, avg=1320.00, stdev=294.16, samples=2
   iops        : min=  278, max=  382, avg=330.00, stdev=73.54, samples=2
  write: IOPS=376, BW=1508KiB/s (1544kB/s)(1636KiB/1085msec)
    clat (usec): min=84, max=9435, avg=1064.39, stdev=1286.76
     lat (usec): min=85, max=9436, avg=1065.05, stdev=1286.77
    clat percentiles (usec):
     |  1.00th=[   94],  5.00th=[  111], 10.00th=[  118], 20.00th=[  143],
     | 30.00th=[  167], 40.00th=[  204], 50.00th=[  742], 60.00th=[  979],
     | 70.00th=[ 1205], 80.00th=[ 1942], 90.00th=[ 2704], 95.00th=[ 3621],
     | 99.00th=[ 5276], 99.50th=[ 7111], 99.90th=[ 9372], 99.95th=[ 9372],
     | 99.99th=[ 9372]
   bw (  KiB/s): min= 1376, max= 1576, per=25.42%, avg=1476.00, stdev=141.42, samples=2
   iops        : min=  344, max=  394, avg=369.00, stdev=35.36, samples=2
  lat (usec)   : 4=0.26%, 10=8.85%, 20=6.64%, 50=0.78%, 100=1.17%
  lat (usec)   : 250=22.66%, 500=1.82%, 750=1.56%, 1000=7.55%
  lat (msec)   : 2=19.01%, 4=24.87%, 10=4.43%, 20=0.13%, 50=0.26%
  cpu          : usr=0.74%, sys=16.77%, ctx=884, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19132: Wed Jan 15 20:02:37 2025
   read: IOPS=373, BW=1495KiB/s (1531kB/s)(1564KiB/1046msec)
    clat (usec): min=3, max=21884, avg=1602.70, stdev=2155.23
     lat (usec): min=3, max=21885, avg=1603.25, stdev=2155.27
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[  848], 50.00th=[ 1336], 60.00th=[ 1795],
     | 70.00th=[ 2245], 80.00th=[ 2638], 90.00th=[ 3228], 95.00th=[ 4424],
     | 99.00th=[ 9503], 99.50th=[19268], 99.90th=[21890], 99.95th=[21890],
     | 99.99th=[21890]
   bw (  KiB/s): min= 1152, max= 1744, per=26.24%, avg=1448.00, stdev=418.61, samples=2
   iops        : min=  288, max=  436, avg=362.00, stdev=104.65, samples=2
  write: IOPS=360, BW=1442KiB/s (1476kB/s)(1508KiB/1046msec)
    clat (usec): min=93, max=9039, avg=1086.54, stdev=1269.33
     lat (usec): min=93, max=9039, avg=1087.23, stdev=1269.35
    clat percentiles (usec):
     |  1.00th=[   99],  5.00th=[  109], 10.00th=[  119], 20.00th=[  147],
     | 30.00th=[  178], 40.00th=[  225], 50.00th=[  725], 60.00th=[  947],
     | 70.00th=[ 1254], 80.00th=[ 1991], 90.00th=[ 2606], 95.00th=[ 3425],
     | 99.00th=[ 5866], 99.50th=[ 7570], 99.90th=[ 8979], 99.95th=[ 8979],
     | 99.99th=[ 8979]
   bw (  KiB/s): min= 1088, max= 1656, per=23.63%, avg=1372.00, stdev=401.64, samples=2
   iops        : min=  272, max=  414, avg=343.00, stdev=100.41, samples=2
  lat (usec)   : 4=0.39%, 10=10.29%, 20=8.98%, 50=0.39%, 100=0.78%
  lat (usec)   : 250=19.79%, 500=2.60%, 750=1.69%, 1000=6.64%
  lat (msec)   : 2=20.31%, 4=23.57%, 10=4.17%, 20=0.26%, 50=0.13%
  cpu          : usr=0.10%, sys=15.79%, ctx=881, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19134: Wed Jan 15 20:02:37 2025
   read: IOPS=372, BW=1492KiB/s (1527kB/s)(1508KiB/1011msec)
    clat (usec): min=4, max=22252, avg=1647.04, stdev=2309.43
     lat (usec): min=4, max=22252, avg=1647.72, stdev=2309.39
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   11],
     | 30.00th=[   13], 40.00th=[  153], 50.00th=[ 1418], 60.00th=[ 1876],
     | 70.00th=[ 2376], 80.00th=[ 2802], 90.00th=[ 3359], 95.00th=[ 4359],
     | 99.00th=[15401], 99.50th=[20317], 99.90th=[22152], 99.95th=[22152],
     | 99.99th=[22152]
   bw (  KiB/s): min= 1304, max= 1624, per=26.53%, avg=1464.00, stdev=226.27, samples=2
   iops        : min=  326, max=  406, avg=366.00, stdev=56.57, samples=2
  write: IOPS=386, BW=1547KiB/s (1584kB/s)(1564KiB/1011msec)
    clat (usec): min=86, max=10354, avg=960.53, stdev=1218.52
     lat (usec): min=86, max=10355, avg=961.22, stdev=1218.55
    clat percentiles (usec):
     |  1.00th=[   89],  5.00th=[  102], 10.00th=[  110], 20.00th=[  127],
     | 30.00th=[  159], 40.00th=[  186], 50.00th=[  326], 60.00th=[  873],
     | 70.00th=[ 1074], 80.00th=[ 1680], 90.00th=[ 2671], 95.00th=[ 3523],
     | 99.00th=[ 5014], 99.50th=[ 6521], 99.90th=[10421], 99.95th=[10421],
     | 99.99th=[10421]
   bw (  KiB/s): min= 1280, max= 1768, per=26.25%, avg=1524.00, stdev=345.07, samples=2
   iops        : min=  320, max=  442, avg=381.00, stdev=86.27, samples=2
  lat (usec)   : 10=9.51%, 20=9.24%, 50=0.26%, 100=2.47%, 250=22.53%
  lat (usec)   : 500=2.21%, 750=1.82%, 1000=7.16%
  lat (msec)   : 2=17.71%, 4=22.92%, 10=3.52%, 20=0.39%, 50=0.26%
  cpu          : usr=0.59%, sys=15.45%, ctx=824, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=5519KiB/s (5651kB/s), 1324KiB/s-1515KiB/s (1355kB/s-1551kB/s), io=5988KiB (6132kB), run=977-1085msec
  WRITE: bw=5806KiB/s (5946kB/s), 1442KiB/s-1629KiB/s (1476kB/s-1669kB/s), io=6300KiB (6451kB), run=977-1085msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=19135: Wed Jan 15 20:02:38 2025
   read: IOPS=222, BW=892KiB/s (913kB/s)(1480KiB/1660msec)
    clat (usec): min=563, max=24695, avg=1497.72, stdev=1808.77
     lat (usec): min=563, max=24697, avg=1498.36, stdev=1808.83
    clat percentiles (usec):
     |  1.00th=[  603],  5.00th=[  685], 10.00th=[  734], 20.00th=[  775],
     | 30.00th=[  824], 40.00th=[  881], 50.00th=[  930], 60.00th=[ 1029],
     | 70.00th=[ 1188], 80.00th=[ 1663], 90.00th=[ 3261], 95.00th=[ 4228],
     | 99.00th=[ 7701], 99.50th=[14222], 99.90th=[24773], 99.95th=[24773],
     | 99.99th=[24773]
   bw (  KiB/s): min=  624, max= 1168, per=23.71%, avg=840.00, stdev=288.78, samples=3
   iops        : min=  156, max=  292, avg=210.00, stdev=72.19, samples=3
  write: IOPS=239, BW=959KiB/s (982kB/s)(1592KiB/1660msec)
    clat (usec): min=1222, max=18465, avg=2754.87, stdev=1537.54
     lat (usec): min=1223, max=18466, avg=2755.84, stdev=1537.35
    clat percentiles (usec):
     |  1.00th=[ 1270],  5.00th=[ 1483], 10.00th=[ 1713], 20.00th=[ 1876],
     | 30.00th=[ 1975], 40.00th=[ 2057], 50.00th=[ 2212], 60.00th=[ 2409],
     | 70.00th=[ 2769], 80.00th=[ 3589], 90.00th=[ 4686], 95.00th=[ 5669],
     | 99.00th=[ 8029], 99.50th=[10945], 99.90th=[18482], 99.95th=[18482],
     | 99.99th=[18482]
   bw (  KiB/s): min=  648, max= 1352, per=24.47%, avg=912.00, stdev=383.58, samples=3
   iops        : min=  162, max=  338, avg=228.00, stdev=95.90, samples=3
  lat (usec)   : 750=6.90%, 1000=20.57%
  lat (msec)   : 2=30.34%, 4=31.12%, 10=10.55%, 20=0.39%, 50=0.13%
  cpu          : usr=0.48%, sys=6.15%, ctx=795, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19136: Wed Jan 15 20:02:38 2025
   read: IOPS=212, BW=850KiB/s (870kB/s)(1436KiB/1690msec)
    clat (usec): min=538, max=23900, avg=1567.00, stdev=1734.77
     lat (usec): min=538, max=23902, avg=1567.62, stdev=1734.81
    clat percentiles (usec):
     |  1.00th=[  644],  5.00th=[  701], 10.00th=[  742], 20.00th=[  799],
     | 30.00th=[  840], 40.00th=[  898], 50.00th=[  955], 60.00th=[ 1057],
     | 70.00th=[ 1237], 80.00th=[ 1909], 90.00th=[ 3523], 95.00th=[ 4686],
     | 99.00th=[ 6652], 99.50th=[ 8029], 99.90th=[23987], 99.95th=[23987],
     | 99.99th=[23987]
   bw (  KiB/s): min=  560, max= 1240, per=22.50%, avg=797.33, stdev=383.69, samples=3
   iops        : min=  140, max=  310, avg=199.33, stdev=95.92, samples=3
  write: IOPS=242, BW=968KiB/s (991kB/s)(1636KiB/1690msec)
    clat (usec): min=1167, max=13154, avg=2732.36, stdev=1487.01
     lat (usec): min=1167, max=13155, avg=2733.09, stdev=1487.03
    clat percentiles (usec):
     |  1.00th=[ 1237],  5.00th=[ 1401], 10.00th=[ 1532], 20.00th=[ 1811],
     | 30.00th=[ 1926], 40.00th=[ 2073], 50.00th=[ 2212], 60.00th=[ 2409],
     | 70.00th=[ 2835], 80.00th=[ 3458], 90.00th=[ 4490], 95.00th=[ 5800],
     | 99.00th=[ 8225], 99.50th=[ 9372], 99.90th=[13173], 99.95th=[13173],
     | 99.99th=[13173]
   bw (  KiB/s): min=  656, max= 1256, per=24.11%, avg=898.67, stdev=316.01, samples=3
   iops        : min=  164, max=  314, avg=224.67, stdev=79.00, samples=3
  lat (usec)   : 750=5.34%, 1000=20.18%
  lat (msec)   : 2=31.64%, 4=31.64%, 10=10.94%, 20=0.13%, 50=0.13%
  cpu          : usr=0.36%, sys=6.16%, ctx=805, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19137: Wed Jan 15 20:02:38 2025
   read: IOPS=239, BW=958KiB/s (981kB/s)(1564KiB/1633msec)
    clat (usec): min=651, max=18328, avg=1446.66, stdev=1431.90
     lat (usec): min=652, max=18328, avg=1447.29, stdev=1431.90
    clat percentiles (usec):
     |  1.00th=[  660],  5.00th=[  701], 10.00th=[  742], 20.00th=[  807],
     | 30.00th=[  848], 40.00th=[  889], 50.00th=[  963], 60.00th=[ 1057],
     | 70.00th=[ 1237], 80.00th=[ 1549], 90.00th=[ 2704], 95.00th=[ 4359],
     | 99.00th=[ 7308], 99.50th=[ 8160], 99.90th=[18220], 99.95th=[18220],
     | 99.99th=[18220]
   bw (  KiB/s): min=  584, max= 1328, per=25.52%, avg=904.00, stdev=382.75, samples=3
   iops        : min=  146, max=  332, avg=226.00, stdev=95.69, samples=3
  write: IOPS=230, BW=923KiB/s (946kB/s)(1508KiB/1633msec)
    clat (usec): min=1152, max=32776, avg=2803.32, stdev=2043.34
     lat (usec): min=1152, max=32778, avg=2804.15, stdev=2043.41
    clat percentiles (usec):
     |  1.00th=[ 1270],  5.00th=[ 1467], 10.00th=[ 1680], 20.00th=[ 1860],
     | 30.00th=[ 1975], 40.00th=[ 2114], 50.00th=[ 2278], 60.00th=[ 2442],
     | 70.00th=[ 2802], 80.00th=[ 3458], 90.00th=[ 4686], 95.00th=[ 5538],
     | 99.00th=[ 8586], 99.50th=[11731], 99.90th=[32900], 99.95th=[32900],
     | 99.99th=[32900]
   bw (  KiB/s): min=  688, max= 1264, per=23.75%, avg=885.33, stdev=328.03, samples=3
   iops        : min=  172, max=  316, avg=221.33, stdev=82.01, samples=3
  lat (usec)   : 750=5.21%, 1000=22.79%
  lat (msec)   : 2=31.12%, 4=31.38%, 10=9.11%, 20=0.26%, 50=0.13%
  cpu          : usr=0.49%, sys=6.19%, ctx=806, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19138: Wed Jan 15 20:02:38 2025
   read: IOPS=227, BW=911KiB/s (932kB/s)(1508KiB/1656msec)
    clat (usec): min=617, max=15328, avg=1506.44, stdev=1341.11
     lat (usec): min=617, max=15329, avg=1507.63, stdev=1340.87
    clat percentiles (usec):
     |  1.00th=[  635],  5.00th=[  709], 10.00th=[  742], 20.00th=[  799],
     | 30.00th=[  873], 40.00th=[  922], 50.00th=[  988], 60.00th=[ 1139],
     | 70.00th=[ 1352], 80.00th=[ 1860], 90.00th=[ 3228], 95.00th=[ 4293],
     | 99.00th=[ 6194], 99.50th=[ 6587], 99.90th=[15270], 99.95th=[15270],
     | 99.99th=[15270]
   bw (  KiB/s): min=  568, max= 1224, per=24.31%, avg=861.33, stdev=333.45, samples=3
   iops        : min=  142, max=  306, avg=215.33, stdev=83.36, samples=3
  write: IOPS=236, BW=944KiB/s (967kB/s)(1564KiB/1656msec)
    clat (usec): min=1212, max=37020, avg=2762.88, stdev=2166.74
     lat (usec): min=1212, max=37023, avg=2763.64, stdev=2166.84
    clat percentiles (usec):
     |  1.00th=[ 1254],  5.00th=[ 1483], 10.00th=[ 1696], 20.00th=[ 1860],
     | 30.00th=[ 1975], 40.00th=[ 2057], 50.00th=[ 2212], 60.00th=[ 2376],
     | 70.00th=[ 2638], 80.00th=[ 3392], 90.00th=[ 4555], 95.00th=[ 5342],
     | 99.00th=[ 8848], 99.50th=[ 8979], 99.90th=[36963], 99.95th=[36963],
     | 99.99th=[36963]
   bw (  KiB/s): min=  608, max= 1336, per=24.04%, avg=896.00, stdev=387.07, samples=3
   iops        : min=  152, max=  334, avg=224.00, stdev=96.77, samples=3
  lat (usec)   : 750=5.47%, 1000=19.53%
  lat (msec)   : 2=32.29%, 4=32.03%, 10=10.42%, 20=0.13%, 50=0.13%
  cpu          : usr=0.36%, sys=6.28%, ctx=814, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3543KiB/s (3628kB/s), 850KiB/s-958KiB/s (870kB/s-981kB/s), io=5988KiB (6132kB), run=1633-1690msec
  WRITE: bw=3728KiB/s (3817kB/s), 923KiB/s-968KiB/s (946kB/s-991kB/s), io=6300KiB (6451kB), run=1633-1690msec