== sanity test 398b: DIO and buffer IO race ============== 13:42:24 (1721410944)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.114296 s, 110 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=17910: Fri Jul 19 13:42:27 2024
   read: IOPS=376, BW=1504KiB/s (1540kB/s)(1480KiB/984msec)
    clat (usec): min=3, max=24565, avg=1525.82, stdev=2154.57
     lat (usec): min=3, max=24566, avg=1526.44, stdev=2154.81
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    6], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[   21], 50.00th=[ 1037], 60.00th=[ 1762],
     | 70.00th=[ 2212], 80.00th=[ 2573], 90.00th=[ 3163], 95.00th=[ 4424],
     | 99.00th=[ 9896], 99.50th=[11600], 99.90th=[24511], 99.95th=[24511],
     | 99.99th=[24511]
   bw (  KiB/s): min= 1160, max= 1160, per=20.87%, avg=1160.00, stdev= 0.00, samples=1
   iops        : min=  290, max=  290, avg=290.00, stdev= 0.00, samples=1
  write: IOPS=404, BW=1618KiB/s (1657kB/s)(1592KiB/984msec)
    clat (usec): min=89, max=25670, avg=1026.43, stdev=1711.25
     lat (usec): min=90, max=25671, avg=1027.40, stdev=1711.58
    clat percentiles (usec):
     |  1.00th=[   95],  5.00th=[  103], 10.00th=[  110], 20.00th=[  123],
     | 30.00th=[  157], 40.00th=[  204], 50.00th=[  461], 60.00th=[  922],
     | 70.00th=[ 1156], 80.00th=[ 1696], 90.00th=[ 2343], 95.00th=[ 3130],
     | 99.00th=[ 6587], 99.50th=[ 8291], 99.90th=[25560], 99.95th=[25560],
     | 99.99th=[25560]
   bw (  KiB/s): min= 1104, max= 1104, per=18.88%, avg=1104.00, stdev= 0.00, samples=1
   iops        : min=  276, max=  276, avg=276.00, stdev= 0.00, samples=1
  lat (usec)   : 4=0.39%, 10=11.07%, 20=7.68%, 50=0.26%, 100=2.08%
  lat (usec)   : 250=22.14%, 500=3.12%, 750=3.12%, 1000=7.03%
  lat (msec)   : 2=17.19%, 4=21.48%, 10=3.91%, 20=0.26%, 50=0.26%
  cpu          : usr=0.51%, sys=16.99%, ctx=951, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17911: Fri Jul 19 13:42:27 2024
   read: IOPS=346, BW=1386KiB/s (1419kB/s)(1436KiB/1036msec)
    clat (usec): min=3, max=39320, avg=1588.00, stdev=2728.26
     lat (usec): min=3, max=39321, avg=1588.86, stdev=2728.35
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   11], 40.00th=[  562], 50.00th=[ 1270], 60.00th=[ 1614],
     | 70.00th=[ 2114], 80.00th=[ 2606], 90.00th=[ 3228], 95.00th=[ 4228],
     | 99.00th=[10683], 99.50th=[14746], 99.90th=[39060], 99.95th=[39060],
     | 99.99th=[39060]
   bw (  KiB/s): min=  966, max= 1800, per=24.88%, avg=1383.00, stdev=589.73, samples=2
   iops        : min=  241, max=  450, avg=345.50, stdev=147.79, samples=2
  write: IOPS=394, BW=1579KiB/s (1617kB/s)(1636KiB/1036msec)
    clat (usec): min=90, max=27300, avg=1106.79, stdev=1800.45
     lat (usec): min=90, max=27301, avg=1107.46, stdev=1800.46
    clat percentiles (usec):
     |  1.00th=[   96],  5.00th=[  104], 10.00th=[  113], 20.00th=[  129],
     | 30.00th=[  161], 40.00th=[  217], 50.00th=[  635], 60.00th=[  857],
     | 70.00th=[ 1237], 80.00th=[ 1893], 90.00th=[ 2769], 95.00th=[ 3621],
     | 99.00th=[ 5735], 99.50th=[ 6325], 99.90th=[27395], 99.95th=[27395],
     | 99.99th=[27395]
   bw (  KiB/s): min= 1189, max= 1936, per=26.71%, avg=1562.50, stdev=528.21, samples=2
   iops        : min=  297, max=  484, avg=390.50, stdev=132.23, samples=2
  lat (usec)   : 4=0.78%, 10=11.59%, 20=4.95%, 50=0.65%, 100=1.69%
  lat (usec)   : 250=21.61%, 500=2.47%, 750=3.52%, 1000=8.85%
  lat (msec)   : 2=19.66%, 4=19.66%, 10=3.91%, 20=0.39%, 50=0.26%
  cpu          : usr=0.39%, sys=18.07%, ctx=1004, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17912: Fri Jul 19 13:42:27 2024
   read: IOPS=363, BW=1452KiB/s (1487kB/s)(1564KiB/1077msec)
    clat (usec): min=3, max=29724, avg=1663.57, stdev=2422.57
     lat (usec): min=3, max=29725, avg=1664.68, stdev=2422.37
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    8], 20.00th=[   11],
     | 30.00th=[   14], 40.00th=[  930], 50.00th=[ 1532], 60.00th=[ 1876],
     | 70.00th=[ 2212], 80.00th=[ 2638], 90.00th=[ 3228], 95.00th=[ 4621],
     | 99.00th=[ 8717], 99.50th=[25035], 99.90th=[29754], 99.95th=[29754],
     | 99.99th=[29754]
   bw (  KiB/s): min= 1048, max= 1736, per=25.04%, avg=1392.00, stdev=486.49, samples=2
   iops        : min=  262, max=  434, avg=348.00, stdev=121.62, samples=2
  write: IOPS=350, BW=1400KiB/s (1434kB/s)(1508KiB/1077msec)
    clat (usec): min=86, max=14668, avg=1096.44, stdev=1390.53
     lat (usec): min=86, max=14668, avg=1097.49, stdev=1390.47
    clat percentiles (usec):
     |  1.00th=[   96],  5.00th=[  109], 10.00th=[  119], 20.00th=[  153],
     | 30.00th=[  194], 40.00th=[  314], 50.00th=[  783], 60.00th=[  996],
     | 70.00th=[ 1254], 80.00th=[ 1893], 90.00th=[ 2507], 95.00th=[ 3425],
     | 99.00th=[ 6390], 99.50th=[10028], 99.90th=[14615], 99.95th=[14615],
     | 99.99th=[14615]
   bw (  KiB/s): min= 1040, max= 1656, per=23.05%, avg=1348.00, stdev=435.58, samples=2
   iops        : min=  260, max=  414, avg=337.00, stdev=108.89, samples=2
  lat (usec)   : 4=0.52%, 10=9.24%, 20=8.07%, 50=0.52%, 100=1.43%
  lat (usec)   : 250=16.80%, 500=3.91%, 750=2.47%, 1000=7.42%
  lat (msec)   : 2=22.53%, 4=22.53%, 10=4.04%, 20=0.26%, 50=0.26%
  cpu          : usr=0.37%, sys=18.12%, ctx=1041, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17913: Fri Jul 19 13:42:27 2024
   read: IOPS=374, BW=1499KiB/s (1535kB/s)(1508KiB/1006msec)
    clat (usec): min=3, max=23642, avg=1579.59, stdev=2131.76
     lat (usec): min=3, max=23643, avg=1581.31, stdev=2131.15
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   14], 40.00th=[  840], 50.00th=[ 1434], 60.00th=[ 1860],
     | 70.00th=[ 2147], 80.00th=[ 2540], 90.00th=[ 3097], 95.00th=[ 4178],
     | 99.00th=[11207], 99.50th=[13698], 99.90th=[23725], 99.95th=[23725],
     | 99.99th=[23725]
   bw (  KiB/s): min= 1064, max= 1912, per=26.77%, avg=1488.00, stdev=599.63, samples=2
   iops        : min=  266, max=  478, avg=372.00, stdev=149.91, samples=2
  write: IOPS=388, BW=1555KiB/s (1592kB/s)(1564KiB/1006msec)
    clat (usec): min=88, max=8787, avg=1016.61, stdev=1157.87
     lat (usec): min=88, max=8789, avg=1017.89, stdev=1157.52
    clat percentiles (usec):
     |  1.00th=[   92],  5.00th=[  102], 10.00th=[  111], 20.00th=[  129],
     | 30.00th=[  174], 40.00th=[  262], 50.00th=[  676], 60.00th=[  914],
     | 70.00th=[ 1172], 80.00th=[ 1827], 90.00th=[ 2638], 95.00th=[ 3261],
     | 99.00th=[ 5014], 99.50th=[ 5342], 99.90th=[ 8848], 99.95th=[ 8848],
     | 99.99th=[ 8848]
   bw (  KiB/s): min= 1056, max= 2056, per=26.60%, avg=1556.00, stdev=707.11, samples=2
   iops        : min=  264, max=  514, avg=389.00, stdev=176.78, samples=2
  lat (usec)   : 4=0.13%, 10=11.46%, 20=6.12%, 50=0.52%, 100=2.47%
  lat (usec)   : 250=17.97%, 500=4.30%, 750=2.86%, 1000=8.33%
  lat (msec)   : 2=19.66%, 4=21.74%, 10=3.91%, 20=0.39%, 50=0.13%
  cpu          : usr=0.50%, sys=18.21%, ctx=1029, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=5560KiB/s (5693kB/s), 1386KiB/s-1504KiB/s (1419kB/s-1540kB/s), io=5988KiB (6132kB), run=984-1077msec
  WRITE: bw=5850KiB/s (5990kB/s), 1400KiB/s-1618KiB/s (1434kB/s-1657kB/s), io=6300KiB (6451kB), run=984-1077msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=17914: Fri Jul 19 13:42:28 2024
   read: IOPS=221, BW=886KiB/s (907kB/s)(1480KiB/1671msec)
    clat (usec): min=570, max=22074, avg=1519.37, stdev=1806.54
     lat (usec): min=570, max=22076, avg=1520.03, stdev=1806.61
    clat percentiles (usec):
     |  1.00th=[  603],  5.00th=[  685], 10.00th=[  750], 20.00th=[  824],
     | 30.00th=[  881], 40.00th=[  938], 50.00th=[  996], 60.00th=[ 1123],
     | 70.00th=[ 1352], 80.00th=[ 1778], 90.00th=[ 2802], 95.00th=[ 4113],
     | 99.00th=[ 5276], 99.50th=[21890], 99.90th=[22152], 99.95th=[22152],
     | 99.99th=[22152]
   bw (  KiB/s): min=  624, max= 1096, per=24.13%, avg=840.00, stdev=238.53, samples=3
   iops        : min=  156, max=  274, avg=210.00, stdev=59.63, samples=3
  write: IOPS=238, BW=953KiB/s (976kB/s)(1592KiB/1671msec)
    clat (usec): min=1284, max=12003, avg=2761.71, stdev=1378.46
     lat (usec): min=1285, max=12004, avg=2762.53, stdev=1378.44
    clat percentiles (usec):
     |  1.00th=[ 1336],  5.00th=[ 1500], 10.00th=[ 1696], 20.00th=[ 1926],
     | 30.00th=[ 2040], 40.00th=[ 2147], 50.00th=[ 2343], 60.00th=[ 2573],
     | 70.00th=[ 2868], 80.00th=[ 3359], 90.00th=[ 4293], 95.00th=[ 5473],
     | 99.00th=[ 9372], 99.50th=[ 9765], 99.90th=[11994], 99.95th=[11994],
     | 99.99th=[11994]
   bw (  KiB/s): min=  640, max= 1264, per=24.98%, avg=914.67, stdev=318.63, samples=3
   iops        : min=  160, max=  316, avg=228.67, stdev=79.66, samples=3
  lat (usec)   : 750=4.82%, 1000=19.40%
  lat (msec)   : 2=29.95%, 4=36.98%, 10=8.46%, 20=0.13%, 50=0.26%
  cpu          : usr=0.24%, sys=6.29%, ctx=805, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17915: Fri Jul 19 13:42:28 2024
   read: IOPS=210, BW=842KiB/s (862kB/s)(1436KiB/1705msec)
    clat (usec): min=547, max=21206, avg=1453.77, stdev=1486.20
     lat (usec): min=547, max=21208, avg=1455.20, stdev=1485.96
    clat percentiles (usec):
     |  1.00th=[  627],  5.00th=[  701], 10.00th=[  742], 20.00th=[  824],
     | 30.00th=[  873], 40.00th=[  922], 50.00th=[  979], 60.00th=[ 1057],
     | 70.00th=[ 1287], 80.00th=[ 1680], 90.00th=[ 2835], 95.00th=[ 3916],
     | 99.00th=[ 5080], 99.50th=[ 8979], 99.90th=[21103], 99.95th=[21103],
     | 99.99th=[21103]
   bw (  KiB/s): min=  616, max= 1160, per=23.06%, avg=802.67, stdev=309.56, samples=3
   iops        : min=  154, max=  290, avg=200.67, stdev=77.39, samples=3
  write: IOPS=239, BW=960KiB/s (983kB/s)(1636KiB/1705msec)
    clat (usec): min=1053, max=18541, avg=2868.72, stdev=1615.97
     lat (usec): min=1054, max=18543, avg=2869.55, stdev=1616.00
    clat percentiles (usec):
     |  1.00th=[ 1188],  5.00th=[ 1467], 10.00th=[ 1663], 20.00th=[ 1893],
     | 30.00th=[ 2040], 40.00th=[ 2180], 50.00th=[ 2343], 60.00th=[ 2573],
     | 70.00th=[ 3032], 80.00th=[ 3654], 90.00th=[ 4817], 95.00th=[ 5604],
     | 99.00th=[ 7898], 99.50th=[10028], 99.90th=[18482], 99.95th=[18482],
     | 99.99th=[18482]
   bw (  KiB/s): min=  672, max= 1168, per=24.69%, avg=904.00, stdev=249.54, samples=3
   iops        : min=  168, max=  292, avg=226.00, stdev=62.39, samples=3
  lat (usec)   : 750=4.95%, 1000=20.05%
  lat (msec)   : 2=28.91%, 4=34.64%, 10=10.94%, 20=0.39%, 50=0.13%
  cpu          : usr=0.23%, sys=6.92%, ctx=826, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17916: Fri Jul 19 13:42:28 2024
   read: IOPS=231, BW=924KiB/s (947kB/s)(1564KiB/1692msec)
    clat (usec): min=599, max=8826, avg=1431.05, stdev=1061.88
     lat (usec): min=599, max=8826, avg=1431.89, stdev=1062.03
    clat percentiles (usec):
     |  1.00th=[  627],  5.00th=[  701], 10.00th=[  758], 20.00th=[  824],
     | 30.00th=[  881], 40.00th=[  947], 50.00th=[ 1004], 60.00th=[ 1106],
     | 70.00th=[ 1336], 80.00th=[ 1778], 90.00th=[ 2900], 95.00th=[ 3884],
     | 99.00th=[ 5604], 99.50th=[ 6063], 99.90th=[ 8848], 99.95th=[ 8848],
     | 99.99th=[ 8848]
   bw (  KiB/s): min=  568, max= 1192, per=24.51%, avg=853.33, stdev=315.40, samples=3
   iops        : min=  142, max=  298, avg=213.33, stdev=78.85, samples=3
  write: IOPS=222, BW=891KiB/s (913kB/s)(1508KiB/1692msec)
    clat (usec): min=1167, max=27408, avg=2977.41, stdev=2246.21
     lat (usec): min=1168, max=27409, avg=2978.26, stdev=2246.33
    clat percentiles (usec):
     |  1.00th=[ 1221],  5.00th=[ 1450], 10.00th=[ 1647], 20.00th=[ 1926],
     | 30.00th=[ 2040], 40.00th=[ 2180], 50.00th=[ 2343], 60.00th=[ 2573],
     | 70.00th=[ 2933], 80.00th=[ 3621], 90.00th=[ 5014], 95.00th=[ 6128],
     | 99.00th=[11207], 99.50th=[24511], 99.90th=[27395], 99.95th=[27395],
     | 99.99th=[27395]
   bw (  KiB/s): min=  664, max= 1184, per=23.23%, avg=850.67, stdev=289.37, samples=3
   iops        : min=  166, max=  296, avg=212.67, stdev=72.34, samples=3
  lat (usec)   : 750=4.82%, 1000=19.92%
  lat (msec)   : 2=30.99%, 4=34.24%, 10=9.51%, 20=0.26%, 50=0.26%
  cpu          : usr=0.41%, sys=6.45%, ctx=811, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17917: Fri Jul 19 13:42:28 2024
   read: IOPS=219, BW=877KiB/s (898kB/s)(1508KiB/1720msec)
    clat (usec): min=591, max=7805, avg=1509.10, stdev=1152.79
     lat (usec): min=591, max=7806, avg=1509.76, stdev=1152.79
    clat percentiles (usec):
     |  1.00th=[  627],  5.00th=[  693], 10.00th=[  742], 20.00th=[  832],
     | 30.00th=[  906], 40.00th=[  955], 50.00th=[ 1029], 60.00th=[ 1123],
     | 70.00th=[ 1352], 80.00th=[ 2008], 90.00th=[ 3097], 95.00th=[ 4047],
     | 99.00th=[ 6325], 99.50th=[ 7308], 99.90th=[ 7832], 99.95th=[ 7832],
     | 99.99th=[ 7832]
   bw (  KiB/s): min=  584, max= 1040, per=23.29%, avg=810.67, stdev=228.01, samples=3
   iops        : min=  146, max=  260, avg=202.67, stdev=57.00, samples=3
  write: IOPS=227, BW=909KiB/s (931kB/s)(1564KiB/1720msec)
    clat (usec): min=1114, max=38994, avg=2915.86, stdev=2368.14
     lat (usec): min=1115, max=38996, avg=2916.86, stdev=2368.07
    clat percentiles (usec):
     |  1.00th=[ 1319],  5.00th=[ 1500], 10.00th=[ 1631], 20.00th=[ 1926],
     | 30.00th=[ 2040], 40.00th=[ 2212], 50.00th=[ 2311], 60.00th=[ 2540],
     | 70.00th=[ 2900], 80.00th=[ 3556], 90.00th=[ 4686], 95.00th=[ 5735],
     | 99.00th=[12649], 99.50th=[13698], 99.90th=[39060], 99.95th=[39060],
     | 99.99th=[39060]
   bw (  KiB/s): min=  624, max= 1208, per=23.52%, avg=861.33, stdev=306.97, samples=3
   iops        : min=  156, max=  302, avg=215.33, stdev=76.74, samples=3
  lat (usec)   : 750=5.21%, 1000=17.84%
  lat (msec)   : 2=29.30%, 4=37.11%, 10=10.03%, 20=0.39%, 50=0.13%
  cpu          : usr=0.12%, sys=6.40%, ctx=836, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3481KiB/s (3565kB/s), 842KiB/s-924KiB/s (862kB/s-947kB/s), io=5988KiB (6132kB), run=1671-1720msec
  WRITE: bw=3663KiB/s (3751kB/s), 891KiB/s-960KiB/s (913kB/s-983kB/s), io=6300KiB (6451kB), run=1671-1720msec