== sanity test 398b: DIO and buffer IO race ============== 11:00:50 (1721228450)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.106525 s, 118 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=19769: Wed Jul 17 11:00:52 2024
   read: IOPS=361, BW=1447KiB/s (1481kB/s)(1480KiB/1023msec)
    clat (usec): min=3, max=36724, avg=1518.53, stdev=3022.40
     lat (usec): min=3, max=36725, avg=1520.74, stdev=3022.19
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   10], 40.00th=[   22], 50.00th=[ 1057], 60.00th=[ 1549],
     | 70.00th=[ 1876], 80.00th=[ 2311], 90.00th=[ 2737], 95.00th=[ 3752],
     | 99.00th=[11863], 99.50th=[32375], 99.90th=[36963], 99.95th=[36963],
     | 99.99th=[36963]
   bw (  KiB/s): min=  760, max= 2080, per=25.99%, avg=1420.00, stdev=933.38, samples=2
   iops        : min=  190, max=  520, avg=355.00, stdev=233.35, samples=2
  write: IOPS=389, BW=1556KiB/s (1594kB/s)(1592KiB/1023msec)
    clat (usec): min=77, max=51877, avg=1134.01, stdev=2916.49
     lat (usec): min=78, max=51878, avg=1134.60, stdev=2916.52
    clat percentiles (usec):
     |  1.00th=[   87],  5.00th=[   93], 10.00th=[   98], 20.00th=[  112],
     | 30.00th=[  123], 40.00th=[  159], 50.00th=[  553], 60.00th=[  840],
     | 70.00th=[ 1106], 80.00th=[ 1614], 90.00th=[ 2474], 95.00th=[ 3687],
     | 99.00th=[ 7504], 99.50th=[10552], 99.90th=[51643], 99.95th=[51643],
     | 99.99th=[51643]
   bw (  KiB/s): min=  800, max= 2296, per=26.93%, avg=1548.00, stdev=1057.83, samples=2
   iops        : min=  200, max=  574, avg=387.00, stdev=264.46, samples=2
  lat (usec)   : 4=1.56%, 10=14.06%, 20=3.52%, 50=0.52%, 100=6.25%
  lat (usec)   : 250=18.23%, 500=2.08%, 750=4.43%, 1000=7.81%
  lat (msec)   : 2=20.96%, 4=16.15%, 10=3.39%, 20=0.65%, 50=0.26%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.10%, sys=15.75%, ctx=794, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19770: Wed Jul 17 11:00:52 2024
   read: IOPS=327, BW=1310KiB/s (1342kB/s)(1436KiB/1096msec)
    clat (usec): min=3, max=26611, avg=1727.23, stdev=2774.48
     lat (usec): min=3, max=26612, avg=1728.65, stdev=2774.05
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   10], 40.00th=[  783], 50.00th=[ 1287], 60.00th=[ 1680],
     | 70.00th=[ 2278], 80.00th=[ 2704], 90.00th=[ 3490], 95.00th=[ 4555],
     | 99.00th=[17171], 99.50th=[22152], 99.90th=[26608], 99.95th=[26608],
     | 99.99th=[26608]
   bw (  KiB/s): min=  712, max= 1824, per=23.21%, avg=1268.00, stdev=786.30, samples=2
   iops        : min=  178, max=  456, avg=317.00, stdev=196.58, samples=2
  write: IOPS=373, BW=1493KiB/s (1529kB/s)(1636KiB/1096msec)
    clat (usec): min=76, max=33062, avg=1116.98, stdev=2086.05
     lat (usec): min=76, max=33063, avg=1117.59, stdev=2086.08
    clat percentiles (usec):
     |  1.00th=[   90],  5.00th=[   97], 10.00th=[  103], 20.00th=[  115],
     | 30.00th=[  135], 40.00th=[  233], 50.00th=[  660], 60.00th=[  857],
     | 70.00th=[ 1172], 80.00th=[ 1811], 90.00th=[ 2540], 95.00th=[ 3621],
     | 99.00th=[ 6980], 99.50th=[ 9503], 99.90th=[33162], 99.95th=[33162],
     | 99.99th=[33162]
   bw (  KiB/s): min=  808, max= 2040, per=24.77%, avg=1424.00, stdev=871.16, samples=2
   iops        : min=  202, max=  510, avg=356.00, stdev=217.79, samples=2
  lat (usec)   : 4=1.04%, 10=13.41%, 20=3.12%, 50=0.13%, 100=3.91%
  lat (usec)   : 250=18.62%, 500=2.21%, 750=5.47%, 1000=8.72%
  lat (msec)   : 2=18.23%, 4=19.66%, 10=4.56%, 20=0.39%, 50=0.52%
  cpu          : usr=0.37%, sys=15.43%, ctx=875, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19771: Wed Jul 17 11:00:52 2024
   read: IOPS=375, BW=1504KiB/s (1540kB/s)(1564KiB/1040msec)
    clat (usec): min=3, max=33645, avg=1663.17, stdev=3317.76
     lat (usec): min=3, max=33646, avg=1664.00, stdev=3317.70
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    6], 20.00th=[    8],
     | 30.00th=[   10], 40.00th=[   22], 50.00th=[ 1139], 60.00th=[ 1631],
     | 70.00th=[ 1942], 80.00th=[ 2376], 90.00th=[ 3195], 95.00th=[ 4621],
     | 99.00th=[22938], 99.50th=[29754], 99.90th=[33817], 99.95th=[33817],
     | 99.99th=[33817]
   bw (  KiB/s): min=  720, max= 2072, per=25.55%, avg=1396.00, stdev=956.01, samples=2
   iops        : min=  180, max=  518, avg=349.00, stdev=239.00, samples=2
  write: IOPS=362, BW=1450KiB/s (1485kB/s)(1508KiB/1040msec)
    clat (usec): min=80, max=12358, avg=994.74, stdev=1449.16
     lat (usec): min=80, max=12359, avg=995.33, stdev=1449.21
    clat percentiles (usec):
     |  1.00th=[   89],  5.00th=[   97], 10.00th=[  102], 20.00th=[  114],
     | 30.00th=[  133], 40.00th=[  196], 50.00th=[  594], 60.00th=[  758],
     | 70.00th=[  938], 80.00th=[ 1598], 90.00th=[ 2474], 95.00th=[ 3621],
     | 99.00th=[ 7242], 99.50th=[ 8717], 99.90th=[12387], 99.95th=[12387],
     | 99.99th=[12387]
   bw (  KiB/s): min=  824, max= 1872, per=23.45%, avg=1348.00, stdev=741.05, samples=2
   iops        : min=  206, max=  468, avg=337.00, stdev=185.26, samples=2
  lat (usec)   : 4=1.95%, 10=15.62%, 20=2.73%, 50=0.78%, 100=4.30%
  lat (usec)   : 250=18.10%, 500=2.34%, 750=5.73%, 1000=7.81%
  lat (msec)   : 2=18.88%, 4=16.67%, 10=3.91%, 20=0.65%, 50=0.52%
  cpu          : usr=0.67%, sys=14.73%, ctx=854, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19772: Wed Jul 17 11:00:52 2024
   read: IOPS=363, BW=1453KiB/s (1488kB/s)(1508KiB/1038msec)
    clat (usec): min=3, max=52177, avg=1793.36, stdev=3814.56
     lat (usec): min=3, max=52178, avg=1794.06, stdev=3814.60
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   10], 40.00th=[  725], 50.00th=[ 1319], 60.00th=[ 1680],
     | 70.00th=[ 2114], 80.00th=[ 2507], 90.00th=[ 3195], 95.00th=[ 4555],
     | 99.00th=[15401], 99.50th=[30802], 99.90th=[52167], 99.95th=[52167],
     | 99.99th=[52167]
   bw (  KiB/s): min=  768, max= 1896, per=24.38%, avg=1332.00, stdev=797.62, samples=2
   iops        : min=  192, max=  474, avg=333.00, stdev=199.40, samples=2
  write: IOPS=376, BW=1507KiB/s (1543kB/s)(1564KiB/1038msec)
    clat (usec): min=75, max=12188, avg=885.45, stdev=1246.63
     lat (usec): min=75, max=12189, avg=886.55, stdev=1246.41
    clat percentiles (usec):
     |  1.00th=[   85],  5.00th=[   92], 10.00th=[  101], 20.00th=[  112],
     | 30.00th=[  122], 40.00th=[  147], 50.00th=[  241], 60.00th=[  734],
     | 70.00th=[  938], 80.00th=[ 1614], 90.00th=[ 2212], 95.00th=[ 3294],
     | 99.00th=[ 5145], 99.50th=[ 5997], 99.90th=[12125], 99.95th=[12125],
     | 99.99th=[12125]
   bw (  KiB/s): min=  808, max= 2016, per=24.57%, avg=1412.00, stdev=854.18, samples=2
   iops        : min=  202, max=  504, avg=353.00, stdev=213.55, samples=2
  lat (usec)   : 4=0.91%, 10=14.97%, 20=2.47%, 50=0.65%, 100=4.82%
  lat (usec)   : 250=20.83%, 500=2.08%, 750=4.43%, 1000=6.25%
  lat (msec)   : 2=19.79%, 4=17.97%, 10=3.91%, 20=0.52%, 50=0.26%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.58%, sys=16.01%, ctx=791, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=5464KiB/s (5595kB/s), 1310KiB/s-1504KiB/s (1342kB/s-1540kB/s), io=5988KiB (6132kB), run=1023-1096msec
  WRITE: bw=5748KiB/s (5886kB/s), 1450KiB/s-1556KiB/s (1485kB/s-1594kB/s), io=6300KiB (6451kB), run=1023-1096msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=19773: Wed Jul 17 11:00:53 2024
   read: IOPS=216, BW=865KiB/s (886kB/s)(1480KiB/1711msec)
    clat (usec): min=499, max=50755, avg=1599.66, stdev=3050.38
     lat (usec): min=499, max=50757, avg=1600.35, stdev=3050.42
    clat percentiles (usec):
     |  1.00th=[  562],  5.00th=[  611], 10.00th=[  660], 20.00th=[  717],
     | 30.00th=[  758], 40.00th=[  816], 50.00th=[  914], 60.00th=[ 1020],
     | 70.00th=[ 1188], 80.00th=[ 1811], 90.00th=[ 3130], 95.00th=[ 4228],
     | 99.00th=[ 9765], 99.50th=[20579], 99.90th=[50594], 99.95th=[50594],
     | 99.99th=[50594]
   bw (  KiB/s): min=  352, max= 1152, per=22.56%, avg=789.33, stdev=405.19, samples=3
   iops        : min=   88, max=  288, avg=197.33, stdev=101.30, samples=3
  write: IOPS=232, BW=930KiB/s (953kB/s)(1592KiB/1711msec)
    clat (usec): min=1115, max=51738, avg=2784.90, stdev=3098.08
     lat (usec): min=1116, max=51739, avg=2785.63, stdev=3098.13
    clat percentiles (usec):
     |  1.00th=[ 1172],  5.00th=[ 1352], 10.00th=[ 1549], 20.00th=[ 1729],
     | 30.00th=[ 1844], 40.00th=[ 1909], 50.00th=[ 2024], 60.00th=[ 2212],
     | 70.00th=[ 2573], 80.00th=[ 3097], 90.00th=[ 4424], 95.00th=[ 6259],
     | 99.00th=[11338], 99.50th=[22938], 99.90th=[51643], 99.95th=[51643],
     | 99.99th=[51643]
   bw (  KiB/s): min=  408, max= 1408, per=23.39%, avg=861.33, stdev=506.49, samples=3
   iops        : min=  102, max=  352, avg=215.33, stdev=126.62, samples=3
  lat (usec)   : 500=0.13%, 750=12.89%, 1000=14.71%
  lat (msec)   : 2=36.72%, 4=25.65%, 10=8.72%, 20=0.65%, 50=0.26%
  lat (msec)   : 100=0.26%
  cpu          : usr=0.12%, sys=6.20%, ctx=822, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19774: Wed Jul 17 11:00:53 2024
   read: IOPS=211, BW=845KiB/s (865kB/s)(1436KiB/1699msec)
    clat (usec): min=535, max=52161, avg=1573.94, stdev=3142.78
     lat (usec): min=535, max=52163, avg=1574.57, stdev=3142.83
    clat percentiles (usec):
     |  1.00th=[  570],  5.00th=[  627], 10.00th=[  668], 20.00th=[  717],
     | 30.00th=[  766], 40.00th=[  832], 50.00th=[  906], 60.00th=[  988],
     | 70.00th=[ 1156], 80.00th=[ 1663], 90.00th=[ 3097], 95.00th=[ 4424],
     | 99.00th=[ 9241], 99.50th=[20579], 99.90th=[52167], 99.95th=[52167],
     | 99.99th=[52167]
   bw (  KiB/s): min=  408, max= 1272, per=22.71%, avg=794.67, stdev=439.08, samples=3
   iops        : min=  102, max=  318, avg=198.67, stdev=109.77, samples=3
  write: IOPS=240, BW=963KiB/s (986kB/s)(1636KiB/1699msec)
    clat (usec): min=964, max=14685, avg=2747.91, stdev=1812.71
     lat (usec): min=965, max=14686, avg=2748.65, stdev=1812.76
    clat percentiles (usec):
     |  1.00th=[ 1172],  5.00th=[ 1287], 10.00th=[ 1500], 20.00th=[ 1745],
     | 30.00th=[ 1827], 40.00th=[ 1958], 50.00th=[ 2114], 60.00th=[ 2343],
     | 70.00th=[ 2835], 80.00th=[ 3425], 90.00th=[ 4621], 95.00th=[ 5669],
     | 99.00th=[11469], 99.50th=[12649], 99.90th=[14746], 99.95th=[14746],
     | 99.99th=[14746]
   bw (  KiB/s): min=  456, max= 1272, per=23.90%, avg=880.00, stdev=408.94, samples=3
   iops        : min=  114, max=  318, avg=220.00, stdev=102.24, samples=3
  lat (usec)   : 750=12.63%, 1000=16.02%
  lat (msec)   : 2=33.72%, 4=27.34%, 10=8.85%, 20=1.17%, 50=0.13%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.41%, sys=5.59%, ctx=818, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19775: Wed Jul 17 11:00:53 2024
   read: IOPS=231, BW=928KiB/s (950kB/s)(1564KiB/1686msec)
    clat (usec): min=520, max=23307, avg=1569.37, stdev=2158.11
     lat (usec): min=520, max=23308, avg=1570.01, stdev=2158.15
    clat percentiles (usec):
     |  1.00th=[  537],  5.00th=[  611], 10.00th=[  652], 20.00th=[  701],
     | 30.00th=[  758], 40.00th=[  807], 50.00th=[  873], 60.00th=[  947],
     | 70.00th=[ 1205], 80.00th=[ 1795], 90.00th=[ 3359], 95.00th=[ 4424],
     | 99.00th=[10945], 99.50th=[21627], 99.90th=[23200], 99.95th=[23200],
     | 99.99th=[23200]
   bw (  KiB/s): min=  328, max= 1288, per=23.93%, avg=837.33, stdev=482.68, samples=3
   iops        : min=   82, max=  322, avg=209.33, stdev=120.67, samples=3
  write: IOPS=223, BW=894KiB/s (916kB/s)(1508KiB/1686msec)
    clat (usec): min=1109, max=56127, avg=2813.63, stdev=3233.88
     lat (usec): min=1110, max=56129, avg=2814.33, stdev=3233.94
    clat percentiles (usec):
     |  1.00th=[ 1156],  5.00th=[ 1287], 10.00th=[ 1418], 20.00th=[ 1680],
     | 30.00th=[ 1827], 40.00th=[ 1926], 50.00th=[ 2040], 60.00th=[ 2212],
     | 70.00th=[ 2573], 80.00th=[ 3359], 90.00th=[ 4948], 95.00th=[ 6587],
     | 99.00th=[ 9765], 99.50th=[12518], 99.90th=[56361], 99.95th=[56361],
     | 99.99th=[56361]
   bw (  KiB/s): min=  416, max= 1320, per=22.45%, avg=826.67, stdev=457.63, samples=3
   iops        : min=  104, max=  330, avg=206.67, stdev=114.41, samples=3
  lat (usec)   : 750=14.45%, 1000=17.19%
  lat (msec)   : 2=33.07%, 4=24.61%, 10=9.64%, 20=0.65%, 50=0.26%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.00%, sys=6.35%, ctx=836, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=19776: Wed Jul 17 11:00:53 2024
   read: IOPS=224, BW=898KiB/s (920kB/s)(1508KiB/1679msec)
    clat (usec): min=494, max=44041, avg=1490.76, stdev=2501.62
     lat (usec): min=494, max=44042, avg=1491.41, stdev=2501.73
    clat percentiles (usec):
     |  1.00th=[  562],  5.00th=[  627], 10.00th=[  668], 20.00th=[  734],
     | 30.00th=[  783], 40.00th=[  832], 50.00th=[  914], 60.00th=[ 1029],
     | 70.00th=[ 1303], 80.00th=[ 1680], 90.00th=[ 2769], 95.00th=[ 3949],
     | 99.00th=[ 7373], 99.50th=[ 8979], 99.90th=[44303], 99.95th=[44303],
     | 99.99th=[44303]
   bw (  KiB/s): min=  368, max= 1200, per=23.40%, avg=818.67, stdev=420.31, samples=3
   iops        : min=   92, max=  300, avg=204.67, stdev=105.08, samples=3
  write: IOPS=232, BW=932KiB/s (954kB/s)(1564KiB/1679msec)
    clat (usec): min=1012, max=48035, avg=2823.17, stdev=3262.81
     lat (usec): min=1013, max=48037, avg=2824.01, stdev=3262.86
    clat percentiles (usec):
     |  1.00th=[ 1156],  5.00th=[ 1287], 10.00th=[ 1549], 20.00th=[ 1713],
     | 30.00th=[ 1844], 40.00th=[ 1926], 50.00th=[ 2040], 60.00th=[ 2147],
     | 70.00th=[ 2540], 80.00th=[ 3458], 90.00th=[ 4424], 95.00th=[ 5604],
     | 99.00th=[12649], 99.50th=[34341], 99.90th=[47973], 99.95th=[47973],
     | 99.99th=[47973]
   bw (  KiB/s): min=  416, max= 1336, per=23.61%, avg=869.33, stdev=460.14, samples=3
   iops        : min=  104, max=  334, avg=217.33, stdev=115.04, samples=3
  lat (usec)   : 500=0.13%, 750=11.72%, 1000=16.28%
  lat (msec)   : 2=37.50%, 4=24.74%, 10=8.72%, 20=0.52%, 50=0.39%
  cpu          : usr=0.18%, sys=5.90%, ctx=828, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3500KiB/s (3584kB/s), 845KiB/s-928KiB/s (865kB/s-950kB/s), io=5988KiB (6132kB), run=1679-1711msec
  WRITE: bw=3682KiB/s (3770kB/s), 894KiB/s-963KiB/s (916kB/s-986kB/s), io=6300KiB (6451kB), run=1679-1711msec