== sanity test 398b: DIO and buffer IO race ============== 14:08:25 (1721412505)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0889489 s, 141 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=21663: Fri Jul 19 14:08:28 2024
   read: IOPS=346, BW=1386KiB/s (1419kB/s)(1480KiB/1068msec)
    clat (usec): min=3, max=40871, avg=1683.11, stdev=3375.35
     lat (usec): min=3, max=40871, avg=1683.76, stdev=3375.38
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    7], 20.00th=[   10],
     | 30.00th=[   11], 40.00th=[   24], 50.00th=[ 1221], 60.00th=[ 1647],
     | 70.00th=[ 2147], 80.00th=[ 2671], 90.00th=[ 3294], 95.00th=[ 4883],
     | 99.00th=[13698], 99.50th=[37487], 99.90th=[40633], 99.95th=[40633],
     | 99.99th=[40633]
   bw (  KiB/s): min=  712, max= 1944, per=24.24%, avg=1328.00, stdev=871.16, samples=2
   iops        : min=  178, max=  486, avg=332.00, stdev=217.79, samples=2
  write: IOPS=372, BW=1491KiB/s (1526kB/s)(1592KiB/1068msec)
    clat (usec): min=90, max=13590, avg=1096.72, stdev=1543.62
     lat (usec): min=90, max=13591, avg=1097.62, stdev=1543.82
    clat percentiles (usec):
     |  1.00th=[   95],  5.00th=[  103], 10.00th=[  110], 20.00th=[  123],
     | 30.00th=[  155], 40.00th=[  190], 50.00th=[  586], 60.00th=[  857],
     | 70.00th=[ 1123], 80.00th=[ 1827], 90.00th=[ 2704], 95.00th=[ 4359],
     | 99.00th=[ 7701], 99.50th=[ 8225], 99.90th=[13566], 99.95th=[13566],
     | 99.99th=[13566]
   bw (  KiB/s): min=  752, max= 2136, per=25.06%, avg=1444.00, stdev=978.64, samples=2
   iops        : min=  188, max=  534, avg=361.00, stdev=244.66, samples=2
  lat (usec)   : 4=1.30%, 10=11.20%, 20=6.38%, 50=0.91%, 100=2.08%
  lat (usec)   : 250=20.83%, 500=3.12%, 750=2.60%, 1000=9.24%
  lat (msec)   : 2=17.84%, 4=18.75%, 10=5.08%, 20=0.39%, 50=0.26%
  cpu          : usr=0.47%, sys=16.59%, ctx=900, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21664: Fri Jul 19 14:08:28 2024
   read: IOPS=339, BW=1357KiB/s (1390kB/s)(1436KiB/1058msec)
    clat (usec): min=3, max=26141, avg=1476.15, stdev=2202.08
     lat (usec): min=3, max=26142, avg=1476.74, stdev=2202.08
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   13], 40.00th=[   78], 50.00th=[ 1057], 60.00th=[ 1647],
     | 70.00th=[ 2180], 80.00th=[ 2606], 90.00th=[ 3064], 95.00th=[ 3884],
     | 99.00th=[11863], 99.50th=[14484], 99.90th=[26084], 99.95th=[26084],
     | 99.99th=[26084]
   bw (  KiB/s): min=  840, max= 1800, per=24.10%, avg=1320.00, stdev=678.82, samples=2
   iops        : min=  210, max=  450, avg=330.00, stdev=169.71, samples=2
  write: IOPS=386, BW=1546KiB/s (1583kB/s)(1636KiB/1058msec)
    clat (usec): min=90, max=50393, avg=1263.79, stdev=3364.87
     lat (usec): min=90, max=50394, avg=1264.47, stdev=3364.91
    clat percentiles (usec):
     |  1.00th=[   94],  5.00th=[  103], 10.00th=[  110], 20.00th=[  130],
     | 30.00th=[  159], 40.00th=[  194], 50.00th=[  322], 60.00th=[  832],
     | 70.00th=[ 1139], 80.00th=[ 1860], 90.00th=[ 2638], 95.00th=[ 4080],
     | 99.00th=[11076], 99.50th=[14484], 99.90th=[50594], 99.95th=[50594],
     | 99.99th=[50594]
   bw (  KiB/s): min=  936, max= 2016, per=25.61%, avg=1476.00, stdev=763.68, samples=2
   iops        : min=  234, max=  504, avg=369.00, stdev=190.92, samples=2
  lat (usec)   : 4=0.39%, 10=11.20%, 20=5.99%, 50=0.65%, 100=2.86%
  lat (usec)   : 250=22.92%, 500=3.39%, 750=3.78%, 1000=7.42%
  lat (msec)   : 2=16.67%, 4=19.92%, 10=3.65%, 20=0.78%, 50=0.26%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.38%, sys=15.99%, ctx=834, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21665: Fri Jul 19 14:08:28 2024
   read: IOPS=357, BW=1431KiB/s (1465kB/s)(1564KiB/1093msec)
    clat (usec): min=3, max=35399, avg=1556.77, stdev=2567.70
     lat (usec): min=3, max=35399, avg=1557.36, stdev=2567.71
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[   21], 50.00th=[ 1221], 60.00th=[ 1745],
     | 70.00th=[ 2180], 80.00th=[ 2606], 90.00th=[ 3294], 95.00th=[ 4146],
     | 99.00th=[ 9634], 99.50th=[19268], 99.90th=[35390], 99.95th=[35390],
     | 99.99th=[35390]
   bw (  KiB/s): min=  704, max= 1896, per=23.73%, avg=1300.00, stdev=842.87, samples=2
   iops        : min=  176, max=  474, avg=325.00, stdev=210.72, samples=2
  write: IOPS=344, BW=1380KiB/s (1413kB/s)(1508KiB/1093msec)
    clat (usec): min=93, max=51967, avg=1263.44, stdev=3034.97
     lat (usec): min=93, max=51968, avg=1264.14, stdev=3035.01
    clat percentiles (usec):
     |  1.00th=[   96],  5.00th=[  108], 10.00th=[  116], 20.00th=[  143],
     | 30.00th=[  172], 40.00th=[  221], 50.00th=[  494], 60.00th=[  971],
     | 70.00th=[ 1254], 80.00th=[ 1876], 90.00th=[ 2999], 95.00th=[ 4178],
     | 99.00th=[ 6521], 99.50th=[15664], 99.90th=[52167], 99.95th=[52167],
     | 99.99th=[52167]
   bw (  KiB/s): min=  792, max= 1784, per=22.35%, avg=1288.00, stdev=701.45, samples=2
   iops        : min=  198, max=  446, avg=322.00, stdev=175.36, samples=2
  lat (usec)   : 4=0.91%, 10=13.41%, 20=5.73%, 50=0.78%, 100=1.43%
  lat (usec)   : 250=20.70%, 500=3.78%, 750=1.56%, 1000=5.47%
  lat (msec)   : 2=19.53%, 4=20.96%, 10=5.08%, 20=0.39%, 50=0.13%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.64%, sys=15.11%, ctx=925, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21666: Fri Jul 19 14:08:28 2024
   read: IOPS=353, BW=1413KiB/s (1447kB/s)(1508KiB/1067msec)
    clat (usec): min=3, max=34261, avg=1688.06, stdev=3366.74
     lat (usec): min=3, max=34261, avg=1690.14, stdev=3367.60
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   10], 40.00th=[   16], 50.00th=[ 1139], 60.00th=[ 1680],
     | 70.00th=[ 2040], 80.00th=[ 2704], 90.00th=[ 3490], 95.00th=[ 5407],
     | 99.00th=[12780], 99.50th=[33817], 99.90th=[34341], 99.95th=[34341],
     | 99.99th=[34341]
   bw (  KiB/s): min=  784, max= 1880, per=24.32%, avg=1332.00, stdev=774.99, samples=2
   iops        : min=  196, max=  470, avg=333.00, stdev=193.75, samples=2
  write: IOPS=366, BW=1466KiB/s (1501kB/s)(1564KiB/1067msec)
    clat (usec): min=81, max=9221, avg=1072.09, stdev=1376.98
     lat (usec): min=82, max=9222, avg=1073.52, stdev=1376.84
    clat percentiles (usec):
     |  1.00th=[   91],  5.00th=[   99], 10.00th=[  109], 20.00th=[  123],
     | 30.00th=[  159], 40.00th=[  215], 50.00th=[  717], 60.00th=[  971],
     | 70.00th=[ 1270], 80.00th=[ 1795], 90.00th=[ 2376], 95.00th=[ 3687],
     | 99.00th=[ 7373], 99.50th=[ 8586], 99.90th=[ 9241], 99.95th=[ 9241],
     | 99.99th=[ 9241]
   bw (  KiB/s): min=  864, max= 1952, per=24.43%, avg=1408.00, stdev=769.33, samples=2
   iops        : min=  216, max=  488, avg=352.00, stdev=192.33, samples=2
  lat (usec)   : 4=1.56%, 10=14.19%, 20=4.43%, 50=1.04%, 100=3.52%
  lat (usec)   : 250=18.36%, 500=2.60%, 750=2.86%, 1000=6.51%
  lat (msec)   : 2=21.22%, 4=17.97%, 10=4.95%, 20=0.39%, 50=0.39%
  cpu          : usr=0.38%, sys=16.89%, ctx=868, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=5478KiB/s (5610kB/s), 1357KiB/s-1431KiB/s (1390kB/s-1465kB/s), io=5988KiB (6132kB), run=1058-1093msec
  WRITE: bw=5764KiB/s (5902kB/s), 1380KiB/s-1546KiB/s (1413kB/s-1583kB/s), io=6300KiB (6451kB), run=1058-1093msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=21668: Fri Jul 19 14:08:28 2024
   read: IOPS=208, BW=833KiB/s (853kB/s)(1480KiB/1777msec)
    clat (usec): min=580, max=51842, avg=1665.41, stdev=3688.20
     lat (usec): min=581, max=51844, avg=1666.32, stdev=3688.22
    clat percentiles (usec):
     |  1.00th=[  603],  5.00th=[  660], 10.00th=[  709], 20.00th=[  775],
     | 30.00th=[  816], 40.00th=[  865], 50.00th=[  930], 60.00th=[ 1045],
     | 70.00th=[ 1205], 80.00th=[ 1745], 90.00th=[ 2900], 95.00th=[ 4424],
     | 99.00th=[ 7242], 99.50th=[43779], 99.90th=[51643], 99.95th=[51643],
     | 99.99th=[51643]
   bw (  KiB/s): min=  368, max= 1104, per=22.64%, avg=762.67, stdev=370.89, samples=3
   iops        : min=   92, max=  276, avg=190.67, stdev=92.72, samples=3
  write: IOPS=223, BW=896KiB/s (917kB/s)(1592KiB/1777msec)
    clat (usec): min=1051, max=17655, avg=2893.59, stdev=1717.68
     lat (usec): min=1052, max=17656, avg=2894.31, stdev=1717.71
    clat percentiles (usec):
     |  1.00th=[ 1156],  5.00th=[ 1418], 10.00th=[ 1729], 20.00th=[ 1909],
     | 30.00th=[ 2040], 40.00th=[ 2147], 50.00th=[ 2278], 60.00th=[ 2474],
     | 70.00th=[ 2868], 80.00th=[ 3523], 90.00th=[ 5014], 95.00th=[ 6325],
     | 99.00th=[ 8848], 99.50th=[13698], 99.90th=[17695], 99.95th=[17695],
     | 99.99th=[17695]
   bw (  KiB/s): min=  424, max= 1344, per=23.32%, avg=826.67, stdev=470.60, samples=3
   iops        : min=  106, max=  336, avg=206.67, stdev=117.65, samples=3
  lat (usec)   : 750=8.33%, 1000=19.01%
  lat (msec)   : 2=26.56%, 4=35.03%, 10=10.42%, 20=0.39%, 50=0.13%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.34%, sys=5.41%, ctx=808, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21669: Fri Jul 19 14:08:28 2024
   read: IOPS=202, BW=809KiB/s (829kB/s)(1436KiB/1774msec)
    clat (usec): min=552, max=37297, avg=1635.93, stdev=2921.66
     lat (usec): min=553, max=37299, avg=1636.59, stdev=2921.72
    clat percentiles (usec):
     |  1.00th=[  594],  5.00th=[  660], 10.00th=[  693], 20.00th=[  758],
     | 30.00th=[  807], 40.00th=[  873], 50.00th=[  922], 60.00th=[  988],
     | 70.00th=[ 1205], 80.00th=[ 1680], 90.00th=[ 3523], 95.00th=[ 4490],
     | 99.00th=[ 7308], 99.50th=[33424], 99.90th=[37487], 99.95th=[37487],
     | 99.99th=[37487]
   bw (  KiB/s): min=  400, max= 1280, per=22.95%, avg=773.33, stdev=454.90, samples=3
   iops        : min=  100, max=  320, avg=193.33, stdev=113.72, samples=3
  write: IOPS=230, BW=922KiB/s (944kB/s)(1636KiB/1774msec)
    clat (usec): min=1132, max=16733, avg=2880.01, stdev=1773.86
     lat (usec): min=1132, max=16735, avg=2880.94, stdev=1773.94
    clat percentiles (usec):
     |  1.00th=[ 1205],  5.00th=[ 1418], 10.00th=[ 1598], 20.00th=[ 1893],
     | 30.00th=[ 2024], 40.00th=[ 2147], 50.00th=[ 2311], 60.00th=[ 2442],
     | 70.00th=[ 2900], 80.00th=[ 3523], 90.00th=[ 4948], 95.00th=[ 6325],
     | 99.00th=[10028], 99.50th=[12256], 99.90th=[16712], 99.95th=[16712],
     | 99.99th=[16712]
   bw (  KiB/s): min=  448, max= 1208, per=23.47%, avg=832.00, stdev=380.06, samples=3
   iops        : min=  112, max=  302, avg=208.00, stdev=95.02, samples=3
  lat (usec)   : 750=8.59%, 1000=19.79%
  lat (msec)   : 2=25.52%, 4=33.98%, 10=11.07%, 20=0.78%, 50=0.26%
  cpu          : usr=0.51%, sys=5.25%, ctx=818, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21670: Fri Jul 19 14:08:28 2024
   read: IOPS=222, BW=889KiB/s (910kB/s)(1564KiB/1760msec)
    clat (usec): min=547, max=17693, avg=1442.43, stdev=1449.49
     lat (usec): min=547, max=17694, avg=1443.05, stdev=1449.53
    clat percentiles (usec):
     |  1.00th=[  586],  5.00th=[  644], 10.00th=[  701], 20.00th=[  750],
     | 30.00th=[  807], 40.00th=[  865], 50.00th=[  922], 60.00th=[ 1012],
     | 70.00th=[ 1237], 80.00th=[ 1827], 90.00th=[ 2638], 95.00th=[ 4146],
     | 99.00th=[ 7046], 99.50th=[ 9241], 99.90th=[17695], 99.95th=[17695],
     | 99.99th=[17695]
   bw (  KiB/s): min=  328, max= 1192, per=23.51%, avg=792.00, stdev=435.54, samples=3
   iops        : min=   82, max=  298, avg=198.00, stdev=108.89, samples=3
  write: IOPS=214, BW=857KiB/s (877kB/s)(1508KiB/1760msec)
    clat (usec): min=1098, max=56917, avg=3152.93, stdev=3912.80
     lat (usec): min=1099, max=56918, avg=3153.64, stdev=3912.88
    clat percentiles (usec):
     |  1.00th=[ 1221],  5.00th=[ 1434], 10.00th=[ 1663], 20.00th=[ 1876],
     | 30.00th=[ 2008], 40.00th=[ 2114], 50.00th=[ 2245], 60.00th=[ 2442],
     | 70.00th=[ 2999], 80.00th=[ 3654], 90.00th=[ 4883], 95.00th=[ 6783],
     | 99.00th=[14353], 99.50th=[44827], 99.90th=[56886], 99.95th=[56886],
     | 99.99th=[56886]
   bw (  KiB/s): min=  400, max= 1248, per=22.42%, avg=794.67, stdev=427.03, samples=3
   iops        : min=  100, max=  312, avg=198.67, stdev=106.76, samples=3
  lat (usec)   : 750=10.03%, 1000=19.66%
  lat (msec)   : 2=26.56%, 4=32.29%, 10=10.68%, 20=0.52%, 50=0.13%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.23%, sys=5.46%, ctx=815, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21671: Fri Jul 19 14:08:28 2024
   read: IOPS=215, BW=861KiB/s (882kB/s)(1508KiB/1751msec)
    clat (usec): min=543, max=48456, avg=1565.27, stdev=2826.52
     lat (usec): min=543, max=48457, avg=1565.89, stdev=2826.55
    clat percentiles (usec):
     |  1.00th=[  603],  5.00th=[  660], 10.00th=[  701], 20.00th=[  758],
     | 30.00th=[  807], 40.00th=[  873], 50.00th=[  930], 60.00th=[ 1004],
     | 70.00th=[ 1221], 80.00th=[ 1795], 90.00th=[ 3097], 95.00th=[ 4228],
     | 99.00th=[ 6783], 99.50th=[17433], 99.90th=[48497], 99.95th=[48497],
     | 99.99th=[48497]
   bw (  KiB/s): min=  344, max= 1096, per=22.72%, avg=765.33, stdev=384.11, samples=3
   iops        : min=   86, max=  274, avg=191.33, stdev=96.03, samples=3
  write: IOPS=223, BW=893KiB/s (915kB/s)(1564KiB/1751msec)
    clat (usec): min=1204, max=55967, avg=2946.51, stdev=3659.23
     lat (usec): min=1205, max=55969, avg=2947.21, stdev=3659.31
    clat percentiles (usec):
     |  1.00th=[ 1270],  5.00th=[ 1516], 10.00th=[ 1663], 20.00th=[ 1860],
     | 30.00th=[ 1975], 40.00th=[ 2089], 50.00th=[ 2212], 60.00th=[ 2409],
     | 70.00th=[ 2835], 80.00th=[ 3326], 90.00th=[ 4621], 95.00th=[ 5932],
     | 99.00th=[ 8029], 99.50th=[44827], 99.90th=[55837], 99.95th=[55837],
     | 99.99th=[55837]
   bw (  KiB/s): min=  416, max= 1288, per=23.47%, avg=832.00, stdev=437.37, samples=3
   iops        : min=  104, max=  322, avg=208.00, stdev=109.34, samples=3
  lat (usec)   : 750=9.51%, 1000=19.66%
  lat (msec)   : 2=28.52%, 4=32.94%, 10=8.72%, 20=0.26%, 50=0.26%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.17%, sys=5.54%, ctx=835, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3370KiB/s (3451kB/s), 809KiB/s-889KiB/s (829kB/s-910kB/s), io=5988KiB (6132kB), run=1751-1777msec
  WRITE: bw=3545KiB/s (3630kB/s), 857KiB/s-922KiB/s (877kB/s-944kB/s), io=6300KiB (6451kB), run=1751-1777msec