== sanity test 398b: DIO and buffer IO race ============== 16:18:50 (1715458730)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.112095 s, 112 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=16664: Sat May 11 16:18:53 2024
   read: IOPS=412, BW=1650KiB/s (1690kB/s)(1480KiB/897msec)
    clat (usec): min=3, max=47816, avg=1440.30, stdev=3137.33
     lat (usec): min=3, max=47818, avg=1441.70, stdev=3137.43
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    8],
     | 30.00th=[   10], 40.00th=[   20], 50.00th=[ 1074], 60.00th=[ 1385],
     | 70.00th=[ 1795], 80.00th=[ 2245], 90.00th=[ 2900], 95.00th=[ 3752],
     | 99.00th=[10421], 99.50th=[26084], 99.90th=[47973], 99.95th=[47973],
     | 99.99th=[47973]
   bw (  KiB/s): min= 1192, max= 1192, per=18.87%, avg=1192.00, stdev= 0.00, samples=1
   iops        : min=  298, max=  298, avg=298.00, stdev= 0.00, samples=1
  write: IOPS=443, BW=1775KiB/s (1817kB/s)(1592KiB/897msec)
    clat (usec): min=82, max=6539, avg=884.59, stdev=1153.13
     lat (usec): min=83, max=6540, avg=885.18, stdev=1153.18
    clat percentiles (usec):
     |  1.00th=[   89],  5.00th=[   94], 10.00th=[  101], 20.00th=[  110],
     | 30.00th=[  121], 40.00th=[  149], 50.00th=[  355], 60.00th=[  717],
     | 70.00th=[  996], 80.00th=[ 1467], 90.00th=[ 2245], 95.00th=[ 3490],
     | 99.00th=[ 5735], 99.50th=[ 6325], 99.90th=[ 6521], 99.95th=[ 6521],
     | 99.99th=[ 6521]
   bw (  KiB/s): min= 1144, max= 1144, per=17.22%, avg=1144.00, stdev= 0.00, samples=1
   iops        : min=  286, max=  286, avg=286.00, stdev= 0.00, samples=1
  lat (usec)   : 4=1.30%, 10=15.36%, 20=2.73%, 50=0.26%, 100=5.08%
  lat (usec)   : 250=20.31%, 500=1.82%, 750=6.77%, 1000=5.99%
  lat (msec)   : 2=21.61%, 4=14.84%, 10=3.39%, 20=0.26%, 50=0.26%
  cpu          : usr=0.33%, sys=17.52%, ctx=891, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=16666: Sat May 11 16:18:53 2024
   read: IOPS=378, BW=1515KiB/s (1551kB/s)(1436KiB/948msec)
    clat (usec): min=2, max=14295, avg=1258.34, stdev=1539.18
     lat (usec): min=3, max=14297, avg=1258.89, stdev=1539.17
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    8],
     | 30.00th=[   10], 40.00th=[   69], 50.00th=[ 1074], 60.00th=[ 1418],
     | 70.00th=[ 1860], 80.00th=[ 2278], 90.00th=[ 2802], 95.00th=[ 3490],
     | 99.00th=[ 6718], 99.50th=[ 9241], 99.90th=[14353], 99.95th=[14353],
     | 99.99th=[14353]
   bw (  KiB/s): min=  984, max=  984, per=15.58%, avg=984.00, stdev= 0.00, samples=1
   iops        : min=  246, max=  246, avg=246.00, stdev= 0.00, samples=1
  write: IOPS=431, BW=1726KiB/s (1767kB/s)(1636KiB/948msec)
    clat (usec): min=82, max=52506, avg=1151.17, stdev=2969.58
     lat (usec): min=83, max=52507, avg=1152.90, stdev=2969.40
    clat percentiles (usec):
     |  1.00th=[   87],  5.00th=[   94], 10.00th=[  100], 20.00th=[  111],
     | 30.00th=[  127], 40.00th=[  186], 50.00th=[  627], 60.00th=[  816],
     | 70.00th=[ 1045], 80.00th=[ 1614], 90.00th=[ 2442], 95.00th=[ 3589],
     | 99.00th=[ 9241], 99.50th=[ 9765], 99.90th=[52691], 99.95th=[52691],
     | 99.99th=[52691]
   bw (  KiB/s): min= 1192, max= 1192, per=17.94%, avg=1192.00, stdev= 0.00, samples=1
   iops        : min=  298, max=  298, avg=298.00, stdev= 0.00, samples=1
  lat (usec)   : 4=2.08%, 10=13.67%, 20=2.60%, 50=0.13%, 100=5.73%
  lat (usec)   : 250=17.97%, 500=2.21%, 750=5.73%, 1000=9.37%
  lat (msec)   : 2=21.35%, 4=15.36%, 10=3.39%, 20=0.26%, 100=0.13%
  cpu          : usr=0.42%, sys=15.21%, ctx=893, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=16668: Sat May 11 16:18:53 2024
   read: IOPS=433, BW=1732KiB/s (1774kB/s)(1564KiB/903msec)
    clat (usec): min=3, max=33962, avg=1324.37, stdev=2227.73
     lat (usec): min=3, max=33963, avg=1324.96, stdev=2227.93
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    4], 10.00th=[    6], 20.00th=[    7],
     | 30.00th=[    9], 40.00th=[   11], 50.00th=[  914], 60.00th=[ 1450],
     | 70.00th=[ 1876], 80.00th=[ 2376], 90.00th=[ 2999], 95.00th=[ 3752],
     | 99.00th=[ 7111], 99.50th=[11207], 99.90th=[33817], 99.95th=[33817],
     | 99.99th=[33817]
   bw (  KiB/s): min= 1064, max= 1064, per=16.85%, avg=1064.00, stdev= 0.00, samples=1
   iops        : min=  266, max=  266, avg=266.00, stdev= 0.00, samples=1
  write: IOPS=417, BW=1670KiB/s (1710kB/s)(1508KiB/903msec)
    clat (usec): min=84, max=24808, avg=991.22, stdev=1835.57
     lat (usec): min=84, max=24809, avg=991.82, stdev=1835.67
    clat percentiles (usec):
     |  1.00th=[   90],  5.00th=[   95], 10.00th=[  101], 20.00th=[  112],
     | 30.00th=[  120], 40.00th=[  176], 50.00th=[  437], 60.00th=[  742],
     | 70.00th=[  996], 80.00th=[ 1663], 90.00th=[ 2343], 95.00th=[ 3130],
     | 99.00th=[ 5932], 99.50th=[15533], 99.90th=[24773], 99.95th=[24773],
     | 99.99th=[24773]
   bw (  KiB/s): min= 1040, max= 1040, per=15.65%, avg=1040.00, stdev= 0.00, samples=1
   iops        : min=  260, max=  260, avg=260.00, stdev= 0.00, samples=1
  lat (usec)   : 4=3.12%, 10=16.02%, 20=2.60%, 50=0.26%, 100=4.56%
  lat (usec)   : 250=18.62%, 500=2.08%, 750=5.86%, 1000=7.29%
  lat (msec)   : 2=18.23%, 4=17.58%, 10=3.26%, 20=0.26%, 50=0.26%
  cpu          : usr=0.89%, sys=15.30%, ctx=856, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=16669: Sat May 11 16:18:53 2024
   read: IOPS=420, BW=1683KiB/s (1723kB/s)(1508KiB/896msec)
    clat (usec): min=3, max=43905, avg=1263.75, stdev=2513.29
     lat (usec): min=3, max=43906, avg=1264.73, stdev=2513.32
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    8],
     | 30.00th=[   10], 40.00th=[   18], 50.00th=[ 1029], 60.00th=[ 1418],
     | 70.00th=[ 1795], 80.00th=[ 2147], 90.00th=[ 2802], 95.00th=[ 3425],
     | 99.00th=[ 5407], 99.50th=[ 5997], 99.90th=[43779], 99.95th=[43779],
     | 99.99th=[43779]
   bw (  KiB/s): min= 1208, max= 1208, per=19.13%, avg=1208.00, stdev= 0.00, samples=1
   iops        : min=  302, max=  302, avg=302.00, stdev= 0.00, samples=1
  write: IOPS=436, BW=1746KiB/s (1787kB/s)(1564KiB/896msec)
    clat (usec): min=84, max=26044, avg=1009.41, stdev=1841.65
     lat (usec): min=84, max=26046, avg=1010.20, stdev=1842.02
    clat percentiles (usec):
     |  1.00th=[   90],  5.00th=[   95], 10.00th=[  101], 20.00th=[  112],
     | 30.00th=[  123], 40.00th=[  176], 50.00th=[  359], 60.00th=[  717],
     | 70.00th=[  938], 80.00th=[ 1647], 90.00th=[ 2540], 95.00th=[ 3556],
     | 99.00th=[ 7701], 99.50th=[10683], 99.90th=[26084], 99.95th=[26084],
     | 99.99th=[26084]
   bw (  KiB/s): min= 1216, max= 1216, per=18.30%, avg=1216.00, stdev= 0.00, samples=1
   iops        : min=  304, max=  304, avg=304.00, stdev= 0.00, samples=1
  lat (usec)   : 4=1.04%, 10=15.89%, 20=2.86%, 50=0.26%, 100=4.82%
  lat (usec)   : 250=20.05%, 500=2.21%, 750=5.73%, 1000=7.42%
  lat (msec)   : 2=20.83%, 4=16.15%, 10=2.34%, 20=0.13%, 50=0.26%
  cpu          : usr=0.00%, sys=15.98%, ctx=872, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=6316KiB/s (6468kB/s), 1515KiB/s-1732KiB/s (1551kB/s-1774kB/s), io=5988KiB (6132kB), run=896-948msec
  WRITE: bw=6646KiB/s (6805kB/s), 1670KiB/s-1775KiB/s (1710kB/s-1817kB/s), io=6300KiB (6451kB), run=896-948msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=16662: Sat May 11 16:18:54 2024
   read: IOPS=244, BW=976KiB/s (1000kB/s)(1480KiB/1516msec)
    clat (usec): min=489, max=45056, avg=1521.11, stdev=2943.29
     lat (usec): min=490, max=45057, avg=1521.67, stdev=2943.37
    clat percentiles (usec):
     |  1.00th=[  523],  5.00th=[  578], 10.00th=[  611], 20.00th=[  652],
     | 30.00th=[  709], 40.00th=[  758], 50.00th=[  824], 60.00th=[  930],
     | 70.00th=[ 1123], 80.00th=[ 1647], 90.00th=[ 2966], 95.00th=[ 4080],
     | 99.00th=[ 7635], 99.50th=[28705], 99.90th=[44827], 99.95th=[44827],
     | 99.99th=[44827]
   bw (  KiB/s): min=  536, max= 1392, per=24.87%, avg=970.67, stdev=428.16, samples=3
   iops        : min=  134, max=  348, avg=242.67, stdev=107.04, samples=3
  write: IOPS=262, BW=1050KiB/s (1075kB/s)(1592KiB/1516msec)
    clat (usec): min=1045, max=11601, avg=2373.78, stdev=1391.52
     lat (usec): min=1045, max=11602, avg=2374.45, stdev=1391.57
    clat percentiles (usec):
     |  1.00th=[ 1106],  5.00th=[ 1237], 10.00th=[ 1418], 20.00th=[ 1614],
     | 30.00th=[ 1729], 40.00th=[ 1811], 50.00th=[ 1926], 60.00th=[ 2024],
     | 70.00th=[ 2245], 80.00th=[ 2769], 90.00th=[ 4047], 95.00th=[ 5342],
     | 99.00th=[ 9110], 99.50th=[10683], 99.90th=[11600], 99.95th=[11600],
     | 99.99th=[11600]
   bw (  KiB/s): min=  560, max= 1600, per=25.52%, avg=1048.00, stdev=522.95, samples=3
   iops        : min=  140, max=  400, avg=262.00, stdev=130.74, samples=3
  lat (usec)   : 500=0.13%, 750=18.75%, 1000=12.50%
  lat (msec)   : 2=38.41%, 4=22.40%, 10=7.03%, 20=0.52%, 50=0.26%
  cpu          : usr=0.26%, sys=6.07%, ctx=832, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=16663: Sat May 11 16:18:54 2024
   read: IOPS=234, BW=936KiB/s (959kB/s)(1436KiB/1534msec)
    clat (usec): min=467, max=13784, avg=1292.43, stdev=1296.21
     lat (usec): min=468, max=13785, avg=1292.99, stdev=1296.26
    clat percentiles (usec):
     |  1.00th=[  529],  5.00th=[  578], 10.00th=[  611], 20.00th=[  668],
     | 30.00th=[  734], 40.00th=[  775], 50.00th=[  848], 60.00th=[  922],
     | 70.00th=[ 1074], 80.00th=[ 1582], 90.00th=[ 2507], 95.00th=[ 3818],
     | 99.00th=[ 6259], 99.50th=[ 9503], 99.90th=[13829], 99.95th=[13829],
     | 99.99th=[13829]
   bw (  KiB/s): min=  528, max= 1480, per=24.05%, avg=938.67, stdev=489.27, samples=3
   iops        : min=  132, max=  370, avg=234.67, stdev=122.32, samples=3
  write: IOPS=266, BW=1066KiB/s (1092kB/s)(1636KiB/1534msec)
    clat (usec): min=970, max=53164, avg=2556.78, stdev=2903.53
     lat (usec): min=971, max=53166, avg=2557.46, stdev=2903.60
    clat percentiles (usec):
     |  1.00th=[ 1090],  5.00th=[ 1254], 10.00th=[ 1434], 20.00th=[ 1631],
     | 30.00th=[ 1729], 40.00th=[ 1827], 50.00th=[ 1958], 60.00th=[ 2114],
     | 70.00th=[ 2474], 80.00th=[ 2999], 90.00th=[ 3884], 95.00th=[ 5211],
     | 99.00th=[ 9241], 99.50th=[12518], 99.90th=[53216], 99.95th=[53216],
     | 99.99th=[53216]
   bw (  KiB/s): min=  608, max= 1560, per=25.78%, avg=1058.67, stdev=478.02, samples=3
   iops        : min=  152, max=  390, avg=264.67, stdev=119.50, samples=3
  lat (usec)   : 500=0.13%, 750=15.76%, 1000=15.49%
  lat (msec)   : 2=37.63%, 4=24.09%, 10=6.25%, 20=0.52%, 100=0.13%
  cpu          : usr=0.26%, sys=6.13%, ctx=823, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=16665: Sat May 11 16:18:54 2024
   read: IOPS=263, BW=1055KiB/s (1080kB/s)(1564KiB/1483msec)
    clat (usec): min=490, max=11023, avg=1230.29, stdev=1074.81
     lat (usec): min=490, max=11024, avg=1230.86, stdev=1074.83
    clat percentiles (usec):
     |  1.00th=[  523],  5.00th=[  594], 10.00th=[  627], 20.00th=[  685],
     | 30.00th=[  734], 40.00th=[  766], 50.00th=[  807], 60.00th=[  873],
     | 70.00th=[ 1029], 80.00th=[ 1467], 90.00th=[ 2442], 95.00th=[ 3654],
     | 99.00th=[ 5604], 99.50th=[ 6652], 99.90th=[11076], 99.95th=[11076],
     | 99.99th=[11076]
   bw (  KiB/s): min=  495, max= 1048, per=19.77%, avg=771.50, stdev=391.03, samples=2
   iops        : min=  123, max=  262, avg=192.50, stdev=98.29, samples=2
  write: IOPS=254, BW=1017KiB/s (1041kB/s)(1508KiB/1483msec)
    clat (usec): min=1034, max=60361, avg=2590.17, stdev=3535.40
     lat (usec): min=1035, max=60362, avg=2591.25, stdev=3535.44
    clat percentiles (usec):
     |  1.00th=[ 1090],  5.00th=[ 1287], 10.00th=[ 1483], 20.00th=[ 1631],
     | 30.00th=[ 1729], 40.00th=[ 1827], 50.00th=[ 1926], 60.00th=[ 2089],
     | 70.00th=[ 2311], 80.00th=[ 2933], 90.00th=[ 3949], 95.00th=[ 5407],
     | 99.00th=[ 8291], 99.50th=[30802], 99.90th=[60556], 99.95th=[60556],
     | 99.99th=[60556]
   bw (  KiB/s): min=  630, max=  936, per=19.07%, avg=783.00, stdev=216.37, samples=2
   iops        : min=  157, max=  234, avg=195.50, stdev=54.45, samples=2
  lat (usec)   : 500=0.13%, 750=17.97%, 1000=16.54%
  lat (msec)   : 2=36.20%, 4=22.66%, 10=6.12%, 20=0.13%, 50=0.13%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.34%, sys=6.21%, ctx=835, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=16667: Sat May 11 16:18:54 2024
   read: IOPS=250, BW=1002KiB/s (1026kB/s)(1508KiB/1505msec)
    clat (usec): min=508, max=44491, avg=1343.34, stdev=2467.32
     lat (usec): min=509, max=44492, avg=1344.53, stdev=2467.39
    clat percentiles (usec):
     |  1.00th=[  515],  5.00th=[  578], 10.00th=[  627], 20.00th=[  668],
     | 30.00th=[  709], 40.00th=[  766], 50.00th=[  832], 60.00th=[  906],
     | 70.00th=[ 1045], 80.00th=[ 1467], 90.00th=[ 2573], 95.00th=[ 3720],
     | 99.00th=[ 6849], 99.50th=[ 7963], 99.90th=[44303], 99.95th=[44303],
     | 99.99th=[44303]
   bw (  KiB/s): min=  512, max= 1448, per=25.48%, avg=994.67, stdev=468.69, samples=3
   iops        : min=  128, max=  362, avg=248.67, stdev=117.17, samples=3
  write: IOPS=259, BW=1039KiB/s (1064kB/s)(1564KiB/1505msec)
    clat (usec): min=992, max=39703, avg=2529.92, stdev=2381.92
     lat (usec): min=992, max=39705, avg=2530.64, stdev=2381.98
    clat percentiles (usec):
     |  1.00th=[ 1074],  5.00th=[ 1237], 10.00th=[ 1434], 20.00th=[ 1631],
     | 30.00th=[ 1729], 40.00th=[ 1811], 50.00th=[ 1909], 60.00th=[ 2024],
     | 70.00th=[ 2343], 80.00th=[ 2933], 90.00th=[ 4293], 95.00th=[ 5276],
     | 99.00th=[10290], 99.50th=[11469], 99.90th=[39584], 99.95th=[39584],
     | 99.99th=[39584]
   bw (  KiB/s): min=  568, max= 1600, per=25.33%, avg=1040.00, stdev=521.60, samples=3
   iops        : min=  142, max=  400, avg=260.00, stdev=130.40, samples=3
  lat (usec)   : 750=18.36%, 1000=14.84%
  lat (msec)   : 2=39.71%, 4=19.40%, 10=7.03%, 20=0.39%, 50=0.26%
  cpu          : usr=0.73%, sys=5.91%, ctx=843, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3904KiB/s (3997kB/s), 936KiB/s-1055KiB/s (959kB/s-1080kB/s), io=5988KiB (6132kB), run=1483-1534msec
  WRITE: bw=4107KiB/s (4205kB/s), 1017KiB/s-1066KiB/s (1041kB/s-1092kB/s), io=6300KiB (6451kB), run=1483-1534msec