== sanity test 398b: DIO and buffer IO race ============== 21:06:40 (1718932000)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0782409 s, 161 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=15874: Thu Jun 20 21:06:43 2024
   read: IOPS=369, BW=1479KiB/s (1514kB/s)(1480KiB/1001msec)
    clat (usec): min=3, max=31660, avg=1540.36, stdev=2660.33
     lat (usec): min=3, max=31660, avg=1542.16, stdev=2660.62
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[   17], 50.00th=[ 1254], 60.00th=[ 1745],
     | 70.00th=[ 2114], 80.00th=[ 2573], 90.00th=[ 3064], 95.00th=[ 3785],
     | 99.00th=[11600], 99.50th=[27919], 99.90th=[31589], 99.95th=[31589],
     | 99.99th=[31589]
   bw (  KiB/s): min= 1208, max= 1208, per=21.85%, avg=1208.00, stdev= 0.00, samples=1
   iops        : min=  302, max=  302, avg=302.00, stdev= 0.00, samples=1
  write: IOPS=397, BW=1590KiB/s (1629kB/s)(1592KiB/1001msec)
    clat (usec): min=85, max=12478, avg=1058.01, stdev=1420.09
     lat (usec): min=86, max=12479, avg=1059.15, stdev=1419.86
    clat percentiles (usec):
     |  1.00th=[   95],  5.00th=[  109], 10.00th=[  119], 20.00th=[  145],
     | 30.00th=[  167], 40.00th=[  192], 50.00th=[  668], 60.00th=[  848],
     | 70.00th=[ 1106], 80.00th=[ 1762], 90.00th=[ 2933], 95.00th=[ 3752],
     | 99.00th=[ 7439], 99.50th=[ 9503], 99.90th=[12518], 99.95th=[12518],
     | 99.99th=[12518]
   bw (  KiB/s): min= 1176, max= 1176, per=20.22%, avg=1176.00, stdev= 0.00, samples=1
   iops        : min=  294, max=  294, avg=294.00, stdev= 0.00, samples=1
  lat (usec)   : 4=0.26%, 10=9.64%, 20=9.77%, 50=0.78%, 100=1.56%
  lat (usec)   : 250=22.27%, 500=1.56%, 750=2.60%, 1000=8.07%
  lat (msec)   : 2=18.62%, 4=20.57%, 10=3.65%, 20=0.39%, 50=0.26%
  cpu          : usr=0.50%, sys=17.30%, ctx=862, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15875: Thu Jun 20 21:06:43 2024
   read: IOPS=347, BW=1391KiB/s (1425kB/s)(1436KiB/1032msec)
    clat (usec): min=3, max=29019, avg=1649.92, stdev=2544.62
     lat (usec): min=3, max=29019, avg=1650.57, stdev=2544.65
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[   23], 50.00th=[ 1221], 60.00th=[ 1811],
     | 70.00th=[ 2278], 80.00th=[ 2671], 90.00th=[ 3359], 95.00th=[ 5211],
     | 99.00th=[11207], 99.50th=[19530], 99.90th=[28967], 99.95th=[28967],
     | 99.99th=[28967]
   bw (  KiB/s): min= 1032, max= 1688, per=24.60%, avg=1360.00, stdev=463.86, samples=2
   iops        : min=  258, max=  422, avg=340.00, stdev=115.97, samples=2
  write: IOPS=396, BW=1585KiB/s (1623kB/s)(1636KiB/1032msec)
    clat (usec): min=88, max=28321, avg=1037.60, stdev=1994.84
     lat (usec): min=89, max=28322, avg=1038.33, stdev=1994.86
    clat percentiles (usec):
     |  1.00th=[   93],  5.00th=[  103], 10.00th=[  111], 20.00th=[  131],
     | 30.00th=[  163], 40.00th=[  186], 50.00th=[  367], 60.00th=[  824],
     | 70.00th=[  988], 80.00th=[ 1450], 90.00th=[ 2573], 95.00th=[ 3490],
     | 99.00th=[ 6915], 99.50th=[12780], 99.90th=[28443], 99.95th=[28443],
     | 99.99th=[28443]
   bw (  KiB/s): min= 1272, max= 1784, per=26.27%, avg=1528.00, stdev=362.04, samples=2
   iops        : min=  318, max=  446, avg=382.00, stdev=90.51, samples=2
  lat (usec)   : 4=0.65%, 10=10.42%, 20=7.55%, 50=0.39%, 100=2.60%
  lat (usec)   : 250=22.79%, 500=2.73%, 750=2.86%, 1000=9.51%
  lat (msec)   : 2=16.02%, 4=19.66%, 10=3.78%, 20=0.78%, 50=0.26%
  cpu          : usr=0.58%, sys=17.85%, ctx=806, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15876: Thu Jun 20 21:06:43 2024
   read: IOPS=361, BW=1444KiB/s (1479kB/s)(1564KiB/1083msec)
    clat (usec): min=3, max=23381, avg=1636.90, stdev=2218.27
     lat (usec): min=3, max=23381, avg=1637.74, stdev=2218.47
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[   64], 50.00th=[ 1369], 60.00th=[ 1778],
     | 70.00th=[ 2278], 80.00th=[ 2769], 90.00th=[ 3326], 95.00th=[ 5080],
     | 99.00th=[10552], 99.50th=[14877], 99.90th=[23462], 99.95th=[23462],
     | 99.99th=[23462]
   bw (  KiB/s): min= 1088, max= 1592, per=24.24%, avg=1340.00, stdev=356.38, samples=2
   iops        : min=  272, max=  398, avg=335.00, stdev=89.10, samples=2
  write: IOPS=348, BW=1392KiB/s (1426kB/s)(1508KiB/1083msec)
    clat (usec): min=90, max=17983, avg=1140.26, stdev=1695.40
     lat (usec): min=90, max=17984, avg=1141.22, stdev=1695.33
    clat percentiles (usec):
     |  1.00th=[   91],  5.00th=[  102], 10.00th=[  114], 20.00th=[  147],
     | 30.00th=[  172], 40.00th=[  200], 50.00th=[  709], 60.00th=[  947],
     | 70.00th=[ 1270], 80.00th=[ 1893], 90.00th=[ 2900], 95.00th=[ 3785],
     | 99.00th=[ 5997], 99.50th=[15533], 99.90th=[17957], 99.95th=[17957],
     | 99.99th=[17957]
   bw (  KiB/s): min= 1064, max= 1584, per=22.76%, avg=1324.00, stdev=367.70, samples=2
   iops        : min=  266, max=  396, avg=331.00, stdev=91.92, samples=2
  lat (usec)   : 4=0.26%, 10=10.29%, 20=9.37%, 50=0.26%, 100=2.34%
  lat (usec)   : 250=19.53%, 500=2.86%, 750=1.56%, 1000=7.42%
  lat (msec)   : 2=18.75%, 4=22.01%, 10=4.43%, 20=0.78%, 50=0.13%
  cpu          : usr=0.18%, sys=15.99%, ctx=921, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15877: Thu Jun 20 21:06:43 2024
   read: IOPS=371, BW=1487KiB/s (1523kB/s)(1508KiB/1014msec)
    clat (usec): min=3, max=24991, avg=1570.14, stdev=2570.88
     lat (usec): min=3, max=24992, avg=1570.69, stdev=2570.93
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    8], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[   15], 50.00th=[ 1012], 60.00th=[ 1680],
     | 70.00th=[ 2245], 80.00th=[ 2671], 90.00th=[ 3228], 95.00th=[ 4621],
     | 99.00th=[15533], 99.50th=[22938], 99.90th=[25035], 99.95th=[25035],
     | 99.99th=[25035]
   bw (  KiB/s): min= 1176, max= 1712, per=26.12%, avg=1444.00, stdev=379.01, samples=2
   iops        : min=  294, max=  428, avg=361.00, stdev=94.75, samples=2
  write: IOPS=385, BW=1542KiB/s (1579kB/s)(1564KiB/1014msec)
    clat (usec): min=92, max=11622, avg=1038.56, stdev=1432.15
     lat (usec): min=93, max=11623, avg=1039.22, stdev=1432.19
    clat percentiles (usec):
     |  1.00th=[   95],  5.00th=[  104], 10.00th=[  115], 20.00th=[  137],
     | 30.00th=[  163], 40.00th=[  186], 50.00th=[  330], 60.00th=[  816],
     | 70.00th=[ 1074], 80.00th=[ 1860], 90.00th=[ 2802], 95.00th=[ 3490],
     | 99.00th=[ 6456], 99.50th=[ 9372], 99.90th=[11600], 99.95th=[11600],
     | 99.99th=[11600]
   bw (  KiB/s): min= 1168, max= 1880, per=26.20%, avg=1524.00, stdev=503.46, samples=2
   iops        : min=  292, max=  470, avg=381.00, stdev=125.87, samples=2
  lat (usec)   : 4=0.91%, 10=11.46%, 20=8.72%, 50=0.39%, 100=2.21%
  lat (usec)   : 250=22.53%, 500=2.60%, 750=2.47%, 1000=7.29%
  lat (msec)   : 2=15.62%, 4=20.18%, 10=4.95%, 20=0.39%, 50=0.26%
  cpu          : usr=0.30%, sys=18.16%, ctx=867, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=5529KiB/s (5662kB/s), 1391KiB/s-1487KiB/s (1425kB/s-1523kB/s), io=5988KiB (6132kB), run=1001-1083msec
  WRITE: bw=5817KiB/s (5957kB/s), 1392KiB/s-1590KiB/s (1426kB/s-1629kB/s), io=6300KiB (6451kB), run=1001-1083msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=15878: Thu Jun 20 21:06:43 2024
   read: IOPS=216, BW=864KiB/s (885kB/s)(1480KiB/1712msec)
    clat (usec): min=527, max=40408, avg=1583.23, stdev=2538.37
     lat (usec): min=527, max=40410, avg=1583.90, stdev=2538.43
    clat percentiles (usec):
     |  1.00th=[  635],  5.00th=[  676], 10.00th=[  709], 20.00th=[  766],
     | 30.00th=[  840], 40.00th=[  898], 50.00th=[  955], 60.00th=[ 1012],
     | 70.00th=[ 1172], 80.00th=[ 1565], 90.00th=[ 3195], 95.00th=[ 4424],
     | 99.00th=[ 9372], 99.50th=[15795], 99.90th=[40633], 99.95th=[40633],
     | 99.99th=[40633]
   bw (  KiB/s): min=  592, max= 1109, per=22.92%, avg=801.67, stdev=271.99, samples=3
   iops        : min=  148, max=  277, avg=200.33, stdev=67.86, samples=3
  write: IOPS=232, BW=930KiB/s (952kB/s)(1592KiB/1712msec)
    clat (usec): min=1163, max=25434, avg=2806.99, stdev=1843.58
     lat (usec): min=1164, max=25434, avg=2807.90, stdev=1843.63
    clat percentiles (usec):
     |  1.00th=[ 1270],  5.00th=[ 1532], 10.00th=[ 1713], 20.00th=[ 1909],
     | 30.00th=[ 2008], 40.00th=[ 2089], 50.00th=[ 2212], 60.00th=[ 2343],
     | 70.00th=[ 2671], 80.00th=[ 3458], 90.00th=[ 4752], 95.00th=[ 5866],
     | 99.00th=[ 8586], 99.50th=[14615], 99.90th=[25560], 99.95th=[25560],
     | 99.99th=[25560]
   bw (  KiB/s): min=  584, max= 1317, per=23.75%, avg=873.67, stdev=389.91, samples=3
   iops        : min=  146, max=  329, avg=218.33, stdev=97.34, samples=3
  lat (usec)   : 750=8.07%, 1000=19.92%
  lat (msec)   : 2=26.82%, 4=33.46%, 10=11.07%, 20=0.39%, 50=0.26%
  cpu          : usr=0.53%, sys=5.49%, ctx=817, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15879: Thu Jun 20 21:06:43 2024
   read: IOPS=210, BW=843KiB/s (863kB/s)(1436KiB/1704msec)
    clat (usec): min=589, max=38836, avg=1523.41, stdev=2546.11
     lat (usec): min=589, max=38837, avg=1524.08, stdev=2546.13
    clat percentiles (usec):
     |  1.00th=[  635],  5.00th=[  693], 10.00th=[  742], 20.00th=[  783],
     | 30.00th=[  832], 40.00th=[  889], 50.00th=[  955], 60.00th=[ 1029],
     | 70.00th=[ 1205], 80.00th=[ 1582], 90.00th=[ 2343], 95.00th=[ 3916],
     | 99.00th=[10290], 99.50th=[17433], 99.90th=[39060], 99.95th=[39060],
     | 99.99th=[39060]
   bw (  KiB/s): min=  544, max= 1248, per=22.80%, avg=797.33, stdev=391.29, samples=3
   iops        : min=  136, max=  312, avg=199.33, stdev=97.82, samples=3
  write: IOPS=240, BW=960KiB/s (983kB/s)(1636KiB/1704msec)
    clat (usec): min=1077, max=21778, avg=2808.25, stdev=1824.48
     lat (usec): min=1078, max=21779, avg=2809.03, stdev=1824.52
    clat percentiles (usec):
     |  1.00th=[ 1319],  5.00th=[ 1500], 10.00th=[ 1680], 20.00th=[ 1876],
     | 30.00th=[ 1991], 40.00th=[ 2089], 50.00th=[ 2212], 60.00th=[ 2376],
     | 70.00th=[ 2835], 80.00th=[ 3523], 90.00th=[ 4686], 95.00th=[ 5538],
     | 99.00th=[ 8979], 99.50th=[10552], 99.90th=[21890], 99.95th=[21890],
     | 99.99th=[21890]
   bw (  KiB/s): min=  608, max= 1256, per=24.28%, avg=893.33, stdev=330.85, samples=3
   iops        : min=  152, max=  314, avg=223.33, stdev=82.71, samples=3
  lat (usec)   : 750=5.60%, 1000=21.22%
  lat (msec)   : 2=29.69%, 4=33.98%, 10=8.46%, 20=0.78%, 50=0.26%
  cpu          : usr=0.18%, sys=5.81%, ctx=807, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15880: Thu Jun 20 21:06:43 2024
   read: IOPS=233, BW=933KiB/s (955kB/s)(1564KiB/1677msec)
    clat (usec): min=576, max=22817, avg=1508.54, stdev=1820.66
     lat (usec): min=576, max=22818, avg=1509.18, stdev=1820.67
    clat percentiles (usec):
     |  1.00th=[  627],  5.00th=[  685], 10.00th=[  742], 20.00th=[  799],
     | 30.00th=[  832], 40.00th=[  889], 50.00th=[  947], 60.00th=[ 1012],
     | 70.00th=[ 1205], 80.00th=[ 1745], 90.00th=[ 3064], 95.00th=[ 4080],
     | 99.00th=[10290], 99.50th=[14746], 99.90th=[22938], 99.95th=[22938],
     | 99.99th=[22938]
   bw (  KiB/s): min=  504, max= 1296, per=24.63%, avg=861.33, stdev=401.62, samples=3
   iops        : min=  126, max=  324, avg=215.33, stdev=100.41, samples=3
  write: IOPS=224, BW=899KiB/s (921kB/s)(1508KiB/1677msec)
    clat (usec): min=1114, max=50028, avg=2863.45, stdev=2778.36
     lat (usec): min=1115, max=50031, avg=2864.20, stdev=2778.46
    clat percentiles (usec):
     |  1.00th=[ 1221],  5.00th=[ 1418], 10.00th=[ 1647], 20.00th=[ 1860],
     | 30.00th=[ 2008], 40.00th=[ 2114], 50.00th=[ 2245], 60.00th=[ 2442],
     | 70.00th=[ 2802], 80.00th=[ 3490], 90.00th=[ 4686], 95.00th=[ 5538],
     | 99.00th=[ 7832], 99.50th=[ 9896], 99.90th=[50070], 99.95th=[50070],
     | 99.99th=[50070]
   bw (  KiB/s): min=  632, max= 1280, per=23.27%, avg=856.00, stdev=367.39, samples=3
   iops        : min=  158, max=  320, avg=214.00, stdev=91.85, samples=3
  lat (usec)   : 750=6.12%, 1000=23.18%
  lat (msec)   : 2=27.60%, 4=32.55%, 10=9.90%, 20=0.39%, 50=0.13%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.24%, sys=6.32%, ctx=813, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=15881: Thu Jun 20 21:06:43 2024
   read: IOPS=221, BW=887KiB/s (908kB/s)(1508KiB/1701msec)
    clat (usec): min=581, max=21318, avg=1519.02, stdev=1614.74
     lat (usec): min=581, max=21319, avg=1519.65, stdev=1614.74
    clat percentiles (usec):
     |  1.00th=[  644],  5.00th=[  693], 10.00th=[  725], 20.00th=[  799],
     | 30.00th=[  857], 40.00th=[  906], 50.00th=[  971], 60.00th=[ 1057],
     | 70.00th=[ 1237], 80.00th=[ 1795], 90.00th=[ 3261], 95.00th=[ 4359],
     | 99.00th=[ 6718], 99.50th=[10028], 99.90th=[21365], 99.95th=[21365],
     | 99.99th=[21365]
   bw (  KiB/s): min=  534, max= 1168, per=23.32%, avg=815.33, stdev=322.96, samples=3
   iops        : min=  133, max=  292, avg=203.67, stdev=80.96, samples=3
  write: IOPS=229, BW=919KiB/s (942kB/s)(1564KiB/1701msec)
    clat (usec): min=1037, max=52492, avg=2863.25, stdev=3006.30
     lat (usec): min=1038, max=52493, avg=2864.16, stdev=3006.33
    clat percentiles (usec):
     |  1.00th=[ 1303],  5.00th=[ 1450], 10.00th=[ 1663], 20.00th=[ 1844],
     | 30.00th=[ 1975], 40.00th=[ 2057], 50.00th=[ 2180], 60.00th=[ 2376],
     | 70.00th=[ 2737], 80.00th=[ 3490], 90.00th=[ 4555], 95.00th=[ 5538],
     | 99.00th=[ 8586], 99.50th=[21365], 99.90th=[52691], 99.95th=[52691],
     | 99.99th=[52691]
   bw (  KiB/s): min=  590, max= 1304, per=23.39%, avg=860.67, stdev=387.05, samples=3
   iops        : min=  147, max=  326, avg=215.00, stdev=96.94, samples=3
  lat (usec)   : 750=6.51%, 1000=20.31%
  lat (msec)   : 2=30.73%, 4=31.77%, 10=10.03%, 20=0.26%, 50=0.26%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.47%, sys=5.76%, ctx=821, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3498KiB/s (3582kB/s), 843KiB/s-933KiB/s (863kB/s-955kB/s), io=5988KiB (6132kB), run=1677-1712msec
  WRITE: bw=3680KiB/s (3768kB/s), 899KiB/s-960KiB/s (921kB/s-983kB/s), io=6300KiB (6451kB), run=1677-1712msec