== sanity test 398b: DIO and buffer IO race ============== 08:23:10 (1717158190)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0845552 s, 149 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=14291: Fri May 31 08:23:13 2024
   read: IOPS=329, BW=1319KiB/s (1351kB/s)(1480KiB/1122msec)
    clat (usec): min=3, max=29405, avg=1753.21, stdev=3121.96
     lat (usec): min=3, max=29406, avg=1753.98, stdev=3122.12
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[    9],
     | 30.00th=[   11], 40.00th=[  701], 50.00th=[ 1287], 60.00th=[ 1696],
     | 70.00th=[ 2073], 80.00th=[ 2573], 90.00th=[ 3752], 95.00th=[ 5080],
     | 99.00th=[25035], 99.50th=[26608], 99.90th=[29492], 99.95th=[29492],
     | 99.99th=[29492]
   bw (  KiB/s): min=  616, max= 1640, per=22.40%, avg=1128.00, stdev=724.08, samples=2
   iops        : min=  154, max=  410, avg=282.00, stdev=181.02, samples=2
  write: IOPS=354, BW=1419KiB/s (1453kB/s)(1592KiB/1122msec)
    clat (usec): min=78, max=38515, avg=1140.97, stdev=2357.00
     lat (usec): min=78, max=38516, avg=1141.56, stdev=2357.05
    clat percentiles (usec):
     |  1.00th=[   86],  5.00th=[   93], 10.00th=[  101], 20.00th=[  111],
     | 30.00th=[  131], 40.00th=[  184], 50.00th=[  619], 60.00th=[  807],
     | 70.00th=[ 1057], 80.00th=[ 1745], 90.00th=[ 2835], 95.00th=[ 3916],
     | 99.00th=[ 7111], 99.50th=[10945], 99.90th=[38536], 99.95th=[38536],
     | 99.99th=[38536]
   bw (  KiB/s): min=  624, max= 1800, per=22.88%, avg=1212.00, stdev=831.56, samples=2
   iops        : min=  156, max=  450, avg=303.00, stdev=207.89, samples=2
  lat (usec)   : 4=0.65%, 10=12.37%, 20=4.82%, 50=0.78%, 100=5.21%
  lat (usec)   : 250=18.23%, 500=1.56%, 750=5.34%, 1000=8.33%
  lat (msec)   : 2=18.10%, 4=17.45%, 10=6.12%, 20=0.39%, 50=0.65%
  cpu          : usr=0.09%, sys=16.95%, ctx=843, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14292: Fri May 31 08:23:13 2024
   read: IOPS=321, BW=1286KiB/s (1316kB/s)(1436KiB/1117msec)
    clat (usec): min=3, max=37729, avg=1598.20, stdev=2708.30
     lat (usec): min=3, max=37730, avg=1598.72, stdev=2708.38
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    7], 20.00th=[   10],
     | 30.00th=[   11], 40.00th=[  392], 50.00th=[ 1303], 60.00th=[ 1713],
     | 70.00th=[ 2040], 80.00th=[ 2442], 90.00th=[ 3064], 95.00th=[ 5014],
     | 99.00th=[ 9896], 99.50th=[17957], 99.90th=[37487], 99.95th=[37487],
     | 99.99th=[37487]
   bw (  KiB/s): min=  592, max= 1728, per=23.03%, avg=1160.00, stdev=803.27, samples=2
   iops        : min=  148, max=  432, avg=290.00, stdev=200.82, samples=2
  write: IOPS=366, BW=1465KiB/s (1500kB/s)(1636KiB/1117msec)
    clat (usec): min=82, max=27794, avg=1259.80, stdev=2479.17
     lat (usec): min=83, max=27796, avg=1261.45, stdev=2479.46
    clat percentiles (usec):
     |  1.00th=[   90],  5.00th=[   98], 10.00th=[  103], 20.00th=[  116],
     | 30.00th=[  137], 40.00th=[  223], 50.00th=[  701], 60.00th=[  865],
     | 70.00th=[ 1172], 80.00th=[ 1713], 90.00th=[ 2835], 95.00th=[ 4178],
     | 99.00th=[12256], 99.50th=[17433], 99.90th=[27919], 99.95th=[27919],
     | 99.99th=[27919]
   bw (  KiB/s): min=  648, max= 1832, per=23.41%, avg=1240.00, stdev=837.21, samples=2
   iops        : min=  162, max=  458, avg=310.00, stdev=209.30, samples=2
  lat (usec)   : 4=0.52%, 10=11.59%, 20=5.47%, 50=0.65%, 100=3.78%
  lat (usec)   : 250=18.36%, 500=1.56%, 750=5.60%, 1000=8.20%
  lat (msec)   : 2=21.48%, 4=16.41%, 10=5.34%, 20=0.65%, 50=0.39%
  cpu          : usr=0.72%, sys=14.70%, ctx=851, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14294: Fri May 31 08:23:13 2024
   read: IOPS=335, BW=1342KiB/s (1375kB/s)(1564KiB/1165msec)
    clat (usec): min=3, max=35825, avg=1842.59, stdev=2943.75
     lat (usec): min=3, max=35826, avg=1844.97, stdev=2943.53
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    7], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[  914], 50.00th=[ 1450], 60.00th=[ 1844],
     | 70.00th=[ 2212], 80.00th=[ 2704], 90.00th=[ 3785], 95.00th=[ 5080],
     | 99.00th=[16057], 99.50th=[19268], 99.90th=[35914], 99.95th=[35914],
     | 99.99th=[35914]
   bw (  KiB/s): min=  784, max= 1464, per=22.32%, avg=1124.00, stdev=480.83, samples=2
   iops        : min=  196, max=  366, avg=281.00, stdev=120.21, samples=2
  write: IOPS=323, BW=1294KiB/s (1325kB/s)(1508KiB/1165msec)
    clat (usec): min=76, max=15309, avg=1150.84, stdev=1737.15
     lat (usec): min=77, max=15310, avg=1151.72, stdev=1737.08
    clat percentiles (usec):
     |  1.00th=[   91],  5.00th=[   97], 10.00th=[  105], 20.00th=[  119],
     | 30.00th=[  147], 40.00th=[  212], 50.00th=[  619], 60.00th=[  848],
     | 70.00th=[ 1287], 80.00th=[ 1778], 90.00th=[ 3163], 95.00th=[ 4228],
     | 99.00th=[ 8029], 99.50th=[12387], 99.90th=[15270], 99.95th=[15270],
     | 99.99th=[15270]
   bw (  KiB/s): min=  872, max= 1296, per=20.46%, avg=1084.00, stdev=299.81, samples=2
   iops        : min=  218, max=  324, avg=271.00, stdev=74.95, samples=2
  lat (usec)   : 4=0.78%, 10=11.46%, 20=5.21%, 50=0.52%, 100=3.12%
  lat (usec)   : 250=18.49%, 500=2.86%, 750=4.04%, 1000=6.90%
  lat (msec)   : 2=19.53%, 4=20.18%, 10=5.60%, 20=1.17%, 50=0.13%
  cpu          : usr=0.34%, sys=15.89%, ctx=930, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14296: Fri May 31 08:23:13 2024
   read: IOPS=317, BW=1268KiB/s (1299kB/s)(1508KiB/1189msec)
    clat (usec): min=3, max=52204, avg=1817.30, stdev=3319.26
     lat (usec): min=3, max=52205, avg=1817.84, stdev=3319.34
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    6], 10.00th=[    7], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[  922], 50.00th=[ 1434], 60.00th=[ 1811],
     | 70.00th=[ 2212], 80.00th=[ 2704], 90.00th=[ 3556], 95.00th=[ 5211],
     | 99.00th=[11600], 99.50th=[17695], 99.90th=[52167], 99.95th=[52167],
     | 99.99th=[52167]
   bw (  KiB/s): min=  704, max= 1448, per=21.37%, avg=1076.00, stdev=526.09, samples=2
   iops        : min=  176, max=  362, avg=269.00, stdev=131.52, samples=2
  write: IOPS=328, BW=1315KiB/s (1347kB/s)(1564KiB/1189msec)
    clat (usec): min=82, max=11389, avg=1216.77, stdev=1585.23
     lat (usec): min=82, max=11390, avg=1217.41, stdev=1585.33
    clat percentiles (usec):
     |  1.00th=[   92],  5.00th=[  101], 10.00th=[  108], 20.00th=[  122],
     | 30.00th=[  165], 40.00th=[  277], 50.00th=[  750], 60.00th=[  963],
     | 70.00th=[ 1565], 80.00th=[ 1991], 90.00th=[ 2802], 95.00th=[ 3851],
     | 99.00th=[ 8160], 99.50th=[10290], 99.90th=[11338], 99.95th=[11338],
     | 99.99th=[11338]
   bw (  KiB/s): min=  728, max= 1584, per=21.82%, avg=1156.00, stdev=605.28, samples=2
   iops        : min=  182, max=  396, avg=289.00, stdev=151.32, samples=2
  lat (usec)   : 4=1.17%, 10=11.33%, 20=4.04%, 50=0.65%, 100=2.21%
  lat (usec)   : 250=17.45%, 500=3.65%, 750=3.39%, 1000=7.03%
  lat (msec)   : 2=21.61%, 4=21.22%, 10=5.47%, 20=0.65%, 100=0.13%
  cpu          : usr=0.17%, sys=16.67%, ctx=980, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=5036KiB/s (5157kB/s), 1268KiB/s-1342KiB/s (1299kB/s-1375kB/s), io=5988KiB (6132kB), run=1117-1189msec
  WRITE: bw=5299KiB/s (5426kB/s), 1294KiB/s-1465KiB/s (1325kB/s-1500kB/s), io=6300KiB (6451kB), run=1117-1189msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=14293: Fri May 31 08:23:13 2024
   read: IOPS=213, BW=852KiB/s (872kB/s)(1480KiB/1737msec)
    clat (usec): min=505, max=38236, avg=1701.66, stdev=2803.57
     lat (usec): min=506, max=38237, avg=1702.32, stdev=2803.65
    clat percentiles (usec):
     |  1.00th=[  562],  5.00th=[  627], 10.00th=[  676], 20.00th=[  750],
     | 30.00th=[  799], 40.00th=[  848], 50.00th=[  906], 60.00th=[ 1057],
     | 70.00th=[ 1336], 80.00th=[ 1893], 90.00th=[ 3326], 95.00th=[ 5014],
     | 99.00th=[14746], 99.50th=[21890], 99.90th=[38011], 99.95th=[38011],
     | 99.99th=[38011]
   bw (  KiB/s): min=  432, max= 1072, per=22.68%, avg=768.00, stdev=321.20, samples=3
   iops        : min=  108, max=  268, avg=192.00, stdev=80.30, samples=3
  write: IOPS=229, BW=917KiB/s (939kB/s)(1592KiB/1737msec)
    clat (usec): min=1157, max=17052, avg=2708.51, stdev=1660.08
     lat (usec): min=1157, max=17054, avg=2709.31, stdev=1660.15
    clat percentiles (usec):
     |  1.00th=[ 1221],  5.00th=[ 1401], 10.00th=[ 1598], 20.00th=[ 1778],
     | 30.00th=[ 1876], 40.00th=[ 1975], 50.00th=[ 2089], 60.00th=[ 2278],
     | 70.00th=[ 2606], 80.00th=[ 3359], 90.00th=[ 4883], 95.00th=[ 5866],
     | 99.00th=[ 9896], 99.50th=[12256], 99.90th=[17171], 99.95th=[17171],
     | 99.99th=[17171]
   bw (  KiB/s): min=  472, max= 1280, per=23.43%, avg=834.67, stdev=410.29, samples=3
   iops        : min=  118, max=  320, avg=208.67, stdev=102.57, samples=3
  lat (usec)   : 750=9.51%, 1000=18.49%
  lat (msec)   : 2=32.94%, 4=28.26%, 10=9.77%, 20=0.78%, 50=0.26%
  cpu          : usr=0.40%, sys=5.93%, ctx=838, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14295: Fri May 31 08:23:13 2024
   read: IOPS=203, BW=812KiB/s (832kB/s)(1436KiB/1768msec)
    clat (usec): min=475, max=55751, avg=1649.92, stdev=3571.55
     lat (usec): min=475, max=55752, avg=1650.70, stdev=3571.56
    clat percentiles (usec):
     |  1.00th=[  553],  5.00th=[  611], 10.00th=[  668], 20.00th=[  725],
     | 30.00th=[  766], 40.00th=[  816], 50.00th=[  898], 60.00th=[  988],
     | 70.00th=[ 1123], 80.00th=[ 1549], 90.00th=[ 2933], 95.00th=[ 4686],
     | 99.00th=[12256], 99.50th=[28705], 99.90th=[55837], 99.95th=[55837],
     | 99.99th=[55837]
   bw (  KiB/s): min=  384, max= 1261, per=22.42%, avg=759.00, stdev=452.08, samples=3
   iops        : min=   96, max=  315, avg=189.67, stdev=112.88, samples=3
  write: IOPS=231, BW=925KiB/s (948kB/s)(1636KiB/1768msec)
    clat (usec): min=1052, max=27230, avg=2857.41, stdev=2367.53
     lat (usec): min=1052, max=27232, avg=2858.13, stdev=2367.61
    clat percentiles (usec):
     |  1.00th=[ 1188],  5.00th=[ 1336], 10.00th=[ 1582], 20.00th=[ 1778],
     | 30.00th=[ 1893], 40.00th=[ 1975], 50.00th=[ 2089], 60.00th=[ 2278],
     | 70.00th=[ 2802], 80.00th=[ 3523], 90.00th=[ 4817], 95.00th=[ 5669],
     | 99.00th=[14353], 99.50th=[17433], 99.90th=[27132], 99.95th=[27132],
     | 99.99th=[27132]
   bw (  KiB/s): min=  392, max= 1213, per=22.87%, avg=815.00, stdev=411.07, samples=3
   iops        : min=   98, max=  303, avg=203.67, stdev=102.65, samples=3
  lat (usec)   : 500=0.13%, 750=11.59%, 1000=17.19%
  lat (msec)   : 2=33.59%, 4=25.91%, 10=9.77%, 20=1.43%, 50=0.26%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.57%, sys=5.32%, ctx=831, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14297: Fri May 31 08:23:13 2024
   read: IOPS=224, BW=899KiB/s (920kB/s)(1564KiB/1740msec)
    clat (usec): min=517, max=24817, avg=1631.14, stdev=2120.13
     lat (usec): min=517, max=24818, avg=1631.72, stdev=2120.17
    clat percentiles (usec):
     |  1.00th=[  562],  5.00th=[  635], 10.00th=[  685], 20.00th=[  750],
     | 30.00th=[  799], 40.00th=[  889], 50.00th=[  963], 60.00th=[ 1074],
     | 70.00th=[ 1434], 80.00th=[ 1991], 90.00th=[ 3163], 95.00th=[ 4359],
     | 99.00th=[ 9503], 99.50th=[22414], 99.90th=[24773], 99.95th=[24773],
     | 99.99th=[24773]
   bw (  KiB/s): min=  384, max= 1128, per=23.31%, avg=789.33, stdev=376.45, samples=3
   iops        : min=   96, max=  282, avg=197.33, stdev=94.11, samples=3
  write: IOPS=216, BW=867KiB/s (887kB/s)(1508KiB/1740msec)
    clat (usec): min=1050, max=41751, avg=2846.36, stdev=2581.75
     lat (usec): min=1051, max=41752, avg=2847.05, stdev=2581.83
    clat percentiles (usec):
     |  1.00th=[ 1156],  5.00th=[ 1336], 10.00th=[ 1598], 20.00th=[ 1762],
     | 30.00th=[ 1860], 40.00th=[ 1958], 50.00th=[ 2089], 60.00th=[ 2311],
     | 70.00th=[ 2835], 80.00th=[ 3687], 90.00th=[ 5014], 95.00th=[ 5800],
     | 99.00th=[ 9241], 99.50th=[15008], 99.90th=[41681], 99.95th=[41681],
     | 99.99th=[41681]
   bw (  KiB/s): min=  480, max= 1168, per=22.08%, avg=786.67, stdev=350.02, samples=3
   iops        : min=  120, max=  292, avg=196.67, stdev=87.51, samples=3
  lat (usec)   : 750=10.42%, 1000=17.71%
  lat (msec)   : 2=34.37%, 4=25.26%, 10=11.46%, 20=0.39%, 50=0.39%
  cpu          : usr=0.46%, sys=5.81%, ctx=846, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=14298: Fri May 31 08:23:13 2024
   read: IOPS=215, BW=861KiB/s (882kB/s)(1508KiB/1751msec)
    clat (usec): min=518, max=27477, avg=1622.50, stdev=2143.11
     lat (usec): min=519, max=27478, avg=1623.32, stdev=2143.06
    clat percentiles (usec):
     |  1.00th=[  553],  5.00th=[  635], 10.00th=[  668], 20.00th=[  742],
     | 30.00th=[  791], 40.00th=[  865], 50.00th=[  938], 60.00th=[ 1074],
     | 70.00th=[ 1336], 80.00th=[ 1876], 90.00th=[ 3392], 95.00th=[ 4948],
     | 99.00th=[12649], 99.50th=[15401], 99.90th=[27395], 99.95th=[27395],
     | 99.99th=[27395]
   bw (  KiB/s): min=  376, max= 1072, per=21.97%, avg=744.00, stdev=349.72, samples=3
   iops        : min=   94, max=  268, avg=186.00, stdev=87.43, samples=3
  write: IOPS=223, BW=893KiB/s (915kB/s)(1564KiB/1751msec)
    clat (usec): min=1015, max=44627, avg=2836.16, stdev=2919.17
     lat (usec): min=1015, max=44630, avg=2836.99, stdev=2919.30
    clat percentiles (usec):
     |  1.00th=[ 1172],  5.00th=[ 1385], 10.00th=[ 1647], 20.00th=[ 1795],
     | 30.00th=[ 1876], 40.00th=[ 1958], 50.00th=[ 2057], 60.00th=[ 2278],
     | 70.00th=[ 2835], 80.00th=[ 3425], 90.00th=[ 4555], 95.00th=[ 5473],
     | 99.00th=[11076], 99.50th=[28967], 99.90th=[44827], 99.95th=[44827],
     | 99.99th=[44827]
   bw (  KiB/s): min=  448, max= 1248, per=22.53%, avg=802.67, stdev=407.63, samples=3
   iops        : min=  112, max=  312, avg=200.67, stdev=101.91, samples=3
  lat (usec)   : 750=10.55%, 1000=17.58%
  lat (msec)   : 2=34.24%, 4=26.30%, 10=10.29%, 20=0.65%, 50=0.39%
  cpu          : usr=0.51%, sys=5.94%, ctx=830, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3387KiB/s (3468kB/s), 812KiB/s-899KiB/s (832kB/s-920kB/s), io=5988KiB (6132kB), run=1737-1768msec
  WRITE: bw=3563KiB/s (3649kB/s), 867KiB/s-925KiB/s (887kB/s-948kB/s), io=6300KiB (6451kB), run=1737-1768msec