== sanity test 398b: DIO and buffer IO race ============== 09:35:00 (1721223300)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0915489 s, 137 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=21547: Wed Jul 17 09:35:03 2024
   read: IOPS=320, BW=1281KiB/s (1312kB/s)(1480KiB/1155msec)
    clat (usec): min=3, max=26895, avg=1766.30, stdev=2527.86
     lat (usec): min=3, max=26896, avg=1766.87, stdev=2527.94
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   16], 40.00th=[ 1020], 50.00th=[ 1434], 60.00th=[ 1696],
     | 70.00th=[ 2114], 80.00th=[ 2671], 90.00th=[ 3654], 95.00th=[ 5407],
     | 99.00th=[10945], 99.50th=[23200], 99.90th=[26870], 99.95th=[26870],
     | 99.99th=[26870]
   bw (  KiB/s): min=  776, max= 1540, per=22.78%, avg=1158.00, stdev=540.23, samples=2
   iops        : min=  194, max=  385, avg=289.50, stdev=135.06, samples=2
  write: IOPS=344, BW=1378KiB/s (1411kB/s)(1592KiB/1155msec)
    clat (usec): min=83, max=58107, avg=1236.74, stdev=3128.21
     lat (usec): min=83, max=58108, avg=1237.40, stdev=3128.25
    clat percentiles (usec):
     |  1.00th=[   94],  5.00th=[  102], 10.00th=[  115], 20.00th=[  137],
     | 30.00th=[  174], 40.00th=[  289], 50.00th=[  725], 60.00th=[  955],
     | 70.00th=[ 1270], 80.00th=[ 1844], 90.00th=[ 2868], 95.00th=[ 3720],
     | 99.00th=[ 6783], 99.50th=[ 7111], 99.90th=[57934], 99.95th=[57934],
     | 99.99th=[57934]
   bw (  KiB/s): min=  800, max= 1716, per=23.52%, avg=1258.00, stdev=647.71, samples=2
   iops        : min=  200, max=  429, avg=314.50, stdev=161.93, samples=2
  lat (usec)   : 4=0.13%, 10=7.29%, 20=7.42%, 50=0.91%, 100=1.95%
  lat (usec)   : 250=18.49%, 500=2.60%, 750=4.69%, 1000=8.07%
  lat (msec)   : 2=23.57%, 4=18.49%, 10=5.60%, 20=0.39%, 50=0.26%
  lat (msec)   : 100=0.13%
  cpu          : usr=0.43%, sys=15.42%, ctx=895, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21548: Wed Jul 17 09:35:03 2024
   read: IOPS=307, BW=1232KiB/s (1261kB/s)(1436KiB/1166msec)
    clat (usec): min=3, max=35693, avg=1860.94, stdev=3088.11
     lat (usec): min=4, max=35694, avg=1862.18, stdev=3088.24
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   14], 40.00th=[  807], 50.00th=[ 1483], 60.00th=[ 1827],
     | 70.00th=[ 2147], 80.00th=[ 2671], 90.00th=[ 3785], 95.00th=[ 5276],
     | 99.00th=[15401], 99.50th=[24511], 99.90th=[35914], 99.95th=[35914],
     | 99.99th=[35914]
   bw (  KiB/s): min=  800, max= 1261, per=20.27%, avg=1030.50, stdev=325.98, samples=2
   iops        : min=  200, max=  315, avg=257.50, stdev=81.32, samples=2
  write: IOPS=350, BW=1403KiB/s (1437kB/s)(1636KiB/1166msec)
    clat (usec): min=83, max=9938, avg=1173.86, stdev=1500.42
     lat (usec): min=83, max=9939, avg=1174.64, stdev=1500.91
    clat percentiles (usec):
     |  1.00th=[   93],  5.00th=[  104], 10.00th=[  114], 20.00th=[  128],
     | 30.00th=[  167], 40.00th=[  227], 50.00th=[  701], 60.00th=[  906],
     | 70.00th=[ 1270], 80.00th=[ 1860], 90.00th=[ 3097], 95.00th=[ 4424],
     | 99.00th=[ 6980], 99.50th=[ 6980], 99.90th=[ 9896], 99.95th=[ 9896],
     | 99.99th=[ 9896]
   bw (  KiB/s): min=  920, max= 1341, per=21.14%, avg=1130.50, stdev=297.69, samples=2
   iops        : min=  230, max=  335, avg=282.50, stdev=74.25, samples=2
  lat (usec)   : 4=0.13%, 10=8.98%, 20=6.25%, 50=0.65%, 100=2.21%
  lat (usec)   : 250=19.79%, 500=2.60%, 750=4.82%, 1000=7.94%
  lat (msec)   : 2=20.70%, 4=18.36%, 10=6.77%, 20=0.39%, 50=0.39%
  cpu          : usr=0.34%, sys=17.68%, ctx=974, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21549: Wed Jul 17 09:35:03 2024
   read: IOPS=331, BW=1328KiB/s (1360kB/s)(1564KiB/1178msec)
    clat (usec): min=3, max=28430, avg=1910.13, stdev=2993.88
     lat (usec): min=4, max=28431, avg=1910.77, stdev=2994.03
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   13], 40.00th=[  938], 50.00th=[ 1450], 60.00th=[ 1827],
     | 70.00th=[ 2245], 80.00th=[ 2737], 90.00th=[ 3818], 95.00th=[ 5800],
     | 99.00th=[17171], 99.50th=[24511], 99.90th=[28443], 99.95th=[28443],
     | 99.99th=[28443]
   bw (  KiB/s): min=  768, max= 1440, per=21.72%, avg=1104.00, stdev=475.18, samples=2
   iops        : min=  192, max=  360, avg=276.00, stdev=118.79, samples=2
  write: IOPS=320, BW=1280KiB/s (1311kB/s)(1508KiB/1178msec)
    clat (usec): min=88, max=11172, avg=1107.86, stdev=1408.75
     lat (usec): min=88, max=11173, avg=1109.35, stdev=1408.33
    clat percentiles (usec):
     |  1.00th=[   91],  5.00th=[  108], 10.00th=[  114], 20.00th=[  130],
     | 30.00th=[  159], 40.00th=[  243], 50.00th=[  660], 60.00th=[  914],
     | 70.00th=[ 1352], 80.00th=[ 1827], 90.00th=[ 2769], 95.00th=[ 3752],
     | 99.00th=[ 6652], 99.50th=[ 7963], 99.90th=[11207], 99.95th=[11207],
     | 99.99th=[11207]
   bw (  KiB/s): min=  856, max= 1264, per=19.82%, avg=1060.00, stdev=288.50, samples=2
   iops        : min=  214, max=  316, avg=265.00, stdev=72.12, samples=2
  lat (usec)   : 4=0.13%, 10=10.55%, 20=6.25%, 50=0.65%, 100=1.17%
  lat (usec)   : 250=19.27%, 500=2.73%, 750=4.69%, 1000=5.60%
  lat (msec)   : 2=22.66%, 4=19.53%, 10=5.60%, 20=0.91%, 50=0.26%
  cpu          : usr=0.34%, sys=18.44%, ctx=976, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21550: Wed Jul 17 09:35:03 2024
   read: IOPS=332, BW=1330KiB/s (1362kB/s)(1508KiB/1134msec)
    clat (usec): min=3, max=48909, avg=1953.29, stdev=3437.63
     lat (usec): min=3, max=48910, avg=1955.03, stdev=3437.48
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   13], 40.00th=[  971], 50.00th=[ 1467], 60.00th=[ 1975],
     | 70.00th=[ 2343], 80.00th=[ 2737], 90.00th=[ 3818], 95.00th=[ 5800],
     | 99.00th=[11731], 99.50th=[25560], 99.90th=[49021], 99.95th=[49021],
     | 99.99th=[49021]
   bw (  KiB/s): min=  718, max= 1456, per=21.39%, avg=1087.00, stdev=521.84, samples=2
   iops        : min=  179, max=  364, avg=271.50, stdev=130.81, samples=2
  write: IOPS=344, BW=1379KiB/s (1412kB/s)(1564KiB/1134msec)
    clat (usec): min=87, max=10747, avg=983.87, stdev=1336.58
     lat (usec): min=87, max=10748, avg=984.57, stdev=1336.64
    clat percentiles (usec):
     |  1.00th=[   90],  5.00th=[  102], 10.00th=[  109], 20.00th=[  120],
     | 30.00th=[  141], 40.00th=[  190], 50.00th=[  457], 60.00th=[  799],
     | 70.00th=[ 1106], 80.00th=[ 1631], 90.00th=[ 2573], 95.00th=[ 3425],
     | 99.00th=[ 6194], 99.50th=[ 9896], 99.90th=[10683], 99.95th=[10683],
     | 99.99th=[10683]
   bw (  KiB/s): min=  758, max= 1624, per=22.27%, avg=1191.00, stdev=612.35, samples=2
   iops        : min=  189, max=  406, avg=297.50, stdev=153.44, samples=2
  lat (usec)   : 4=0.52%, 10=10.29%, 20=5.08%, 50=0.52%, 100=2.73%
  lat (usec)   : 250=21.09%, 500=2.21%, 750=5.21%, 1000=6.38%
  lat (msec)   : 2=19.14%, 4=20.31%, 10=5.47%, 20=0.78%, 50=0.26%
  cpu          : usr=0.09%, sys=17.21%, ctx=919, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=5083KiB/s (5205kB/s), 1232KiB/s-1330KiB/s (1261kB/s-1362kB/s), io=5988KiB (6132kB), run=1134-1178msec
  WRITE: bw=5348KiB/s (5476kB/s), 1280KiB/s-1403KiB/s (1311kB/s-1437kB/s), io=6300KiB (6451kB), run=1134-1178msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=21552: Wed Jul 17 09:35:03 2024
   read: IOPS=228, BW=914KiB/s (936kB/s)(1480KiB/1619msec)
    clat (usec): min=474, max=27442, avg=1600.86, stdev=2202.48
     lat (usec): min=475, max=27445, avg=1601.44, stdev=2202.59
    clat percentiles (usec):
     |  1.00th=[  515],  5.00th=[  570], 10.00th=[  619], 20.00th=[  676],
     | 30.00th=[  717], 40.00th=[  783], 50.00th=[  840], 60.00th=[  979],
     | 70.00th=[ 1401], 80.00th=[ 1926], 90.00th=[ 3523], 95.00th=[ 4752],
     | 99.00th=[ 8225], 99.50th=[20579], 99.90th=[27395], 99.95th=[27395],
     | 99.99th=[27395]
   bw (  KiB/s): min=  488, max= 1296, per=23.78%, avg=864.00, stdev=406.90, samples=3
   iops        : min=  122, max=  324, avg=216.00, stdev=101.73, samples=3
  write: IOPS=245, BW=983KiB/s (1007kB/s)(1592KiB/1619msec)
    clat (usec): min=1004, max=46436, avg=2556.53, stdev=2647.02
     lat (usec): min=1005, max=46437, avg=2557.26, stdev=2647.08
    clat percentiles (usec):
     |  1.00th=[ 1045],  5.00th=[ 1156], 10.00th=[ 1303], 20.00th=[ 1532],
     | 30.00th=[ 1631], 40.00th=[ 1729], 50.00th=[ 1860], 60.00th=[ 2040],
     | 70.00th=[ 2507], 80.00th=[ 3359], 90.00th=[ 4359], 95.00th=[ 5407],
     | 99.00th=[ 9372], 99.50th=[10290], 99.90th=[46400], 99.95th=[46400],
     | 99.99th=[46400]
   bw (  KiB/s): min=  512, max= 1504, per=24.56%, avg=938.67, stdev=510.33, samples=3
   iops        : min=  128, max=  376, avg=234.67, stdev=127.58, samples=3
  lat (usec)   : 500=0.26%, 750=16.41%, 1000=12.76%
  lat (msec)   : 2=39.32%, 4=21.09%, 10=9.51%, 20=0.26%, 50=0.39%
  cpu          : usr=0.37%, sys=6.67%, ctx=830, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21553: Wed Jul 17 09:35:03 2024
   read: IOPS=218, BW=872KiB/s (893kB/s)(1436KiB/1646msec)
    clat (usec): min=487, max=26322, avg=1571.06, stdev=2325.71
     lat (usec): min=488, max=26323, avg=1571.79, stdev=2325.84
    clat percentiles (usec):
     |  1.00th=[  529],  5.00th=[  578], 10.00th=[  635], 20.00th=[  676],
     | 30.00th=[  734], 40.00th=[  799], 50.00th=[  873], 60.00th=[  979],
     | 70.00th=[ 1139], 80.00th=[ 1778], 90.00th=[ 3359], 95.00th=[ 5014],
     | 99.00th=[ 9241], 99.50th=[25035], 99.90th=[26346], 99.95th=[26346],
     | 99.99th=[26346]
   bw (  KiB/s): min=  456, max= 1336, per=22.83%, avg=829.33, stdev=454.90, samples=3
   iops        : min=  114, max=  334, avg=207.33, stdev=113.72, samples=3
  write: IOPS=248, BW=994KiB/s (1018kB/s)(1636KiB/1646msec)
    clat (usec): min=1005, max=15514, avg=2627.25, stdev=1843.28
     lat (usec): min=1006, max=15516, avg=2628.25, stdev=1843.30
    clat percentiles (usec):
     |  1.00th=[ 1074],  5.00th=[ 1319], 10.00th=[ 1483], 20.00th=[ 1598],
     | 30.00th=[ 1680], 40.00th=[ 1795], 50.00th=[ 1958], 60.00th=[ 2212],
     | 70.00th=[ 2606], 80.00th=[ 3294], 90.00th=[ 4555], 95.00th=[ 5538],
     | 99.00th=[10028], 99.50th=[14222], 99.90th=[15533], 99.95th=[15533],
     | 99.99th=[15533]
   bw (  KiB/s): min=  536, max= 1384, per=24.56%, avg=938.67, stdev=425.61, samples=3
   iops        : min=  134, max=  346, avg=234.67, stdev=106.40, samples=3
  lat (usec)   : 500=0.13%, 750=15.23%, 1000=13.80%
  lat (msec)   : 2=37.37%, 4=21.74%, 10=10.81%, 20=0.65%, 50=0.26%
  cpu          : usr=0.30%, sys=6.08%, ctx=856, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21554: Wed Jul 17 09:35:03 2024
   read: IOPS=243, BW=973KiB/s (996kB/s)(1564KiB/1608msec)
    clat (usec): min=469, max=13296, avg=1469.93, stdev=1563.43
     lat (usec): min=470, max=13297, avg=1470.53, stdev=1563.49
    clat percentiles (usec):
     |  1.00th=[  519],  5.00th=[  586], 10.00th=[  619], 20.00th=[  676],
     | 30.00th=[  725], 40.00th=[  783], 50.00th=[  865], 60.00th=[  988],
     | 70.00th=[ 1319], 80.00th=[ 1876], 90.00th=[ 3261], 95.00th=[ 4621],
     | 99.00th=[ 9372], 99.50th=[12125], 99.90th=[13304], 99.95th=[13304],
     | 99.99th=[13304]
   bw (  KiB/s): min=  432, max= 1496, per=25.54%, avg=928.00, stdev=535.64, samples=3
   iops        : min=  108, max=  374, avg=232.00, stdev=133.91, samples=3
  write: IOPS=234, BW=938KiB/s (960kB/s)(1508KiB/1608msec)
    clat (usec): min=948, max=25950, avg=2718.08, stdev=2256.78
     lat (usec): min=948, max=25953, avg=2718.80, stdev=2256.90
    clat percentiles (usec):
     |  1.00th=[ 1045],  5.00th=[ 1188], 10.00th=[ 1369], 20.00th=[ 1532],
     | 30.00th=[ 1647], 40.00th=[ 1745], 50.00th=[ 1926], 60.00th=[ 2180],
     | 70.00th=[ 2671], 80.00th=[ 3556], 90.00th=[ 4948], 95.00th=[ 6587],
     | 99.00th=[11600], 99.50th=[17957], 99.90th=[25822], 99.95th=[25822],
     | 99.99th=[25822]
   bw (  KiB/s): min=  520, max= 1384, per=23.37%, avg=893.33, stdev=443.79, samples=3
   iops        : min=  130, max=  346, avg=223.33, stdev=110.95, samples=3
  lat (usec)   : 500=0.26%, 750=16.93%, 1000=14.19%
  lat (msec)   : 2=36.07%, 4=21.22%, 10=10.16%, 20=1.04%, 50=0.13%
  cpu          : usr=0.31%, sys=6.22%, ctx=838, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=21555: Wed Jul 17 09:35:03 2024
   read: IOPS=228, BW=915KiB/s (937kB/s)(1508KiB/1648msec)
    clat (usec): min=484, max=24989, avg=1606.57, stdev=2079.24
     lat (usec): min=484, max=24991, avg=1607.24, stdev=2079.30
    clat percentiles (usec):
     |  1.00th=[  502],  5.00th=[  570], 10.00th=[  619], 20.00th=[  685],
     | 30.00th=[  750], 40.00th=[  799], 50.00th=[  889], 60.00th=[ 1004],
     | 70.00th=[ 1369], 80.00th=[ 1893], 90.00th=[ 3785], 95.00th=[ 4621],
     | 99.00th=[ 7767], 99.50th=[19530], 99.90th=[25035], 99.95th=[25035],
     | 99.99th=[25035]
   bw (  KiB/s): min=  471, max= 1288, per=23.55%, avg=855.67, stdev=410.58, samples=3
   iops        : min=  117, max=  322, avg=213.67, stdev=103.00, samples=3
  write: IOPS=237, BW=949KiB/s (972kB/s)(1564KiB/1648msec)
    clat (usec): min=964, max=32219, avg=2646.58, stdev=2367.76
     lat (usec): min=965, max=32222, avg=2647.30, stdev=2367.91
    clat percentiles (usec):
     |  1.00th=[ 1029],  5.00th=[ 1172], 10.00th=[ 1352], 20.00th=[ 1565],
     | 30.00th=[ 1680], 40.00th=[ 1745], 50.00th=[ 1893], 60.00th=[ 2114],
     | 70.00th=[ 2606], 80.00th=[ 3228], 90.00th=[ 4621], 95.00th=[ 5669],
     | 99.00th=[12256], 99.50th=[15401], 99.90th=[32113], 99.95th=[32113],
     | 99.99th=[32113]
   bw (  KiB/s): min=  518, max= 1392, per=23.29%, avg=890.00, stdev=451.27, samples=3
   iops        : min=  129, max=  348, avg=222.33, stdev=113.02, samples=3
  lat (usec)   : 500=0.39%, 750=15.36%, 1000=13.93%
  lat (msec)   : 2=38.15%, 4=21.35%, 10=9.51%, 20=1.04%, 50=0.26%
  cpu          : usr=0.36%, sys=5.89%, ctx=838, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3633KiB/s (3721kB/s), 872KiB/s-973KiB/s (893kB/s-996kB/s), io=5988KiB (6132kB), run=1608-1648msec
  WRITE: bw=3823KiB/s (3915kB/s), 938KiB/s-994KiB/s (960kB/s-1018kB/s), io=6300KiB (6451kB), run=1608-1648msec