== sanity test 398b: DIO and buffer IO race ============== 12:46:42 (1737136002)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0868641 s, 145 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=20727: Fri Jan 17 12:46:45 2025
   read: IOPS=217, BW=868KiB/s (889kB/s)(1480KiB/1705msec)
    clat (usec): min=4, max=34170, avg=2647.83, stdev=3197.21
     lat (usec): min=5, max=34171, avg=2648.48, stdev=3197.24
    clat percentiles (usec):
     |  1.00th=[    7],  5.00th=[    9], 10.00th=[   10], 20.00th=[   13],
     | 30.00th=[   15], 40.00th=[   39], 50.00th=[ 2311], 60.00th=[ 3261],
     | 70.00th=[ 3785], 80.00th=[ 4948], 90.00th=[ 5866], 95.00th=[ 7767],
     | 99.00th=[11469], 99.50th=[16581], 99.90th=[34341], 99.95th=[34341],
     | 99.99th=[34341]
   bw (  KiB/s): min=  512, max= 1008, per=22.94%, avg=792.00, stdev=254.12, samples=3
   iops        : min=  128, max=  252, avg=198.00, stdev=63.53, samples=3
  write: IOPS=233, BW=934KiB/s (956kB/s)(1592KiB/1705msec)
    clat (usec): min=82, max=11786, avg=1790.62, stdev=2290.71
     lat (usec): min=82, max=11787, avg=1792.75, stdev=2291.69
    clat percentiles (usec):
     |  1.00th=[  110],  5.00th=[  133], 10.00th=[  143], 20.00th=[  155],
     | 30.00th=[  169], 40.00th=[  206], 50.00th=[  938], 60.00th=[ 1532],
     | 70.00th=[ 2147], 80.00th=[ 3130], 90.00th=[ 4948], 95.00th=[ 6521],
     | 99.00th=[10683], 99.50th=[11207], 99.90th=[11731], 99.95th=[11731],
     | 99.99th=[11731]
   bw (  KiB/s): min=  544, max= 1160, per=23.71%, avg=861.33, stdev=308.42, samples=3
   iops        : min=  136, max=  290, avg=215.33, stdev=77.11, samples=3
  lat (usec)   : 10=5.47%, 20=12.89%, 50=1.04%, 100=0.52%, 250=22.92%
  lat (usec)   : 500=1.82%, 750=0.52%, 1000=1.04%
  lat (msec)   : 2=11.85%, 4=20.70%, 10=19.66%, 20=1.43%, 50=0.13%
  cpu          : usr=0.53%, sys=12.62%, ctx=821, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20729: Fri Jan 17 12:46:45 2025
   read: IOPS=209, BW=837KiB/s (857kB/s)(1436KiB/1716msec)
    clat (usec): min=3, max=23229, avg=2556.95, stdev=3124.15
     lat (usec): min=3, max=23230, avg=2557.70, stdev=3124.27
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[   10], 20.00th=[   12],
     | 30.00th=[   15], 40.00th=[   42], 50.00th=[ 1876], 60.00th=[ 2769],
     | 70.00th=[ 3556], 80.00th=[ 4686], 90.00th=[ 6521], 95.00th=[ 7701],
     | 99.00th=[13435], 99.50th=[18744], 99.90th=[23200], 99.95th=[23200],
     | 99.99th=[23200]
   bw (  KiB/s): min=  440, max= 1000, per=21.39%, avg=738.67, stdev=281.86, samples=3
   iops        : min=  110, max=  250, avg=184.67, stdev=70.47, samples=3
  write: IOPS=238, BW=953KiB/s (976kB/s)(1636KiB/1716msec)
    clat (usec): min=83, max=31238, avg=1926.21, stdev=2983.62
     lat (usec): min=83, max=31238, avg=1927.28, stdev=2983.44
    clat percentiles (usec):
     |  1.00th=[  100],  5.00th=[  131], 10.00th=[  143], 20.00th=[  153],
     | 30.00th=[  172], 40.00th=[  229], 50.00th=[ 1074], 60.00th=[ 1745],
     | 70.00th=[ 2180], 80.00th=[ 3097], 90.00th=[ 4752], 95.00th=[ 6587],
     | 99.00th=[14615], 99.50th=[17171], 99.90th=[31327], 99.95th=[31327],
     | 99.99th=[31327]
   bw (  KiB/s): min=  504, max=  992, per=22.09%, avg=802.67, stdev=261.73, samples=3
   iops        : min=  126, max=  248, avg=200.67, stdev=65.43, samples=3
  lat (usec)   : 4=0.26%, 10=6.38%, 20=11.07%, 50=1.17%, 100=1.17%
  lat (usec)   : 250=21.61%, 500=3.65%, 750=0.52%, 1000=0.52%
  lat (msec)   : 2=13.02%, 4=21.09%, 10=17.32%, 20=1.95%, 50=0.26%
  cpu          : usr=0.76%, sys=12.01%, ctx=787, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20730: Fri Jan 17 12:46:45 2025
   read: IOPS=225, BW=902KiB/s (924kB/s)(1564KiB/1734msec)
    clat (usec): min=3, max=21983, avg=2525.41, stdev=3053.11
     lat (usec): min=3, max=21984, avg=2526.05, stdev=3053.16
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   14], 40.00th=[   21], 50.00th=[ 1893], 60.00th=[ 2769],
     | 70.00th=[ 3916], 80.00th=[ 4752], 90.00th=[ 5932], 95.00th=[ 7111],
     | 99.00th=[14353], 99.50th=[15270], 99.90th=[21890], 99.95th=[21890],
     | 99.99th=[21890]
   bw (  KiB/s): min=  504, max=  984, per=21.93%, avg=757.33, stdev=241.11, samples=3
   iops        : min=  126, max=  246, avg=189.33, stdev=60.28, samples=3
  write: IOPS=217, BW=870KiB/s (891kB/s)(1508KiB/1734msec)
    clat (usec): min=88, max=25269, avg=1949.66, stdev=2629.15
     lat (usec): min=89, max=25270, avg=1950.48, stdev=2629.19
    clat percentiles (usec):
     |  1.00th=[  111],  5.00th=[  133], 10.00th=[  143], 20.00th=[  155],
     | 30.00th=[  180], 40.00th=[  273], 50.00th=[ 1287], 60.00th=[ 1614],
     | 70.00th=[ 2180], 80.00th=[ 3294], 90.00th=[ 5145], 95.00th=[ 7177],
     | 99.00th=[ 9765], 99.50th=[15533], 99.90th=[25297], 99.95th=[25297],
     | 99.99th=[25297]
   bw (  KiB/s): min=  640, max=  912, per=20.26%, avg=736.00, stdev=152.63, samples=3
   iops        : min=  160, max=  228, avg=184.00, stdev=38.16, samples=3
  lat (usec)   : 4=0.13%, 10=7.68%, 20=12.50%, 50=0.65%, 100=0.65%
  lat (usec)   : 250=19.01%, 500=3.12%, 750=0.65%, 1000=0.78%
  lat (msec)   : 2=12.89%, 4=20.05%, 10=20.05%, 20=1.56%, 50=0.26%
  cpu          : usr=0.92%, sys=12.52%, ctx=826, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20731: Fri Jan 17 12:46:45 2025
   read: IOPS=219, BW=879KiB/s (900kB/s)(1508KiB/1715msec)
    clat (usec): min=4, max=12481, avg=2421.85, stdev=2570.72
     lat (usec): min=4, max=12482, avg=2422.53, stdev=2570.75
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    8], 10.00th=[    9], 20.00th=[   11],
     | 30.00th=[   13], 40.00th=[   25], 50.00th=[ 2040], 60.00th=[ 2966],
     | 70.00th=[ 3720], 80.00th=[ 4424], 90.00th=[ 5866], 95.00th=[ 7308],
     | 99.00th=[10159], 99.50th=[11600], 99.90th=[12518], 99.95th=[12518],
     | 99.99th=[12518]
   bw (  KiB/s): min=  534, max=  896, per=21.89%, avg=756.00, stdev=194.43, samples=3
   iops        : min=  133, max=  224, avg=188.67, stdev=48.79, samples=3
  write: IOPS=227, BW=912KiB/s (934kB/s)(1564KiB/1715msec)
    clat (usec): min=85, max=38724, avg=2021.26, stdev=3402.43
     lat (usec): min=86, max=38725, avg=2022.63, stdev=3402.25
    clat percentiles (usec):
     |  1.00th=[  100],  5.00th=[  130], 10.00th=[  139], 20.00th=[  155],
     | 30.00th=[  169], 40.00th=[  212], 50.00th=[  494], 60.00th=[ 1565],
     | 70.00th=[ 2008], 80.00th=[ 3458], 90.00th=[ 5014], 95.00th=[ 7308],
     | 99.00th=[13698], 99.50th=[27132], 99.90th=[38536], 99.95th=[38536],
     | 99.99th=[38536]
   bw (  KiB/s): min=  598, max= 1037, per=22.49%, avg=817.00, stdev=219.50, samples=3
   iops        : min=  149, max=  259, avg=204.00, stdev=55.00, samples=3
  lat (usec)   : 10=8.33%, 20=10.94%, 50=0.78%, 100=0.39%, 250=22.14%
  lat (usec)   : 500=3.12%, 750=0.39%, 1000=0.78%
  lat (msec)   : 2=13.02%, 4=17.32%, 10=20.96%, 20=1.56%, 50=0.26%
  cpu          : usr=0.64%, sys=11.61%, ctx=804, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3453KiB/s (3536kB/s), 837KiB/s-902KiB/s (857kB/s-924kB/s), io=5988KiB (6132kB), run=1705-1734msec
  WRITE: bw=3633KiB/s (3720kB/s), 870KiB/s-953KiB/s (891kB/s-976kB/s), io=6300KiB (6451kB), run=1705-1734msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=20724: Fri Jan 17 12:46:46 2025
   read: IOPS=127, BW=511KiB/s (524kB/s)(1480KiB/2894msec)
    clat (usec): min=720, max=17354, avg=2442.72, stdev=2262.08
     lat (usec): min=721, max=17356, avg=2443.57, stdev=2262.11
    clat percentiles (usec):
     |  1.00th=[  807],  5.00th=[  947], 10.00th=[ 1020], 20.00th=[ 1123],
     | 30.00th=[ 1270], 40.00th=[ 1418], 50.00th=[ 1663], 60.00th=[ 1860],
     | 70.00th=[ 2311], 80.00th=[ 3032], 90.00th=[ 4621], 95.00th=[ 8225],
     | 99.00th=[11731], 99.50th=[12649], 99.90th=[17433], 99.95th=[17433],
     | 99.99th=[17433]
   bw (  KiB/s): min=  264, max=  736, per=23.39%, avg=478.40, stdev=173.93, samples=5
   iops        : min=   66, max=  184, avg=119.60, stdev=43.48, samples=5
  write: IOPS=137, BW=550KiB/s (563kB/s)(1592KiB/2894msec)
    clat (usec): min=1603, max=42752, avg=4964.17, stdev=3354.96
     lat (usec): min=1603, max=42754, avg=4965.57, stdev=3354.92
    clat percentiles (usec):
     |  1.00th=[ 1729],  5.00th=[ 2278], 10.00th=[ 2573], 20.00th=[ 2966],
     | 30.00th=[ 3261], 40.00th=[ 3523], 50.00th=[ 3949], 60.00th=[ 4490],
     | 70.00th=[ 5342], 80.00th=[ 6456], 90.00th=[ 8717], 95.00th=[10421],
     | 99.00th=[13435], 99.50th=[27657], 99.90th=[42730], 99.95th=[42730],
     | 99.99th=[42730]
   bw (  KiB/s): min=  312, max=  792, per=24.10%, avg=518.40, stdev=226.66, samples=5
   iops        : min=   78, max=  198, avg=129.60, stdev=56.66, samples=5
  lat (usec)   : 750=0.13%, 1000=3.91%
  lat (msec)   : 2=27.86%, 4=36.33%, 10=27.60%, 20=3.91%, 50=0.26%
  cpu          : usr=0.35%, sys=4.80%, ctx=803, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20725: Fri Jan 17 12:46:46 2025
   read: IOPS=122, BW=490KiB/s (502kB/s)(1436KiB/2928msec)
    clat (usec): min=771, max=19428, avg=2467.51, stdev=2238.16
     lat (usec): min=771, max=19430, avg=2468.28, stdev=2238.22
    clat percentiles (usec):
     |  1.00th=[  799],  5.00th=[  930], 10.00th=[ 1020], 20.00th=[ 1156],
     | 30.00th=[ 1319], 40.00th=[ 1483], 50.00th=[ 1663], 60.00th=[ 1958],
     | 70.00th=[ 2245], 80.00th=[ 3097], 90.00th=[ 4752], 95.00th=[ 7701],
     | 99.00th=[11076], 99.50th=[13435], 99.90th=[19530], 99.95th=[19530],
     | 99.99th=[19530]
   bw (  KiB/s): min=  256, max=  768, per=23.08%, avg=472.00, stdev=223.86, samples=5
   iops        : min=   64, max=  192, avg=118.00, stdev=55.96, samples=5
  write: IOPS=139, BW=559KiB/s (572kB/s)(1636KiB/2928msec)
    clat (usec): min=1564, max=39321, avg=4962.41, stdev=3001.06
     lat (usec): min=1565, max=39322, avg=4963.53, stdev=3001.28
    clat percentiles (usec):
     |  1.00th=[ 1860],  5.00th=[ 2409], 10.00th=[ 2606], 20.00th=[ 2933],
     | 30.00th=[ 3359], 40.00th=[ 3818], 50.00th=[ 4228], 60.00th=[ 4621],
     | 70.00th=[ 5276], 80.00th=[ 6325], 90.00th=[ 8356], 95.00th=[10159],
     | 99.00th=[14222], 99.50th=[15401], 99.90th=[39060], 99.95th=[39060],
     | 99.99th=[39060]
   bw (  KiB/s): min=  312, max=  776, per=23.95%, avg=515.20, stdev=190.21, samples=5
   iops        : min=   78, max=  194, avg=128.80, stdev=47.55, samples=5
  lat (usec)   : 1000=3.78%
  lat (msec)   : 2=25.65%, 4=34.11%, 10=33.07%, 20=3.26%, 50=0.13%
  cpu          : usr=0.89%, sys=4.17%, ctx=801, majf=0, minf=31
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20726: Fri Jan 17 12:46:46 2025
   read: IOPS=135, BW=544KiB/s (557kB/s)(1564KiB/2877msec)
    clat (usec): min=689, max=14099, avg=2495.61, stdev=2184.84
     lat (usec): min=689, max=14448, avg=2497.29, stdev=2189.68
    clat percentiles (usec):
     |  1.00th=[  824],  5.00th=[  971], 10.00th=[ 1057], 20.00th=[ 1172],
     | 30.00th=[ 1352], 40.00th=[ 1549], 50.00th=[ 1713], 60.00th=[ 1958],
     | 70.00th=[ 2278], 80.00th=[ 2868], 90.00th=[ 5473], 95.00th=[ 7504],
     | 99.00th=[11731], 99.50th=[13566], 99.90th=[14091], 99.95th=[14091],
     | 99.99th=[14091]
   bw (  KiB/s): min=  232, max=  832, per=24.25%, avg=496.00, stdev=229.64, samples=5
   iops        : min=   58, max=  208, avg=124.00, stdev=57.41, samples=5
  write: IOPS=131, BW=524KiB/s (537kB/s)(1508KiB/2877msec)
    clat (usec): min=1404, max=39331, avg=5000.90, stdev=3323.51
     lat (usec): min=1404, max=39332, avg=5001.89, stdev=3323.58
    clat percentiles (usec):
     |  1.00th=[ 1762],  5.00th=[ 2245], 10.00th=[ 2573], 20.00th=[ 2966],
     | 30.00th=[ 3294], 40.00th=[ 3720], 50.00th=[ 3982], 60.00th=[ 4555],
     | 70.00th=[ 5211], 80.00th=[ 6390], 90.00th=[ 8455], 95.00th=[10814],
     | 99.00th=[15795], 99.50th=[24511], 99.90th=[39584], 99.95th=[39584],
     | 99.99th=[39584]
   bw (  KiB/s): min=  312, max=  768, per=22.91%, avg=492.80, stdev=206.04, samples=5
   iops        : min=   78, max=  192, avg=123.20, stdev=51.51, samples=5
  lat (usec)   : 750=0.13%, 1000=3.39%
  lat (msec)   : 2=28.91%, 4=35.68%, 10=27.86%, 20=3.78%, 50=0.26%
  cpu          : usr=0.76%, sys=4.69%, ctx=826, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=20728: Fri Jan 17 12:46:46 2025
   read: IOPS=130, BW=523KiB/s (535kB/s)(1508KiB/2886msec)
    clat (usec): min=665, max=11758, avg=2631.11, stdev=2359.98
     lat (usec): min=666, max=11759, avg=2632.33, stdev=2359.86
    clat percentiles (usec):
     |  1.00th=[  848],  5.00th=[  922], 10.00th=[ 1020], 20.00th=[ 1172],
     | 30.00th=[ 1336], 40.00th=[ 1516], 50.00th=[ 1663], 60.00th=[ 1991],
     | 70.00th=[ 2442], 80.00th=[ 3326], 90.00th=[ 6325], 95.00th=[ 8717],
     | 99.00th=[11469], 99.50th=[11731], 99.90th=[11731], 99.95th=[11731],
     | 99.99th=[11731]
   bw (  KiB/s): min=  263, max=  664, per=22.90%, avg=468.40, stdev=152.43, samples=5
   iops        : min=   65, max=  166, avg=116.80, stdev=38.39, samples=5
  write: IOPS=135, BW=542KiB/s (555kB/s)(1564KiB/2886msec)
    clat (usec): min=1582, max=31857, avg=4802.15, stdev=3097.10
     lat (usec): min=1583, max=31858, avg=4803.92, stdev=3097.44
    clat percentiles (usec):
     |  1.00th=[ 1876],  5.00th=[ 2311], 10.00th=[ 2573], 20.00th=[ 2933],
     | 30.00th=[ 3228], 40.00th=[ 3621], 50.00th=[ 3916], 60.00th=[ 4228],
     | 70.00th=[ 4817], 80.00th=[ 5997], 90.00th=[ 7963], 95.00th=[ 9896],
     | 99.00th=[19530], 99.50th=[24773], 99.90th=[31851], 99.95th=[31851],
     | 99.99th=[31851]
   bw (  KiB/s): min=  343, max=  808, per=23.64%, avg=508.40, stdev=207.21, samples=5
   iops        : min=   85, max=  202, avg=126.80, stdev=52.05, samples=5
  lat (usec)   : 750=0.13%, 1000=4.43%
  lat (msec)   : 2=25.91%, 4=38.15%, 10=27.60%, 20=3.52%, 50=0.26%
  cpu          : usr=0.35%, sys=4.75%, ctx=814, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=2045KiB/s (2094kB/s), 490KiB/s-544KiB/s (502kB/s-557kB/s), io=5988KiB (6132kB), run=2877-2928msec
  WRITE: bw=2152KiB/s (2203kB/s), 524KiB/s-559KiB/s (537kB/s-572kB/s), io=6300KiB (6451kB), run=2877-2928msec