== sanity test 398b: DIO and buffer IO race ============== 18:33:15 (1747261995)
/usr/bin/fio
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0969515 s, 130 MB/s
mix direct rw 12M to OST0 by fio with 4 jobs...
mix buffer rw 12M to OST0 by fio with 4 jobs...
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes
rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16
...
fio-3.7
Starting 4 processes

rand-rw: (groupid=0, jobs=1): err= 0: pid=17686: Wed May 14 18:33:17 2025
   read: IOPS=384, BW=1538KiB/s (1575kB/s)(1480KiB/962msec)
    clat (usec): min=3, max=12550, avg=1423.41, stdev=1836.15
     lat (usec): min=3, max=12551, avg=1423.97, stdev=1836.22
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    6], 10.00th=[    7], 20.00th=[   10],
     | 30.00th=[   11], 40.00th=[   16], 50.00th=[ 1090], 60.00th=[ 1598],
     | 70.00th=[ 1991], 80.00th=[ 2540], 90.00th=[ 3228], 95.00th=[ 3982],
     | 99.00th=[ 8717], 99.50th=[11863], 99.90th=[12518], 99.95th=[12518],
     | 99.99th=[12518]
   bw (  KiB/s): min=  960, max=  960, per=18.12%, avg=960.00, stdev= 0.00, samples=1
   iops        : min=  240, max=  240, avg=240.00, stdev= 0.00, samples=1
  write: IOPS=413, BW=1655KiB/s (1695kB/s)(1592KiB/962msec)
    clat (usec): min=87, max=11390, avg=1075.62, stdev=1503.97
     lat (usec): min=88, max=11391, avg=1076.37, stdev=1504.01
    clat percentiles (usec):
     |  1.00th=[   91],  5.00th=[  103], 10.00th=[  109], 20.00th=[  120],
     | 30.00th=[  133], 40.00th=[  169], 50.00th=[  343], 60.00th=[  832],
     | 70.00th=[ 1188], 80.00th=[ 1811], 90.00th=[ 2737], 95.00th=[ 4178],
     | 99.00th=[ 6456], 99.50th=[11338], 99.90th=[11338], 99.95th=[11338],
     | 99.99th=[11338]
   bw (  KiB/s): min=  920, max=  920, per=16.50%, avg=920.00, stdev= 0.00, samples=1
   iops        : min=  230, max=  230, avg=230.00, stdev= 0.00, samples=1
  lat (usec)   : 4=0.39%, 10=12.89%, 20=6.77%, 50=0.65%, 100=1.95%
  lat (usec)   : 250=22.79%, 500=1.82%, 750=3.39%, 1000=6.12%
  lat (msec)   : 2=20.18%, 4=17.71%, 10=4.69%, 20=0.65%
  cpu          : usr=0.21%, sys=16.86%, ctx=786, majf=0, minf=35
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17687: Wed May 14 18:33:17 2025
   read: IOPS=350, BW=1401KiB/s (1435kB/s)(1436KiB/1025msec)
    clat (usec): min=3, max=27978, avg=1547.96, stdev=2375.49
     lat (usec): min=4, max=27978, avg=1548.52, stdev=2375.49
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    8], 10.00th=[    8], 20.00th=[   11],
     | 30.00th=[   12], 40.00th=[  807], 50.00th=[ 1401], 60.00th=[ 1729],
     | 70.00th=[ 2114], 80.00th=[ 2474], 90.00th=[ 3097], 95.00th=[ 3752],
     | 99.00th=[ 9110], 99.50th=[23725], 99.90th=[27919], 99.95th=[27919],
     | 99.99th=[27919]
   bw (  KiB/s): min=  824, max= 1976, per=26.42%, avg=1400.00, stdev=814.59, samples=2
   iops        : min=  206, max=  494, avg=350.00, stdev=203.65, samples=2
  write: IOPS=399, BW=1596KiB/s (1634kB/s)(1636KiB/1025msec)
    clat (usec): min=90, max=19283, avg=1122.88, stdev=1746.94
     lat (usec): min=90, max=19284, avg=1123.76, stdev=1747.37
    clat percentiles (usec):
     |  1.00th=[   95],  5.00th=[  105], 10.00th=[  112], 20.00th=[  123],
     | 30.00th=[  145], 40.00th=[  184], 50.00th=[  396], 60.00th=[  873],
     | 70.00th=[ 1205], 80.00th=[ 1926], 90.00th=[ 2900], 95.00th=[ 3949],
     | 99.00th=[ 6194], 99.50th=[10159], 99.90th=[19268], 99.95th=[19268],
     | 99.99th=[19268]
   bw (  KiB/s): min=  936, max= 2208, per=28.20%, avg=1572.00, stdev=899.44, samples=2
   iops        : min=  234, max=  552, avg=393.00, stdev=224.86, samples=2
  lat (usec)   : 4=0.13%, 10=8.98%, 20=7.81%, 50=0.13%, 100=1.56%
  lat (usec)   : 250=23.96%, 500=2.21%, 750=2.99%, 1000=6.51%
  lat (msec)   : 2=20.57%, 4=20.96%, 10=3.52%, 20=0.39%, 50=0.26%
  cpu          : usr=0.20%, sys=18.07%, ctx=845, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17688: Wed May 14 18:33:17 2025
   read: IOPS=346, BW=1384KiB/s (1417kB/s)(1564KiB/1130msec)
    clat (usec): min=3, max=17655, avg=1616.61, stdev=1932.33
     lat (usec): min=3, max=17656, avg=1618.23, stdev=1933.12
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   12], 40.00th=[  873], 50.00th=[ 1434], 60.00th=[ 1876],
     | 70.00th=[ 2311], 80.00th=[ 2671], 90.00th=[ 3425], 95.00th=[ 4555],
     | 99.00th=[11994], 99.50th=[13304], 99.90th=[17695], 99.95th=[17695],
     | 99.99th=[17695]
   bw (  KiB/s): min=  640, max= 1888, per=23.85%, avg=1264.00, stdev=882.47, samples=2
   iops        : min=  160, max=  472, avg=316.00, stdev=220.62, samples=2
  write: IOPS=333, BW=1335KiB/s (1367kB/s)(1508KiB/1130msec)
    clat (usec): min=86, max=17348, avg=1294.64, stdev=1871.58
     lat (usec): min=86, max=17349, avg=1296.53, stdev=1871.01
    clat percentiles (usec):
     |  1.00th=[   93],  5.00th=[  104], 10.00th=[  112], 20.00th=[  125],
     | 30.00th=[  147], 40.00th=[  206], 50.00th=[  791], 60.00th=[  996],
     | 70.00th=[ 1450], 80.00th=[ 2024], 90.00th=[ 3032], 95.00th=[ 4555],
     | 99.00th=[ 9765], 99.50th=[11469], 99.90th=[17433], 99.95th=[17433],
     | 99.99th=[17433]
   bw (  KiB/s): min=  752, max= 1752, per=22.46%, avg=1252.00, stdev=707.11, samples=2
   iops        : min=  188, max=  438, avg=313.00, stdev=176.78, samples=2
  lat (usec)   : 4=0.65%, 10=11.59%, 20=5.73%, 50=0.13%, 100=2.08%
  lat (usec)   : 250=18.88%, 500=0.78%, 750=2.86%, 1000=8.72%
  lat (msec)   : 2=19.66%, 4=22.14%, 10=5.86%, 20=0.91%
  cpu          : usr=0.35%, sys=15.23%, ctx=936, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17689: Wed May 14 18:33:17 2025
   read: IOPS=348, BW=1394KiB/s (1427kB/s)(1508KiB/1082msec)
    clat (usec): min=3, max=19429, avg=1597.21, stdev=1857.15
     lat (usec): min=3, max=19430, avg=1598.14, stdev=1857.18
    clat percentiles (usec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[   10],
     | 30.00th=[   14], 40.00th=[  955], 50.00th=[ 1549], 60.00th=[ 1909],
     | 70.00th=[ 2212], 80.00th=[ 2671], 90.00th=[ 3294], 95.00th=[ 4424],
     | 99.00th=[ 7373], 99.50th=[12780], 99.90th=[19530], 99.95th=[19530],
     | 99.99th=[19530]
   bw (  KiB/s): min=  888, max= 1672, per=24.16%, avg=1280.00, stdev=554.37, samples=2
   iops        : min=  222, max=  418, avg=320.00, stdev=138.59, samples=2
  write: IOPS=361, BW=1445KiB/s (1480kB/s)(1564KiB/1082msec)
    clat (usec): min=88, max=19186, avg=1201.19, stdev=1844.75
     lat (usec): min=89, max=19189, avg=1202.30, stdev=1844.58
    clat percentiles (usec):
     |  1.00th=[   94],  5.00th=[  103], 10.00th=[  110], 20.00th=[  127],
     | 30.00th=[  161], 40.00th=[  253], 50.00th=[  750], 60.00th=[  898],
     | 70.00th=[ 1270], 80.00th=[ 1844], 90.00th=[ 3032], 95.00th=[ 3949],
     | 99.00th=[ 9110], 99.50th=[15270], 99.90th=[19268], 99.95th=[19268],
     | 99.99th=[19268]
   bw (  KiB/s): min=  912, max= 1752, per=23.89%, avg=1332.00, stdev=593.97, samples=2
   iops        : min=  228, max=  438, avg=333.00, stdev=148.49, samples=2
  lat (usec)   : 4=0.13%, 10=10.29%, 20=6.25%, 50=0.78%, 100=2.08%
  lat (usec)   : 250=18.49%, 500=2.47%, 750=3.39%, 1000=9.24%
  lat (msec)   : 2=20.70%, 4=21.22%, 10=4.30%, 20=0.65%
  cpu          : usr=0.37%, sys=16.00%, ctx=926, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=5299KiB/s (5426kB/s), 1384KiB/s-1538KiB/s (1417kB/s-1575kB/s), io=5988KiB (6132kB), run=962-1130msec
  WRITE: bw=5575KiB/s (5709kB/s), 1335KiB/s-1655KiB/s (1367kB/s-1695kB/s), io=6300KiB (6451kB), run=962-1130msec

rand-rw: (groupid=0, jobs=1): err= 0: pid=17681: Wed May 14 18:33:18 2025
   read: IOPS=222, BW=888KiB/s (910kB/s)(1480KiB/1666msec)
    clat (usec): min=448, max=26266, avg=1558.33, stdev=2094.89
     lat (usec): min=448, max=26269, avg=1558.92, stdev=2095.01
    clat percentiles (usec):
     |  1.00th=[  537],  5.00th=[  619], 10.00th=[  668], 20.00th=[  717],
     | 30.00th=[  766], 40.00th=[  840], 50.00th=[  906], 60.00th=[ 1012],
     | 70.00th=[ 1336], 80.00th=[ 1909], 90.00th=[ 3097], 95.00th=[ 4424],
     | 99.00th=[ 9896], 99.50th=[20055], 99.90th=[26346], 99.95th=[26346],
     | 99.99th=[26346]
   bw (  KiB/s): min=  472, max= 1168, per=23.47%, avg=821.33, stdev=348.01, samples=3
   iops        : min=  118, max=  292, avg=205.33, stdev=87.00, samples=3
  write: IOPS=238, BW=956KiB/s (979kB/s)(1592KiB/1666msec)
    clat (usec): min=961, max=13173, avg=2698.23, stdev=1610.19
     lat (usec): min=962, max=13175, avg=2699.01, stdev=1610.28
    clat percentiles (usec):
     |  1.00th=[ 1139],  5.00th=[ 1336], 10.00th=[ 1532], 20.00th=[ 1713],
     | 30.00th=[ 1811], 40.00th=[ 1942], 50.00th=[ 2057], 60.00th=[ 2311],
     | 70.00th=[ 2868], 80.00th=[ 3392], 90.00th=[ 4621], 95.00th=[ 5997],
     | 99.00th=[ 9372], 99.50th=[11469], 99.90th=[13173], 99.95th=[13173],
     | 99.99th=[13173]
   bw (  KiB/s): min=  496, max= 1416, per=24.62%, avg=906.67, stdev=467.87, samples=3
   iops        : min=  124, max=  354, avg=226.67, stdev=116.97, samples=3
  lat (usec)   : 500=0.13%, 750=12.63%, 1000=15.89%
  lat (msec)   : 2=34.90%, 4=26.04%, 10=9.77%, 20=0.39%, 50=0.26%
  cpu          : usr=0.18%, sys=6.13%, ctx=838, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17682: Wed May 14 18:33:18 2025
   read: IOPS=210, BW=842KiB/s (862kB/s)(1436KiB/1706msec)
    clat (usec): min=520, max=26380, avg=1560.44, stdev=2077.00
     lat (usec): min=520, max=26382, avg=1561.01, stdev=2077.07
    clat percentiles (usec):
     |  1.00th=[  562],  5.00th=[  619], 10.00th=[  652], 20.00th=[  709],
     | 30.00th=[  766], 40.00th=[  832], 50.00th=[  881], 60.00th=[ 1004],
     | 70.00th=[ 1254], 80.00th=[ 1942], 90.00th=[ 3392], 95.00th=[ 4228],
     | 99.00th=[ 8717], 99.50th=[19006], 99.90th=[26346], 99.95th=[26346],
     | 99.99th=[26346]
   bw (  KiB/s): min=  440, max= 1272, per=22.71%, avg=794.67, stdev=429.35, samples=3
   iops        : min=  110, max=  318, avg=198.67, stdev=107.34, samples=3
  write: IOPS=239, BW=959KiB/s (982kB/s)(1636KiB/1706msec)
    clat (usec): min=1002, max=19373, avg=2764.81, stdev=1824.78
     lat (usec): min=1002, max=19374, avg=2765.50, stdev=1824.88
    clat percentiles (usec):
     |  1.00th=[ 1156],  5.00th=[ 1418], 10.00th=[ 1565], 20.00th=[ 1729],
     | 30.00th=[ 1860], 40.00th=[ 1942], 50.00th=[ 2089], 60.00th=[ 2278],
     | 70.00th=[ 2835], 80.00th=[ 3490], 90.00th=[ 4817], 95.00th=[ 6259],
     | 99.00th=[ 9765], 99.50th=[10421], 99.90th=[19268], 99.95th=[19268],
     | 99.99th=[19268]
   bw (  KiB/s): min=  496, max= 1288, per=23.97%, avg=882.67, stdev=396.33, samples=3
   iops        : min=  124, max=  322, avg=220.67, stdev=99.08, samples=3
  lat (usec)   : 750=13.02%, 1000=14.97%
  lat (msec)   : 2=33.59%, 4=27.86%, 10=9.90%, 20=0.52%, 50=0.13%
  cpu          : usr=0.29%, sys=6.04%, ctx=827, majf=0, minf=34
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17683: Wed May 14 18:33:18 2025
   read: IOPS=230, BW=923KiB/s (945kB/s)(1564KiB/1694msec)
    clat (usec): min=523, max=20221, avg=1525.19, stdev=1646.01
     lat (usec): min=524, max=20221, avg=1525.92, stdev=1645.97
    clat percentiles (usec):
     |  1.00th=[  578],  5.00th=[  635], 10.00th=[  668], 20.00th=[  717],
     | 30.00th=[  783], 40.00th=[  840], 50.00th=[  947], 60.00th=[ 1057],
     | 70.00th=[ 1385], 80.00th=[ 1909], 90.00th=[ 3359], 95.00th=[ 4424],
     | 99.00th=[ 8586], 99.50th=[ 9765], 99.90th=[20317], 99.95th=[20317],
     | 99.99th=[20317]
   bw (  KiB/s): min=  368, max= 1264, per=24.08%, avg=842.67, stdev=450.37, samples=3
   iops        : min=   92, max=  316, avg=210.67, stdev=112.59, samples=3
  write: IOPS=222, BW=890KiB/s (912kB/s)(1508KiB/1694msec)
    clat (usec): min=1128, max=41238, avg=2884.66, stdev=2578.03
     lat (usec): min=1129, max=41241, avg=2885.36, stdev=2578.17
    clat percentiles (usec):
     |  1.00th=[ 1188],  5.00th=[ 1385], 10.00th=[ 1549], 20.00th=[ 1713],
     | 30.00th=[ 1827], 40.00th=[ 1942], 50.00th=[ 2114], 60.00th=[ 2507],
     | 70.00th=[ 2999], 80.00th=[ 3752], 90.00th=[ 5145], 95.00th=[ 6128],
     | 99.00th=[ 8979], 99.50th=[17433], 99.90th=[41157], 99.95th=[41157],
     | 99.99th=[41157]
   bw (  KiB/s): min=  480, max= 1264, per=22.67%, avg=834.67, stdev=397.30, samples=3
   iops        : min=  120, max=  316, avg=208.67, stdev=99.32, samples=3
  lat (usec)   : 750=12.63%, 1000=15.89%
  lat (msec)   : 2=34.37%, 4=24.61%, 10=12.11%, 20=0.13%, 50=0.26%
  cpu          : usr=0.30%, sys=6.08%, ctx=845, majf=0, minf=32
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16
rand-rw: (groupid=0, jobs=1): err= 0: pid=17684: Wed May 14 18:33:18 2025
   read: IOPS=220, BW=881KiB/s (903kB/s)(1508KiB/1711msec)
    clat (usec): min=517, max=11395, avg=1479.18, stdev=1346.84
     lat (usec): min=517, max=11397, avg=1479.76, stdev=1346.88
    clat percentiles (usec):
     |  1.00th=[  537],  5.00th=[  627], 10.00th=[  685], 20.00th=[  725],
     | 30.00th=[  783], 40.00th=[  832], 50.00th=[  898], 60.00th=[ 1020],
     | 70.00th=[ 1303], 80.00th=[ 1926], 90.00th=[ 3228], 95.00th=[ 4424],
     | 99.00th=[ 6390], 99.50th=[ 9372], 99.90th=[11338], 99.95th=[11338],
     | 99.99th=[11338]
   bw (  KiB/s): min=  432, max= 1168, per=22.94%, avg=802.67, stdev=368.03, samples=3
   iops        : min=  108, max=  292, avg=200.67, stdev=92.01, samples=3
  write: IOPS=228, BW=914KiB/s (936kB/s)(1564KiB/1711msec)
    clat (usec): min=988, max=44403, avg=2907.39, stdev=2975.75
     lat (usec): min=988, max=44406, avg=2908.20, stdev=2975.92
    clat percentiles (usec):
     |  1.00th=[ 1139],  5.00th=[ 1303], 10.00th=[ 1467], 20.00th=[ 1696],
     | 30.00th=[ 1827], 40.00th=[ 1926], 50.00th=[ 2057], 60.00th=[ 2311],
     | 70.00th=[ 2769], 80.00th=[ 3392], 90.00th=[ 4948], 95.00th=[ 6390],
     | 99.00th=[15795], 99.50th=[19530], 99.90th=[44303], 99.95th=[44303],
     | 99.99th=[44303]
   bw (  KiB/s): min=  456, max= 1360, per=23.32%, avg=858.67, stdev=460.01, samples=3
   iops        : min=  114, max=  340, avg=214.67, stdev=115.00, samples=3
  lat (usec)   : 750=12.11%, 1000=17.32%
  lat (msec)   : 2=33.33%, 4=26.30%, 10=9.77%, 20=1.04%, 50=0.13%
  cpu          : usr=0.53%, sys=5.85%, ctx=833, majf=0, minf=33
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=3500KiB/s (3584kB/s), 842KiB/s-923KiB/s (862kB/s-945kB/s), io=5988KiB (6132kB), run=1666-1711msec
  WRITE: bw=3682KiB/s (3770kB/s), 890KiB/s-959KiB/s (912kB/s-982kB/s), io=6300KiB (6451kB), run=1666-1711msec