== sanity test 398b: DIO and buffer IO race ============== 06:23:23 (1781259803) /usr/bin/fio 12+0 records in 12+0 records out 12582912 bytes (13 MB, 12 MiB) copied, 0.313616 s, 40.1 MB/s mix direct rw 12M to OST0 by fio with 4 jobs... mix buffer rw 12M to OST0 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147920: Fri Jun 12 06:23:50 2026 read: IOPS=23, BW=93.2KiB/s (95.4kB/s)(1480KiB/15878msec) clat (usec): min=7, max=148551, avg=24832.62, stdev=26010.53 lat (usec): min=7, max=148552, avg=24833.65, stdev=26010.79 clat percentiles (usec): | 1.00th=[ 9], 5.00th=[ 15], 10.00th=[ 20], 20.00th=[ 27], | 30.00th=[ 38], 40.00th=[ 13566], 50.00th=[ 22414], 60.00th=[ 27657], | 70.00th=[ 34866], 80.00th=[ 42730], 90.00th=[ 58459], 95.00th=[ 68682], | 99.00th=[115868], 99.50th=[133694], 99.90th=[147850], 99.95th=[147850], | 99.99th=[147850] bw ( KiB/s): min= 31, max= 163, per=24.33%, avg=91.74, stdev=37.15, samples=31 iops : min= 7, max= 40, avg=22.32, stdev= 9.31, samples=31 write: IOPS=25, BW=100KiB/s (103kB/s)(1592KiB/15878msec); 0 zone resets clat (usec): min=166, max=105946, avg=16759.93, stdev=20364.72 lat (usec): min=167, max=105947, avg=16760.94, stdev=20364.79 clat percentiles (usec): | 1.00th=[ 174], 5.00th=[ 200], 10.00th=[ 219], 20.00th=[ 281], | 30.00th=[ 396], 40.00th=[ 2966], 50.00th=[ 11076], 60.00th=[ 14746], | 70.00th=[ 22938], 80.00th=[ 29492], 90.00th=[ 45351], 95.00th=[ 56361], | 99.00th=[ 96994], 99.50th=[104334], 99.90th=[106431], 99.95th=[106431], | 99.99th=[106431] bw ( KiB/s): min= 23, max= 176, per=25.23%, avg=99.90, stdev=41.63, samples=31 iops : min= 5, max= 44, avg=24.39, stdev=10.45, samples=31 lat (usec) : 10=1.30%, 20=3.65%, 50=10.68%, 100=0.39%, 250=8.59% lat (usec) : 500=9.38%, 750=1.04%, 1000=1.17% lat (msec) : 2=1.30%, 4=0.65%, 10=4.69%, 20=14.19%, 50=31.51% lat (msec) : 100=10.16%, 250=1.30% cpu : usr=0.09%, sys=2.49%, ctx=817, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147921: Fri Jun 12 06:23:50 2026 read: IOPS=22, BW=90.0KiB/s (93.2kB/s)(1436KiB/15784msec) clat (usec): min=7, max=154895, avg=25087.28, stdev=24971.49 lat (usec): min=8, max=154896, avg=25088.26, stdev=24971.59 clat percentiles (usec): | 1.00th=[ 11], 5.00th=[ 15], 10.00th=[ 22], 20.00th=[ 31], | 30.00th=[ 39], 40.00th=[ 14877], 50.00th=[ 22676], 60.00th=[ 30016], | 70.00th=[ 35914], 80.00th=[ 44303], 90.00th=[ 55313], 95.00th=[ 70779], | 99.00th=[106431], 99.50th=[133694], 99.90th=[154141], 99.95th=[154141], | 99.99th=[154141] bw ( KiB/s): min= 29, max= 142, per=23.76%, avg=89.57, stdev=29.27, samples=30 iops : min= 7, max= 35, avg=21.80, stdev= 7.31, samples=30 write: IOPS=25, BW=104KiB/s (106kB/s)(1636KiB/15784msec); 0 zone resets clat (usec): min=153, max=189637, avg=16517.13, stdev=21134.84 lat (usec): min=153, max=189638, avg=16518.67, stdev=21134.49 clat percentiles (usec): | 1.00th=[ 174], 5.00th=[ 198], 10.00th=[ 221], 20.00th=[ 277], | 30.00th=[ 367], 40.00th=[ 6128], 50.00th=[ 10552], 60.00th=[ 14746], | 70.00th=[ 20579], 80.00th=[ 30016], 90.00th=[ 42206], 95.00th=[ 55837], | 99.00th=[ 77071], 99.50th=[108528], 99.90th=[189793], 99.95th=[189793], | 99.99th=[189793] bw ( KiB/s): min= 39, max= 187, per=25.25%, avg=100.00, stdev=32.42, samples=30 iops : min= 9, max= 46, avg=24.40, stdev= 8.12, samples=30 lat (usec) : 10=0.26%, 20=3.65%, 50=11.20%, 100=0.13%, 250=7.94% lat (usec) : 500=9.51%, 750=0.91%, 1000=1.17% lat (msec) : 2=0.78%, 4=0.39%, 10=6.90%, 20=15.10%, 50=32.16% lat (msec) : 100=8.85%, 250=1.04% cpu : usr=0.11%, sys=2.41%, ctx=848, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147922: Fri Jun 12 06:23:50 2026 read: IOPS=24, BW=99.8KiB/s (102kB/s)(1564KiB/15672msec) clat (usec): min=7, max=169805, avg=24502.08, stdev=24659.08 lat (usec): min=7, max=169806, avg=24502.98, stdev=24659.08 clat percentiles (usec): | 1.00th=[ 9], 5.00th=[ 18], 10.00th=[ 22], 20.00th=[ 32], | 30.00th=[ 3163], 40.00th=[ 14091], 50.00th=[ 21103], 60.00th=[ 27132], | 70.00th=[ 34341], 80.00th=[ 42206], 90.00th=[ 56361], 95.00th=[ 65799], | 99.00th=[112722], 99.50th=[152044], 99.90th=[168821], 99.95th=[168821], | 99.99th=[168821] bw ( KiB/s): min= 29, max= 200, per=26.76%, avg=100.90, stdev=37.38, samples=30 iops : min= 7, max= 50, avg=24.63, stdev= 9.43, samples=30 write: IOPS=24, BW=96.2KiB/s (98.5kB/s)(1508KiB/15672msec); 0 zone resets clat (usec): min=162, max=206823, avg=16102.25, stdev=21729.50 lat (usec): min=162, max=206824, avg=16103.59, stdev=21729.64 clat percentiles (usec): | 1.00th=[ 174], 5.00th=[ 196], 10.00th=[ 223], 20.00th=[ 289], | 30.00th=[ 367], 40.00th=[ 1418], 50.00th=[ 9110], 60.00th=[ 15401], | 70.00th=[ 21103], 80.00th=[ 28967], 90.00th=[ 42206], 95.00th=[ 57934], | 99.00th=[ 80217], 99.50th=[ 99091], 99.90th=[206570], 99.95th=[206570], | 99.99th=[206570] bw ( KiB/s): min= 35, max= 176, per=24.22%, avg=95.90, stdev=36.55, samples=30 iops : min= 8, max= 44, avg=23.37, stdev= 9.17, samples=30 lat (usec) : 10=1.04%, 20=3.12%, 50=9.77%, 100=1.04%, 250=7.55% lat (usec) : 500=10.42%, 750=1.04%, 1000=0.26% lat (msec) : 2=1.04%, 4=0.26%, 10=7.29%, 20=14.84%, 50=31.77% lat (msec) : 100=9.90%, 250=0.65% cpu : usr=0.06%, sys=3.01%, ctx=886, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147923: Fri Jun 12 06:23:50 2026 read: IOPS=24, BW=99.9KiB/s (102kB/s)(1508KiB/15097msec) clat (usec): min=5, max=104311, avg=23203.93, stdev=23810.88 lat (usec): min=5, max=104314, avg=23204.80, stdev=23810.91 clat percentiles (usec): | 1.00th=[ 9], 5.00th=[ 16], 10.00th=[ 19], 20.00th=[ 27], | 30.00th=[ 35], 40.00th=[ 7111], 50.00th=[ 18482], 60.00th=[ 27132], | 70.00th=[ 36439], 80.00th=[ 45876], 90.00th=[ 56886], 95.00th=[ 65274], | 99.00th=[ 82314], 99.50th=[ 98042], 99.90th=[104334], 99.95th=[104334], | 99.99th=[104334] bw ( KiB/s): min= 29, max= 158, per=26.04%, avg=98.17, stdev=32.72, samples=29 iops : min= 7, max= 39, avg=23.97, stdev= 8.21, samples=29 write: IOPS=25, BW=104KiB/s (106kB/s)(1564KiB/15097msec); 0 zone resets clat (usec): min=174, max=173305, avg=16183.43, stdev=21652.82 lat (usec): min=174, max=173307, avg=16184.46, stdev=21652.83 clat percentiles (usec): | 1.00th=[ 186], 5.00th=[ 206], 10.00th=[ 233], 20.00th=[ 273], | 30.00th=[ 363], 40.00th=[ 1418], 50.00th=[ 7832], 60.00th=[ 13566], | 70.00th=[ 21627], 80.00th=[ 31589], 90.00th=[ 42730], 95.00th=[ 55837], | 99.00th=[ 84411], 99.50th=[127402], 99.90th=[173016], 99.95th=[173016], | 99.99th=[173016] bw ( KiB/s): min= 41, max= 167, per=26.07%, avg=103.24, stdev=33.22, samples=29 iops : min= 10, max= 41, avg=25.24, stdev= 8.35, samples=29 lat (usec) : 10=1.04%, 20=5.21%, 50=11.46%, 100=0.65%, 250=8.59% lat (usec) : 500=9.77%, 750=0.78%, 1000=0.65% lat (msec) : 2=2.08%, 4=1.43%, 10=5.34%, 20=12.63%, 50=28.39% lat (msec) : 100=11.46%, 250=0.52% cpu : usr=0.10%, sys=2.33%, ctx=800, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=377KiB/s (386kB/s), 90.0KiB/s-99.9KiB/s (93.2kB/s-102kB/s), io=5988KiB (6132kB), run=15097-15878msec WRITE: bw=397KiB/s (406kB/s), 96.2KiB/s-104KiB/s (98.5kB/s-106kB/s), io=6300KiB (6451kB), run=15097-15878msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147915: Fri Jun 12 06:23:56 2026 read: IOPS=15, BW=62.2KiB/s (63.7kB/s)(1480KiB/23785msec) clat (msec): min=2, max=299, avg=22.92, stdev=25.60 lat (msec): min=2, max=299, avg=22.92, stdev=25.60 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 34], 90.00th=[ 48], 95.00th=[ 77], | 99.00th=[ 108], 99.50th=[ 112], 99.90th=[ 300], 99.95th=[ 300], | 99.99th=[ 300] bw ( KiB/s): min= 7, max= 159, per=23.79%, avg=59.72, stdev=38.16, samples=46 iops : min= 1, max= 39, avg=14.28, stdev= 9.59, samples=46 write: IOPS=16, BW=66.9KiB/s (68.5kB/s)(1592KiB/23785msec); 0 zone resets clat (msec): min=5, max=173, avg=38.19, stdev=27.78 lat (msec): min=5, max=173, avg=38.19, stdev=27.78 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 17], | 30.00th=[ 21], 40.00th=[ 24], 50.00th=[ 29], 60.00th=[ 38], | 70.00th=[ 46], 80.00th=[ 57], 90.00th=[ 73], 95.00th=[ 90], | 99.00th=[ 150], 99.50th=[ 161], 99.90th=[ 174], 99.95th=[ 174], | 99.99th=[ 174] bw ( KiB/s): min= 15, max= 192, per=24.82%, avg=65.52, stdev=38.81, samples=46 iops : min= 3, max= 48, avg=15.74, stdev= 9.79, samples=46 lat (msec) : 4=0.91%, 10=17.32%, 20=27.08%, 50=36.72%, 100=15.10% lat (msec) : 250=2.73%, 500=0.13% cpu : usr=0.08%, sys=0.98%, ctx=903, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147916: Fri Jun 12 06:23:56 2026 read: IOPS=15, BW=60.0KiB/s (62.4kB/s)(1436KiB/23553msec) clat (msec): min=2, max=305, avg=23.08, stdev=25.27 lat (msec): min=2, max=305, avg=23.08, stdev=25.27 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 15], 60.00th=[ 19], | 70.00th=[ 26], 80.00th=[ 35], 90.00th=[ 51], 95.00th=[ 69], | 99.00th=[ 105], 99.50th=[ 118], 99.90th=[ 305], 99.95th=[ 305], | 99.99th=[ 305] bw ( KiB/s): min= 11, max= 159, per=24.03%, avg=60.33, stdev=36.56, samples=46 iops : min= 2, max= 39, avg=14.43, stdev= 9.17, samples=46 write: IOPS=17, BW=69.5KiB/s (71.1kB/s)(1636KiB/23553msec); 0 zone resets clat (msec): min=5, max=209, avg=37.16, stdev=25.72 lat (msec): min=5, max=209, avg=37.17, stdev=25.72 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 14], 20.00th=[ 17], | 30.00th=[ 21], 40.00th=[ 25], 50.00th=[ 31], 60.00th=[ 37], | 70.00th=[ 44], 80.00th=[ 54], 90.00th=[ 73], 95.00th=[ 88], | 99.00th=[ 120], 99.50th=[ 127], 99.90th=[ 211], 99.95th=[ 211], | 99.99th=[ 211] bw ( KiB/s): min= 23, max= 208, per=25.95%, avg=68.50, stdev=40.58, samples=46 iops : min= 5, max= 52, avg=16.48, stdev=10.23, samples=46 lat (msec) : 4=1.30%, 10=14.19%, 20=29.56%, 50=38.80%, 100=14.06% lat (msec) : 250=1.95%, 500=0.13% cpu : usr=0.07%, sys=0.88%, ctx=875, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147917: Fri Jun 12 06:23:56 2026 read: IOPS=16, BW=66.5KiB/s (68.1kB/s)(1564KiB/23525msec) clat (msec): min=2, max=167, avg=23.63, stdev=22.62 lat (msec): min=2, max=167, avg=23.64, stdev=22.63 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 13], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 26], 80.00th=[ 36], 90.00th=[ 57], 95.00th=[ 71], | 99.00th=[ 105], 99.50th=[ 138], 99.90th=[ 167], 99.95th=[ 167], | 99.99th=[ 167] bw ( KiB/s): min= 23, max= 239, per=26.53%, avg=66.59, stdev=45.16, samples=46 iops : min= 5, max= 59, avg=16.00, stdev=11.35, samples=46 write: IOPS=16, BW=64.1KiB/s (65.6kB/s)(1508KiB/23525msec); 0 zone resets clat (msec): min=5, max=170, avg=37.75, stdev=26.97 lat (msec): min=5, max=170, avg=37.75, stdev=26.97 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 18], | 30.00th=[ 21], 40.00th=[ 25], 50.00th=[ 31], 60.00th=[ 35], | 70.00th=[ 43], 80.00th=[ 55], 90.00th=[ 78], 95.00th=[ 91], | 99.00th=[ 144], 99.50th=[ 153], 99.90th=[ 171], 99.95th=[ 171], | 99.99th=[ 171] bw ( KiB/s): min= 15, max= 192, per=23.95%, avg=63.24, stdev=38.67, samples=46 iops : min= 3, max= 48, avg=15.17, stdev= 9.76, samples=46 lat (msec) : 4=1.04%, 10=16.41%, 20=27.60%, 50=37.76%, 100=14.58% lat (msec) : 250=2.60% cpu : usr=0.14%, sys=0.99%, ctx=887, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147918: Fri Jun 12 06:23:56 2026 read: IOPS=15, BW=63.3KiB/s (64.8kB/s)(1508KiB/23818msec) clat (msec): min=3, max=118, avg=23.36, stdev=21.58 lat (msec): min=3, max=118, avg=23.36, stdev=21.58 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 26], 80.00th=[ 36], 90.00th=[ 53], 95.00th=[ 73], | 99.00th=[ 102], 99.50th=[ 105], 99.90th=[ 120], 99.95th=[ 120], | 99.99th=[ 120] bw ( KiB/s): min= 15, max= 191, per=24.25%, avg=60.87, stdev=38.82, samples=46 iops : min= 3, max= 47, avg=14.59, stdev= 9.77, samples=46 write: IOPS=16, BW=65.7KiB/s (67.2kB/s)(1564KiB/23818msec); 0 zone resets clat (msec): min=5, max=181, avg=38.27, stdev=27.29 lat (msec): min=5, max=181, avg=38.27, stdev=27.29 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 11], 10.00th=[ 14], 20.00th=[ 18], | 30.00th=[ 21], 40.00th=[ 25], 50.00th=[ 31], 60.00th=[ 37], | 70.00th=[ 45], 80.00th=[ 56], 90.00th=[ 75], 95.00th=[ 92], | 99.00th=[ 131], 99.50th=[ 171], 99.90th=[ 182], 99.95th=[ 182], | 99.99th=[ 182] bw ( KiB/s): min= 23, max= 208, per=24.17%, avg=63.80, stdev=37.81, samples=46 iops : min= 5, max= 52, avg=15.30, stdev= 9.53, samples=46 lat (msec) : 4=1.17%, 10=16.15%, 20=27.73%, 50=36.33%, 100=16.02% lat (msec) : 250=2.60% cpu : usr=0.09%, sys=1.01%, ctx=881, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=251KiB/s (257kB/s), 60.0KiB/s-66.5KiB/s (62.4kB/s-68.1kB/s), io=5988KiB (6132kB), run=23525-23818msec WRITE: bw=265KiB/s (271kB/s), 64.1KiB/s-69.5KiB/s (65.6kB/s-71.1kB/s), io=6300KiB (6451kB), run=23525-23818msec