== sanity test 398b: DIO and buffer IO race ============== 11:54:29 (1776268469) /usr/bin/fio 12+0 records in 12+0 records out 12582912 bytes (13 MB, 12 MiB) copied, 0.446058 s, 28.2 MB/s mix direct rw 12M to OST0 by fio with 4 jobs... mix buffer rw 12M to OST0 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=122931: Wed Apr 15 11:55:00 2026 read: IOPS=26, BW=105KiB/s (108kB/s)(1480KiB/14064msec) clat (usec): min=6, max=127512, avg=14539.06, stdev=21722.13 lat (usec): min=6, max=127513, avg=14539.97, stdev=21722.09 clat percentiles (usec): | 1.00th=[ 8], 5.00th=[ 12], 10.00th=[ 16], 20.00th=[ 25], | 30.00th=[ 33], 40.00th=[ 44], 50.00th=[ 72], 60.00th=[ 9765], | 70.00th=[ 19006], 80.00th=[ 30278], 90.00th=[ 44303], 95.00th=[ 60556], | 99.00th=[ 87557], 99.50th=[110625], 99.90th=[127402], 99.95th=[127402], | 99.99th=[127402] bw ( KiB/s): min= 31, max= 202, per=26.06%, avg=101.11, stdev=48.72, samples=27 iops : min= 7, max= 50, avg=24.74, stdev=12.26, samples=27 write: IOPS=28, BW=113KiB/s (116kB/s)(1592KiB/14064msec); 0 zone resets clat (usec): min=170, max=189371, avg=21758.49, stdev=33370.74 lat (usec): min=170, max=189371, avg=21761.13, stdev=33369.75 clat percentiles (usec): | 1.00th=[ 182], 5.00th=[ 202], 10.00th=[ 233], 20.00th=[ 273], | 30.00th=[ 355], 40.00th=[ 1483], 50.00th=[ 8094], 60.00th=[ 14746], | 70.00th=[ 23200], 80.00th=[ 37487], 90.00th=[ 62129], 95.00th=[ 90702], | 99.00th=[158335], 99.50th=[175113], 99.90th=[189793], 99.95th=[189793], | 99.99th=[189793] bw ( KiB/s): min= 28, max= 239, per=26.61%, avg=108.85, stdev=48.53, samples=27 iops : min= 7, max= 59, avg=26.70, stdev=12.10, samples=27 lat (usec) : 10=1.82%, 20=5.47%, 50=13.54%, 100=4.17%, 250=8.59% lat (usec) : 500=10.94%, 750=1.43%, 1000=0.65% lat (msec) : 2=1.82%, 4=1.82%, 10=5.99%, 20=12.37%, 50=19.66% lat (msec) : 100=9.24%, 250=2.47% cpu : usr=0.17%, sys=2.77%, ctx=756, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=122932: Wed Apr 15 11:55:00 2026 read: IOPS=24, BW=96.8KiB/s (99.1kB/s)(1436KiB/14836msec) clat (usec): min=6, max=206238, avg=18237.56, stdev=30788.38 lat (usec): min=6, max=206239, avg=18238.50, stdev=30788.28 clat percentiles (usec): | 1.00th=[ 9], 5.00th=[ 12], 10.00th=[ 15], 20.00th=[ 24], | 30.00th=[ 32], 40.00th=[ 49], 50.00th=[ 135], 60.00th=[ 12911], | 70.00th=[ 23200], 80.00th=[ 34341], 90.00th=[ 45876], 95.00th=[ 66323], | 99.00th=[170918], 99.50th=[204473], 99.90th=[206570], 99.95th=[206570], | 99.99th=[206570] bw ( KiB/s): min= 28, max= 175, per=24.56%, avg=95.31, stdev=34.28, samples=29 iops : min= 7, max= 43, avg=23.34, stdev= 8.54, samples=29 write: IOPS=27, BW=110KiB/s (113kB/s)(1636KiB/14836msec); 0 zone resets clat (usec): min=126, max=198461, avg=20156.97, stdev=28393.64 lat (usec): min=127, max=198462, avg=20158.01, stdev=28393.60 clat percentiles (usec): | 1.00th=[ 182], 5.00th=[ 204], 10.00th=[ 231], 20.00th=[ 302], | 30.00th=[ 416], 40.00th=[ 2671], 50.00th=[ 9896], 60.00th=[ 15008], | 70.00th=[ 23200], 80.00th=[ 38011], 90.00th=[ 52167], 95.00th=[ 80217], | 99.00th=[122160], 99.50th=[135267], 99.90th=[198181], 99.95th=[198181], | 99.99th=[198181] bw ( KiB/s): min= 15, max= 191, per=26.24%, avg=107.31, stdev=46.40, samples=29 iops : min= 3, max= 47, avg=26.38, stdev=11.59, samples=29 lat (usec) : 10=1.43%, 20=6.12%, 50=11.59%, 100=4.17%, 250=7.81% lat (usec) : 500=10.29%, 750=1.69%, 1000=1.30% lat (msec) : 2=1.04%, 4=1.69%, 10=5.86%, 20=12.63%, 50=24.61% lat (msec) : 100=7.03%, 250=2.73% cpu : usr=0.13%, sys=2.60%, ctx=813, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=122933: Wed Apr 15 11:55:00 2026 read: IOPS=25, BW=102KiB/s (104kB/s)(1564KiB/15399msec) clat (usec): min=6, max=205388, avg=17411.50, stdev=27984.49 lat (usec): min=6, max=205389, avg=17412.76, stdev=27984.84 clat percentiles (usec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 13], 20.00th=[ 22], | 30.00th=[ 32], 40.00th=[ 43], 50.00th=[ 204], 60.00th=[ 14091], | 70.00th=[ 23462], 80.00th=[ 33424], 90.00th=[ 48497], 95.00th=[ 64226], | 99.00th=[145753], 99.50th=[204473], 99.90th=[204473], 99.95th=[204473], | 99.99th=[204473] bw ( KiB/s): min= 28, max= 320, per=26.40%, avg=102.43, stdev=61.94, samples=30 iops : min= 7, max= 80, avg=25.13, stdev=15.54, samples=30 write: IOPS=24, BW=97.9KiB/s (100kB/s)(1508KiB/15399msec); 0 zone resets clat (usec): min=177, max=188075, avg=22729.68, stdev=32935.32 lat (usec): min=177, max=188078, avg=22730.83, stdev=32935.41 clat percentiles (usec): | 1.00th=[ 186], 5.00th=[ 204], 10.00th=[ 239], 20.00th=[ 310], | 30.00th=[ 412], 40.00th=[ 1860], 50.00th=[ 9634], 60.00th=[ 17433], | 70.00th=[ 26084], 80.00th=[ 36963], 90.00th=[ 66847], 95.00th=[ 92799], | 99.00th=[170918], 99.50th=[187696], 99.90th=[187696], 99.95th=[187696], | 99.99th=[187696] bw ( KiB/s): min= 28, max= 191, per=23.80%, avg=97.33, stdev=43.33, samples=30 iops : min= 7, max= 47, avg=23.87, stdev=10.85, samples=30 lat (usec) : 10=2.86%, 20=6.38%, 50=13.41%, 100=1.95%, 250=6.64% lat (usec) : 500=10.42%, 750=0.91%, 1000=1.30% lat (msec) : 2=2.08%, 4=1.56%, 10=5.86%, 20=11.20%, 50=23.70% lat (msec) : 100=8.98%, 250=2.73% cpu : usr=0.07%, sys=3.34%, ctx=837, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=122934: Wed Apr 15 11:55:00 2026 read: IOPS=24, BW=98.6KiB/s (101kB/s)(1508KiB/15298msec) clat (usec): min=5, max=231917, avg=17560.37, stdev=27758.24 lat (usec): min=5, max=231918, avg=17561.24, stdev=27758.22 clat percentiles (usec): | 1.00th=[ 7], 5.00th=[ 10], 10.00th=[ 15], 20.00th=[ 23], | 30.00th=[ 30], 40.00th=[ 41], 50.00th=[ 87], 60.00th=[ 13960], | 70.00th=[ 25297], 80.00th=[ 33817], 90.00th=[ 50070], 95.00th=[ 64750], | 99.00th=[129500], 99.50th=[183501], 99.90th=[231736], 99.95th=[231736], | 99.99th=[231736] bw ( KiB/s): min= 23, max= 214, per=25.09%, avg=97.37, stdev=51.16, samples=30 iops : min= 5, max= 53, avg=23.87, stdev=12.86, samples=30 write: IOPS=25, BW=102KiB/s (105kB/s)(1564KiB/15298msec); 0 zone resets clat (usec): min=165, max=179843, avg=22114.28, stdev=31423.77 lat (usec): min=166, max=179844, avg=22119.49, stdev=31424.50 clat percentiles (usec): | 1.00th=[ 176], 5.00th=[ 200], 10.00th=[ 227], 20.00th=[ 273], | 30.00th=[ 363], 40.00th=[ 1156], 50.00th=[ 9241], 60.00th=[ 17171], | 70.00th=[ 26608], 80.00th=[ 38011], 90.00th=[ 72877], 95.00th=[ 90702], | 99.00th=[141558], 99.50th=[166724], 99.90th=[179307], 99.95th=[179307], | 99.99th=[179307] bw ( KiB/s): min= 28, max= 248, per=24.90%, avg=101.83, stdev=50.67, samples=30 iops : min= 7, max= 62, avg=24.97, stdev=12.69, samples=30 lat (usec) : 10=2.60%, 20=5.47%, 50=13.54%, 100=2.99%, 250=8.72% lat (usec) : 500=10.42%, 750=1.56%, 1000=0.78% lat (msec) : 2=1.04%, 4=1.69%, 10=5.21%, 20=10.16%, 50=23.83% lat (msec) : 100=9.64%, 250=2.34% cpu : usr=0.20%, sys=2.77%, ctx=831, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=389KiB/s (398kB/s), 96.8KiB/s-105KiB/s (99.1kB/s-108kB/s), io=5988KiB (6132kB), run=14064-15399msec WRITE: bw=409KiB/s (419kB/s), 97.9KiB/s-113KiB/s (100kB/s-116kB/s), io=6300KiB (6451kB), run=14064-15399msec rand-rw: (groupid=0, jobs=1): err= 0: pid=122927: Wed Apr 15 11:55:43 2026 read: IOPS=6, BW=25.6KiB/s (26.3kB/s)(1480KiB/57726msec) clat (msec): min=3, max=273, avg=26.82, stdev=35.37 lat (msec): min=3, max=273, avg=26.82, stdev=35.37 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 17], 60.00th=[ 21], | 70.00th=[ 26], 80.00th=[ 33], 90.00th=[ 51], 95.00th=[ 72], | 99.00th=[ 209], 99.50th=[ 226], 99.90th=[ 275], 99.95th=[ 275], | 99.99th=[ 275] bw ( KiB/s): min= 7, max= 104, per=26.17%, avg=26.95, stdev=18.21, samples=106 iops : min= 1, max= 26, avg= 6.12, stdev= 4.64, samples=106 write: IOPS=6, BW=27.6KiB/s (28.2kB/s)(1592KiB/57726msec); 0 zone resets clat (msec): min=36, max=449, avg=119.81, stdev=49.50 lat (msec): min=36, max=449, avg=119.82, stdev=49.50 clat percentiles (msec): | 1.00th=[ 44], 5.00th=[ 58], 10.00th=[ 66], 20.00th=[ 79], | 30.00th=[ 88], 40.00th=[ 101], 50.00th=[ 112], 60.00th=[ 125], | 70.00th=[ 140], 80.00th=[ 159], 90.00th=[ 182], 95.00th=[ 211], | 99.00th=[ 262], 99.50th=[ 284], 99.90th=[ 451], 99.95th=[ 451], | 99.99th=[ 451] bw ( KiB/s): min= 7, max= 55, per=25.11%, avg=27.12, stdev= 9.92, samples=113 iops : min= 1, max= 13, avg= 6.16, stdev= 2.51, samples=113 lat (msec) : 4=0.78%, 10=11.98%, 20=16.02%, 50=15.49%, 100=22.53% lat (msec) : 250=32.42%, 500=0.78% cpu : usr=0.05%, sys=0.63%, ctx=992, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=122928: Wed Apr 15 11:55:43 2026 read: IOPS=6, BW=24.8KiB/s (25.4kB/s)(1436KiB/57933msec) clat (msec): min=2, max=244, avg=24.78, stdev=28.47 lat (msec): min=2, max=244, avg=24.78, stdev=28.47 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 27], 80.00th=[ 35], 90.00th=[ 49], 95.00th=[ 70], | 99.00th=[ 142], 99.50th=[ 220], 99.90th=[ 245], 99.95th=[ 245], | 99.99th=[ 245] bw ( KiB/s): min= 7, max= 88, per=25.63%, avg=26.40, stdev=17.07, samples=104 iops : min= 1, max= 22, avg= 5.99, stdev= 4.31, samples=104 write: IOPS=7, BW=28.2KiB/s (28.9kB/s)(1636KiB/57933msec); 0 zone resets clat (msec): min=23, max=362, avg=119.77, stdev=51.54 lat (msec): min=23, max=362, avg=119.78, stdev=51.54 clat percentiles (msec): | 1.00th=[ 49], 5.00th=[ 61], 10.00th=[ 66], 20.00th=[ 78], | 30.00th=[ 89], 40.00th=[ 101], 50.00th=[ 111], 60.00th=[ 121], | 70.00th=[ 136], 80.00th=[ 155], 90.00th=[ 186], 95.00th=[ 211], | 99.00th=[ 296], 99.50th=[ 338], 99.90th=[ 363], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 7, max= 62, per=25.39%, avg=27.42, stdev= 9.90, samples=114 iops : min= 1, max= 15, avg= 6.24, stdev= 2.47, samples=114 lat (msec) : 4=1.04%, 10=12.11%, 20=15.36%, 50=14.45%, 100=23.83% lat (msec) : 250=31.64%, 500=1.56% cpu : usr=0.08%, sys=0.70%, ctx=989, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=122929: Wed Apr 15 11:55:43 2026 read: IOPS=7, BW=28.5KiB/s (29.2kB/s)(1564KiB/54801msec) clat (msec): min=3, max=231, avg=20.78, stdev=21.55 lat (msec): min=3, max=231, avg=20.78, stdev=21.55 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 15], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 28], 90.00th=[ 44], 95.00th=[ 54], | 99.00th=[ 112], 99.50th=[ 157], 99.90th=[ 232], 99.95th=[ 232], | 99.99th=[ 232] bw ( KiB/s): min= 7, max= 95, per=29.20%, avg=30.08, stdev=20.97, samples=101 iops : min= 1, max= 23, avg= 6.90, stdev= 5.28, samples=101 write: IOPS=6, BW=27.5KiB/s (28.2kB/s)(1508KiB/54801msec); 0 zone resets clat (msec): min=45, max=363, avg=123.49, stdev=49.78 lat (msec): min=45, max=363, avg=123.49, stdev=49.78 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 67], 10.00th=[ 72], 20.00th=[ 84], | 30.00th=[ 93], 40.00th=[ 105], 50.00th=[ 115], 60.00th=[ 126], | 70.00th=[ 136], 80.00th=[ 157], 90.00th=[ 188], 95.00th=[ 230], | 99.00th=[ 288], 99.50th=[ 347], 99.90th=[ 363], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 7, max= 47, per=24.87%, avg=26.86, stdev= 8.24, samples=108 iops : min= 1, max= 11, avg= 6.09, stdev= 2.05, samples=108 lat (msec) : 4=0.39%, 10=14.32%, 20=19.79%, 50=14.19%, 100=19.79% lat (msec) : 250=30.34%, 500=1.17% cpu : usr=0.06%, sys=0.61%, ctx=957, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=122930: Wed Apr 15 11:55:43 2026 read: IOPS=6, BW=26.9KiB/s (27.5kB/s)(1508KiB/56072msec) clat (msec): min=3, max=216, avg=23.20, stdev=25.88 lat (msec): min=3, max=216, avg=23.20, stdev=25.88 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 31], 90.00th=[ 45], 95.00th=[ 62], | 99.00th=[ 169], 99.50th=[ 180], 99.90th=[ 218], 99.95th=[ 218], | 99.99th=[ 218] bw ( KiB/s): min= 7, max= 95, per=27.29%, avg=28.11, stdev=19.58, samples=103 iops : min= 1, max= 23, avg= 6.42, stdev= 4.91, samples=103 write: IOPS=6, BW=27.9KiB/s (28.6kB/s)(1564KiB/56072msec); 0 zone resets clat (msec): min=40, max=409, avg=120.78, stdev=50.91 lat (msec): min=40, max=409, avg=120.78, stdev=50.91 clat percentiles (msec): | 1.00th=[ 47], 5.00th=[ 59], 10.00th=[ 69], 20.00th=[ 84], | 30.00th=[ 91], 40.00th=[ 103], 50.00th=[ 115], 60.00th=[ 125], | 70.00th=[ 136], 80.00th=[ 150], 90.00th=[ 178], 95.00th=[ 209], | 99.00th=[ 321], 99.50th=[ 388], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 7, max= 55, per=25.25%, avg=27.27, stdev= 8.89, samples=111 iops : min= 1, max= 13, avg= 6.20, stdev= 2.25, samples=111 lat (msec) : 4=0.39%, 10=12.89%, 20=18.23%, 50=14.58%, 100=20.96% lat (msec) : 250=31.51%, 500=1.43% cpu : usr=0.06%, sys=0.63%, ctx=962, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=103KiB/s (106kB/s), 24.8KiB/s-28.5KiB/s (25.4kB/s-29.2kB/s), io=5988KiB (6132kB), run=54801-57933msec WRITE: bw=109KiB/s (111kB/s), 27.5KiB/s-28.2KiB/s (28.2kB/s-28.9kB/s), io=6300KiB (6451kB), run=54801-57933msec