== sanity test 398b: DIO and buffer IO race ============== 05:54:04 (1781258044) /usr/bin/fio 12+0 records in 12+0 records out 12582912 bytes (13 MB, 12 MiB) copied, 0.371673 s, 33.9 MB/s mix direct rw 12M to OST0 by fio with 4 jobs... mix buffer rw 12M to OST0 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=124397: Fri Jun 12 05:54:31 2026 read: IOPS=28, BW=116KiB/s (118kB/s)(1480KiB/12799msec) clat (usec): min=6, max=178083, avg=15000.29, stdev=21792.48 lat (usec): min=6, max=178083, avg=15000.99, stdev=21792.49 clat percentiles (usec): | 1.00th=[ 7], 5.00th=[ 10], 10.00th=[ 17], 20.00th=[ 25], | 30.00th=[ 36], 40.00th=[ 53], 50.00th=[ 3982], 60.00th=[ 14877], | 70.00th=[ 20055], 80.00th=[ 27132], 90.00th=[ 41157], 95.00th=[ 53216], | 99.00th=[119014], 99.50th=[125305], 99.90th=[177210], 99.95th=[177210], | 99.99th=[177210] bw ( KiB/s): min= 47, max= 264, per=28.64%, avg=116.29, stdev=51.38, samples=24 iops : min= 11, max= 66, avg=28.54, stdev=12.93, samples=24 write: IOPS=31, BW=124KiB/s (127kB/s)(1592KiB/12799msec); 0 zone resets clat (usec): min=169, max=199012, avg=18019.90, stdev=26660.04 lat (usec): min=170, max=199012, avg=18021.12, stdev=26659.90 clat percentiles (usec): | 1.00th=[ 182], 5.00th=[ 202], 10.00th=[ 235], 20.00th=[ 310], | 30.00th=[ 453], 40.00th=[ 1811], 50.00th=[ 6652], 60.00th=[ 12518], | 70.00th=[ 21365], 80.00th=[ 31065], 90.00th=[ 51643], 95.00th=[ 73925], | 99.00th=[129500], 99.50th=[139461], 99.90th=[198181], 99.95th=[198181], | 99.99th=[198181] bw ( KiB/s): min= 55, max= 230, per=29.53%, avg=126.08, stdev=47.47, samples=24 iops : min= 13, max= 57, avg=31.00, stdev=11.93, samples=24 lat (usec) : 10=2.47%, 20=4.82%, 50=11.20%, 100=2.86%, 250=7.42% lat (usec) : 500=10.68%, 750=1.95%, 1000=1.43% lat (msec) : 2=1.82%, 4=2.21%, 10=7.42%, 20=15.10%, 50=21.88% lat (msec) : 100=7.16%, 250=1.56% cpu : usr=0.09%, sys=2.77%, ctx=850, majf=0, minf=19 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=124398: Fri Jun 12 05:54:31 2026 read: IOPS=25, BW=102KiB/s (105kB/s)(1436KiB/14049msec) clat (usec): min=6, max=230071, avg=14608.48, stdev=24274.88 lat (usec): min=7, max=230072, avg=14610.52, stdev=24274.26 clat percentiles (usec): | 1.00th=[ 8], 5.00th=[ 12], 10.00th=[ 17], 20.00th=[ 28], | 30.00th=[ 35], 40.00th=[ 49], 50.00th=[ 478], 60.00th=[ 14615], | 70.00th=[ 21627], 80.00th=[ 28181], 90.00th=[ 37487], 95.00th=[ 45351], | 99.00th=[160433], 99.50th=[162530], 99.90th=[229639], 99.95th=[229639], | 99.99th=[229639] bw ( KiB/s): min= 31, max= 175, per=25.28%, avg=102.63, stdev=40.17, samples=27 iops : min= 7, max= 43, avg=25.11, stdev=10.08, samples=27 write: IOPS=29, BW=116KiB/s (119kB/s)(1636KiB/14049msec); 0 zone resets clat (usec): min=177, max=187456, avg=21240.95, stdev=28581.40 lat (usec): min=177, max=187457, avg=21241.94, stdev=28581.38 clat percentiles (usec): | 1.00th=[ 200], 5.00th=[ 225], 10.00th=[ 258], 20.00th=[ 338], | 30.00th=[ 734], 40.00th=[ 2769], 50.00th=[ 9110], 60.00th=[ 16909], | 70.00th=[ 27395], 80.00th=[ 38536], 90.00th=[ 61604], 95.00th=[ 77071], | 99.00th=[125305], 99.50th=[133694], 99.90th=[187696], 99.95th=[187696], | 99.99th=[187696] bw ( KiB/s): min= 63, max= 174, per=27.28%, avg=116.48, stdev=28.08, samples=27 iops : min= 15, max= 43, avg=28.59, stdev= 6.97, samples=27 lat (usec) : 10=1.04%, 20=5.21%, 50=12.89%, 100=3.12%, 250=5.34% lat (usec) : 500=9.90%, 750=2.08%, 1000=1.30% lat (msec) : 2=3.65%, 4=2.21%, 10=6.90%, 20=11.46%, 50=25.91% lat (msec) : 100=7.16%, 250=1.82% cpu : usr=0.15%, sys=2.71%, ctx=872, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=124400: Fri Jun 12 05:54:31 2026 read: IOPS=27, BW=110KiB/s (112kB/s)(1564KiB/14254msec) clat (usec): min=6, max=198643, avg=15622.81, stdev=20842.80 lat (usec): min=6, max=198645, avg=15623.64, stdev=20842.80 clat percentiles (usec): | 1.00th=[ 8], 5.00th=[ 12], 10.00th=[ 19], 20.00th=[ 28], | 30.00th=[ 37], 40.00th=[ 58], 50.00th=[ 8848], 60.00th=[ 17695], | 70.00th=[ 22414], 80.00th=[ 29492], 90.00th=[ 40633], 95.00th=[ 54264], | 99.00th=[ 77071], 99.50th=[105382], 99.90th=[198181], 99.95th=[198181], | 99.99th=[198181] bw ( KiB/s): min= 39, max= 216, per=27.36%, avg=111.07, stdev=39.59, samples=27 iops : min= 9, max= 54, avg=27.22, stdev= 9.98, samples=27 write: IOPS=26, BW=106KiB/s (108kB/s)(1508KiB/14254msec); 0 zone resets clat (usec): min=185, max=141395, avg=21490.65, stdev=28400.78 lat (usec): min=185, max=141396, avg=21491.76, stdev=28400.85 clat percentiles (usec): | 1.00th=[ 188], 5.00th=[ 239], 10.00th=[ 273], 20.00th=[ 351], | 30.00th=[ 1123], 40.00th=[ 6325], 50.00th=[ 11076], 60.00th=[ 17433], | 70.00th=[ 25035], 80.00th=[ 36963], 90.00th=[ 60556], 95.00th=[ 87557], | 99.00th=[124257], 99.50th=[135267], 99.90th=[141558], 99.95th=[141558], | 99.99th=[141558] bw ( KiB/s): min= 31, max= 200, per=24.52%, avg=104.70, stdev=35.62, samples=27 iops : min= 7, max= 50, avg=25.63, stdev= 8.97, samples=27 lat (usec) : 10=1.95%, 20=3.65%, 50=13.41%, 100=2.99%, 250=3.65% lat (usec) : 500=9.90%, 750=1.56%, 1000=0.65% lat (msec) : 2=1.82%, 4=2.99%, 10=6.64%, 20=16.15%, 50=24.87% lat (msec) : 100=7.68%, 250=2.08% cpu : usr=0.11%, sys=2.88%, ctx=936, majf=0, minf=19 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=124401: Fri Jun 12 05:54:31 2026 read: IOPS=25, BW=102KiB/s (105kB/s)(1508KiB/14729msec) clat (usec): min=6, max=228837, avg=15387.54, stdev=24716.04 lat (usec): min=6, max=228837, avg=15388.25, stdev=24716.01 clat percentiles (usec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 14], 20.00th=[ 23], | 30.00th=[ 32], 40.00th=[ 42], 50.00th=[ 5538], 60.00th=[ 15270], | 70.00th=[ 21103], 80.00th=[ 28443], 90.00th=[ 37487], 95.00th=[ 46924], | 99.00th=[115868], 99.50th=[206570], 99.90th=[229639], 99.95th=[229639], | 99.99th=[229639] bw ( KiB/s): min= 47, max= 191, per=25.75%, avg=104.56, stdev=33.19, samples=27 iops : min= 11, max= 47, avg=25.59, stdev= 8.40, samples=27 write: IOPS=26, BW=106KiB/s (109kB/s)(1564KiB/14729msec); 0 zone resets clat (usec): min=172, max=276077, avg=22563.47, stdev=33375.15 lat (usec): min=173, max=276080, avg=22564.86, stdev=33375.03 clat percentiles (usec): | 1.00th=[ 206], 5.00th=[ 243], 10.00th=[ 293], 20.00th=[ 367], | 30.00th=[ 783], 40.00th=[ 4555], 50.00th=[ 10945], 60.00th=[ 16581], | 70.00th=[ 25035], 80.00th=[ 39584], 90.00th=[ 63177], 95.00th=[ 84411], | 99.00th=[166724], 99.50th=[202376], 99.90th=[274727], 99.95th=[274727], | 99.99th=[274727] bw ( KiB/s): min= 16, max= 167, per=24.49%, avg=104.57, stdev=36.30, samples=28 iops : min= 4, max= 41, avg=25.61, stdev= 9.10, samples=28 lat (usec) : 10=2.99%, 20=5.60%, 50=11.85%, 100=2.21%, 250=4.17% lat (usec) : 500=10.29%, 750=2.08%, 1000=1.17% lat (msec) : 2=1.43%, 4=1.95%, 10=7.03%, 20=16.15%, 50=23.83% lat (msec) : 100=6.51%, 250=2.60%, 500=0.13% cpu : usr=0.16%, sys=2.74%, ctx=872, majf=0, minf=19 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=407KiB/s (416kB/s), 102KiB/s-116KiB/s (105kB/s-118kB/s), io=5988KiB (6132kB), run=12799-14729msec WRITE: bw=428KiB/s (438kB/s), 106KiB/s-124KiB/s (108kB/s-127kB/s), io=6300KiB (6451kB), run=12799-14729msec rand-rw: (groupid=0, jobs=1): err= 0: pid=124396: Fri Jun 12 05:55:03 2026 read: IOPS=8, BW=32.2KiB/s (32.0kB/s)(1480KiB/45982msec) clat (msec): min=2, max=170, avg=18.32, stdev=20.98 lat (msec): min=2, max=170, avg=18.33, stdev=20.98 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 15], | 70.00th=[ 17], 80.00th=[ 23], 90.00th=[ 37], 95.00th=[ 56], | 99.00th=[ 133], 99.50th=[ 148], 99.90th=[ 171], 99.95th=[ 171], | 99.99th=[ 171] bw ( KiB/s): min= 7, max= 102, per=25.67%, avg=32.35, stdev=18.91, samples=89 iops : min= 1, max= 25, avg= 7.54, stdev= 4.83, samples=89 write: IOPS=8, BW=34.6KiB/s (35.5kB/s)(1592KiB/45982msec); 0 zone resets clat (msec): min=25, max=256, avg=98.23, stdev=37.35 lat (msec): min=25, max=256, avg=98.23, stdev=37.35 clat percentiles (msec): | 1.00th=[ 43], 5.00th=[ 52], 10.00th=[ 56], 20.00th=[ 67], | 30.00th=[ 78], 40.00th=[ 85], 50.00th=[ 92], 60.00th=[ 100], | 70.00th=[ 112], 80.00th=[ 124], 90.00th=[ 150], 95.00th=[ 174], | 99.00th=[ 224], 99.50th=[ 232], 99.90th=[ 257], 99.95th=[ 257], | 99.99th=[ 257] bw ( KiB/s): min= 7, max= 55, per=25.65%, avg=34.11, stdev=10.32, samples=91 iops : min= 1, max= 13, avg= 7.97, stdev= 2.61, samples=91 lat (msec) : 4=1.17%, 10=18.10%, 20=17.58%, 50=10.29%, 100=31.90% lat (msec) : 250=20.83%, 500=0.13% cpu : usr=0.10%, sys=0.57%, ctx=862, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=124399: Fri Jun 12 05:55:03 2026 read: IOPS=7, BW=30.3KiB/s (31.1kB/s)(1436KiB/47325msec) clat (msec): min=3, max=134, avg=16.95, stdev=17.84 lat (msec): min=3, max=134, avg=16.95, stdev=17.84 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 16], 80.00th=[ 22], 90.00th=[ 33], 95.00th=[ 54], | 99.00th=[ 97], 99.50th=[ 128], 99.90th=[ 136], 99.95th=[ 136], | 99.99th=[ 136] bw ( KiB/s): min= 5, max= 119, per=25.84%, avg=32.55, stdev=20.43, samples=85 iops : min= 1, max= 29, avg= 7.59, stdev= 5.12, samples=85 write: IOPS=8, BW=34.6KiB/s (35.4kB/s)(1636KiB/47325msec); 0 zone resets clat (msec): min=35, max=251, avg=100.53, stdev=38.18 lat (msec): min=35, max=251, avg=100.54, stdev=38.18 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 51], 10.00th=[ 58], 20.00th=[ 66], | 30.00th=[ 78], 40.00th=[ 88], 50.00th=[ 95], 60.00th=[ 105], | 70.00th=[ 115], 80.00th=[ 129], 90.00th=[ 150], 95.00th=[ 176], | 99.00th=[ 224], 99.50th=[ 230], 99.90th=[ 251], 99.95th=[ 251], | 99.99th=[ 251] bw ( KiB/s): min= 15, max= 64, per=25.45%, avg=33.85, stdev=10.41, samples=93 iops : min= 3, max= 16, avg= 7.91, stdev= 2.62, samples=93 lat (msec) : 4=0.52%, 10=19.01%, 20=16.41%, 50=10.68%, 100=29.95% lat (msec) : 250=23.31%, 500=0.13% cpu : usr=0.07%, sys=0.61%, ctx=858, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=124402: Fri Jun 12 05:55:03 2026 read: IOPS=8, BW=34.6KiB/s (35.4kB/s)(1564KiB/45189msec) clat (msec): min=2, max=107, avg=16.40, stdev=15.35 lat (msec): min=2, max=107, avg=16.40, stdev=15.35 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 21], 90.00th=[ 30], 95.00th=[ 52], | 99.00th=[ 87], 99.50th=[ 91], 99.90th=[ 108], 99.95th=[ 108], | 99.99th=[ 108] bw ( KiB/s): min= 7, max= 119, per=27.93%, avg=35.20, stdev=24.54, samples=87 iops : min= 1, max= 29, avg= 8.24, stdev= 6.16, samples=87 write: IOPS=8, BW=33.4KiB/s (34.2kB/s)(1508KiB/45189msec); 0 zone resets clat (msec): min=40, max=271, avg=102.60, stdev=39.67 lat (msec): min=40, max=271, avg=102.60, stdev=39.67 clat percentiles (msec): | 1.00th=[ 44], 5.00th=[ 52], 10.00th=[ 60], 20.00th=[ 70], | 30.00th=[ 80], 40.00th=[ 88], 50.00th=[ 95], 60.00th=[ 105], | 70.00th=[ 115], 80.00th=[ 130], 90.00th=[ 155], 95.00th=[ 182], | 99.00th=[ 241], 99.50th=[ 271], 99.90th=[ 271], 99.95th=[ 271], | 99.99th=[ 271] bw ( KiB/s): min= 11, max= 48, per=24.68%, avg=32.82, stdev= 9.69, samples=89 iops : min= 2, max= 12, avg= 7.64, stdev= 2.46, samples=89 lat (msec) : 4=1.82%, 10=19.01%, 20=18.75%, 50=10.55%, 100=28.13% lat (msec) : 250=21.48%, 500=0.26% cpu : usr=0.09%, sys=0.70%, ctx=850, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=124403: Fri Jun 12 05:55:03 2026 read: IOPS=8, BW=32.9KiB/s (33.7kB/s)(1508KiB/45832msec) clat (msec): min=2, max=180, avg=16.12, stdev=17.54 lat (msec): min=2, max=180, avg=16.12, stdev=17.54 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 21], 90.00th=[ 32], 95.00th=[ 48], | 99.00th=[ 106], 99.50th=[ 127], 99.90th=[ 182], 99.95th=[ 182], | 99.99th=[ 182] bw ( KiB/s): min= 7, max= 87, per=26.63%, avg=33.55, stdev=19.10, samples=87 iops : min= 1, max= 21, avg= 7.83, stdev= 4.82, samples=87 write: IOPS=8, BW=34.1KiB/s (34.9kB/s)(1564KiB/45832msec); 0 zone resets clat (msec): min=37, max=269, avg=101.44, stdev=38.63 lat (msec): min=37, max=269, avg=101.44, stdev=38.63 clat percentiles (msec): | 1.00th=[ 47], 5.00th=[ 55], 10.00th=[ 61], 20.00th=[ 69], | 30.00th=[ 78], 40.00th=[ 85], 50.00th=[ 93], 60.00th=[ 104], | 70.00th=[ 115], 80.00th=[ 132], 90.00th=[ 157], 95.00th=[ 171], | 99.00th=[ 224], 99.50th=[ 264], 99.90th=[ 271], 99.95th=[ 271], | 99.99th=[ 271] bw ( KiB/s): min= 15, max= 64, per=25.37%, avg=33.74, stdev= 9.80, samples=90 iops : min= 3, max= 16, avg= 7.88, stdev= 2.49, samples=90 lat (msec) : 4=1.17%, 10=20.70%, 20=17.58%, 50=8.98%, 100=29.56% lat (msec) : 250=21.74%, 500=0.26% cpu : usr=0.06%, sys=0.63%, ctx=858, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=127KiB/s (130kB/s), 30.3KiB/s-34.6KiB/s (31.1kB/s-35.4kB/s), io=5988KiB (6132kB), run=45189-47325msec WRITE: bw=133KiB/s (136kB/s), 33.4KiB/s-34.6KiB/s (34.2kB/s-35.5kB/s), io=6300KiB (6451kB), run=45189-47325msec