== sanity test 398b: DIO and buffer IO race ============== 17:42:15 (1722548535) /usr/bin/fio 12+0 records in 12+0 records out 12582912 bytes (13 MB) copied, 0.173329 s, 72.6 MB/s mix direct rw 12M to OST0 by fio with 4 jobs... mix buffer rw 12M to OST0 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=27501: Thu Aug 1 17:42:23 2024 read: IOPS=60, BW=244KiB/s (250kB/s)(1480KiB/6068msec) clat (usec): min=3, max=118684, avg=3736.07, stdev=9036.04 lat (usec): min=3, max=118685, avg=3736.86, stdev=9036.07 clat percentiles (usec): | 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 17], 50.00th=[ 24], 60.00th=[ 416], | 70.00th=[ 4752], 80.00th=[ 6194], 90.00th=[ 9110], 95.00th=[ 13042], | 99.00th=[ 43779], 99.50th=[ 62129], 99.90th=[119014], 99.95th=[119014], | 99.99th=[119014] bw ( KiB/s): min= 87, max= 336, per=24.58%, avg=237.92, stdev=87.13, samples=12 iops : min= 21, max= 84, avg=59.42, stdev=21.90, samples=12 write: IOPS=65, BW=262KiB/s (269kB/s)(1592KiB/6068msec) clat (usec): min=113, max=118429, avg=11605.26, stdev=18706.31 lat (usec): min=114, max=118430, avg=11606.68, stdev=18706.56 clat percentiles (usec): | 1.00th=[ 127], 5.00th=[ 139], 10.00th=[ 149], 20.00th=[ 176], | 30.00th=[ 229], 40.00th=[ 1729], 50.00th=[ 3326], 60.00th=[ 5866], | 70.00th=[ 9634], 80.00th=[ 20579], 90.00th=[ 36963], 95.00th=[ 55837], | 99.00th=[ 87557], 99.50th=[101188], 99.90th=[117965], 99.95th=[117965], | 99.99th=[117965] bw ( KiB/s): min= 95, max= 368, per=25.31%, avg=257.92, stdev=94.21, samples=12 iops : min= 23, max= 92, avg=64.42, stdev=23.67, samples=12 lat (usec) : 4=0.26%, 10=8.85%, 20=13.67%, 50=4.17%, 100=0.39% lat (usec) : 250=16.67%, 500=2.47%, 750=0.78%, 1000=1.30% lat (msec) : 2=2.60%, 4=8.98%, 10=20.96%, 20=7.16%, 50=8.20% lat (msec) : 100=3.12%, 250=0.39% cpu : usr=1.05%, sys=4.73%, ctx=802, majf=0, minf=37 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=27502: Thu Aug 1 17:42:23 2024 read: IOPS=60, BW=242KiB/s (248kB/s)(1436KiB/5938msec) clat (usec): min=5, max=95272, avg=3657.98, stdev=7830.11 lat (usec): min=5, max=95274, avg=3658.84, stdev=7830.13 clat percentiles (usec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 19], 50.00th=[ 32], 60.00th=[ 2638], | 70.00th=[ 5080], 80.00th=[ 6652], 90.00th=[ 8979], 95.00th=[12780], | 99.00th=[32637], 99.50th=[56361], 99.90th=[94897], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 168, max= 360, per=25.08%, avg=242.73, stdev=62.57, samples=11 iops : min= 42, max= 90, avg=60.55, stdev=15.69, samples=11 write: IOPS=68, BW=276KiB/s (282kB/s)(1636KiB/5938msec) clat (usec): min=109, max=89972, avg=11155.60, stdev=16924.75 lat (usec): min=109, max=89973, avg=11156.56, stdev=16924.73 clat percentiles (usec): | 1.00th=[ 130], 5.00th=[ 143], 10.00th=[ 157], 20.00th=[ 174], | 30.00th=[ 237], 40.00th=[ 1029], 50.00th=[ 3621], 60.00th=[ 5866], | 70.00th=[10159], 80.00th=[22152], 90.00th=[36963], 95.00th=[47973], | 99.00th=[69731], 99.50th=[78119], 99.90th=[89654], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 192, max= 360, per=26.67%, avg=271.82, stdev=52.86, samples=11 iops : min= 48, max= 90, avg=67.82, stdev=13.15, samples=11 lat (usec) : 10=6.12%, 20=13.80%, 50=5.73%, 100=0.65%, 250=16.93% lat (usec) : 500=2.86%, 750=1.82%, 1000=1.04% lat (msec) : 2=2.21%, 4=6.77%, 10=22.01%, 20=7.55%, 50=9.64% lat (msec) : 100=2.86% cpu : usr=0.32%, sys=5.52%, ctx=807, majf=0, minf=35 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=27503: Thu Aug 1 17:42:23 2024 read: IOPS=65, BW=261KiB/s (267kB/s)(1564KiB/6001msec) clat (usec): min=3, max=65634, avg=3799.91, stdev=6599.61 lat (usec): min=4, max=65634, avg=3800.95, stdev=6599.47 clat percentiles (usec): | 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 19], 50.00th=[ 37], 60.00th=[ 3097], | 70.00th=[ 5211], 80.00th=[ 7635], 90.00th=[10159], 95.00th=[13304], | 99.00th=[35914], 99.50th=[46400], 99.90th=[65799], 99.95th=[65799], | 99.99th=[65799] bw ( KiB/s): min= 160, max= 456, per=26.91%, avg=260.50, stdev=83.01, samples=12 iops : min= 40, max= 114, avg=65.00, stdev=20.84, samples=12 write: IOPS=62, BW=251KiB/s (257kB/s)(1508KiB/6001msec) clat (usec): min=97, max=84058, avg=11931.23, stdev=16672.24 lat (usec): min=97, max=84059, avg=11932.70, stdev=16672.77 clat percentiles (usec): | 1.00th=[ 120], 5.00th=[ 143], 10.00th=[ 161], 20.00th=[ 194], | 30.00th=[ 586], 40.00th=[ 2311], 50.00th=[ 4490], 60.00th=[ 6849], | 70.00th=[12780], 80.00th=[23725], 90.00th=[34866], 95.00th=[50594], | 99.00th=[69731], 99.50th=[71828], 99.90th=[84411], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 159, max= 376, per=24.58%, avg=250.50, stdev=54.51, samples=12 iops : min= 39, max= 94, avg=62.50, stdev=13.71, samples=12 lat (usec) : 4=0.13%, 10=8.20%, 20=13.41%, 50=5.47%, 100=1.04% lat (usec) : 250=12.89%, 500=2.34%, 750=1.56%, 1000=0.65% lat (msec) : 2=2.47%, 4=7.81%, 10=22.27%, 20=9.51%, 50=9.64% lat (msec) : 100=2.60% cpu : usr=0.00%, sys=6.30%, ctx=875, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=27504: Thu Aug 1 17:42:23 2024 read: IOPS=60, BW=244KiB/s (250kB/s)(1508KiB/6181msec) clat (usec): min=4, max=76008, avg=3873.12, stdev=9090.67 lat (usec): min=4, max=76008, avg=3873.93, stdev=9090.64 clat percentiles (usec): | 1.00th=[ 6], 5.00th=[ 6], 10.00th=[ 8], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 49], | 70.00th=[ 4686], 80.00th=[ 6849], 90.00th=[ 9896], 95.00th=[13304], | 99.00th=[57934], 99.50th=[74974], 99.90th=[76022], 99.95th=[76022], | 99.99th=[76022] bw ( KiB/s): min= 104, max= 424, per=24.09%, avg=233.17, stdev=87.05, samples=12 iops : min= 26, max= 106, avg=58.17, stdev=21.84, samples=12 write: IOPS=63, BW=253KiB/s (259kB/s)(1564KiB/6181msec) clat (usec): min=121, max=122109, avg=11895.71, stdev=18621.15 lat (usec): min=122, max=122112, avg=11897.23, stdev=18620.88 clat percentiles (usec): | 1.00th=[ 128], 5.00th=[ 147], 10.00th=[ 161], 20.00th=[ 188], | 30.00th=[ 249], 40.00th=[ 1418], 50.00th=[ 3425], 60.00th=[ 5211], | 70.00th=[ 10945], 80.00th=[ 23725], 90.00th=[ 38536], 95.00th=[ 50070], | 99.00th=[ 83362], 99.50th=[106431], 99.90th=[122160], 99.95th=[122160], | 99.99th=[122160] bw ( KiB/s): min= 112, max= 400, per=24.12%, avg=245.83, stdev=83.41, samples=12 iops : min= 28, max= 100, avg=61.33, stdev=20.95, samples=12 lat (usec) : 10=9.37%, 20=15.49%, 50=4.69%, 100=0.39%, 250=15.62% lat (usec) : 500=1.95%, 750=1.56%, 1000=1.17% lat (msec) : 2=2.60%, 4=8.20%, 10=17.97%, 20=8.98%, 50=8.72% lat (msec) : 100=2.99%, 250=0.26% cpu : usr=0.00%, sys=5.50%, ctx=745, majf=0, minf=36 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=969KiB/s (992kB/s), 242KiB/s-261KiB/s (248kB/s-267kB/s), io=5988KiB (6132kB), run=5938-6181msec WRITE: bw=1019KiB/s (1044kB/s), 251KiB/s-276KiB/s (257kB/s-282kB/s), io=6300KiB (6451kB), run=5938-6181msec rand-rw: (groupid=0, jobs=1): err= 0: pid=27505: Thu Aug 1 17:42:38 2024 read: IOPS=17, BW=69.4KiB/s (71.0kB/s)(1480KiB/21338msec) clat (usec): min=883, max=111531, avg=5288.79, stdev=9540.81 lat (usec): min=884, max=111532, avg=5290.18, stdev=9540.82 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 1270], 10.00th=[ 1500], 20.00th=[ 1778], | 30.00th=[ 2089], 40.00th=[ 2409], 50.00th=[ 2802], 60.00th=[ 3163], | 70.00th=[ 3851], 80.00th=[ 5276], 90.00th=[ 9110], 95.00th=[ 16712], | 99.00th=[ 50070], 99.50th=[ 64750], 99.90th=[111674], 99.95th=[111674], | 99.99th=[111674] bw ( KiB/s): min= 23, max= 176, per=24.40%, avg=67.83, stdev=32.88, samples=42 iops : min= 5, max= 44, avg=16.69, stdev= 8.30, samples=42 write: IOPS=18, BW=74.6KiB/s (76.4kB/s)(1592KiB/21338msec) clat (msec): min=15, max=158, avg=48.53, stdev=24.54 lat (msec): min=15, max=158, avg=48.53, stdev=24.54 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 22], 10.00th=[ 25], 20.00th=[ 30], | 30.00th=[ 34], 40.00th=[ 38], 50.00th=[ 42], 60.00th=[ 48], | 70.00th=[ 55], 80.00th=[ 66], 90.00th=[ 82], 95.00th=[ 101], | 99.00th=[ 140], 99.50th=[ 159], 99.90th=[ 159], 99.95th=[ 159], | 99.99th=[ 159] bw ( KiB/s): min= 16, max= 136, per=25.17%, avg=73.74, stdev=30.10, samples=42 iops : min= 4, max= 34, avg=18.17, stdev= 7.54, samples=42 lat (usec) : 1000=0.39% lat (msec) : 2=12.50%, 4=21.22%, 10=9.64%, 20=3.65%, 50=33.98% lat (msec) : 100=15.89%, 250=2.73% cpu : usr=0.15%, sys=1.24%, ctx=823, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=27506: Thu Aug 1 17:42:38 2024 read: IOPS=16, BW=66.8KiB/s (68.4kB/s)(1436KiB/21492msec) clat (usec): min=989, max=87723, avg=5339.00, stdev=9127.52 lat (usec): min=990, max=87724, avg=5342.01, stdev=9127.57 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 1254], 10.00th=[ 1532], 20.00th=[ 1827], | 30.00th=[ 2057], 40.00th=[ 2409], 50.00th=[ 2737], 60.00th=[ 3228], | 70.00th=[ 4228], 80.00th=[ 5342], 90.00th=[ 8455], 95.00th=[19268], | 99.00th=[58459], 99.50th=[64750], 99.90th=[87557], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 7, max= 144, per=23.88%, avg=66.38, stdev=35.61, samples=42 iops : min= 1, max= 36, avg=16.38, stdev= 8.96, samples=42 write: IOPS=19, BW=76.1KiB/s (77.9kB/s)(1636KiB/21492msec) clat (msec): min=14, max=156, avg=47.70, stdev=23.06 lat (msec): min=14, max=156, avg=47.70, stdev=23.06 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 25], 20.00th=[ 29], | 30.00th=[ 35], 40.00th=[ 39], 50.00th=[ 43], 60.00th=[ 47], | 70.00th=[ 55], 80.00th=[ 63], 90.00th=[ 79], 95.00th=[ 94], | 99.00th=[ 133], 99.50th=[ 136], 99.90th=[ 157], 99.95th=[ 157], | 99.99th=[ 157] bw ( KiB/s): min= 39, max= 152, per=25.45%, avg=74.57, stdev=28.63, samples=42 iops : min= 9, max= 38, avg=18.43, stdev= 7.20, samples=42 lat (usec) : 1000=0.13% lat (msec) : 2=11.72%, 4=20.18%, 10=10.81%, 20=4.43%, 50=32.81% lat (msec) : 100=18.10%, 250=1.82% cpu : usr=0.23%, sys=1.16%, ctx=830, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=27507: Thu Aug 1 17:42:38 2024 read: IOPS=18, BW=75.1KiB/s (76.9kB/s)(1564KiB/20817msec) clat (usec): min=896, max=102950, avg=5207.90, stdev=9373.78 lat (usec): min=896, max=102951, avg=5209.63, stdev=9374.86 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 1221], 10.00th=[ 1434], 20.00th=[ 1745], | 30.00th=[ 2114], 40.00th=[ 2409], 50.00th=[ 2671], 60.00th=[ 3195], | 70.00th=[ 3752], 80.00th=[ 5014], 90.00th=[ 8717], 95.00th=[ 18482], | 99.00th=[ 54789], 99.50th=[ 68682], 99.90th=[103285], 99.95th=[103285], | 99.99th=[103285] bw ( KiB/s): min= 15, max= 224, per=27.29%, avg=75.88, stdev=48.64, samples=41 iops : min= 3, max= 56, avg=18.80, stdev=12.25, samples=41 write: IOPS=18, BW=72.4KiB/s (74.2kB/s)(1508KiB/20817msec) clat (msec): min=15, max=199, avg=49.74, stdev=23.13 lat (msec): min=15, max=199, avg=49.74, stdev=23.13 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 23], 10.00th=[ 27], 20.00th=[ 32], | 30.00th=[ 35], 40.00th=[ 40], 50.00th=[ 44], 60.00th=[ 51], | 70.00th=[ 58], 80.00th=[ 67], 90.00th=[ 79], 95.00th=[ 90], | 99.00th=[ 127], 99.50th=[ 146], 99.90th=[ 201], 99.95th=[ 201], | 99.99th=[ 201] bw ( KiB/s): min= 31, max= 136, per=24.57%, avg=71.98, stdev=27.40, samples=41 iops : min= 7, max= 34, avg=17.83, stdev= 6.91, samples=41 lat (usec) : 1000=0.39% lat (msec) : 2=13.67%, 4=23.18%, 10=9.11%, 20=3.39%, 50=30.21% lat (msec) : 100=18.49%, 250=1.56% cpu : usr=0.00%, sys=1.33%, ctx=820, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=27508: Thu Aug 1 17:42:38 2024 read: IOPS=17, BW=71.1KiB/s (72.8kB/s)(1508KiB/21198msec) clat (usec): min=1029, max=98151, avg=4982.84, stdev=10190.90 lat (usec): min=1030, max=98153, avg=4984.14, stdev=10190.90 clat percentiles (usec): | 1.00th=[ 1123], 5.00th=[ 1319], 10.00th=[ 1467], 20.00th=[ 1745], | 30.00th=[ 1991], 40.00th=[ 2278], 50.00th=[ 2638], 60.00th=[ 3195], | 70.00th=[ 3818], 80.00th=[ 4752], 90.00th=[ 7767], 95.00th=[13829], | 99.00th=[76022], 99.50th=[92799], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 8, max= 160, per=25.33%, avg=70.43, stdev=35.23, samples=42 iops : min= 2, max= 40, avg=17.43, stdev= 8.84, samples=42 write: IOPS=18, BW=73.8KiB/s (75.6kB/s)(1564KiB/21198msec) clat (msec): min=16, max=219, avg=49.27, stdev=24.87 lat (msec): min=16, max=219, avg=49.27, stdev=24.87 clat percentiles (msec): | 1.00th=[ 19], 5.00th=[ 25], 10.00th=[ 27], 20.00th=[ 32], | 30.00th=[ 36], 40.00th=[ 40], 50.00th=[ 44], 60.00th=[ 47], | 70.00th=[ 54], 80.00th=[ 62], 90.00th=[ 80], 95.00th=[ 95], | 99.00th=[ 138], 99.50th=[ 186], 99.90th=[ 220], 99.95th=[ 220], | 99.99th=[ 220] bw ( KiB/s): min= 32, max= 120, per=25.08%, avg=73.48, stdev=27.54, samples=42 iops : min= 8, max= 30, avg=18.19, stdev= 6.95, samples=42 lat (msec) : 2=14.84%, 4=20.57%, 10=9.77%, 20=3.39%, 50=32.81% lat (msec) : 100=16.28%, 250=2.34% cpu : usr=0.15%, sys=1.24%, ctx=837, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=279KiB/s (285kB/s), 66.8KiB/s-75.1KiB/s (68.4kB/s-76.9kB/s), io=5988KiB (6132kB), run=20817-21492msec WRITE: bw=293KiB/s (300kB/s), 72.4KiB/s-76.1KiB/s (74.2kB/s-77.9kB/s), io=6300KiB (6451kB), run=20817-21492msec