== sanity test 398b: DIO and buffer IO race ============== 11:45:25 (1736268325) /usr/bin/fio 12+0 records in 12+0 records out 12582912 bytes (13 MB) copied, 0.103282 s, 122 MB/s mix direct rw 12M to OST0 by fio with 4 jobs... mix buffer rw 12M to OST0 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=19633: Tue Jan 7 11:45:28 2025 read: IOPS=321, BW=1286KiB/s (1317kB/s)(1480KiB/1151msec) clat (usec): min=3, max=51194, avg=1811.73, stdev=3604.13 lat (usec): min=3, max=51196, avg=1812.35, stdev=3604.15 clat percentiles (usec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 12], 40.00th=[ 18], 50.00th=[ 1303], 60.00th=[ 1795], | 70.00th=[ 2147], 80.00th=[ 2737], 90.00th=[ 3621], 95.00th=[ 5145], | 99.00th=[13829], 99.50th=[24511], 99.90th=[51119], 99.95th=[51119], | 99.99th=[51119] bw ( KiB/s): min= 688, max= 1720, per=23.69%, avg=1204.00, stdev=729.73, samples=2 iops : min= 172, max= 430, avg=301.00, stdev=182.43, samples=2 write: IOPS=345, BW=1383KiB/s (1416kB/s)(1592KiB/1151msec) clat (usec): min=82, max=17103, avg=1175.48, stdev=1769.85 lat (usec): min=83, max=17104, avg=1176.69, stdev=1769.63 clat percentiles (usec): | 1.00th=[ 94], 5.00th=[ 104], 10.00th=[ 113], 20.00th=[ 147], | 30.00th=[ 163], 40.00th=[ 215], 50.00th=[ 635], 60.00th=[ 963], | 70.00th=[ 1303], 80.00th=[ 1942], 90.00th=[ 2802], 95.00th=[ 4178], | 99.00th=[ 9372], 99.50th=[13960], 99.90th=[17171], 99.95th=[17171], | 99.99th=[17171] bw ( KiB/s): min= 704, max= 1928, per=24.61%, avg=1316.00, stdev=865.50, samples=2 iops : min= 176, max= 482, avg=329.00, stdev=216.37, samples=2 lat (usec) : 4=0.39%, 10=10.94%, 20=7.94%, 50=0.39%, 100=1.95% lat (usec) : 250=21.48%, 500=2.60%, 750=2.08%, 1000=5.99% lat (msec) : 2=20.83%, 4=18.88%, 10=5.21%, 20=0.91%, 50=0.26% lat (msec) : 100=0.13% cpu : usr=0.52%, sys=15.57%, ctx=888, majf=0, minf=35 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=19634: Tue Jan 7 11:45:28 2025 read: IOPS=324, BW=1297KiB/s (1328kB/s)(1436KiB/1107msec) clat (usec): min=3, max=31951, avg=1721.50, stdev=3486.49 lat (usec): min=3, max=31952, avg=1722.64, stdev=3486.27 clat percentiles (usec): | 1.00th=[ 4], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 16], 50.00th=[ 1106], 60.00th=[ 1631], | 70.00th=[ 2114], 80.00th=[ 2573], 90.00th=[ 3359], 95.00th=[ 4228], | 99.00th=[26346], 99.50th=[29230], 99.90th=[31851], 99.95th=[31851], | 99.99th=[31851] bw ( KiB/s): min= 680, max= 1808, per=24.47%, avg=1244.00, stdev=797.62, samples=2 iops : min= 170, max= 452, avg=311.00, stdev=199.40, samples=2 write: IOPS=369, BW=1478KiB/s (1513kB/s)(1636KiB/1107msec) clat (usec): min=89, max=24075, avg=1162.81, stdev=2050.93 lat (usec): min=89, max=24076, avg=1163.62, stdev=2051.13 clat percentiles (usec): | 1.00th=[ 97], 5.00th=[ 103], 10.00th=[ 113], 20.00th=[ 128], | 30.00th=[ 155], 40.00th=[ 190], 50.00th=[ 603], 60.00th=[ 848], | 70.00th=[ 1074], 80.00th=[ 1598], 90.00th=[ 2900], 95.00th=[ 4424], | 99.00th=[10552], 99.50th=[11600], 99.90th=[23987], 99.95th=[23987], | 99.99th=[23987] bw ( KiB/s): min= 696, max= 2112, per=26.25%, avg=1404.00, stdev=1001.26, samples=2 iops : min= 174, max= 528, avg=351.00, stdev=250.32, samples=2 lat (usec) : 4=1.17%, 10=11.72%, 20=6.25%, 50=0.39%, 100=2.08% lat (usec) : 250=22.40%, 500=2.08%, 750=3.39%, 1000=9.11% lat (msec) : 2=17.32%, 4=18.36%, 10=4.30%, 20=0.65%, 50=0.78% cpu : usr=0.72%, sys=16.09%, ctx=789, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=19635: Tue Jan 7 11:45:28 2025 read: IOPS=331, BW=1328KiB/s (1360kB/s)(1564KiB/1178msec) clat (usec): min=3, max=29453, avg=1803.54, stdev=3225.44 lat (usec): min=3, max=29454, avg=1804.09, stdev=3225.47 clat percentiles (usec): | 1.00th=[ 4], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 12], 40.00th=[ 30], 50.00th=[ 1237], 60.00th=[ 1663], | 70.00th=[ 2147], 80.00th=[ 2671], 90.00th=[ 3654], 95.00th=[ 5080], | 99.00th=[20579], 99.50th=[28181], 99.90th=[29492], 99.95th=[29492], | 99.99th=[29492] bw ( KiB/s): min= 654, max= 1632, per=22.49%, avg=1143.00, stdev=691.55, samples=2 iops : min= 163, max= 408, avg=285.50, stdev=173.24, samples=2 write: IOPS=320, BW=1280KiB/s (1311kB/s)(1508KiB/1178msec) clat (usec): min=89, max=23358, avg=1233.55, stdev=2062.54 lat (usec): min=90, max=23359, avg=1234.42, stdev=2062.83 clat percentiles (usec): | 1.00th=[ 91], 5.00th=[ 99], 10.00th=[ 113], 20.00th=[ 143], | 30.00th=[ 167], 40.00th=[ 198], 50.00th=[ 619], 60.00th=[ 906], | 70.00th=[ 1237], 80.00th=[ 1975], 90.00th=[ 3195], 95.00th=[ 4555], | 99.00th=[ 8586], 99.50th=[16450], 99.90th=[23462], 99.95th=[23462], | 99.99th=[23462] bw ( KiB/s): min= 758, max= 1456, per=20.70%, avg=1107.00, stdev=493.56, samples=2 iops : min= 189, max= 364, avg=276.50, stdev=123.74, samples=2 lat (usec) : 4=0.65%, 10=12.24%, 20=7.03%, 50=0.52%, 100=3.12% lat (usec) : 250=19.01%, 500=2.86%, 750=2.08%, 1000=7.68% lat (msec) : 2=19.01%, 4=17.84%, 10=6.38%, 20=0.91%, 50=0.65% cpu : usr=0.25%, sys=16.99%, ctx=840, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=19636: Tue Jan 7 11:45:28 2025 read: IOPS=343, BW=1372KiB/s (1405kB/s)(1508KiB/1099msec) clat (usec): min=3, max=37728, avg=1688.77, stdev=3237.58 lat (usec): min=3, max=37729, avg=1689.69, stdev=3237.43 clat percentiles (usec): | 1.00th=[ 4], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 12], 40.00th=[ 38], 50.00th=[ 1352], 60.00th=[ 1762], | 70.00th=[ 2089], 80.00th=[ 2474], 90.00th=[ 3163], 95.00th=[ 4293], | 99.00th=[17171], 99.50th=[27395], 99.90th=[37487], 99.95th=[37487], | 99.99th=[37487] bw ( KiB/s): min= 760, max= 1784, per=25.02%, avg=1272.00, stdev=724.08, samples=2 iops : min= 190, max= 446, avg=318.00, stdev=181.02, samples=2 write: IOPS=355, BW=1423KiB/s (1457kB/s)(1564KiB/1099msec) clat (usec): min=79, max=40895, avg=1149.22, stdev=2454.12 lat (usec): min=80, max=40897, avg=1150.18, stdev=2454.16 clat percentiles (usec): | 1.00th=[ 91], 5.00th=[ 103], 10.00th=[ 114], 20.00th=[ 145], | 30.00th=[ 172], 40.00th=[ 221], 50.00th=[ 709], 60.00th=[ 922], | 70.00th=[ 1188], 80.00th=[ 1713], 90.00th=[ 2442], 95.00th=[ 3589], | 99.00th=[ 8094], 99.50th=[11338], 99.90th=[41157], 99.95th=[41157], | 99.99th=[41157] bw ( KiB/s): min= 800, max= 1856, per=24.83%, avg=1328.00, stdev=746.70, samples=2 iops : min= 200, max= 464, avg=332.00, stdev=186.68, samples=2 lat (usec) : 4=0.65%, 10=12.11%, 20=6.25%, 50=0.65%, 100=2.08% lat (usec) : 250=20.18%, 500=2.34%, 750=3.26%, 1000=7.68% lat (msec) : 2=21.48%, 4=17.71%, 10=4.56%, 20=0.52%, 50=0.52% cpu : usr=0.36%, sys=16.03%, ctx=858, majf=0, minf=35 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5083KiB/s (5205kB/s), 1286KiB/s-1372KiB/s (1317kB/s-1405kB/s), io=5988KiB (6132kB), run=1099-1178msec WRITE: bw=5348KiB/s (5476kB/s), 1280KiB/s-1478KiB/s (1311kB/s-1513kB/s), io=6300KiB (6451kB), run=1099-1178msec rand-rw: (groupid=0, jobs=1): err= 0: pid=19629: Tue Jan 7 11:45:29 2025 read: IOPS=202, BW=811KiB/s (831kB/s)(1480KiB/1824msec) clat (usec): min=511, max=60406, avg=1818.16, stdev=3941.72 lat (usec): min=512, max=60407, avg=1818.81, stdev=3941.76 clat percentiles (usec): | 1.00th=[ 578], 5.00th=[ 676], 10.00th=[ 717], 20.00th=[ 758], | 30.00th=[ 816], 40.00th=[ 881], 50.00th=[ 930], 60.00th=[ 1020], | 70.00th=[ 1221], 80.00th=[ 1663], 90.00th=[ 2999], 95.00th=[ 5407], | 99.00th=[15664], 99.50th=[28705], 99.90th=[60556], 99.95th=[60556], | 99.99th=[60556] bw ( KiB/s): min= 384, max= 928, per=21.88%, avg=712.00, stdev=288.78, samples=3 iops : min= 96, max= 232, avg=178.00, stdev=72.19, samples=3 write: IOPS=218, BW=873KiB/s (894kB/s)(1592KiB/1824msec) clat (usec): min=1156, max=10900, avg=2868.89, stdev=1616.43 lat (usec): min=1156, max=10900, avg=2869.61, stdev=1616.47 clat percentiles (usec): | 1.00th=[ 1237], 5.00th=[ 1450], 10.00th=[ 1614], 20.00th=[ 1876], | 30.00th=[ 2024], 40.00th=[ 2114], 50.00th=[ 2245], 60.00th=[ 2474], | 70.00th=[ 2868], 80.00th=[ 3621], 90.00th=[ 5145], 95.00th=[ 6390], | 99.00th=[10028], 99.50th=[10552], 99.90th=[10945], 99.95th=[10945], | 99.99th=[10945] bw ( KiB/s): min= 448, max= 1144, per=22.59%, avg=773.33, stdev=350.21, samples=3 iops : min= 112, max= 286, avg=193.33, stdev=87.55, samples=3 lat (usec) : 750=8.20%, 1000=20.18% lat (msec) : 2=27.60%, 4=32.55%, 10=10.03%, 20=1.17%, 50=0.13% lat (msec) : 100=0.13% cpu : usr=0.38%, sys=5.38%, ctx=798, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=19630: Tue Jan 7 11:45:29 2025 read: IOPS=195, BW=782KiB/s (800kB/s)(1436KiB/1837msec) clat (usec): min=539, max=35574, avg=1706.20, stdev=2732.44 lat (usec): min=540, max=35575, avg=1707.03, stdev=2732.58 clat percentiles (usec): | 1.00th=[ 553], 5.00th=[ 652], 10.00th=[ 693], 20.00th=[ 775], | 30.00th=[ 816], 40.00th=[ 873], 50.00th=[ 914], 60.00th=[ 988], | 70.00th=[ 1172], 80.00th=[ 1876], 90.00th=[ 3687], 95.00th=[ 4948], | 99.00th=[15401], 99.50th=[21627], 99.90th=[35390], 99.95th=[35390], | 99.99th=[35390] bw ( KiB/s): min= 384, max= 1120, per=21.88%, avg=712.00, stdev=374.46, samples=3 iops : min= 96, max= 280, avg=178.00, stdev=93.62, samples=3 write: IOPS=222, BW=891KiB/s (912kB/s)(1636KiB/1837msec) clat (usec): min=1071, max=19305, avg=2958.37, stdev=2106.05 lat (usec): min=1072, max=19307, avg=2959.35, stdev=2105.98 clat percentiles (usec): | 1.00th=[ 1237], 5.00th=[ 1434], 10.00th=[ 1614], 20.00th=[ 1909], | 30.00th=[ 2040], 40.00th=[ 2180], 50.00th=[ 2311], 60.00th=[ 2507], | 70.00th=[ 2868], 80.00th=[ 3654], 90.00th=[ 4752], 95.00th=[ 6194], | 99.00th=[13960], 99.50th=[16909], 99.90th=[19268], 99.95th=[19268], | 99.99th=[19268] bw ( KiB/s): min= 448, max= 1088, per=22.75%, avg=778.67, stdev=320.53, samples=3 iops : min= 112, max= 272, avg=194.67, stdev=80.13, samples=3 lat (usec) : 750=7.68%, 1000=21.35% lat (msec) : 2=23.31%, 4=36.07%, 10=10.16%, 20=1.17%, 50=0.26% cpu : usr=0.49%, sys=5.28%, ctx=820, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=19631: Tue Jan 7 11:45:29 2025 read: IOPS=217, BW=870KiB/s (891kB/s)(1564KiB/1798msec) clat (usec): min=568, max=51972, avg=1605.59, stdev=2919.69 lat (usec): min=568, max=51973, avg=1606.27, stdev=2919.72 clat percentiles (usec): | 1.00th=[ 586], 5.00th=[ 652], 10.00th=[ 685], 20.00th=[ 758], | 30.00th=[ 807], 40.00th=[ 857], 50.00th=[ 914], 60.00th=[ 1012], | 70.00th=[ 1336], 80.00th=[ 1811], 90.00th=[ 3195], 95.00th=[ 4424], | 99.00th=[ 6521], 99.50th=[16319], 99.90th=[52167], 99.95th=[52167], | 99.99th=[52167] bw ( KiB/s): min= 368, max= 1064, per=23.44%, avg=762.67, stdev=357.26, samples=3 iops : min= 92, max= 266, avg=190.67, stdev=89.32, samples=3 write: IOPS=209, BW=839KiB/s (859kB/s)(1508KiB/1798msec) clat (usec): min=1139, max=41351, avg=3062.81, stdev=2821.51 lat (usec): min=1139, max=41353, avg=3063.70, stdev=2821.51 clat percentiles (usec): | 1.00th=[ 1205], 5.00th=[ 1401], 10.00th=[ 1647], 20.00th=[ 1860], | 30.00th=[ 1991], 40.00th=[ 2089], 50.00th=[ 2278], 60.00th=[ 2507], | 70.00th=[ 2868], 80.00th=[ 3785], 90.00th=[ 5080], 95.00th=[ 6259], | 99.00th=[15139], 99.50th=[19530], 99.90th=[41157], 99.95th=[41157], | 99.99th=[41157] bw ( KiB/s): min= 464, max= 1088, per=21.81%, avg=746.67, stdev=316.11, samples=3 iops : min= 116, max= 272, avg=186.67, stdev=79.03, samples=3 lat (usec) : 750=9.77%, 1000=20.05% lat (msec) : 2=27.34%, 4=30.60%, 10=11.20%, 20=0.78%, 50=0.13% lat (msec) : 100=0.13% cpu : usr=0.17%, sys=5.62%, ctx=809, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=19632: Tue Jan 7 11:45:29 2025 read: IOPS=204, BW=820KiB/s (839kB/s)(1508KiB/1840msec) clat (usec): min=471, max=41616, avg=1786.04, stdev=2979.85 lat (usec): min=471, max=41617, avg=1786.67, stdev=2979.88 clat percentiles (usec): | 1.00th=[ 586], 5.00th=[ 652], 10.00th=[ 725], 20.00th=[ 799], | 30.00th=[ 857], 40.00th=[ 898], 50.00th=[ 955], 60.00th=[ 1074], | 70.00th=[ 1352], 80.00th=[ 2008], 90.00th=[ 3687], 95.00th=[ 4883], | 99.00th=[10814], 99.50th=[28705], 99.90th=[41681], 99.95th=[41681], | 99.99th=[41681] bw ( KiB/s): min= 368, max= 944, per=21.39%, avg=696.00, stdev=296.22, samples=3 iops : min= 92, max= 236, avg=174.00, stdev=74.05, samples=3 write: IOPS=212, BW=850KiB/s (870kB/s)(1564KiB/1840msec) clat (usec): min=984, max=40220, avg=2942.09, stdev=2602.22 lat (usec): min=984, max=40221, avg=2942.86, stdev=2602.32 clat percentiles (usec): | 1.00th=[ 1172], 5.00th=[ 1483], 10.00th=[ 1647], 20.00th=[ 1860], | 30.00th=[ 2024], 40.00th=[ 2147], 50.00th=[ 2278], 60.00th=[ 2507], | 70.00th=[ 2868], 80.00th=[ 3425], 90.00th=[ 4686], 95.00th=[ 6456], | 99.00th=[10159], 99.50th=[21890], 99.90th=[40109], 99.95th=[40109], | 99.99th=[40109] bw ( KiB/s): min= 424, max= 1104, per=21.81%, avg=746.67, stdev=341.32, samples=3 iops : min= 106, max= 276, avg=186.67, stdev=85.33, samples=3 lat (usec) : 500=0.13%, 750=6.12%, 1000=20.57% lat (msec) : 2=27.21%, 4=35.16%, 10=9.77%, 20=0.52%, 50=0.52% cpu : usr=0.11%, sys=5.71%, ctx=811, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3254KiB/s (3332kB/s), 782KiB/s-870KiB/s (800kB/s-891kB/s), io=5988KiB (6132kB), run=1798-1840msec WRITE: bw=3424KiB/s (3506kB/s), 839KiB/s-891KiB/s (859kB/s-912kB/s), io=6300KiB (6451kB), run=1798-1840msec