== sanity test 398b: DIO and buffer IO race ============== 07:43:26 (1756813406) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.21049 s, 15.7 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=186630: Tue Sep 2 07:44:43 2025 read: IOPS=22, BW=91.1KiB/s (93.3kB/s)(5992KiB/65791msec) clat (usec): min=375, max=794235, avg=24171.54, stdev=22823.06 lat (usec): min=376, max=794236, avg=24174.18, stdev=22822.73 clat percentiles (usec): | 1.00th=[ 742], 5.00th=[ 8979], 10.00th=[ 11207], 20.00th=[ 14484], | 30.00th=[ 16712], 40.00th=[ 19530], 50.00th=[ 21890], 60.00th=[ 24773], | 70.00th=[ 28181], 80.00th=[ 32637], 90.00th=[ 38011], 95.00th=[ 43779], | 99.00th=[ 59507], 99.50th=[ 65799], 99.90th=[ 79168], 99.95th=[792724], | 99.99th=[792724] bw ( KiB/s): min= 6, max= 223, per=24.95%, avg=90.08, stdev=27.98, samples=130 iops : min= 1, max= 55, avg=22.12, stdev= 7.05, samples=130 write: IOPS=23, BW=95.7KiB/s (97.0kB/s)(6296KiB/65791msec); 0 zone resets clat (usec): min=696, max=85982, avg=18628.55, stdev=13246.14 lat (usec): min=697, max=85983, avg=18633.87, stdev=13249.97 clat percentiles (usec): | 1.00th=[ 1483], 5.00th=[ 3130], 10.00th=[ 4490], 20.00th=[ 6390], | 30.00th=[ 9634], 40.00th=[13173], 50.00th=[15926], 60.00th=[19006], | 70.00th=[23462], 80.00th=[28967], 90.00th=[35914], 95.00th=[43779], | 99.00th=[62129], 99.50th=[66323], 99.90th=[73925], 99.95th=[85459], | 99.99th=[85459] bw ( KiB/s): min= 32, max= 232, per=25.15%, avg=95.05, stdev=30.54, samples=129 iops : min= 8, max= 58, avg=23.36, stdev= 7.66, samples=129 lat (usec) : 500=0.20%, 750=0.36%, 1000=0.36% lat (msec) : 2=0.55%, 4=3.45%, 10=14.16%, 20=33.33%, 50=44.89% lat (msec) : 100=2.67%, 1000=0.03% cpu : usr=0.10%, sys=28.19%, ctx=9811, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186631: Tue Sep 2 07:44:43 2025 read: IOPS=22, BW=90.2KiB/s (92.3kB/s)(5940KiB/65888msec) clat (usec): min=392, max=93638, avg=23927.96, stdev=11121.11 lat (usec): min=393, max=93639, avg=23929.01, stdev=11121.23 clat percentiles (usec): | 1.00th=[ 857], 5.00th=[ 9503], 10.00th=[11731], 20.00th=[14877], | 30.00th=[17171], 40.00th=[19530], 50.00th=[22414], 60.00th=[25297], | 70.00th=[28443], 80.00th=[32375], 90.00th=[37487], 95.00th=[43779], | 99.00th=[57934], 99.50th=[66847], 99.90th=[87557], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 6, max= 208, per=24.92%, avg=89.95, stdev=27.74, samples=131 iops : min= 1, max= 52, avg=22.08, stdev= 6.95, samples=131 write: IOPS=24, BW=96.3KiB/s (98.7kB/s)(6348KiB/65888msec); 0 zone resets clat (usec): min=718, max=791058, avg=18965.87, stdev=23530.54 lat (usec): min=718, max=791060, avg=18966.96, stdev=23530.58 clat percentiles (usec): | 1.00th=[ 1319], 5.00th=[ 3228], 10.00th=[ 4490], 20.00th=[ 6259], | 30.00th=[ 9372], 40.00th=[ 12780], 50.00th=[ 15926], 60.00th=[ 19006], | 70.00th=[ 22938], 80.00th=[ 28705], 90.00th=[ 37487], 95.00th=[ 45351], | 99.00th=[ 57934], 99.50th=[ 65274], 99.90th=[104334], 99.95th=[792724], | 99.99th=[792724] bw ( KiB/s): min= 23, max= 312, per=25.61%, avg=96.82, stdev=35.91, samples=130 iops : min= 5, max= 78, avg=23.80, stdev= 9.06, samples=130 lat (usec) : 500=0.16%, 750=0.20%, 1000=0.49% lat (msec) : 2=0.68%, 4=3.55%, 10=14.26%, 20=32.81%, 50=45.38% lat (msec) : 100=2.41%, 250=0.03%, 1000=0.03% cpu : usr=0.12%, sys=27.71%, ctx=9688, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186632: Tue Sep 2 07:44:43 2025 read: IOPS=22, BW=91.2KiB/s (93.4kB/s)(6028KiB/66112msec) clat (usec): min=305, max=797915, avg=24644.20, stdev=23329.66 lat (usec): min=305, max=797916, avg=24647.30, stdev=23328.93 clat percentiles (usec): | 1.00th=[ 725], 5.00th=[ 8586], 10.00th=[ 11207], 20.00th=[ 14353], | 30.00th=[ 17171], 40.00th=[ 19530], 50.00th=[ 22414], 60.00th=[ 25297], | 70.00th=[ 28967], 80.00th=[ 32900], 90.00th=[ 39060], 95.00th=[ 45351], | 99.00th=[ 62653], 99.50th=[ 69731], 99.90th=[139461], 99.95th=[801113], | 99.99th=[801113] bw ( KiB/s): min= 6, max= 208, per=25.04%, avg=90.38, stdev=31.24, samples=131 iops : min= 1, max= 52, avg=22.19, stdev= 7.84, samples=131 write: IOPS=23, BW=94.7KiB/s (96.0kB/s)(6260KiB/66112msec); 0 zone resets clat (usec): min=695, max=74976, avg=18357.08, stdev=12982.82 lat (usec): min=695, max=74978, avg=18358.27, stdev=12982.85 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 3064], 10.00th=[ 4293], 20.00th=[ 6128], | 30.00th=[ 9634], 40.00th=[12911], 50.00th=[15795], 60.00th=[19006], | 70.00th=[23200], 80.00th=[28443], 90.00th=[36439], 95.00th=[42206], | 99.00th=[57410], 99.50th=[61604], 99.90th=[72877], 99.95th=[74974], | 99.99th=[74974] bw ( KiB/s): min= 20, max= 240, per=24.77%, avg=93.63, stdev=30.25, samples=131 iops : min= 5, max= 60, avg=23.02, stdev= 7.63, samples=131 lat (usec) : 500=0.20%, 750=0.52%, 1000=0.26% lat (msec) : 2=0.49%, 4=3.71%, 10=14.23%, 20=33.24%, 50=44.34% lat (msec) : 100=2.96%, 250=0.03%, 1000=0.03% cpu : usr=0.12%, sys=27.88%, ctx=9832, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186633: Tue Sep 2 07:44:43 2025 read: IOPS=22, BW=91.1KiB/s (93.3kB/s)(6052KiB/66457msec) clat (usec): min=175, max=112790, avg=24005.53, stdev=12214.52 lat (usec): min=176, max=112790, avg=24008.24, stdev=12214.43 clat percentiles (usec): | 1.00th=[ 701], 5.00th=[ 8455], 10.00th=[ 10683], 20.00th=[ 14484], | 30.00th=[ 16909], 40.00th=[ 19530], 50.00th=[ 22152], 60.00th=[ 25297], | 70.00th=[ 28967], 80.00th=[ 32637], 90.00th=[ 38536], 95.00th=[ 45351], | 99.00th=[ 63177], 99.50th=[ 71828], 99.90th=[ 85459], 99.95th=[112722], | 99.99th=[112722] bw ( KiB/s): min= 8, max= 216, per=25.36%, avg=91.56, stdev=28.59, samples=131 iops : min= 2, max= 54, avg=22.49, stdev= 7.18, samples=131 write: IOPS=23, BW=93.8KiB/s (96.1kB/s)(6236KiB/66457msec); 0 zone resets clat (usec): min=737, max=766279, avg=19166.86, stdev=23153.49 lat (usec): min=738, max=766282, avg=19172.57, stdev=23153.29 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 3130], 10.00th=[ 4293], 20.00th=[ 6259], | 30.00th=[ 9634], 40.00th=[ 12911], 50.00th=[ 15795], 60.00th=[ 19006], | 70.00th=[ 23200], 80.00th=[ 29230], 90.00th=[ 38011], 95.00th=[ 45351], | 99.00th=[ 59507], 99.50th=[ 62653], 99.90th=[ 72877], 99.95th=[767558], | 99.99th=[767558] bw ( KiB/s): min= 32, max= 232, per=24.90%, avg=94.12, stdev=33.34, samples=131 iops : min= 8, max= 58, avg=23.13, stdev= 8.40, samples=131 lat (usec) : 250=0.07%, 500=0.13%, 750=0.52%, 1000=0.49% lat (msec) : 2=0.52%, 4=3.65%, 10=14.23%, 20=32.81%, 50=44.47% lat (msec) : 100=3.06%, 250=0.03%, 1000=0.03% cpu : usr=0.08%, sys=27.66%, ctx=9679, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=361KiB/s (370kB/s), 90.2KiB/s-91.2KiB/s (92.3kB/s-93.4kB/s), io=23.4MiB (24.6MB), run=65791-66457msec WRITE: bw=378KiB/s (387kB/s), 93.8KiB/s-96.3KiB/s (96.1kB/s-98.7kB/s), io=24.6MiB (25.7MB), run=65791-66457msec rand-rw: (groupid=0, jobs=1): err= 0: pid=186634: Tue Sep 2 07:44:49 2025 read: IOPS=20, BW=82.5KiB/s (84.5kB/s)(5992KiB/72610msec) clat (msec): min=2, max=807, avg=21.67, stdev=26.66 lat (msec): min=2, max=807, avg=21.67, stdev=26.66 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 7], | 30.00th=[ 10], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 21], | 70.00th=[ 26], 80.00th=[ 33], 90.00th=[ 46], 95.00th=[ 55], | 99.00th=[ 81], 99.50th=[ 95], 99.90th=[ 120], 99.95th=[ 810], | 99.99th=[ 810] bw ( KiB/s): min= 7, max= 368, per=25.04%, avg=82.39, stdev=68.19, samples=143 iops : min= 1, max= 92, avg=20.22, stdev=17.16, samples=143 write: IOPS=21, BW=86.7KiB/s (88.8kB/s)(6296KiB/72610msec); 0 zone resets clat (msec): min=4, max=107, avg=25.39, stdev=18.33 lat (msec): min=4, max=107, avg=25.39, stdev=18.33 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 12], 40.00th=[ 17], 50.00th=[ 22], 60.00th=[ 26], | 70.00th=[ 33], 80.00th=[ 41], 90.00th=[ 52], 95.00th=[ 59], | 99.00th=[ 86], 99.50th=[ 91], 99.90th=[ 107], 99.95th=[ 108], | 99.99th=[ 108] bw ( KiB/s): min= 15, max= 392, per=25.03%, avg=86.36, stdev=71.04, samples=143 iops : min= 3, max= 98, avg=21.21, stdev=17.90, samples=143 lat (msec) : 4=2.08%, 10=26.92%, 20=24.41%, 50=37.47%, 100=8.89% lat (msec) : 250=0.20%, 1000=0.03% cpu : usr=0.10%, sys=13.90%, ctx=4683, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186635: Tue Sep 2 07:44:49 2025 read: IOPS=20, BW=82.1KiB/s (84.1kB/s)(5940KiB/72363msec) clat (msec): min=2, max=772, avg=21.18, stdev=25.81 lat (msec): min=2, max=772, avg=21.18, stdev=25.81 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 7], | 30.00th=[ 10], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 25], 80.00th=[ 32], 90.00th=[ 46], 95.00th=[ 58], | 99.00th=[ 79], 99.50th=[ 84], 99.90th=[ 104], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 8, max= 432, per=25.06%, avg=82.43, stdev=69.89, samples=143 iops : min= 2, max= 108, avg=20.23, stdev=17.59, samples=143 write: IOPS=21, BW=87.7KiB/s (89.8kB/s)(6348KiB/72363msec); 0 zone resets clat (msec): min=3, max=129, avg=25.66, stdev=18.54 lat (msec): min=3, max=129, avg=25.66, stdev=18.54 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 13], 40.00th=[ 17], 50.00th=[ 22], 60.00th=[ 28], | 70.00th=[ 33], 80.00th=[ 42], 90.00th=[ 52], 95.00th=[ 61], | 99.00th=[ 87], 99.50th=[ 90], 99.90th=[ 108], 99.95th=[ 130], | 99.99th=[ 130] bw ( KiB/s): min= 24, max= 384, per=25.55%, avg=88.16, stdev=70.75, samples=143 iops : min= 6, max= 96, avg=21.66, stdev=17.82, samples=143 lat (msec) : 4=2.08%, 10=26.99%, 20=24.84%, 50=36.59%, 100=9.38% lat (msec) : 250=0.10%, 1000=0.03% cpu : usr=0.08%, sys=13.96%, ctx=4679, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186636: Tue Sep 2 07:44:49 2025 read: IOPS=20, BW=83.0KiB/s (85.0kB/s)(6028KiB/72584msec) clat (usec): min=2568, max=95113, avg=20936.38, stdev=17003.21 lat (usec): min=2568, max=95114, avg=20937.24, stdev=17003.34 clat percentiles (usec): | 1.00th=[ 3294], 5.00th=[ 4113], 10.00th=[ 4686], 20.00th=[ 5997], | 30.00th=[ 9372], 40.00th=[13173], 50.00th=[16188], 60.00th=[19792], | 70.00th=[24773], 80.00th=[32113], 90.00th=[46924], 95.00th=[55837], | 99.00th=[74974], 99.50th=[84411], 99.90th=[93848], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 23, max= 408, per=25.22%, avg=82.96, stdev=71.61, samples=143 iops : min= 5, max= 102, avg=20.36, stdev=18.03, samples=143 write: IOPS=21, BW=86.2KiB/s (88.3kB/s)(6260KiB/72584msec); 0 zone resets clat (msec): min=4, max=770, avg=26.09, stdev=26.28 lat (msec): min=4, max=770, avg=26.09, stdev=26.28 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 13], 40.00th=[ 17], 50.00th=[ 22], 60.00th=[ 27], | 70.00th=[ 34], 80.00th=[ 41], 90.00th=[ 52], 95.00th=[ 62], | 99.00th=[ 84], 99.50th=[ 90], 99.90th=[ 136], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 15, max= 368, per=24.90%, avg=85.92, stdev=67.42, samples=143 iops : min= 3, max= 92, avg=21.10, stdev=16.98, samples=143 lat (msec) : 4=2.05%, 10=26.46%, 20=25.36%, 50=36.65%, 100=9.34% lat (msec) : 250=0.10%, 1000=0.03% cpu : usr=0.10%, sys=13.86%, ctx=4727, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186637: Tue Sep 2 07:44:49 2025 read: IOPS=20, BW=83.2KiB/s (85.2kB/s)(6052KiB/72770msec) clat (msec): min=2, max=117, avg=21.45, stdev=17.98 lat (msec): min=2, max=117, avg=21.45, stdev=17.98 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 9], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 26], 80.00th=[ 35], 90.00th=[ 50], 95.00th=[ 59], | 99.00th=[ 81], 99.50th=[ 87], 99.90th=[ 114], 99.95th=[ 118], | 99.99th=[ 118] bw ( KiB/s): min= 8, max= 376, per=24.96%, avg=82.12, stdev=68.93, samples=143 iops : min= 2, max= 94, avg=20.15, stdev=17.35, samples=143 write: IOPS=21, BW=85.7KiB/s (87.8kB/s)(6236KiB/72770msec); 0 zone resets clat (msec): min=4, max=763, avg=25.73, stdev=26.58 lat (msec): min=4, max=763, avg=25.73, stdev=26.58 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 16], 50.00th=[ 21], 60.00th=[ 27], | 70.00th=[ 33], 80.00th=[ 41], 90.00th=[ 53], 95.00th=[ 64], | 99.00th=[ 86], 99.50th=[ 92], 99.90th=[ 128], 99.95th=[ 760], | 99.99th=[ 760] bw ( KiB/s): min= 15, max= 400, per=24.48%, avg=84.46, stdev=70.08, samples=143 iops : min= 3, max= 100, avg=20.73, stdev=17.65, samples=143 lat (msec) : 4=2.44%, 10=27.90%, 20=24.32%, 50=35.22%, 100=9.93% lat (msec) : 250=0.16%, 1000=0.03% cpu : usr=0.10%, sys=13.53%, ctx=4645, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=330KiB/s (338kB/s), 82.1KiB/s-83.2KiB/s (84.1kB/s-85.2kB/s), io=23.4MiB (24.6MB), run=72363-72770msec WRITE: bw=345KiB/s (354kB/s), 85.7KiB/s-87.7KiB/s (87.8kB/s-89.8kB/s), io=24.6MiB (25.7MB), run=72363-72770msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=186661: Tue Sep 2 07:45:09 2025 read: IOPS=24, BW=394KiB/s (403kB/s)(5920KiB/15043msec) clat (usec): min=278, max=56548, avg=20338.68, stdev=10248.42 lat (usec): min=278, max=56549, avg=20339.28, stdev=10248.46 clat percentiles (usec): | 1.00th=[ 367], 5.00th=[ 635], 10.00th=[ 9896], 20.00th=[12387], | 30.00th=[15533], 40.00th=[17433], 50.00th=[19530], 60.00th=[21627], | 70.00th=[23987], 80.00th=[27395], 90.00th=[33817], 95.00th=[38011], | 99.00th=[49546], 99.50th=[55837], 99.90th=[56361], 99.95th=[56361], | 99.99th=[56361] bw ( KiB/s): min= 128, max= 571, per=24.54%, avg=384.03, stdev=106.88, samples=29 iops : min= 8, max= 35, avg=23.48, stdev= 6.68, samples=29 write: IOPS=26, BW=423KiB/s (433kB/s)(6368KiB/15043msec); 0 zone resets clat (usec): min=1170, max=72256, avg=18825.97, stdev=12150.97 lat (usec): min=1171, max=72258, avg=18827.15, stdev=12151.10 clat percentiles (usec): | 1.00th=[ 1893], 5.00th=[ 4621], 10.00th=[ 5538], 20.00th=[ 8225], | 30.00th=[11207], 40.00th=[13566], 50.00th=[15664], 60.00th=[19268], | 70.00th=[23462], 80.00th=[27919], 90.00th=[35914], 95.00th=[42206], | 99.00th=[57410], 99.50th=[69731], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min= 223, max= 670, per=25.53%, avg=420.24, stdev=120.92, samples=29 iops : min= 13, max= 41, avg=25.72, stdev= 7.58, samples=29 lat (usec) : 500=1.95%, 750=0.65%, 1000=0.39% lat (msec) : 2=0.52%, 4=1.30%, 10=13.93%, 20=38.28%, 50=41.80% lat (msec) : 100=1.17% cpu : usr=0.09%, sys=30.10%, ctx=2911, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186662: Tue Sep 2 07:45:09 2025 read: IOPS=24, BW=385KiB/s (394kB/s)(5744KiB/14927msec) clat (usec): min=183, max=82408, avg=21598.09, stdev=10918.95 lat (usec): min=184, max=82409, avg=21598.70, stdev=10919.03 clat percentiles (usec): | 1.00th=[ 367], 5.00th=[ 9634], 10.00th=[10552], 20.00th=[13435], | 30.00th=[14877], 40.00th=[17171], 50.00th=[19530], 60.00th=[21890], | 70.00th=[24773], 80.00th=[28705], 90.00th=[36963], 95.00th=[41681], | 99.00th=[60031], 99.50th=[62653], 99.90th=[82314], 99.95th=[82314], | 99.99th=[82314] bw ( KiB/s): min= 223, max= 636, per=24.56%, avg=384.31, stdev=121.12, samples=29 iops : min= 13, max= 39, avg=23.48, stdev= 7.57, samples=29 write: IOPS=27, BW=438KiB/s (449kB/s)(6544KiB/14927msec); 0 zone resets clat (usec): min=1347, max=69552, avg=17442.00, stdev=12258.98 lat (usec): min=1349, max=69554, avg=17443.25, stdev=12259.12 clat percentiles (usec): | 1.00th=[ 3589], 5.00th=[ 4686], 10.00th=[ 5145], 20.00th=[ 7111], | 30.00th=[ 9372], 40.00th=[11731], 50.00th=[13960], 60.00th=[16581], | 70.00th=[21365], 80.00th=[27132], 90.00th=[32900], 95.00th=[42206], | 99.00th=[61080], 99.50th=[62653], 99.90th=[69731], 99.95th=[69731], | 99.99th=[69731] bw ( KiB/s): min= 190, max= 576, per=26.34%, avg=433.52, stdev=127.39, samples=29 iops : min= 11, max= 36, avg=26.59, stdev= 8.02, samples=29 lat (usec) : 250=0.13%, 500=0.78%, 750=0.13% lat (msec) : 2=0.39%, 4=0.65%, 10=18.49%, 20=39.97%, 50=37.37% lat (msec) : 100=2.08% cpu : usr=0.09%, sys=30.94%, ctx=2767, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186663: Tue Sep 2 07:45:09 2025 read: IOPS=25, BW=409KiB/s (419kB/s)(6256KiB/15301msec) clat (usec): min=132, max=79946, avg=21075.25, stdev=12398.24 lat (usec): min=133, max=79978, avg=21082.59, stdev=12407.95 clat percentiles (usec): | 1.00th=[ 314], 5.00th=[ 453], 10.00th=[ 8356], 20.00th=[12387], | 30.00th=[15139], 40.00th=[16909], 50.00th=[19792], 60.00th=[22152], | 70.00th=[24773], 80.00th=[28705], 90.00th=[35390], 95.00th=[42730], | 99.00th=[72877], 99.50th=[74974], 99.90th=[80217], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 160, max= 670, per=26.01%, avg=407.10, stdev=129.42, samples=30 iops : min= 10, max= 41, avg=24.90, stdev= 8.11, samples=30 write: IOPS=24, BW=394KiB/s (404kB/s)(6032KiB/15301msec); 0 zone resets clat (usec): min=1862, max=79852, avg=18613.36, stdev=12381.78 lat (usec): min=1863, max=79853, avg=18614.64, stdev=12381.87 clat percentiles (usec): | 1.00th=[ 3261], 5.00th=[ 4555], 10.00th=[ 5473], 20.00th=[ 8094], | 30.00th=[10552], 40.00th=[13304], 50.00th=[15270], 60.00th=[19006], | 70.00th=[22938], 80.00th=[27395], 90.00th=[36439], 95.00th=[42730], | 99.00th=[58983], 99.50th=[64750], 99.90th=[80217], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 191, max= 704, per=23.53%, avg=387.23, stdev=115.53, samples=30 iops : min= 11, max= 44, avg=23.67, stdev= 7.32, samples=30 lat (usec) : 250=0.26%, 500=2.47%, 750=1.04%, 1000=0.26% lat (msec) : 2=0.26%, 4=1.17%, 10=14.97%, 20=36.46%, 50=41.28% lat (msec) : 100=1.82% cpu : usr=0.08%, sys=29.50%, ctx=2936, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186664: Tue Sep 2 07:45:09 2025 read: IOPS=25, BW=405KiB/s (415kB/s)(6032KiB/14881msec) clat (usec): min=93, max=72103, avg=20766.29, stdev=11190.06 lat (usec): min=93, max=72104, avg=20766.92, stdev=11190.15 clat percentiles (usec): | 1.00th=[ 322], 5.00th=[ 840], 10.00th=[10159], 20.00th=[12911], | 30.00th=[15139], 40.00th=[17171], 50.00th=[19006], 60.00th=[21365], | 70.00th=[23987], 80.00th=[28705], 90.00th=[34341], 95.00th=[41157], | 99.00th=[64226], 99.50th=[67634], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min= 191, max= 608, per=25.79%, avg=403.66, stdev=114.57, samples=29 iops : min= 11, max= 38, avg=24.69, stdev= 7.24, samples=29 write: IOPS=26, BW=420KiB/s (430kB/s)(6256KiB/14881msec); 0 zone resets clat (usec): min=2028, max=60945, avg=17932.33, stdev=11851.30 lat (usec): min=2029, max=60946, avg=17933.86, stdev=11851.45 clat percentiles (usec): | 1.00th=[ 3752], 5.00th=[ 4686], 10.00th=[ 5342], 20.00th=[ 7439], | 30.00th=[10028], 40.00th=[12387], 50.00th=[15008], 60.00th=[17433], | 70.00th=[21627], 80.00th=[26084], 90.00th=[35390], 95.00th=[42206], | 99.00th=[55837], 99.50th=[60031], 99.90th=[61080], 99.95th=[61080], | 99.99th=[61080] bw ( KiB/s): min= 158, max= 699, per=25.44%, avg=418.76, stdev=129.77, samples=29 iops : min= 9, max= 43, avg=25.66, stdev= 8.11, samples=29 lat (usec) : 100=0.13%, 250=0.13%, 500=1.43%, 750=0.65%, 1000=0.39% lat (msec) : 2=0.13%, 4=0.78%, 10=16.28%, 20=39.58%, 50=38.28% lat (msec) : 100=2.21% cpu : usr=0.08%, sys=30.27%, ctx=2886, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1565KiB/s (1603kB/s), 385KiB/s-409KiB/s (394kB/s-419kB/s), io=23.4MiB (24.5MB), run=14881-15301msec WRITE: bw=1647KiB/s (1686kB/s), 394KiB/s-438KiB/s (404kB/s-449kB/s), io=24.6MiB (25.8MB), run=14881-15301msec rand-rw: (groupid=0, jobs=1): err= 0: pid=186666: Tue Sep 2 07:45:11 2025 read: IOPS=21, BW=348KiB/s (356kB/s)(5920KiB/17014msec) clat (usec): min=2686, max=95683, avg=19803.70, stdev=15930.43 lat (usec): min=2687, max=95684, avg=19804.44, stdev=15930.52 clat percentiles (usec): | 1.00th=[ 4015], 5.00th=[ 5407], 10.00th=[ 6128], 20.00th=[ 8455], | 30.00th=[10028], 40.00th=[11600], 50.00th=[13566], 60.00th=[17171], | 70.00th=[21365], 80.00th=[31065], 90.00th=[41681], 95.00th=[51119], | 99.00th=[80217], 99.50th=[94897], 99.90th=[95945], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 63, max= 862, per=23.98%, avg=337.45, stdev=185.80, samples=33 iops : min= 3, max= 53, avg=20.48, stdev=11.77, samples=33 write: IOPS=23, BW=374KiB/s (383kB/s)(6368KiB/17014msec); 0 zone resets clat (usec): min=3954, max=89699, avg=24230.14, stdev=16593.29 lat (usec): min=3955, max=89701, avg=24231.65, stdev=16593.40 clat percentiles (usec): | 1.00th=[ 5276], 5.00th=[ 7111], 10.00th=[ 8160], 20.00th=[10421], | 30.00th=[12911], 40.00th=[15139], 50.00th=[19006], 60.00th=[23725], | 70.00th=[28967], 80.00th=[35914], 90.00th=[48497], 95.00th=[57410], | 99.00th=[77071], 99.50th=[86508], 99.90th=[89654], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 128, max= 992, per=25.03%, avg=370.67, stdev=204.14, samples=33 iops : min= 8, max= 62, avg=22.55, stdev=12.88, samples=33 lat (msec) : 4=0.52%, 10=23.05%, 20=36.20%, 50=32.81%, 100=7.42% cpu : usr=0.08%, sys=13.86%, ctx=1207, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186667: Tue Sep 2 07:45:11 2025 read: IOPS=21, BW=338KiB/s (346kB/s)(5744KiB/16989msec) clat (usec): min=3439, max=86059, avg=19755.00, stdev=14370.14 lat (usec): min=3439, max=86060, avg=19755.64, stdev=14370.21 clat percentiles (usec): | 1.00th=[ 4293], 5.00th=[ 5604], 10.00th=[ 6652], 20.00th=[ 8586], | 30.00th=[10421], 40.00th=[12911], 50.00th=[14746], 60.00th=[17433], | 70.00th=[22414], 80.00th=[29492], 90.00th=[41157], 95.00th=[49021], | 99.00th=[67634], 99.50th=[76022], 99.90th=[86508], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 95, max= 798, per=23.85%, avg=335.58, stdev=165.44, samples=33 iops : min= 5, max= 49, avg=20.36, stdev=10.47, samples=33 write: IOPS=24, BW=385KiB/s (394kB/s)(6544KiB/16989msec); 0 zone resets clat (usec): min=5421, max=81455, avg=24081.00, stdev=15773.77 lat (usec): min=5422, max=81457, avg=24082.57, stdev=15773.84 clat percentiles (usec): | 1.00th=[ 6128], 5.00th=[ 7504], 10.00th=[ 8717], 20.00th=[10290], | 30.00th=[13173], 40.00th=[15401], 50.00th=[19530], 60.00th=[23462], | 70.00th=[28967], 80.00th=[36439], 90.00th=[49021], 95.00th=[57934], | 99.00th=[71828], 99.50th=[76022], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 160, max= 1120, per=25.55%, avg=378.45, stdev=210.46, samples=33 iops : min= 10, max= 70, avg=23.03, stdev=13.31, samples=33 lat (msec) : 4=0.39%, 10=22.01%, 20=35.81%, 50=34.90%, 100=6.90% cpu : usr=0.08%, sys=13.97%, ctx=1265, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186668: Tue Sep 2 07:45:11 2025 read: IOPS=23, BW=369KiB/s (377kB/s)(6256KiB/16971msec) clat (usec): min=4045, max=92224, avg=19611.74, stdev=15114.82 lat (usec): min=4046, max=92225, avg=19612.49, stdev=15114.95 clat percentiles (usec): | 1.00th=[ 4359], 5.00th=[ 5932], 10.00th=[ 6521], 20.00th=[ 8717], | 30.00th=[10421], 40.00th=[11863], 50.00th=[14615], 60.00th=[16909], | 70.00th=[21365], 80.00th=[28967], 90.00th=[40109], 95.00th=[52167], | 99.00th=[76022], 99.50th=[81265], 99.90th=[91751], 99.95th=[91751], | 99.99th=[91751] bw ( KiB/s): min= 95, max= 1056, per=26.19%, avg=368.48, stdev=220.95, samples=33 iops : min= 5, max= 66, avg=22.42, stdev=13.98, samples=33 write: IOPS=22, BW=355KiB/s (364kB/s)(6032KiB/16971msec); 0 zone resets clat (usec): min=5140, max=82696, avg=24398.92, stdev=16747.03 lat (usec): min=5141, max=82698, avg=24400.77, stdev=16747.59 clat percentiles (usec): | 1.00th=[ 6128], 5.00th=[ 6915], 10.00th=[ 8455], 20.00th=[10159], | 30.00th=[12780], 40.00th=[15401], 50.00th=[19268], 60.00th=[24773], | 70.00th=[28705], 80.00th=[35390], 90.00th=[51119], 95.00th=[60556], | 99.00th=[73925], 99.50th=[79168], 99.90th=[82314], 99.95th=[82314], | 99.99th=[82314] bw ( KiB/s): min= 127, max= 1053, per=23.59%, avg=349.36, stdev=193.72, samples=33 iops : min= 7, max= 65, avg=21.21, stdev=12.23, samples=33 lat (msec) : 10=23.44%, 20=36.33%, 50=31.90%, 100=8.33% cpu : usr=0.09%, sys=14.11%, ctx=1194, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186669: Tue Sep 2 07:45:11 2025 read: IOPS=22, BW=358KiB/s (367kB/s)(6032KiB/16847msec) clat (usec): min=3687, max=95704, avg=19704.74, stdev=14692.97 lat (usec): min=3687, max=95705, avg=19705.43, stdev=14693.03 clat percentiles (usec): | 1.00th=[ 5014], 5.00th=[ 5866], 10.00th=[ 6849], 20.00th=[ 8717], | 30.00th=[10421], 40.00th=[12256], 50.00th=[14091], 60.00th=[17171], | 70.00th=[21890], 80.00th=[29754], 90.00th=[41681], 95.00th=[50070], | 99.00th=[68682], 99.50th=[80217], 99.90th=[95945], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 149, max= 1024, per=25.47%, avg=358.39, stdev=204.91, samples=33 iops : min= 9, max= 64, avg=21.79, stdev=12.97, samples=33 write: IOPS=23, BW=371KiB/s (380kB/s)(6256KiB/16847msec); 0 zone resets clat (usec): min=5133, max=86383, avg=23996.06, stdev=15588.43 lat (usec): min=5134, max=86386, avg=23997.85, stdev=15588.73 clat percentiles (usec): | 1.00th=[ 6128], 5.00th=[ 7570], 10.00th=[ 8717], 20.00th=[10945], | 30.00th=[12780], 40.00th=[15401], 50.00th=[18744], 60.00th=[22938], | 70.00th=[30540], 80.00th=[36439], 90.00th=[46400], 95.00th=[54789], | 99.00th=[73925], 99.50th=[77071], 99.90th=[86508], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 128, max= 926, per=25.23%, avg=373.64, stdev=187.74, samples=33 iops : min= 8, max= 57, avg=22.73, stdev=11.84, samples=33 lat (msec) : 4=0.13%, 10=21.74%, 20=37.63%, 50=34.64%, 100=5.86% cpu : usr=0.09%, sys=14.29%, ctx=1204, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1408KiB/s (1442kB/s), 338KiB/s-369KiB/s (346kB/s-377kB/s), io=23.4MiB (24.5MB), run=16847-17014msec WRITE: bw=1481KiB/s (1517kB/s), 355KiB/s-385KiB/s (364kB/s-394kB/s), io=24.6MiB (25.8MB), run=16847-17014msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=186679: Tue Sep 2 07:45:20 2025 read: IOPS=2, BW=2671KiB/s (2735kB/s)(7168KiB/2684msec) clat (msec): min=55, max=490, avg=259.54, stdev=173.09 lat (msec): min=55, max=490, avg=259.54, stdev=173.09 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 56], 10.00th=[ 56], 20.00th=[ 96], | 30.00th=[ 138], 40.00th=[ 138], 50.00th=[ 230], 60.00th=[ 384], | 70.00th=[ 384], 80.00th=[ 422], 90.00th=[ 489], 95.00th=[ 489], | 99.00th=[ 489], 99.50th=[ 489], 99.90th=[ 489], 99.95th=[ 489], | 99.99th=[ 489] bw ( KiB/s): min= 1882, max= 4096, per=37.66%, avg=2414.80, stdev=942.26, samples=5 iops : min= 1, max= 4, avg= 2.00, stdev= 1.22, samples=5 write: IOPS=1, BW=1908KiB/s (1953kB/s)(5120KiB/2684msec); 0 zone resets clat (msec): min=89, max=279, avg=164.30, stdev=75.53 lat (msec): min=89, max=279, avg=164.36, stdev=75.53 clat percentiles (msec): | 1.00th=[ 90], 5.00th=[ 90], 10.00th=[ 90], 20.00th=[ 90], | 30.00th=[ 125], 40.00th=[ 125], 50.00th=[ 130], 60.00th=[ 130], | 70.00th=[ 199], 80.00th=[ 199], 90.00th=[ 279], 95.00th=[ 279], | 99.00th=[ 279], 99.50th=[ 279], 99.90th=[ 279], 99.95th=[ 279], | 99.99th=[ 279] bw ( KiB/s): min= 2048, max= 4000, per=30.07%, avg=2698.67, stdev=1126.99, samples=3 iops : min= 2, max= 3, avg= 2.33, stdev= 0.58, samples=3 lat (msec) : 100=25.00%, 250=41.67%, 500=33.33% cpu : usr=0.04%, sys=16.96%, ctx=108, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186680: Tue Sep 2 07:45:20 2025 read: IOPS=0, BW=641KiB/s (657kB/s)(2048KiB/3194msec) clat (msec): min=61, max=546, avg=303.85, stdev=342.59 lat (msec): min=61, max=546, avg=303.85, stdev=342.59 clat percentiles (msec): | 1.00th=[ 62], 5.00th=[ 62], 10.00th=[ 62], 20.00th=[ 62], | 30.00th=[ 62], 40.00th=[ 62], 50.00th=[ 62], 60.00th=[ 550], | 70.00th=[ 550], 80.00th=[ 550], 90.00th=[ 550], 95.00th=[ 550], | 99.00th=[ 550], 99.50th=[ 550], 99.90th=[ 550], 99.95th=[ 550], | 99.99th=[ 550] bw ( KiB/s): min= 1822, max= 2000, per=29.80%, avg=1911.00, stdev=125.87, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3206KiB/s (3283kB/s)(10.0MiB/3194msec); 0 zone resets clat (msec): min=76, max=409, avg=254.65, stdev=107.75 lat (msec): min=77, max=409, avg=254.71, stdev=107.75 clat percentiles (msec): | 1.00th=[ 78], 5.00th=[ 78], 10.00th=[ 78], 20.00th=[ 121], | 30.00th=[ 174], 40.00th=[ 247], 50.00th=[ 249], 60.00th=[ 262], | 70.00th=[ 305], 80.00th=[ 326], 90.00th=[ 384], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 1822, max= 6131, per=35.97%, avg=3229.00, stdev=1866.98, samples=5 iops : min= 1, max= 5, avg= 2.80, stdev= 1.64, samples=5 lat (msec) : 100=16.67%, 250=33.33%, 500=41.67%, 750=8.33% cpu : usr=0.00%, sys=23.11%, ctx=166, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186681: Tue Sep 2 07:45:20 2025 read: IOPS=2, BW=3001KiB/s (3074kB/s)(6144KiB/2047msec) clat (msec): min=68, max=259, avg=153.64, stdev=72.53 lat (msec): min=68, max=259, avg=153.64, stdev=72.53 clat percentiles (msec): | 1.00th=[ 69], 5.00th=[ 69], 10.00th=[ 69], 20.00th=[ 77], | 30.00th=[ 77], 40.00th=[ 153], 50.00th=[ 153], 60.00th=[ 182], | 70.00th=[ 186], 80.00th=[ 186], 90.00th=[ 259], 95.00th=[ 259], | 99.00th=[ 259], 99.50th=[ 259], 99.90th=[ 259], 99.95th=[ 259], | 99.99th=[ 259] bw ( KiB/s): min= 4096, max= 4096, per=63.88%, avg=4096.00, stdev= 0.00, samples=2 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=2 write: IOPS=2, BW=3001KiB/s (3074kB/s)(6144KiB/2047msec); 0 zone resets clat (msec): min=51, max=374, avg=185.85, stdev=123.53 lat (msec): min=51, max=374, avg=185.93, stdev=123.53 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 52], 10.00th=[ 52], 20.00th=[ 73], | 30.00th=[ 73], 40.00th=[ 133], 50.00th=[ 133], 60.00th=[ 224], | 70.00th=[ 262], 80.00th=[ 262], 90.00th=[ 376], 95.00th=[ 376], | 99.00th=[ 376], 99.50th=[ 376], 99.90th=[ 376], 99.95th=[ 376], | 99.99th=[ 376] bw ( KiB/s): min= 2048, max= 5485, per=41.96%, avg=3766.50, stdev=2430.33, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 lat (msec) : 100=33.33%, 250=41.67%, 500=25.00% cpu : usr=0.00%, sys=20.43%, ctx=123, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186682: Tue Sep 2 07:45:20 2025 read: IOPS=1, BW=1636KiB/s (1676kB/s)(5120KiB/3129msec) clat (msec): min=48, max=445, avg=243.59, stdev=184.10 lat (msec): min=48, max=445, avg=243.59, stdev=184.10 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 50], 20.00th=[ 50], | 30.00th=[ 56], 40.00th=[ 56], 50.00th=[ 284], 60.00th=[ 284], | 70.00th=[ 384], 80.00th=[ 384], 90.00th=[ 447], 95.00th=[ 447], | 99.00th=[ 447], 99.50th=[ 447], 99.90th=[ 447], 99.95th=[ 447], | 99.99th=[ 447] bw ( KiB/s): min= 2000, max= 4087, per=42.29%, avg=2711.67, stdev=1191.32, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 write: IOPS=2, BW=2291KiB/s (2346kB/s)(7168KiB/3129msec); 0 zone resets clat (msec): min=75, max=666, avg=263.06, stdev=202.14 lat (msec): min=75, max=666, avg=263.13, stdev=202.14 clat percentiles (msec): | 1.00th=[ 75], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 113], | 30.00th=[ 163], 40.00th=[ 163], 50.00th=[ 197], 60.00th=[ 266], | 70.00th=[ 266], 80.00th=[ 363], 90.00th=[ 667], 95.00th=[ 667], | 99.00th=[ 667], 99.50th=[ 667], 99.90th=[ 667], 99.95th=[ 667], | 99.99th=[ 667] bw ( KiB/s): min= 1825, max= 4096, per=33.58%, avg=3014.00, stdev=1247.52, samples=4 iops : min= 1, max= 4, avg= 2.50, stdev= 1.29, samples=4 lat (msec) : 50=8.33%, 100=16.67%, 250=25.00%, 500=41.67%, 750=8.33% cpu : usr=0.06%, sys=17.10%, ctx=123, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6412KiB/s (6566kB/s), 641KiB/s-3001KiB/s (657kB/s-3074kB/s), io=20.0MiB (20.0MB), run=2047-3194msec WRITE: bw=8977KiB/s (9192kB/s), 1908KiB/s-3206KiB/s (1953kB/s-3283kB/s), io=28.0MiB (29.4MB), run=2047-3194msec rand-rw: (groupid=0, jobs=1): err= 0: pid=186683: Tue Sep 2 07:45:20 2025 read: IOPS=3, BW=3519KiB/s (3603kB/s)(7168KiB/2037msec) clat (msec): min=126, max=241, avg=178.71, stdev=49.43 lat (msec): min=126, max=241, avg=178.71, stdev=49.42 clat percentiles (msec): | 1.00th=[ 127], 5.00th=[ 127], 10.00th=[ 127], 20.00th=[ 131], | 30.00th=[ 144], 40.00th=[ 144], 50.00th=[ 169], 60.00th=[ 199], | 70.00th=[ 199], 80.00th=[ 243], 90.00th=[ 243], 95.00th=[ 243], | 99.00th=[ 243], 99.50th=[ 243], 99.90th=[ 243], 99.95th=[ 243], | 99.99th=[ 243] bw ( KiB/s): min= 4087, max= 5398, per=72.16%, avg=4742.50, stdev=927.02, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=2, BW=2514KiB/s (2574kB/s)(5120KiB/2037msec); 0 zone resets clat (msec): min=134, max=182, avg=153.59, stdev=19.90 lat (msec): min=134, max=182, avg=153.65, stdev=19.91 clat percentiles (msec): | 1.00th=[ 134], 5.00th=[ 134], 10.00th=[ 134], 20.00th=[ 134], | 30.00th=[ 136], 40.00th=[ 136], 50.00th=[ 157], 60.00th=[ 157], | 70.00th=[ 161], 80.00th=[ 161], 90.00th=[ 182], 95.00th=[ 182], | 99.00th=[ 182], 99.50th=[ 182], 99.90th=[ 182], 99.95th=[ 182], | 99.99th=[ 182] bw ( KiB/s): min= 2043, max= 6107, per=44.29%, avg=4075.00, stdev=2873.68, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 lat (msec) : 250=100.00% cpu : usr=0.05%, sys=59.09%, ctx=1285, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186684: Tue Sep 2 07:45:20 2025 read: IOPS=0, BW=657KiB/s (673kB/s)(2048KiB/3116msec) clat (msec): min=139, max=205, avg=172.91, stdev=46.80 lat (msec): min=139, max=205, avg=172.91, stdev=46.80 clat percentiles (msec): | 1.00th=[ 140], 5.00th=[ 140], 10.00th=[ 140], 20.00th=[ 140], | 30.00th=[ 140], 40.00th=[ 140], 50.00th=[ 140], 60.00th=[ 207], | 70.00th=[ 207], 80.00th=[ 207], 90.00th=[ 207], 95.00th=[ 207], | 99.00th=[ 207], 99.50th=[ 207], 99.90th=[ 207], 99.95th=[ 207], | 99.99th=[ 207] bw ( KiB/s): min= 1838, max= 2048, per=29.56%, avg=1943.00, stdev=148.49, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3286KiB/s (3365kB/s)(10.0MiB/3116msec); 0 zone resets clat (msec): min=135, max=425, avg=269.51, stdev=103.20 lat (msec): min=135, max=425, avg=269.60, stdev=103.24 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 169], | 30.00th=[ 201], 40.00th=[ 215], 50.00th=[ 224], 60.00th=[ 226], | 70.00th=[ 338], 80.00th=[ 351], 90.00th=[ 414], 95.00th=[ 426], | 99.00th=[ 426], 99.50th=[ 426], 99.90th=[ 426], 99.95th=[ 426], | 99.99th=[ 426] bw ( KiB/s): min= 2035, max= 4096, per=39.86%, avg=3667.40, stdev=913.05, samples=5 iops : min= 1, max= 4, avg= 3.00, stdev= 1.22, samples=5 lat (msec) : 250=66.67%, 500=33.33% cpu : usr=0.03%, sys=42.41%, ctx=535, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186685: Tue Sep 2 07:45:20 2025 read: IOPS=2, BW=2349KiB/s (2405kB/s)(6144KiB/2616msec) clat (usec): min=602, max=153015, avg=65507.19, stdev=72062.72 lat (usec): min=602, max=153016, avg=65507.97, stdev=72062.76 clat percentiles (usec): | 1.00th=[ 603], 5.00th=[ 603], 10.00th=[ 603], 20.00th=[ 742], | 30.00th=[ 742], 40.00th=[ 848], 50.00th=[ 848], 60.00th=[115868], | 70.00th=[121111], 80.00th=[121111], 90.00th=[152044], 95.00th=[152044], | 99.00th=[152044], 99.50th=[152044], 99.90th=[152044], 99.95th=[152044], | 99.99th=[152044] bw ( KiB/s): min= 2011, max= 4096, per=51.62%, avg=3392.67, stdev=1196.62, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=2, BW=2349KiB/s (2405kB/s)(6144KiB/2616msec); 0 zone resets clat (msec): min=148, max=711, avg=354.12, stdev=276.12 lat (msec): min=148, max=711, avg=354.17, stdev=276.13 clat percentiles (msec): | 1.00th=[ 148], 5.00th=[ 148], 10.00th=[ 148], 20.00th=[ 174], | 30.00th=[ 174], 40.00th=[ 176], 50.00th=[ 176], 60.00th=[ 209], | 70.00th=[ 709], 80.00th=[ 709], 90.00th=[ 709], 95.00th=[ 709], | 99.00th=[ 709], 99.50th=[ 709], 99.90th=[ 709], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 2035, max= 4023, per=31.93%, avg=2937.50, stdev=1046.13, samples=4 iops : min= 1, max= 3, avg= 2.25, stdev= 0.96, samples=4 lat (usec) : 750=16.67%, 1000=8.33% lat (msec) : 250=58.33%, 750=16.67% cpu : usr=0.00%, sys=39.54%, ctx=179, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186686: Tue Sep 2 07:45:20 2025 read: IOPS=1, BW=1979KiB/s (2027kB/s)(5120KiB/2587msec) clat (usec): min=599, max=344624, avg=103039.27, stdev=153278.24 lat (usec): min=599, max=344625, avg=103040.03, stdev=153278.28 clat percentiles (usec): | 1.00th=[ 603], 5.00th=[ 603], 10.00th=[ 603], 20.00th=[ 603], | 30.00th=[ 848], 40.00th=[ 848], 50.00th=[ 914], 60.00th=[ 914], | 70.00th=[168821], 80.00th=[168821], 90.00th=[346031], 95.00th=[346031], | 99.00th=[346031], 99.50th=[346031], 99.90th=[346031], 99.95th=[346031], | 99.99th=[346031] bw ( KiB/s): min= 2011, max= 6144, per=62.04%, avg=4077.50, stdev=2922.47, samples=2 iops : min= 1, max= 6, avg= 3.50, stdev= 3.54, samples=2 write: IOPS=2, BW=2771KiB/s (2837kB/s)(7168KiB/2587msec); 0 zone resets clat (msec): min=115, max=829, avg=288.48, stdev=256.21 lat (msec): min=115, max=829, avg=288.55, stdev=256.21 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 128], | 30.00th=[ 144], 40.00th=[ 144], 50.00th=[ 201], 60.00th=[ 211], | 70.00th=[ 211], 80.00th=[ 393], 90.00th=[ 827], 95.00th=[ 827], | 99.00th=[ 827], 99.50th=[ 827], 99.90th=[ 827], 99.95th=[ 827], | 99.99th=[ 827] bw ( KiB/s): min= 1845, max= 4071, per=32.57%, avg=2996.75, stdev=1215.71, samples=4 iops : min= 1, max= 3, avg= 2.25, stdev= 0.96, samples=4 lat (usec) : 750=8.33%, 1000=16.67% lat (msec) : 250=50.00%, 500=16.67%, 1000=8.33% cpu : usr=0.04%, sys=42.46%, ctx=146, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6573KiB/s (6730kB/s), 657KiB/s-3519KiB/s (673kB/s-3603kB/s), io=20.0MiB (20.0MB), run=2037-3116msec WRITE: bw=9202KiB/s (9422kB/s), 2349KiB/s-3286KiB/s (2405kB/s-3365kB/s), io=28.0MiB (29.4MB), run=2037-3116msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=186692: Tue Sep 2 07:45:30 2025 read: IOPS=2, BW=11.2MiB/s (11.8MB/s)(12.0MiB/1068msec) clat (msec): min=40, max=539, avg=340.33, stdev=264.40 lat (msec): min=40, max=539, avg=340.33, stdev=264.40 clat percentiles (msec): | 1.00th=[ 41], 5.00th=[ 41], 10.00th=[ 41], 20.00th=[ 41], | 30.00th=[ 41], 40.00th=[ 443], 50.00th=[ 443], 60.00th=[ 443], | 70.00th=[ 542], 80.00th=[ 542], 90.00th=[ 542], 95.00th=[ 542], | 99.00th=[ 542], 99.50th=[ 542], 99.90th=[ 542], 99.95th=[ 542], | 99.99th=[ 542] bw ( KiB/s): min= 5657, max= 5657, per=83.63%, avg=5657.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 50=33.33%, 500=33.33%, 750=33.33% cpu : usr=0.00%, sys=40.02%, ctx=1518, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186693: Tue Sep 2 07:45:30 2025 read: IOPS=0, BW=1691KiB/s (1732kB/s)(4096KiB/2422msec) clat (nsec): min=440429k, max=440429k, avg=440429415.00, stdev= 0.00 lat (nsec): min=440432k, max=440432k, avg=440431530.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 443], 5.00th=[ 443], 10.00th=[ 443], 20.00th=[ 443], | 30.00th=[ 443], 40.00th=[ 443], 50.00th=[ 443], 60.00th=[ 443], | 70.00th=[ 443], 80.00th=[ 443], 90.00th=[ 443], 95.00th=[ 443], | 99.00th=[ 443], 99.50th=[ 443], 99.90th=[ 443], 99.95th=[ 443], | 99.99th=[ 443] bw ( KiB/s): min= 5736, max= 5736, per=84.80%, avg=5736.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3382KiB/s (3464kB/s)(8192KiB/2422msec); 0 zone resets clat (msec): min=827, max=1118, avg=973.01, stdev=205.18 lat (msec): min=828, max=1118, avg=973.33, stdev=205.11 clat percentiles (msec): | 1.00th=[ 827], 5.00th=[ 827], 10.00th=[ 827], 20.00th=[ 827], | 30.00th=[ 827], 40.00th=[ 827], 50.00th=[ 827], 60.00th=[ 1116], | 70.00th=[ 1116], 80.00th=[ 1116], 90.00th=[ 1116], 95.00th=[ 1116], | 99.00th=[ 1116], 99.50th=[ 1116], 99.90th=[ 1116], 99.95th=[ 1116], | 99.99th=[ 1116] bw ( KiB/s): min= 8192, max= 8192, per=64.95%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=50.89%, ctx=735, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186694: Tue Sep 2 07:45:30 2025 write: IOPS=1, BW=4977KiB/s (5096kB/s)(12.0MiB/2469msec); 0 zone resets clat (msec): min=528, max=954, avg=806.86, stdev=241.07 lat (msec): min=528, max=955, avg=807.16, stdev=241.23 clat percentiles (msec): | 1.00th=[ 531], 5.00th=[ 531], 10.00th=[ 531], 20.00th=[ 531], | 30.00th=[ 531], 40.00th=[ 936], 50.00th=[ 936], 60.00th=[ 936], | 70.00th=[ 953], 80.00th=[ 953], 90.00th=[ 953], 95.00th=[ 953], | 99.00th=[ 953], 99.50th=[ 953], 99.90th=[ 953], 99.95th=[ 953], | 99.99th=[ 953] bw ( KiB/s): min= 8159, max= 8192, per=64.82%, avg=8175.50, stdev=23.33, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 1000=66.67% cpu : usr=0.00%, sys=59.81%, ctx=80, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186695: Tue Sep 2 07:45:30 2025 write: IOPS=1, BW=4730KiB/s (4843kB/s)(12.0MiB/2598msec); 0 zone resets clat (msec): min=660, max=1045, avg=852.10, stdev=192.58 lat (msec): min=660, max=1045, avg=852.41, stdev=192.62 clat percentiles (msec): | 1.00th=[ 659], 5.00th=[ 659], 10.00th=[ 659], 20.00th=[ 659], | 30.00th=[ 659], 40.00th=[ 852], 50.00th=[ 852], 60.00th=[ 852], | 70.00th=[ 1045], 80.00th=[ 1045], 90.00th=[ 1045], 95.00th=[ 1045], | 99.00th=[ 1045], 99.50th=[ 1045], 99.90th=[ 1045], 99.95th=[ 1045], | 99.99th=[ 1045] bw ( KiB/s): min= 8192, max= 8192, per=64.95%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=55.18%, ctx=76, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6765KiB/s (6927kB/s), 1691KiB/s-11.2MiB/s (1732kB/s-11.8MB/s), io=16.0MiB (16.8MB), run=1068-2422msec WRITE: bw=12.3MiB/s (12.9MB/s), 3382KiB/s-4977KiB/s (3464kB/s-5096kB/s), io=32.0MiB (33.6MB), run=2422-2598msec rand-rw: (groupid=0, jobs=1): err= 0: pid=186696: Tue Sep 2 07:45:31 2025 read: IOPS=0, BW=3699KiB/s (3788kB/s)(12.0MiB/3322msec) clat (msec): min=199, max=1749, avg=1087.77, stdev=799.53 lat (msec): min=199, max=1749, avg=1087.77, stdev=799.53 clat percentiles (msec): | 1.00th=[ 201], 5.00th=[ 201], 10.00th=[ 201], 20.00th=[ 201], | 30.00th=[ 201], 40.00th=[ 1318], 50.00th=[ 1318], 60.00th=[ 1318], | 70.00th=[ 1754], 80.00th=[ 1754], 90.00th=[ 1754], 95.00th=[ 1754], | 99.00th=[ 1754], 99.50th=[ 1754], 99.90th=[ 1754], 99.95th=[ 1754], | 99.99th=[ 1754] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 2000=66.67% cpu : usr=0.00%, sys=7.23%, ctx=49, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186697: Tue Sep 2 07:45:31 2025 read: IOPS=0, BW=1257KiB/s (1287kB/s)(4096KiB/3259msec) clat (nsec): min=1862.8M, max=1862.8M, avg=1862785340.00, stdev= 0.00 lat (nsec): min=1862.8M, max=1862.8M, avg=1862787535.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1871], 5.00th=[ 1871], 10.00th=[ 1871], 20.00th=[ 1871], | 30.00th=[ 1871], 40.00th=[ 1871], 50.00th=[ 1871], 60.00th=[ 1871], | 70.00th=[ 1871], 80.00th=[ 1871], 90.00th=[ 1871], 95.00th=[ 1871], | 99.00th=[ 1871], 99.50th=[ 1871], 99.90th=[ 1871], 99.95th=[ 1871], | 99.99th=[ 1871] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2514KiB/s (2574kB/s)(8192KiB/3259msec); 0 zone resets clat (msec): min=222, max=1104, avg=663.62, stdev=623.87 lat (msec): min=222, max=1105, avg=663.90, stdev=623.88 clat percentiles (msec): | 1.00th=[ 224], 5.00th=[ 224], 10.00th=[ 224], 20.00th=[ 224], | 30.00th=[ 224], 40.00th=[ 224], 50.00th=[ 224], 60.00th=[ 1099], | 70.00th=[ 1099], 80.00th=[ 1099], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 8192, max= 8192, per=81.48%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=66.67% cpu : usr=0.00%, sys=13.32%, ctx=84, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186698: Tue Sep 2 07:45:31 2025 write: IOPS=0, BW=3822KiB/s (3914kB/s)(12.0MiB/3215msec); 0 zone resets clat (msec): min=248, max=1636, avg=1050.21, stdev=718.92 lat (msec): min=248, max=1637, avg=1050.46, stdev=718.92 clat percentiles (msec): | 1.00th=[ 249], 5.00th=[ 249], 10.00th=[ 249], 20.00th=[ 249], | 30.00th=[ 249], 40.00th=[ 1267], 50.00th=[ 1267], 60.00th=[ 1267], | 70.00th=[ 1636], 80.00th=[ 1636], 90.00th=[ 1636], 95.00th=[ 1636], | 99.00th=[ 1636], 99.50th=[ 1636], 99.90th=[ 1636], 99.95th=[ 1636], | 99.99th=[ 1636] bw ( KiB/s): min= 8192, max= 8192, per=81.48%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=66.67% cpu : usr=0.00%, sys=26.29%, ctx=111, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=186699: Tue Sep 2 07:45:31 2025 write: IOPS=1, BW=5928KiB/s (6070kB/s)(12.0MiB/2073msec); 0 zone resets clat (msec): min=212, max=1477, avg=683.37, stdev=691.57 lat (msec): min=213, max=1477, avg=683.68, stdev=691.55 clat percentiles (msec): | 1.00th=[ 213], 5.00th=[ 213], 10.00th=[ 213], 20.00th=[ 213], | 30.00th=[ 213], 40.00th=[ 359], 50.00th=[ 359], 60.00th=[ 359], | 70.00th=[ 1485], 80.00th=[ 1485], 90.00th=[ 1485], 95.00th=[ 1485], | 99.00th=[ 1485], 99.50th=[ 1485], 99.90th=[ 1485], 99.95th=[ 1485], | 99.99th=[ 1485] bw ( KiB/s): min=11570, max=11570, per=100.00%, avg=11570.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=14.62%, ctx=65, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4932KiB/s (5050kB/s), 1257KiB/s-3699KiB/s (1287kB/s-3788kB/s), io=16.0MiB (16.8MB), run=3259-3322msec WRITE: bw=9.82MiB/s (10.3MB/s), 2514KiB/s-5928KiB/s (2574kB/s-6070kB/s), io=32.0MiB (33.6MB), run=2073-3259msec