== sanity test 398b: DIO and buffer IO race ============== 22:10:04 (1763349004) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 5.5308 s, 9.1 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147365: Sun Nov 16 22:12:10 2025 read: IOPS=13, BW=54.7KiB/s (55.0kB/s)(5992KiB/109575msec) clat (usec): min=95, max=677308, avg=36386.87, stdev=35537.57 lat (usec): min=96, max=677310, avg=36388.31, stdev=35537.52 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 176], 10.00th=[ 293], 20.00th=[ 16909], | 30.00th=[ 21365], 40.00th=[ 25560], 50.00th=[ 30540], 60.00th=[ 36439], | 70.00th=[ 44827], 80.00th=[ 54264], 90.00th=[ 66847], 95.00th=[ 78119], | 99.00th=[120062], 99.50th=[143655], 99.90th=[455082], 99.95th=[675283], | 99.99th=[675283] bw ( KiB/s): min= 15, max= 133, per=24.89%, avg=54.51, stdev=23.48, samples=214 iops : min= 3, max= 33, avg=13.01, stdev= 5.95, samples=214 write: IOPS=14, BW=57.5KiB/s (58.8kB/s)(6296KiB/109575msec); 0 zone resets clat (usec): min=915, max=422454, avg=34656.05, stdev=36882.85 lat (usec): min=915, max=422456, avg=34667.13, stdev=36881.34 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 1106], 10.00th=[ 1254], 20.00th=[ 2212], | 30.00th=[ 7832], 40.00th=[ 16909], 50.00th=[ 25822], 60.00th=[ 35390], | 70.00th=[ 45351], 80.00th=[ 60556], 90.00th=[ 81265], 95.00th=[101188], | 99.00th=[154141], 99.50th=[175113], 99.90th=[278922], 99.95th=[421528], | 99.99th=[421528] bw ( KiB/s): min= 15, max= 183, per=25.05%, avg=57.36, stdev=24.69, samples=214 iops : min= 3, max= 45, avg=13.72, stdev= 6.20, samples=214 lat (usec) : 100=0.03%, 250=4.56%, 500=0.72%, 750=0.07%, 1000=0.52% lat (msec) : 2=9.54%, 4=3.03%, 10=4.75%, 20=11.72%, 50=39.88% lat (msec) : 100=21.55%, 250=3.35%, 500=0.26%, 750=0.03% cpu : usr=0.11%, sys=6.82%, ctx=6434, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147366: Sun Nov 16 22:12:10 2025 read: IOPS=13, BW=54.6KiB/s (55.9kB/s)(5940KiB/108722msec) clat (usec): min=107, max=471313, avg=35745.86, stdev=27289.15 lat (usec): min=107, max=471313, avg=35748.92, stdev=27289.66 clat percentiles (usec): | 1.00th=[ 130], 5.00th=[ 180], 10.00th=[ 343], 20.00th=[ 17957], | 30.00th=[ 22938], 40.00th=[ 26346], 50.00th=[ 31327], 60.00th=[ 36963], | 70.00th=[ 43779], 80.00th=[ 53216], 90.00th=[ 67634], 95.00th=[ 79168], | 99.00th=[117965], 99.50th=[129500], 99.90th=[254804], 99.95th=[471860], | 99.99th=[471860] bw ( KiB/s): min= 7, max= 128, per=24.82%, avg=54.35, stdev=23.56, samples=212 iops : min= 1, max= 32, avg=12.96, stdev= 5.94, samples=212 write: IOPS=14, BW=58.4KiB/s (59.8kB/s)(6348KiB/108722msec); 0 zone resets clat (usec): min=915, max=870176, avg=34771.35, stdev=42387.73 lat (usec): min=916, max=870177, avg=34775.44, stdev=42386.81 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1156], 10.00th=[ 1270], 20.00th=[ 2638], | 30.00th=[ 6849], 40.00th=[ 14615], 50.00th=[ 25822], 60.00th=[ 34341], | 70.00th=[ 44303], 80.00th=[ 59507], 90.00th=[ 82314], 95.00th=[103285], | 99.00th=[152044], 99.50th=[177210], 99.90th=[413139], 99.95th=[868221], | 99.99th=[868221] bw ( KiB/s): min= 7, max= 158, per=25.16%, avg=57.62, stdev=25.15, samples=213 iops : min= 1, max= 39, avg=13.79, stdev= 6.35, samples=213 lat (usec) : 250=4.39%, 500=0.62%, 750=0.13%, 1000=0.42% lat (msec) : 2=8.76%, 4=3.32%, 10=6.18%, 20=10.32%, 50=41.83% lat (msec) : 100=20.44%, 250=3.39%, 500=0.16%, 1000=0.03% cpu : usr=0.09%, sys=6.53%, ctx=6234, majf=0, minf=19 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147367: Sun Nov 16 22:12:10 2025 read: IOPS=13, BW=55.3KiB/s (56.7kB/s)(6028KiB/108948msec) clat (usec): min=100, max=948575, avg=35069.73, stdev=34025.37 lat (usec): min=101, max=948577, avg=35071.27, stdev=34025.35 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 178], 10.00th=[ 375], 20.00th=[ 16712], | 30.00th=[ 21365], 40.00th=[ 25560], 50.00th=[ 30278], 60.00th=[ 35914], | 70.00th=[ 42730], 80.00th=[ 52691], 90.00th=[ 64750], 95.00th=[ 78119], | 99.00th=[113771], 99.50th=[130548], 99.90th=[258999], 99.95th=[952108], | 99.99th=[952108] bw ( KiB/s): min= 7, max= 134, per=25.28%, avg=55.36, stdev=22.96, samples=213 iops : min= 1, max= 33, avg=13.21, stdev= 5.80, samples=213 write: IOPS=14, BW=57.5KiB/s (58.8kB/s)(6260KiB/108948msec); 0 zone resets clat (usec): min=931, max=444485, avg=35552.13, stdev=37040.33 lat (usec): min=932, max=444487, avg=35559.83, stdev=37035.37 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 1172], 10.00th=[ 1303], 20.00th=[ 2769], | 30.00th=[ 7898], 40.00th=[ 17957], 50.00th=[ 28705], 60.00th=[ 37487], | 70.00th=[ 48497], 80.00th=[ 58983], 90.00th=[ 80217], 95.00th=[101188], | 99.00th=[145753], 99.50th=[187696], 99.90th=[408945], 99.95th=[442500], | 99.99th=[442500] bw ( KiB/s): min= 7, max= 182, per=25.01%, avg=57.27, stdev=23.68, samples=214 iops : min= 1, max= 45, avg=13.69, stdev= 5.97, samples=214 lat (usec) : 250=4.17%, 500=0.91%, 750=0.16%, 1000=0.16% lat (msec) : 2=8.92%, 4=2.86%, 10=5.63%, 20=11.59%, 50=40.30% lat (msec) : 100=21.74%, 250=3.35%, 500=0.16%, 1000=0.03% cpu : usr=0.10%, sys=6.70%, ctx=6399, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147368: Sun Nov 16 22:12:10 2025 read: IOPS=14, BW=56.0KiB/s (57.4kB/s)(6052KiB/108043msec) clat (usec): min=111, max=319660, avg=35195.18, stdev=25665.56 lat (usec): min=111, max=319661, avg=35196.56, stdev=25665.56 clat percentiles (usec): | 1.00th=[ 131], 5.00th=[ 180], 10.00th=[ 359], 20.00th=[ 17433], | 30.00th=[ 21890], 40.00th=[ 25560], 50.00th=[ 30278], 60.00th=[ 36963], | 70.00th=[ 43254], 80.00th=[ 52691], 90.00th=[ 67634], 95.00th=[ 81265], | 99.00th=[115868], 99.50th=[131597], 99.90th=[166724], 99.95th=[320865], | 99.99th=[320865] bw ( KiB/s): min= 7, max= 135, per=25.65%, avg=56.17, stdev=23.26, samples=211 iops : min= 1, max= 33, avg=13.42, stdev= 5.86, samples=211 write: IOPS=14, BW=57.7KiB/s (59.1kB/s)(6236KiB/108043msec); 0 zone resets clat (usec): min=948, max=876619, avg=34836.96, stdev=43245.98 lat (usec): min=949, max=876621, avg=34840.64, stdev=43246.51 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1156], 10.00th=[ 1287], 20.00th=[ 2769], | 30.00th=[ 7308], 40.00th=[ 17171], 50.00th=[ 25035], 60.00th=[ 34341], | 70.00th=[ 44303], 80.00th=[ 58459], 90.00th=[ 82314], 95.00th=[101188], | 99.00th=[145753], 99.50th=[187696], 99.90th=[413139], 99.95th=[876610], | 99.99th=[876610] bw ( KiB/s): min= 7, max= 151, per=25.13%, avg=57.54, stdev=22.00, samples=212 iops : min= 1, max= 37, avg=13.76, stdev= 5.55, samples=212 lat (usec) : 250=4.30%, 500=0.91%, 750=0.16%, 1000=0.39% lat (msec) : 2=8.63%, 4=3.16%, 10=5.08%, 20=12.50%, 50=40.76% lat (msec) : 100=20.61%, 250=3.26%, 500=0.23%, 1000=0.03% cpu : usr=0.11%, sys=6.64%, ctx=6270, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=219KiB/s (224kB/s), 54.6KiB/s-56.0KiB/s (55.9kB/s-57.4kB/s), io=23.4MiB (24.6MB), run=108043-109575msec WRITE: bw=229KiB/s (235kB/s), 57.5KiB/s-58.4KiB/s (58.8kB/s-59.8kB/s), io=24.6MiB (25.7MB), run=108043-109575msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147370: Sun Nov 16 22:14:26 2025 read: IOPS=6, BW=24.5KiB/s (25.1kB/s)(5992KiB/244230msec) clat (msec): min=5, max=712, avg=29.36, stdev=33.03 lat (msec): min=5, max=712, avg=29.36, stdev=33.03 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 22], | 70.00th=[ 27], 80.00th=[ 39], 90.00th=[ 64], 95.00th=[ 84], | 99.00th=[ 144], 99.50th=[ 171], 99.90th=[ 330], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 7, max= 95, per=27.83%, avg=26.99, stdev=17.88, samples=430 iops : min= 1, max= 23, avg= 6.11, stdev= 4.49, samples=430 write: IOPS=6, BW=25.8KiB/s (26.4kB/s)(6296KiB/244230msec); 0 zone resets clat (msec): min=39, max=551, avg=127.00, stdev=59.01 lat (msec): min=39, max=551, avg=127.00, stdev=59.01 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 63], 10.00th=[ 70], 20.00th=[ 83], | 30.00th=[ 93], 40.00th=[ 102], 50.00th=[ 112], 60.00th=[ 126], | 70.00th=[ 142], 80.00th=[ 163], 90.00th=[ 203], 95.00th=[ 241], | 99.00th=[ 330], 99.50th=[ 380], 99.90th=[ 523], 99.95th=[ 550], | 99.99th=[ 550] bw ( KiB/s): min= 7, max= 63, per=24.82%, avg=25.32, stdev=10.98, samples=480 iops : min= 1, max= 15, avg= 5.70, stdev= 2.80, samples=480 lat (msec) : 10=3.58%, 20=22.40%, 50=16.44%, 100=24.51%, 250=30.66% lat (msec) : 500=2.31%, 750=0.10% cpu : usr=0.05%, sys=1.36%, ctx=5350, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147371: Sun Nov 16 22:14:26 2025 read: IOPS=6, BW=24.2KiB/s (24.8kB/s)(5940KiB/245754msec) clat (msec): min=5, max=916, avg=29.26, stdev=38.29 lat (msec): min=5, max=916, avg=29.27, stdev=38.29 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 22], | 70.00th=[ 27], 80.00th=[ 35], 90.00th=[ 60], 95.00th=[ 83], | 99.00th=[ 153], 99.50th=[ 197], 99.90th=[ 468], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 7, max= 96, per=28.12%, avg=27.28, stdev=18.31, samples=421 iops : min= 1, max= 24, avg= 6.19, stdev= 4.61, samples=421 write: IOPS=6, BW=25.8KiB/s (26.5kB/s)(6348KiB/245754msec); 0 zone resets clat (msec): min=37, max=585, avg=127.25, stdev=58.81 lat (msec): min=37, max=585, avg=127.25, stdev=58.81 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 62], 10.00th=[ 70], 20.00th=[ 82], | 30.00th=[ 93], 40.00th=[ 103], 50.00th=[ 112], 60.00th=[ 125], | 70.00th=[ 142], 80.00th=[ 167], 90.00th=[ 205], 95.00th=[ 245], | 99.00th=[ 313], 99.50th=[ 351], 99.90th=[ 472], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 7, max= 55, per=24.81%, avg=25.30, stdev=10.90, samples=484 iops : min= 1, max= 13, avg= 5.69, stdev= 2.77, samples=484 lat (msec) : 10=3.84%, 20=21.88%, 50=16.47%, 100=24.41%, 250=30.76% lat (msec) : 500=2.57%, 750=0.03%, 1000=0.03% cpu : usr=0.05%, sys=1.31%, ctx=5306, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147372: Sun Nov 16 22:14:26 2025 read: IOPS=6, BW=24.6KiB/s (25.2kB/s)(6028KiB/245402msec) clat (msec): min=5, max=469, avg=30.73, stdev=34.98 lat (msec): min=5, max=469, avg=30.73, stdev=34.98 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 16], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 39], 90.00th=[ 65], 95.00th=[ 90], | 99.00th=[ 180], 99.50th=[ 243], 99.90th=[ 342], 99.95th=[ 468], | 99.99th=[ 468] bw ( KiB/s): min= 7, max= 104, per=27.50%, avg=26.68, stdev=18.04, samples=437 iops : min= 1, max= 26, avg= 6.04, stdev= 4.54, samples=437 write: IOPS=6, BW=25.5KiB/s (26.1kB/s)(6260KiB/245402msec); 0 zone resets clat (msec): min=38, max=1004, avg=126.99, stdev=61.26 lat (msec): min=38, max=1004, avg=126.99, stdev=61.26 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 62], 10.00th=[ 68], 20.00th=[ 82], | 30.00th=[ 91], 40.00th=[ 102], 50.00th=[ 113], 60.00th=[ 127], | 70.00th=[ 142], 80.00th=[ 167], 90.00th=[ 203], 95.00th=[ 236], | 99.00th=[ 313], 99.50th=[ 363], 99.90th=[ 592], 99.95th=[ 1003], | 99.99th=[ 1003] bw ( KiB/s): min= 7, max= 62, per=24.85%, avg=25.35, stdev=11.07, samples=476 iops : min= 1, max= 15, avg= 5.70, stdev= 2.82, samples=476 lat (msec) : 10=3.35%, 20=21.97%, 50=17.12%, 100=24.22%, 250=31.45% lat (msec) : 500=1.79%, 750=0.07%, 2000=0.03% cpu : usr=0.05%, sys=1.31%, ctx=5307, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147373: Sun Nov 16 22:14:26 2025 read: IOPS=6, BW=24.8KiB/s (25.4kB/s)(6052KiB/243716msec) clat (msec): min=5, max=336, avg=28.46, stdev=27.31 lat (msec): min=5, max=336, avg=28.46, stdev=27.31 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 38], 90.00th=[ 61], 95.00th=[ 84], | 99.00th=[ 120], 99.50th=[ 133], 99.90th=[ 334], 99.95th=[ 338], | 99.99th=[ 338] bw ( KiB/s): min= 7, max= 111, per=28.37%, avg=27.51, stdev=18.39, samples=426 iops : min= 1, max= 27, avg= 6.25, stdev= 4.61, samples=426 write: IOPS=6, BW=25.6KiB/s (26.2kB/s)(6236KiB/243716msec); 0 zone resets clat (msec): min=36, max=1000, avg=128.46, stdev=62.79 lat (msec): min=36, max=1000, avg=128.47, stdev=62.79 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 63], 10.00th=[ 71], 20.00th=[ 83], | 30.00th=[ 92], 40.00th=[ 103], 50.00th=[ 114], 60.00th=[ 128], | 70.00th=[ 142], 80.00th=[ 165], 90.00th=[ 203], 95.00th=[ 247], | 99.00th=[ 326], 99.50th=[ 405], 99.90th=[ 518], 99.95th=[ 1003], | 99.99th=[ 1003] bw ( KiB/s): min= 7, max= 56, per=24.74%, avg=25.23, stdev=10.65, samples=477 iops : min= 1, max= 14, avg= 5.68, stdev= 2.73, samples=477 lat (msec) : 10=3.45%, 20=22.56%, 50=16.80%, 100=24.48%, 250=30.27% lat (msec) : 500=2.34%, 750=0.07%, 2000=0.03% cpu : usr=0.06%, sys=1.35%, ctx=5230, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=97.7KiB/s (100kB/s), 24.2KiB/s-24.8KiB/s (24.8kB/s-25.4kB/s), io=23.4MiB (24.6MB), run=243716-245754msec WRITE: bw=102KiB/s (105kB/s), 25.5KiB/s-25.8KiB/s (26.1kB/s-26.5kB/s), io=24.6MiB (25.7MB), run=243716-245754msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147428: Sun Nov 16 22:14:55 2025 read: IOPS=19, BW=316KiB/s (324kB/s)(5920KiB/18728msec) clat (usec): min=108, max=121962, avg=23918.59, stdev=20233.00 lat (usec): min=108, max=121964, avg=23919.71, stdev=20233.03 clat percentiles (usec): | 1.00th=[ 118], 5.00th=[ 145], 10.00th=[ 165], 20.00th=[ 265], | 30.00th=[ 3556], 40.00th=[ 19792], 50.00th=[ 23725], 60.00th=[ 28181], | 70.00th=[ 33162], 80.00th=[ 39584], 90.00th=[ 50070], 95.00th=[ 59507], | 99.00th=[ 74974], 99.50th=[ 83362], 99.90th=[122160], 99.95th=[122160], | 99.99th=[122160] bw ( KiB/s): min= 62, max= 574, per=26.19%, avg=311.42, stdev=117.28, samples=36 iops : min= 3, max= 35, avg=18.78, stdev= 7.33, samples=36 write: IOPS=21, BW=340KiB/s (348kB/s)(6368KiB/18728msec); 0 zone resets clat (usec): min=1275, max=128819, avg=24564.60, stdev=25678.24 lat (usec): min=1276, max=128823, avg=24566.75, stdev=25678.19 clat percentiles (usec): | 1.00th=[ 1401], 5.00th=[ 1647], 10.00th=[ 1827], 20.00th=[ 2835], | 30.00th=[ 6063], 40.00th=[ 10028], 50.00th=[ 15533], 60.00th=[ 22414], | 70.00th=[ 32113], 80.00th=[ 43779], 90.00th=[ 60031], 95.00th=[ 77071], | 99.00th=[122160], 99.50th=[128451], 99.90th=[128451], 99.95th=[128451], | 99.99th=[128451] bw ( KiB/s): min= 63, max= 576, per=27.16%, avg=339.83, stdev=120.22, samples=36 iops : min= 3, max= 36, avg=20.58, stdev= 7.62, samples=36 lat (usec) : 250=9.38%, 500=2.73%, 750=0.26%, 1000=0.13% lat (msec) : 2=8.72%, 4=5.34%, 10=9.38%, 20=12.63%, 50=38.67% lat (msec) : 100=11.85%, 250=0.91% cpu : usr=0.10%, sys=10.03%, ctx=1374, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147429: Sun Nov 16 22:14:55 2025 read: IOPS=17, BW=285KiB/s (292kB/s)(5744KiB/20130msec) clat (usec): min=118, max=119990, avg=27066.15, stdev=22084.33 lat (usec): min=118, max=119992, avg=27067.24, stdev=22084.32 clat percentiles (usec): | 1.00th=[ 129], 5.00th=[ 149], 10.00th=[ 169], 20.00th=[ 241], | 30.00th=[ 15139], 40.00th=[ 21627], 50.00th=[ 26870], 60.00th=[ 30540], | 70.00th=[ 36439], 80.00th=[ 44303], 90.00th=[ 54789], 95.00th=[ 66847], | 99.00th=[ 88605], 99.50th=[113771], 99.90th=[120062], 99.95th=[120062], | 99.99th=[120062] bw ( KiB/s): min= 30, max= 542, per=24.01%, avg=285.44, stdev=113.79, samples=39 iops : min= 1, max= 33, avg=17.15, stdev= 7.14, samples=39 write: IOPS=20, BW=325KiB/s (333kB/s)(6544KiB/20130msec); 0 zone resets clat (usec): min=1376, max=159344, avg=24748.08, stdev=27521.62 lat (usec): min=1379, max=159346, avg=24800.90, stdev=27550.45 clat percentiles (usec): | 1.00th=[ 1450], 5.00th=[ 1598], 10.00th=[ 1745], 20.00th=[ 2704], | 30.00th=[ 5276], 40.00th=[ 8356], 50.00th=[ 14746], 60.00th=[ 21103], | 70.00th=[ 32113], 80.00th=[ 46924], 90.00th=[ 63701], 95.00th=[ 73925], | 99.00th=[120062], 99.50th=[137364], 99.90th=[158335], 99.95th=[158335], | 99.99th=[158335] bw ( KiB/s): min= 60, max= 509, per=25.75%, avg=322.08, stdev=103.72, samples=39 iops : min= 3, max= 31, avg=19.44, stdev= 6.48, samples=39 lat (usec) : 250=9.51%, 500=1.43%, 750=0.26%, 1000=0.26% lat (msec) : 2=8.33%, 4=5.73%, 10=10.03%, 20=12.63%, 50=35.42% lat (msec) : 100=14.84%, 250=1.56% cpu : usr=0.09%, sys=10.14%, ctx=1480, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147430: Sun Nov 16 22:14:55 2025 read: IOPS=19, BW=312KiB/s (320kB/s)(6256KiB/20046msec) clat (usec): min=93, max=170353, avg=25696.23, stdev=22376.79 lat (usec): min=93, max=170354, avg=25697.40, stdev=22376.81 clat percentiles (usec): | 1.00th=[ 104], 5.00th=[ 143], 10.00th=[ 165], 20.00th=[ 253], | 30.00th=[ 13698], 40.00th=[ 19530], 50.00th=[ 23462], 60.00th=[ 28705], | 70.00th=[ 35390], 80.00th=[ 43779], 90.00th=[ 55313], 95.00th=[ 64226], | 99.00th=[ 94897], 99.50th=[105382], 99.90th=[170918], 99.95th=[170918], | 99.99th=[170918] bw ( KiB/s): min= 30, max= 541, per=26.21%, avg=311.62, stdev=106.59, samples=39 iops : min= 1, max= 33, avg=18.77, stdev= 6.73, samples=39 write: IOPS=18, BW=301KiB/s (308kB/s)(6032KiB/20046msec); 0 zone resets clat (usec): min=1318, max=147548, avg=25638.13, stdev=26011.27 lat (usec): min=1321, max=147550, avg=25640.33, stdev=26011.29 clat percentiles (usec): | 1.00th=[ 1434], 5.00th=[ 1647], 10.00th=[ 1844], 20.00th=[ 3523], | 30.00th=[ 6587], 40.00th=[ 11600], 50.00th=[ 18220], 60.00th=[ 24249], | 70.00th=[ 31065], 80.00th=[ 44303], 90.00th=[ 61080], 95.00th=[ 76022], | 99.00th=[117965], 99.50th=[137364], 99.90th=[147850], 99.95th=[147850], | 99.99th=[147850] bw ( KiB/s): min= 91, max= 622, per=23.65%, avg=295.87, stdev=118.88, samples=39 iops : min= 5, max= 38, avg=17.79, stdev= 7.42, samples=39 lat (usec) : 100=0.13%, 250=10.03%, 500=2.86%, 750=0.26%, 1000=0.13% lat (msec) : 2=6.64%, 4=4.82%, 10=7.94%, 20=13.41%, 50=38.67% lat (msec) : 100=13.80%, 250=1.30% cpu : usr=0.16%, sys=9.53%, ctx=1500, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147431: Sun Nov 16 22:14:55 2025 read: IOPS=19, BW=313KiB/s (321kB/s)(6032KiB/19260msec) clat (usec): min=106, max=124922, avg=23525.61, stdev=22823.11 lat (usec): min=106, max=124922, avg=23526.72, stdev=22823.20 clat percentiles (usec): | 1.00th=[ 112], 5.00th=[ 131], 10.00th=[ 153], 20.00th=[ 190], | 30.00th=[ 343], 40.00th=[ 14877], 50.00th=[ 21103], 60.00th=[ 27395], | 70.00th=[ 34866], 80.00th=[ 40633], 90.00th=[ 54789], 95.00th=[ 64226], | 99.00th=[ 93848], 99.50th=[123208], 99.90th=[125305], 99.95th=[125305], | 99.99th=[125305] bw ( KiB/s): min= 63, max= 573, per=26.84%, avg=319.08, stdev=114.54, samples=37 iops : min= 3, max= 35, avg=19.24, stdev= 7.21, samples=37 write: IOPS=20, BW=325KiB/s (333kB/s)(6256KiB/19260msec); 0 zone resets clat (usec): min=1266, max=268767, avg=25911.54, stdev=28327.84 lat (usec): min=1267, max=268770, avg=25915.41, stdev=28326.77 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 1663], 10.00th=[ 1876], 20.00th=[ 3425], | 30.00th=[ 6390], 40.00th=[ 9372], 50.00th=[ 17433], 60.00th=[ 24511], | 70.00th=[ 33424], 80.00th=[ 46924], 90.00th=[ 62129], 95.00th=[ 79168], | 99.00th=[107480], 99.50th=[113771], 99.90th=[270533], 99.95th=[270533], | 99.99th=[270533] bw ( KiB/s): min= 90, max= 795, per=25.95%, avg=324.58, stdev=133.12, samples=38 iops : min= 5, max= 49, avg=19.61, stdev= 8.36, samples=38 lat (usec) : 250=13.15%, 500=2.86%, 750=0.52%, 1000=0.13% lat (msec) : 2=6.12%, 4=6.12%, 10=9.90%, 20=12.63%, 50=32.81% lat (msec) : 100=14.45%, 250=1.17%, 500=0.13% cpu : usr=0.12%, sys=9.20%, ctx=1354, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1190KiB/s (1218kB/s), 285KiB/s-316KiB/s (292kB/s-324kB/s), io=23.4MiB (24.5MB), run=18728-20130msec WRITE: bw=1252KiB/s (1282kB/s), 301KiB/s-340KiB/s (308kB/s-348kB/s), io=24.6MiB (25.8MB), run=18728-20130msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147424: Sun Nov 16 22:15:32 2025 read: IOPS=6, BW=105KiB/s (107kB/s)(5920KiB/56526msec) clat (msec): min=7, max=244, avg=30.14, stdev=26.66 lat (msec): min=7, max=244, avg=30.14, stdev=26.66 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 27], | 70.00th=[ 32], 80.00th=[ 41], 90.00th=[ 59], 95.00th=[ 82], | 99.00th=[ 150], 99.50th=[ 171], 99.90th=[ 245], 99.95th=[ 245], | 99.99th=[ 245] bw ( KiB/s): min= 31, max= 412, per=26.28%, avg=110.39, stdev=66.48, samples=104 iops : min= 1, max= 25, avg= 6.12, stdev= 4.20, samples=104 write: IOPS=7, BW=113KiB/s (115kB/s)(6368KiB/56526msec); 0 zone resets clat (msec): min=43, max=307, avg=113.34, stdev=46.47 lat (msec): min=43, max=307, avg=113.34, stdev=46.47 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 61], 10.00th=[ 65], 20.00th=[ 74], | 30.00th=[ 85], 40.00th=[ 93], 50.00th=[ 101], 60.00th=[ 114], | 70.00th=[ 127], 80.00th=[ 146], 90.00th=[ 178], 95.00th=[ 207], | 99.00th=[ 271], 99.50th=[ 296], 99.90th=[ 309], 99.95th=[ 309], | 99.99th=[ 309] bw ( KiB/s): min= 26, max= 256, per=25.34%, avg=112.01, stdev=45.72, samples=111 iops : min= 1, max= 16, avg= 6.22, stdev= 2.88, samples=111 lat (msec) : 10=2.99%, 20=19.01%, 50=20.57%, 100=30.08%, 250=26.56% lat (msec) : 500=0.78% cpu : usr=0.05%, sys=1.75%, ctx=1436, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147425: Sun Nov 16 22:15:32 2025 read: IOPS=6, BW=101KiB/s (103kB/s)(5744KiB/57010msec) clat (msec): min=6, max=179, avg=29.18, stdev=25.75 lat (msec): min=6, max=179, avg=29.18, stdev=25.75 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 24], | 70.00th=[ 31], 80.00th=[ 38], 90.00th=[ 56], 95.00th=[ 83], | 99.00th=[ 144], 99.50th=[ 169], 99.90th=[ 180], 99.95th=[ 180], | 99.99th=[ 180] bw ( KiB/s): min= 26, max= 478, per=25.65%, avg=107.71, stdev=75.92, samples=104 iops : min= 1, max= 29, avg= 5.94, stdev= 4.76, samples=104 write: IOPS=7, BW=115KiB/s (118kB/s)(6544KiB/57010msec); 0 zone resets clat (msec): min=48, max=342, avg=113.38, stdev=47.68 lat (msec): min=48, max=342, avg=113.38, stdev=47.68 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 57], 10.00th=[ 65], 20.00th=[ 77], | 30.00th=[ 84], 40.00th=[ 93], 50.00th=[ 105], 60.00th=[ 114], | 70.00th=[ 128], 80.00th=[ 144], 90.00th=[ 171], 95.00th=[ 218], | 99.00th=[ 279], 99.50th=[ 300], 99.90th=[ 342], 99.95th=[ 342], | 99.99th=[ 342] bw ( KiB/s): min= 31, max= 223, per=25.73%, avg=113.74, stdev=44.77, samples=112 iops : min= 1, max= 13, avg= 6.33, stdev= 2.83, samples=112 lat (msec) : 10=3.65%, 20=18.10%, 50=19.27%, 100=28.65%, 250=29.17% lat (msec) : 500=1.17% cpu : usr=0.06%, sys=1.80%, ctx=1382, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147426: Sun Nov 16 22:15:32 2025 read: IOPS=7, BW=113KiB/s (116kB/s)(6256KiB/55310msec) clat (msec): min=6, max=281, avg=28.37, stdev=27.27 lat (msec): min=6, max=281, avg=28.38, stdev=27.27 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 16], 40.00th=[ 19], 50.00th=[ 22], 60.00th=[ 26], | 70.00th=[ 31], 80.00th=[ 36], 90.00th=[ 51], 95.00th=[ 70], | 99.00th=[ 161], 99.50th=[ 243], 99.90th=[ 284], 99.95th=[ 284], | 99.99th=[ 284] bw ( KiB/s): min= 27, max= 318, per=28.86%, avg=121.20, stdev=80.10, samples=102 iops : min= 1, max= 19, avg= 6.78, stdev= 5.05, samples=102 write: IOPS=6, BW=109KiB/s (112kB/s)(6032KiB/55310msec); 0 zone resets clat (msec): min=42, max=361, avg=117.10, stdev=54.02 lat (msec): min=42, max=361, avg=117.11, stdev=54.02 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 59], 10.00th=[ 67], 20.00th=[ 75], | 30.00th=[ 85], 40.00th=[ 94], 50.00th=[ 103], 60.00th=[ 112], | 70.00th=[ 127], 80.00th=[ 150], 90.00th=[ 190], 95.00th=[ 234], | 99.00th=[ 292], 99.50th=[ 342], 99.90th=[ 363], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 31, max= 224, per=24.76%, avg=109.43, stdev=42.17, samples=108 iops : min= 1, max= 14, avg= 6.06, stdev= 2.65, samples=108 lat (msec) : 10=3.78%, 20=19.53%, 50=23.70%, 100=26.04%, 250=25.00% lat (msec) : 500=1.95% cpu : usr=0.07%, sys=1.85%, ctx=1524, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147427: Sun Nov 16 22:15:32 2025 read: IOPS=6, BW=108KiB/s (111kB/s)(6032KiB/55657msec) clat (msec): min=7, max=178, avg=30.34, stdev=24.07 lat (msec): min=7, max=178, avg=30.34, stdev=24.07 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 21], 50.00th=[ 24], 60.00th=[ 27], | 70.00th=[ 33], 80.00th=[ 39], 90.00th=[ 59], 95.00th=[ 80], | 99.00th=[ 134], 99.50th=[ 157], 99.90th=[ 178], 99.95th=[ 178], | 99.99th=[ 178] bw ( KiB/s): min= 31, max= 319, per=28.67%, avg=120.42, stdev=69.68, samples=98 iops : min= 1, max= 19, avg= 6.74, stdev= 4.39, samples=98 write: IOPS=7, BW=112KiB/s (115kB/s)(6256KiB/55657msec); 0 zone resets clat (msec): min=39, max=315, avg=112.72, stdev=46.53 lat (msec): min=39, max=315, avg=112.73, stdev=46.53 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 56], 10.00th=[ 63], 20.00th=[ 74], | 30.00th=[ 84], 40.00th=[ 95], 50.00th=[ 103], 60.00th=[ 111], | 70.00th=[ 127], 80.00th=[ 150], 90.00th=[ 182], 95.00th=[ 203], | 99.00th=[ 255], 99.50th=[ 279], 99.90th=[ 317], 99.95th=[ 317], | 99.99th=[ 317] bw ( KiB/s): min= 30, max= 224, per=25.36%, avg=112.09, stdev=44.68, samples=110 iops : min= 1, max= 14, avg= 6.23, stdev= 2.81, samples=110 lat (msec) : 10=2.73%, 20=16.67%, 50=24.22%, 100=28.39%, 250=27.34% lat (msec) : 500=0.65% cpu : usr=0.07%, sys=1.84%, ctx=1489, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=420KiB/s (430kB/s), 101KiB/s-113KiB/s (103kB/s-116kB/s), io=23.4MiB (24.5MB), run=55310-57010msec WRITE: bw=442KiB/s (453kB/s), 109KiB/s-115KiB/s (112kB/s-118kB/s), io=24.6MiB (25.8MB), run=55310-57010msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147445: Sun Nov 16 22:15:44 2025 read: IOPS=2, BW=2972KiB/s (3043kB/s)(7168KiB/2412msec) clat (msec): min=60, max=373, avg=126.62, stdev=114.69 lat (msec): min=60, max=373, avg=126.62, stdev=114.69 clat percentiles (msec): | 1.00th=[ 61], 5.00th=[ 61], 10.00th=[ 61], 20.00th=[ 64], | 30.00th=[ 69], 40.00th=[ 69], 50.00th=[ 78], 60.00th=[ 80], | 70.00th=[ 80], 80.00th=[ 165], 90.00th=[ 376], 95.00th=[ 376], | 99.00th=[ 376], 99.50th=[ 376], 99.90th=[ 376], 99.95th=[ 376], | 99.99th=[ 376] bw ( KiB/s): min= 4055, max= 5475, per=100.00%, avg=4765.00, stdev=1004.09, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=2, BW=2123KiB/s (2174kB/s)(5120KiB/2412msec); 0 zone resets clat (msec): min=167, max=638, avg=284.29, stdev=199.51 lat (msec): min=167, max=638, avg=284.40, stdev=199.53 clat percentiles (msec): | 1.00th=[ 167], 5.00th=[ 167], 10.00th=[ 167], 20.00th=[ 167], | 30.00th=[ 176], 40.00th=[ 176], 50.00th=[ 215], 60.00th=[ 215], | 70.00th=[ 226], 80.00th=[ 226], 90.00th=[ 642], 95.00th=[ 642], | 99.00th=[ 642], 99.50th=[ 642], 99.90th=[ 642], 99.95th=[ 642], | 99.99th=[ 642] bw ( KiB/s): min= 1825, max= 2043, per=31.83%, avg=1970.33, stdev=125.86, samples=3 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=3 lat (msec) : 100=41.67%, 250=41.67%, 500=8.33%, 750=8.33% cpu : usr=0.04%, sys=8.63%, ctx=83, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147446: Sun Nov 16 22:15:44 2025 read: IOPS=0, BW=442KiB/s (453kB/s)(2048KiB/4631msec) clat (msec): min=90, max=199, avg=145.15, stdev=77.22 lat (msec): min=90, max=199, avg=145.16, stdev=77.22 clat percentiles (msec): | 1.00th=[ 91], 5.00th=[ 91], 10.00th=[ 91], 20.00th=[ 91], | 30.00th=[ 91], 40.00th=[ 91], 50.00th=[ 91], 60.00th=[ 201], | 70.00th=[ 201], 80.00th=[ 201], 90.00th=[ 201], 95.00th=[ 201], | 99.00th=[ 201], 99.50th=[ 201], 99.90th=[ 201], 99.95th=[ 201], | 99.99th=[ 201] bw ( KiB/s): min= 1845, max= 2043, per=43.96%, avg=1944.00, stdev=140.01, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2211KiB/s (2264kB/s)(10.0MiB/4631msec); 0 zone resets clat (msec): min=87, max=1816, avg=424.00, stdev=515.51 lat (msec): min=87, max=1816, avg=424.10, stdev=515.51 clat percentiles (msec): | 1.00th=[ 88], 5.00th=[ 88], 10.00th=[ 88], 20.00th=[ 126], | 30.00th=[ 169], 40.00th=[ 199], 50.00th=[ 213], 60.00th=[ 249], | 70.00th=[ 288], 80.00th=[ 468], 90.00th=[ 625], 95.00th=[ 1821], | 99.00th=[ 1821], 99.50th=[ 1821], 99.90th=[ 1821], 99.95th=[ 1821], | 99.99th=[ 1821] bw ( KiB/s): min= 2043, max= 6107, per=58.05%, avg=3594.00, stdev=1686.61, samples=5 iops : min= 1, max= 5, avg= 2.60, stdev= 1.67, samples=5 lat (msec) : 100=16.67%, 250=50.00%, 500=16.67%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=6.57%, ctx=94, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147447: Sun Nov 16 22:15:44 2025 read: IOPS=1, BW=1499KiB/s (1535kB/s)(6144KiB/4100msec) clat (msec): min=72, max=342, avg=197.27, stdev=111.85 lat (msec): min=72, max=342, avg=197.27, stdev=111.85 clat percentiles (msec): | 1.00th=[ 72], 5.00th=[ 72], 10.00th=[ 72], 20.00th=[ 113], | 30.00th=[ 113], 40.00th=[ 131], 50.00th=[ 131], 60.00th=[ 209], | 70.00th=[ 317], 80.00th=[ 317], 90.00th=[ 342], 95.00th=[ 342], | 99.00th=[ 342], 99.50th=[ 342], 99.90th=[ 342], 99.95th=[ 342], | 99.99th=[ 342] bw ( KiB/s): min= 2019, max= 4087, per=57.53%, avg=2544.00, stdev=1028.72, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=1, BW=1499KiB/s (1535kB/s)(6144KiB/4100msec); 0 zone resets clat (msec): min=160, max=1840, avg=481.94, stdev=668.71 lat (msec): min=160, max=1840, avg=482.03, stdev=668.72 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 178], | 30.00th=[ 178], 40.00th=[ 180], 50.00th=[ 180], 60.00th=[ 192], | 70.00th=[ 342], 80.00th=[ 342], 90.00th=[ 1838], 95.00th=[ 1838], | 99.00th=[ 1838], 99.50th=[ 1838], 99.90th=[ 1838], 99.95th=[ 1838], | 99.99th=[ 1838] bw ( KiB/s): min= 2027, max= 4039, per=48.03%, avg=2973.50, stdev=1088.66, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 lat (msec) : 100=8.33%, 250=58.33%, 500=25.00%, 2000=8.33% cpu : usr=0.00%, sys=4.95%, ctx=63, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147448: Sun Nov 16 22:15:44 2025 read: IOPS=1, BW=1110KiB/s (1137kB/s)(5120KiB/4613msec) clat (msec): min=71, max=563, avg=183.41, stdev=212.73 lat (msec): min=71, max=563, avg=183.41, stdev=212.73 clat percentiles (msec): | 1.00th=[ 72], 5.00th=[ 72], 10.00th=[ 72], 20.00th=[ 72], | 30.00th=[ 84], 40.00th=[ 84], 50.00th=[ 87], 60.00th=[ 87], | 70.00th=[ 112], 80.00th=[ 112], 90.00th=[ 567], 95.00th=[ 567], | 99.00th=[ 567], 99.50th=[ 567], 99.90th=[ 567], 99.95th=[ 567], | 99.99th=[ 567] bw ( KiB/s): min= 4087, max= 4087, per=92.42%, avg=4087.00, stdev= 0.00, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1554KiB/s (1591kB/s)(7168KiB/4613msec); 0 zone resets clat (msec): min=150, max=1933, avg=514.39, stdev=646.19 lat (msec): min=150, max=1933, avg=514.48, stdev=646.19 clat percentiles (msec): | 1.00th=[ 150], 5.00th=[ 150], 10.00th=[ 150], 20.00th=[ 153], | 30.00th=[ 167], 40.00th=[ 167], 50.00th=[ 199], 60.00th=[ 443], | 70.00th=[ 443], 80.00th=[ 558], 90.00th=[ 1938], 95.00th=[ 1938], | 99.00th=[ 1938], 99.50th=[ 1938], 99.90th=[ 1938], 99.95th=[ 1938], | 99.99th=[ 1938] bw ( KiB/s): min= 2019, max= 4087, per=56.12%, avg=3474.50, stdev=983.11, samples=4 iops : min= 1, max= 3, avg= 2.50, stdev= 1.00, samples=4 lat (msec) : 100=25.00%, 250=41.67%, 500=8.33%, 750=16.67%, 2000=8.33% cpu : usr=0.09%, sys=5.49%, ctx=173, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4422KiB/s (4529kB/s), 442KiB/s-2972KiB/s (453kB/s-3043kB/s), io=20.0MiB (20.0MB), run=2412-4631msec WRITE: bw=6191KiB/s (6340kB/s), 1499KiB/s-2211KiB/s (1535kB/s-2264kB/s), io=28.0MiB (29.4MB), run=2412-4631msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147450: Sun Nov 16 22:15:44 2025 read: IOPS=1, BW=1851KiB/s (1896kB/s)(7168KiB/3872msec) clat (usec): min=962, max=2807.4k, avg=481608.81, stdev=1030286.19 lat (usec): min=963, max=2807.4k, avg=481610.26, stdev=1030286.18 clat percentiles (usec): | 1.00th=[ 963], 5.00th=[ 963], 10.00th=[ 963], | 20.00th=[ 1012], 30.00th=[ 1778], 40.00th=[ 1778], | 50.00th=[ 149947], 60.00th=[ 158335], 70.00th=[ 158335], | 80.00th=[ 250610], 90.00th=[2801796], 95.00th=[2801796], | 99.00th=[2801796], 99.50th=[2801796], 99.90th=[2801796], | 99.95th=[2801796], 99.99th=[2801796] bw ( KiB/s): min= 3459, max= 6035, per=100.00%, avg=4747.00, stdev=1821.51, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=1, BW=1322KiB/s (1354kB/s)(5120KiB/3872msec); 0 zone resets clat (msec): min=81, max=129, avg=98.93, stdev=18.69 lat (msec): min=81, max=129, avg=99.03, stdev=18.74 clat percentiles (msec): | 1.00th=[ 82], 5.00th=[ 82], 10.00th=[ 82], 20.00th=[ 82], | 30.00th=[ 87], 40.00th=[ 87], 50.00th=[ 96], 60.00th=[ 96], | 70.00th=[ 103], 80.00th=[ 103], 90.00th=[ 130], 95.00th=[ 130], | 99.00th=[ 130], 99.50th=[ 130], 99.90th=[ 130], 99.95th=[ 130], | 99.99th=[ 130] bw ( KiB/s): min= 8047, max= 8047, per=100.00%, avg=8047.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 lat (usec) : 1000=8.33% lat (msec) : 2=16.67%, 100=25.00%, 250=33.33%, 500=8.33%, >=2000=8.33% cpu : usr=0.03%, sys=15.71%, ctx=572, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147451: Sun Nov 16 22:15:44 2025 read: IOPS=0, BW=465KiB/s (476kB/s)(2048KiB/4404msec) clat (usec): min=1080, max=251600, avg=126340.51, stdev=177144.05 lat (usec): min=1081, max=251603, avg=126342.57, stdev=177145.44 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 1074], 10.00th=[ 1074], 20.00th=[ 1074], | 30.00th=[ 1074], 40.00th=[ 1074], 50.00th=[ 1074], 60.00th=[250610], | 70.00th=[250610], 80.00th=[250610], 90.00th=[250610], 95.00th=[250610], | 99.00th=[250610], 99.50th=[250610], 99.90th=[250610], 99.95th=[250610], | 99.99th=[250610] bw ( KiB/s): min= 1698, max= 2048, per=40.28%, avg=1873.00, stdev=247.49, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2325KiB/s (2381kB/s)(10.0MiB/4404msec); 0 zone resets clat (msec): min=63, max=1882, avg=414.07, stdev=534.32 lat (msec): min=63, max=1882, avg=414.33, stdev=534.27 clat percentiles (msec): | 1.00th=[ 64], 5.00th=[ 64], 10.00th=[ 64], 20.00th=[ 178], | 30.00th=[ 190], 40.00th=[ 201], 50.00th=[ 209], 60.00th=[ 239], | 70.00th=[ 262], 80.00th=[ 321], 90.00th=[ 600], 95.00th=[ 1888], | 99.00th=[ 1888], 99.50th=[ 1888], 99.90th=[ 1888], 99.95th=[ 1888], | 99.99th=[ 1888] bw ( KiB/s): min= 2039, max= 6144, per=62.85%, avg=4091.50, stdev=1675.86, samples=4 iops : min= 1, max= 6, avg= 3.50, stdev= 2.08, samples=4 lat (msec) : 2=8.33%, 100=8.33%, 250=41.67%, 500=25.00%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=18.19%, ctx=116, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147452: Sun Nov 16 22:15:44 2025 read: IOPS=1, BW=1526KiB/s (1563kB/s)(6144KiB/4025msec) clat (usec): min=975, max=243319, avg=107578.90, stdev=93768.65 lat (usec): min=978, max=243321, avg=107580.93, stdev=93768.41 clat percentiles (usec): | 1.00th=[ 979], 5.00th=[ 979], 10.00th=[ 979], 20.00th=[ 1029], | 30.00th=[ 1029], 40.00th=[117965], 50.00th=[117965], 60.00th=[125305], | 70.00th=[156238], 80.00th=[156238], 90.00th=[244319], 95.00th=[244319], | 99.00th=[244319], 99.50th=[244319], 99.90th=[244319], 99.95th=[244319], | 99.99th=[244319] bw ( KiB/s): min= 4096, max= 4096, per=88.09%, avg=4096.00, stdev= 0.00, samples=2 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1526KiB/s (1563kB/s)(6144KiB/4025msec); 0 zone resets clat (msec): min=159, max=1989, avg=562.15, stdev=708.35 lat (msec): min=159, max=1989, avg=562.29, stdev=708.29 clat percentiles (msec): | 1.00th=[ 159], 5.00th=[ 159], 10.00th=[ 159], 20.00th=[ 194], | 30.00th=[ 194], 40.00th=[ 232], 50.00th=[ 232], 60.00th=[ 321], | 70.00th=[ 481], 80.00th=[ 481], 90.00th=[ 1989], 95.00th=[ 1989], | 99.00th=[ 1989], 99.50th=[ 1989], 99.90th=[ 1989], 99.95th=[ 1989], | 99.99th=[ 1989] bw ( KiB/s): min= 2039, max= 3465, per=38.67%, avg=2517.33, stdev=820.72, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 lat (usec) : 1000=8.33% lat (msec) : 2=8.33%, 250=58.33%, 500=16.67%, 2000=8.33% cpu : usr=0.00%, sys=14.41%, ctx=232, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147453: Sun Nov 16 22:15:44 2025 read: IOPS=1, BW=1190KiB/s (1218kB/s)(5120KiB/4303msec) clat (usec): min=991, max=172817, avg=79319.92, stdev=76525.79 lat (usec): min=993, max=172818, avg=79321.12, stdev=76525.19 clat percentiles (usec): | 1.00th=[ 996], 5.00th=[ 996], 10.00th=[ 996], 20.00th=[ 996], | 30.00th=[ 1074], 40.00th=[ 1074], 50.00th=[ 95945], 60.00th=[ 95945], | 70.00th=[125305], 80.00th=[125305], 90.00th=[173016], 95.00th=[173016], | 99.00th=[173016], 99.50th=[173016], 99.90th=[173016], 99.95th=[173016], | 99.99th=[173016] bw ( KiB/s): min= 4096, max= 4096, per=88.09%, avg=4096.00, stdev= 0.00, samples=2 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1666KiB/s (1706kB/s)(7168KiB/4303msec); 0 zone resets clat (msec): min=103, max=2089, avg=556.40, stdev=685.13 lat (msec): min=103, max=2089, avg=556.51, stdev=685.14 clat percentiles (msec): | 1.00th=[ 104], 5.00th=[ 104], 10.00th=[ 104], 20.00th=[ 192], | 30.00th=[ 359], 40.00th=[ 359], 50.00th=[ 368], 60.00th=[ 393], | 70.00th=[ 393], 80.00th=[ 393], 90.00th=[ 2089], 95.00th=[ 2089], | 99.00th=[ 2089], 99.50th=[ 2089], 99.90th=[ 2089], 99.95th=[ 2089], | 99.99th=[ 2089] bw ( KiB/s): min= 1774, max= 4096, per=38.02%, avg=2475.25, stdev=1086.65, samples=4 iops : min= 1, max= 4, avg= 1.75, stdev= 1.50, samples=4 lat (usec) : 1000=8.33% lat (msec) : 2=8.33%, 100=8.33%, 250=33.33%, 500=33.33%, >=2000=8.33% cpu : usr=0.00%, sys=15.46%, ctx=169, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4650KiB/s (4762kB/s), 465KiB/s-1851KiB/s (476kB/s-1896kB/s), io=20.0MiB (20.0MB), run=3872-4404msec WRITE: bw=6510KiB/s (6667kB/s), 1322KiB/s-2325KiB/s (1354kB/s-2381kB/s), io=28.0MiB (29.4MB), run=3872-4404msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147464: Sun Nov 16 22:15:55 2025 read: IOPS=1, BW=5860KiB/s (6000kB/s)(12.0MiB/2097msec) clat (msec): min=562, max=898, avg=698.65, stdev=176.65 lat (msec): min=562, max=898, avg=698.65, stdev=176.65 clat percentiles (msec): | 1.00th=[ 567], 5.00th=[ 567], 10.00th=[ 567], 20.00th=[ 567], | 30.00th=[ 567], 40.00th=[ 634], 50.00th=[ 634], 60.00th=[ 634], | 70.00th=[ 902], 80.00th=[ 902], 90.00th=[ 902], 95.00th=[ 902], | 99.00th=[ 902], 99.50th=[ 902], 99.90th=[ 902], 99.95th=[ 902], | 99.99th=[ 902] bw ( KiB/s): min= 6059, max= 8175, per=91.09%, avg=7117.00, stdev=1496.24, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=66.67%, 1000=33.33% cpu : usr=0.10%, sys=21.09%, ctx=401, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147465: Sun Nov 16 22:15:55 2025 read: IOPS=0, BW=2462KiB/s (2521kB/s)(4096KiB/1664msec) clat (nsec): min=574527k, max=574527k, avg=574526656.00, stdev= 0.00 lat (nsec): min=574528k, max=574528k, avg=574528309.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 575], 5.00th=[ 575], 10.00th=[ 575], 20.00th=[ 575], | 30.00th=[ 575], 40.00th=[ 575], 50.00th=[ 575], 60.00th=[ 575], | 70.00th=[ 575], 80.00th=[ 575], 90.00th=[ 575], 95.00th=[ 575], | 99.00th=[ 575], 99.50th=[ 575], 99.90th=[ 575], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 6660, max= 6660, per=85.24%, avg=6660.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4923KiB/s (5041kB/s)(8192KiB/1664msec); 0 zone resets clat (msec): min=459, max=625, avg=542.93, stdev=117.43 lat (msec): min=460, max=626, avg=543.46, stdev=117.48 clat percentiles (msec): | 1.00th=[ 460], 5.00th=[ 460], 10.00th=[ 460], 20.00th=[ 460], | 30.00th=[ 460], 40.00th=[ 460], 50.00th=[ 460], 60.00th=[ 625], | 70.00th=[ 625], 80.00th=[ 625], 90.00th=[ 625], 95.00th=[ 625], | 99.00th=[ 625], 99.50th=[ 625], 99.90th=[ 625], 99.95th=[ 625], | 99.99th=[ 625] bw ( KiB/s): min= 8175, max= 8175, per=64.32%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.06%, sys=16.30%, ctx=215, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147466: Sun Nov 16 22:15:55 2025 write: IOPS=1, BW=4766KiB/s (4881kB/s)(12.0MiB/2578msec); 0 zone resets clat (msec): min=282, max=1386, avg=858.36, stdev=553.20 lat (msec): min=283, max=1386, avg=859.00, stdev=552.83 clat percentiles (msec): | 1.00th=[ 284], 5.00th=[ 284], 10.00th=[ 284], 20.00th=[ 284], | 30.00th=[ 284], 40.00th=[ 911], 50.00th=[ 911], 60.00th=[ 911], | 70.00th=[ 1385], 80.00th=[ 1385], 90.00th=[ 1385], 95.00th=[ 1385], | 99.00th=[ 1385], 99.50th=[ 1385], 99.90th=[ 1385], 99.95th=[ 1385], | 99.99th=[ 1385] bw ( KiB/s): min= 8175, max= 8175, per=64.32%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.04%, sys=4.46%, ctx=36, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147467: Sun Nov 16 22:15:55 2025 write: IOPS=1, BW=6986KiB/s (7153kB/s)(12.0MiB/1759msec); 0 zone resets clat (msec): min=284, max=981, avg=585.14, stdev=358.29 lat (msec): min=284, max=982, avg=585.49, stdev=358.36 clat percentiles (msec): | 1.00th=[ 284], 5.00th=[ 284], 10.00th=[ 284], 20.00th=[ 284], | 30.00th=[ 284], 40.00th=[ 489], 50.00th=[ 489], 60.00th=[ 489], | 70.00th=[ 986], 80.00th=[ 986], 90.00th=[ 986], 95.00th=[ 986], | 99.00th=[ 986], 99.50th=[ 986], 99.90th=[ 986], 99.95th=[ 986], | 99.99th=[ 986] bw ( KiB/s): min= 6989, max= 8175, per=59.65%, avg=7582.00, stdev=838.63, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, 1000=33.33% cpu : usr=0.28%, sys=7.96%, ctx=42, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7813KiB/s (8001kB/s), 2462KiB/s-5860KiB/s (2521kB/s-6000kB/s), io=16.0MiB (16.8MB), run=1664-2097msec WRITE: bw=12.4MiB/s (13.0MB/s), 4766KiB/s-6986KiB/s (4881kB/s-7153kB/s), io=32.0MiB (33.6MB), run=1664-2578msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147459: Sun Nov 16 22:15:55 2025 read: IOPS=2, BW=11.1MiB/s (11.6MB/s)(12.0MiB/1085msec) clat (msec): min=137, max=593, avg=316.03, stdev=243.98 lat (msec): min=137, max=593, avg=316.03, stdev=243.98 clat percentiles (msec): | 1.00th=[ 138], 5.00th=[ 138], 10.00th=[ 138], 20.00th=[ 138], | 30.00th=[ 138], 40.00th=[ 218], 50.00th=[ 218], 60.00th=[ 218], | 70.00th=[ 592], 80.00th=[ 592], 90.00th=[ 592], 95.00th=[ 592], | 99.00th=[ 592], 99.50th=[ 592], 99.90th=[ 592], 99.95th=[ 592], | 99.99th=[ 592] bw ( KiB/s): min=14346, max=14346, per=100.00%, avg=14346.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, 750=33.33% cpu : usr=0.00%, sys=20.39%, ctx=66, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147461: Sun Nov 16 22:15:55 2025 read: IOPS=0, BW=1419KiB/s (1453kB/s)(4096KiB/2887msec) clat (nsec): min=930452k, max=930452k, avg=930452237.00, stdev= 0.00 lat (nsec): min=930457k, max=930457k, avg=930456711.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 927], 5.00th=[ 927], 10.00th=[ 927], 20.00th=[ 927], | 30.00th=[ 927], 40.00th=[ 927], 50.00th=[ 927], 60.00th=[ 927], | 70.00th=[ 927], 80.00th=[ 927], 90.00th=[ 927], 95.00th=[ 927], | 99.00th=[ 927], 99.50th=[ 927], 99.90th=[ 927], 99.95th=[ 927], | 99.99th=[ 927] bw ( KiB/s): min= 7816, max= 7816, per=100.00%, avg=7816.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2838KiB/s (2906kB/s)(8192KiB/2887msec); 0 zone resets clat (msec): min=717, max=1136, avg=927.07, stdev=296.79 lat (msec): min=717, max=1137, avg=927.55, stdev=296.74 clat percentiles (msec): | 1.00th=[ 718], 5.00th=[ 718], 10.00th=[ 718], 20.00th=[ 718], | 30.00th=[ 718], 40.00th=[ 718], 50.00th=[ 718], 60.00th=[ 1133], | 70.00th=[ 1133], 80.00th=[ 1133], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 8078, max= 8078, per=71.17%, avg=8078.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=4.54%, ctx=55, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147462: Sun Nov 16 22:15:55 2025 write: IOPS=1, BW=4538KiB/s (4647kB/s)(12.0MiB/2708msec); 0 zone resets clat (msec): min=221, max=1308, avg=842.26, stdev=560.03 lat (msec): min=222, max=1309, avg=843.10, stdev=559.45 clat percentiles (msec): | 1.00th=[ 222], 5.00th=[ 222], 10.00th=[ 222], 20.00th=[ 222], | 30.00th=[ 222], 40.00th=[ 995], 50.00th=[ 995], 60.00th=[ 995], | 70.00th=[ 1318], 80.00th=[ 1318], 90.00th=[ 1318], 95.00th=[ 1318], | 99.00th=[ 1318], 99.50th=[ 1318], 99.90th=[ 1318], 99.95th=[ 1318], | 99.99th=[ 1318] bw ( KiB/s): min= 8015, max= 8175, per=71.32%, avg=8095.00, stdev=113.14, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=8.53%, ctx=35, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147463: Sun Nov 16 22:15:55 2025 write: IOPS=1, BW=6474KiB/s (6630kB/s)(12.0MiB/1898msec); 0 zone resets clat (msec): min=267, max=1016, avg=596.24, stdev=382.97 lat (msec): min=268, max=1017, avg=596.69, stdev=382.96 clat percentiles (msec): | 1.00th=[ 268], 5.00th=[ 268], 10.00th=[ 268], 20.00th=[ 268], | 30.00th=[ 268], 40.00th=[ 506], 50.00th=[ 506], 60.00th=[ 506], | 70.00th=[ 1020], 80.00th=[ 1020], 90.00th=[ 1020], 95.00th=[ 1020], | 99.00th=[ 1020], 99.50th=[ 1020], 99.90th=[ 1020], 99.95th=[ 1020], | 99.99th=[ 1020] bw ( KiB/s): min= 7557, max= 7816, per=67.72%, avg=7686.50, stdev=183.14, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.11%, sys=9.59%, ctx=33, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5675KiB/s (5811kB/s), 1419KiB/s-11.1MiB/s (1453kB/s-11.6MB/s), io=16.0MiB (16.8MB), run=1085-2887msec WRITE: bw=11.1MiB/s (11.6MB/s), 2838KiB/s-6474KiB/s (2906kB/s-6630kB/s), io=32.0MiB (33.6MB), run=1898-2887msec