== sanity test 398b: DIO and buffer IO race ============== 13:45:59 (1763491559) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.27364 s, 15.4 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=182540: Tue Nov 18 13:47:08 2025 read: IOPS=25, BW=102KiB/s (105kB/s)(5992KiB/58702msec) clat (usec): min=115, max=816137, avg=21446.98, stdev=23501.52 lat (usec): min=115, max=816138, avg=21448.03, stdev=23501.56 clat percentiles (usec): | 1.00th=[ 314], 5.00th=[ 7767], 10.00th=[ 9634], 20.00th=[ 11731], | 30.00th=[ 13960], 40.00th=[ 16581], 50.00th=[ 18744], 60.00th=[ 21627], | 70.00th=[ 24773], 80.00th=[ 28705], 90.00th=[ 35390], 95.00th=[ 40633], | 99.00th=[ 63701], 99.50th=[ 68682], 99.90th=[ 94897], 99.95th=[817890], | 99.99th=[817890] bw ( KiB/s): min= 7, max= 174, per=25.19%, avg=101.75, stdev=27.08, samples=116 iops : min= 1, max= 43, avg=24.97, stdev= 6.78, samples=116 write: IOPS=26, BW=107KiB/s (110kB/s)(6296KiB/58702msec); 0 zone resets clat (usec): min=698, max=105248, avg=16708.50, stdev=14154.96 lat (usec): min=699, max=105249, avg=16712.65, stdev=14159.36 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 1663], 10.00th=[ 2343], 20.00th=[ 3490], | 30.00th=[ 5866], 40.00th=[ 10028], 50.00th=[ 13304], 60.00th=[ 17433], | 70.00th=[ 21627], 80.00th=[ 27919], 90.00th=[ 35914], 95.00th=[ 43779], | 99.00th=[ 58983], 99.50th=[ 68682], 99.90th=[ 82314], 99.95th=[105382], | 99.99th=[105382] bw ( KiB/s): min= 47, max= 167, per=25.70%, avg=108.69, stdev=27.51, samples=114 iops : min= 11, max= 41, avg=26.71, stdev= 6.94, samples=114 lat (usec) : 250=0.33%, 500=0.81%, 750=0.13%, 1000=0.26% lat (msec) : 2=3.52%, 4=8.04%, 10=12.86%, 20=34.41%, 50=37.14% lat (msec) : 100=2.44%, 250=0.03%, 1000=0.03% cpu : usr=0.10%, sys=19.58%, ctx=8061, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182542: Tue Nov 18 13:47:08 2025 read: IOPS=25, BW=100KiB/s (103kB/s)(5940KiB/59329msec) clat (usec): min=139, max=93422, avg=21552.39, stdev=11974.99 lat (usec): min=140, max=93423, avg=21554.61, stdev=11975.10 clat percentiles (usec): | 1.00th=[ 322], 5.00th=[ 8094], 10.00th=[ 9896], 20.00th=[12256], | 30.00th=[14222], 40.00th=[16319], 50.00th=[19006], 60.00th=[22152], | 70.00th=[25560], 80.00th=[29754], 90.00th=[36439], 95.00th=[44303], | 99.00th=[61080], 99.50th=[72877], 99.90th=[88605], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 7, max= 159, per=24.92%, avg=100.68, stdev=25.94, samples=117 iops : min= 1, max= 39, avg=24.70, stdev= 6.53, samples=117 write: IOPS=26, BW=107KiB/s (110kB/s)(6348KiB/59329msec); 0 zone resets clat (usec): min=772, max=772148, avg=17031.03, stdev=23733.36 lat (usec): min=773, max=772149, avg=17038.46, stdev=23737.02 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1696], 10.00th=[ 2409], 20.00th=[ 3523], | 30.00th=[ 5669], 40.00th=[ 10290], 50.00th=[ 13304], 60.00th=[ 16909], | 70.00th=[ 20841], 80.00th=[ 27132], 90.00th=[ 36439], 95.00th=[ 44303], | 99.00th=[ 66847], 99.50th=[ 72877], 99.90th=[ 93848], 99.95th=[775947], | 99.99th=[775947] bw ( KiB/s): min= 7, max= 270, per=25.20%, avg=106.58, stdev=35.32, samples=118 iops : min= 1, max= 67, avg=26.18, stdev= 8.85, samples=118 lat (usec) : 250=0.16%, 500=0.72%, 750=0.33%, 1000=0.46% lat (msec) : 2=3.12%, 4=8.40%, 10=12.30%, 20=35.81%, 50=35.94% lat (msec) : 100=2.73%, 1000=0.03% cpu : usr=0.15%, sys=19.24%, ctx=8213, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182544: Tue Nov 18 13:47:08 2025 read: IOPS=25, BW=104KiB/s (106kB/s)(6028KiB/58004msec) clat (usec): min=124, max=787379, avg=21221.90, stdev=22988.07 lat (usec): min=126, max=787379, avg=21222.96, stdev=22988.06 clat percentiles (usec): | 1.00th=[ 322], 5.00th=[ 7832], 10.00th=[ 9503], 20.00th=[ 11600], | 30.00th=[ 14091], 40.00th=[ 16057], 50.00th=[ 18482], 60.00th=[ 21103], | 70.00th=[ 24249], 80.00th=[ 28443], 90.00th=[ 35390], 95.00th=[ 42206], | 99.00th=[ 67634], 99.50th=[ 72877], 99.90th=[ 89654], 99.95th=[784335], | 99.99th=[784335] bw ( KiB/s): min= 15, max= 175, per=25.83%, avg=104.33, stdev=27.17, samples=114 iops : min= 3, max= 43, avg=25.62, stdev= 6.84, samples=114 write: IOPS=26, BW=108KiB/s (111kB/s)(6260KiB/58004msec); 0 zone resets clat (usec): min=796, max=117170, avg=16444.60, stdev=14256.90 lat (usec): min=798, max=117172, avg=16446.57, stdev=14256.99 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1663], 10.00th=[ 2278], 20.00th=[ 3261], | 30.00th=[ 5407], 40.00th=[ 10028], 50.00th=[ 13173], 60.00th=[ 16909], | 70.00th=[ 21890], 80.00th=[ 27132], 90.00th=[ 35390], 95.00th=[ 43254], | 99.00th=[ 63177], 99.50th=[ 70779], 99.90th=[101188], 99.95th=[116917], | 99.99th=[116917] bw ( KiB/s): min= 21, max= 224, per=25.59%, avg=108.25, stdev=30.82, samples=114 iops : min= 5, max= 56, avg=26.61, stdev= 7.70, samples=114 lat (usec) : 250=0.33%, 500=0.88%, 750=0.33%, 1000=0.39% lat (msec) : 2=3.74%, 4=8.33%, 10=12.14%, 20=35.16%, 50=36.30% lat (msec) : 100=2.31%, 250=0.07%, 1000=0.03% cpu : usr=0.14%, sys=19.48%, ctx=8074, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182546: Tue Nov 18 13:47:08 2025 read: IOPS=25, BW=103KiB/s (105kB/s)(6052KiB/58813msec) clat (usec): min=116, max=81586, avg=20365.48, stdev=11673.37 lat (usec): min=116, max=81587, avg=20367.42, stdev=11673.63 clat percentiles (usec): | 1.00th=[ 249], 5.00th=[ 6980], 10.00th=[ 8979], 20.00th=[11338], | 30.00th=[13435], 40.00th=[15533], 50.00th=[18220], 60.00th=[20841], | 70.00th=[24249], 80.00th=[28705], 90.00th=[35390], 95.00th=[41681], | 99.00th=[61604], 99.50th=[70779], 99.90th=[74974], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 15, max= 159, per=25.58%, avg=103.33, stdev=25.32, samples=116 iops : min= 3, max= 39, avg=25.37, stdev= 6.40, samples=116 write: IOPS=26, BW=106KiB/s (109kB/s)(6236KiB/58813msec); 0 zone resets clat (usec): min=631, max=775152, avg=17783.45, stdev=24105.90 lat (usec): min=632, max=775153, avg=17784.90, stdev=24106.07 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1811], 10.00th=[ 2409], 20.00th=[ 3621], | 30.00th=[ 6063], 40.00th=[ 10814], 50.00th=[ 14091], 60.00th=[ 17695], | 70.00th=[ 22414], 80.00th=[ 28705], 90.00th=[ 38011], 95.00th=[ 44303], | 99.00th=[ 60556], 99.50th=[ 79168], 99.90th=[103285], 99.95th=[775947], | 99.99th=[775947] bw ( KiB/s): min= 7, max= 224, per=24.95%, avg=105.53, stdev=31.35, samples=117 iops : min= 1, max= 56, avg=25.91, stdev= 7.84, samples=117 lat (usec) : 250=0.52%, 500=0.91%, 750=0.39%, 1000=0.49% lat (msec) : 2=2.83%, 4=7.94%, 10=13.18%, 20=35.22%, 50=35.94% lat (msec) : 100=2.51%, 250=0.03%, 1000=0.03% cpu : usr=0.10%, sys=19.43%, ctx=8130, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=405KiB/s (414kB/s), 100KiB/s-104KiB/s (103kB/s-106kB/s), io=23.4MiB (24.6MB), run=58004-59329msec WRITE: bw=424KiB/s (434kB/s), 106KiB/s-108KiB/s (109kB/s-111kB/s), io=24.6MiB (25.7MB), run=58004-59329msec rand-rw: (groupid=0, jobs=1): err= 0: pid=182539: Tue Nov 18 13:47:26 2025 read: IOPS=19, BW=76.9KiB/s (78.7kB/s)(5992KiB/77922msec) clat (msec): min=3, max=822, avg=21.24, stdev=26.37 lat (msec): min=3, max=822, avg=21.25, stdev=26.37 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 23], 80.00th=[ 29], 90.00th=[ 44], 95.00th=[ 56], | 99.00th=[ 83], 99.50th=[ 93], 99.90th=[ 131], 99.95th=[ 827], | 99.99th=[ 827] bw ( KiB/s): min= 7, max= 208, per=25.17%, avg=77.53, stdev=39.07, samples=153 iops : min= 1, max= 52, avg=18.95, stdev= 9.78, samples=153 write: IOPS=20, BW=80.8KiB/s (82.7kB/s)(6296KiB/77922msec); 0 zone resets clat (msec): min=7, max=157, avg=29.11, stdev=17.07 lat (msec): min=7, max=157, avg=29.11, stdev=17.07 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 17], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 24], 60.00th=[ 29], | 70.00th=[ 34], 80.00th=[ 42], 90.00th=[ 52], 95.00th=[ 63], | 99.00th=[ 88], 99.50th=[ 97], 99.90th=[ 136], 99.95th=[ 159], | 99.99th=[ 159] bw ( KiB/s): min= 23, max= 199, per=25.28%, avg=81.40, stdev=36.74, samples=153 iops : min= 5, max= 49, avg=19.92, stdev= 9.18, samples=153 lat (msec) : 4=0.07%, 10=13.41%, 20=38.09%, 50=39.13%, 100=8.95% lat (msec) : 250=0.33%, 1000=0.03% cpu : usr=0.11%, sys=7.38%, ctx=5310, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182541: Tue Nov 18 13:47:26 2025 read: IOPS=19, BW=76.9KiB/s (78.7kB/s)(5940KiB/77272msec) clat (msec): min=4, max=783, avg=20.72, stdev=25.05 lat (msec): min=4, max=783, avg=20.72, stdev=25.05 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 28], 90.00th=[ 43], 95.00th=[ 53], | 99.00th=[ 82], 99.50th=[ 91], 99.90th=[ 107], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 7, max= 223, per=24.96%, avg=76.87, stdev=40.89, samples=152 iops : min= 1, max= 55, avg=18.78, stdev=10.24, samples=152 write: IOPS=20, BW=82.2KiB/s (84.1kB/s)(6348KiB/77272msec); 0 zone resets clat (msec): min=7, max=141, avg=29.13, stdev=16.72 lat (msec): min=7, max=141, avg=29.13, stdev=16.72 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 16], | 30.00th=[ 19], 40.00th=[ 22], 50.00th=[ 25], 60.00th=[ 29], | 70.00th=[ 35], 80.00th=[ 42], 90.00th=[ 52], 95.00th=[ 60], | 99.00th=[ 88], 99.50th=[ 102], 99.90th=[ 140], 99.95th=[ 142], | 99.99th=[ 142] bw ( KiB/s): min= 31, max= 183, per=25.63%, avg=82.54, stdev=35.06, samples=151 iops : min= 7, max= 45, avg=20.19, stdev= 8.77, samples=151 lat (msec) : 10=12.50%, 20=38.61%, 50=40.62%, 100=7.94%, 250=0.29% lat (msec) : 1000=0.03% cpu : usr=0.12%, sys=7.77%, ctx=5335, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182543: Tue Nov 18 13:47:26 2025 read: IOPS=19, BW=77.3KiB/s (79.2kB/s)(6028KiB/77950msec) clat (msec): min=4, max=131, avg=21.26, stdev=16.66 lat (msec): min=4, max=131, avg=21.27, stdev=16.66 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 31], 90.00th=[ 46], 95.00th=[ 57], | 99.00th=[ 80], 99.50th=[ 92], 99.90th=[ 115], 99.95th=[ 131], | 99.99th=[ 131] bw ( KiB/s): min= 15, max= 239, per=25.31%, avg=77.95, stdev=38.89, samples=153 iops : min= 3, max= 59, avg=19.05, stdev= 9.74, samples=153 write: IOPS=20, BW=80.3KiB/s (82.2kB/s)(6260KiB/77950msec); 0 zone resets clat (msec): min=6, max=781, avg=29.15, stdev=25.48 lat (msec): min=6, max=781, avg=29.15, stdev=25.48 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 16], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 24], 60.00th=[ 28], | 70.00th=[ 33], 80.00th=[ 41], 90.00th=[ 51], 95.00th=[ 59], | 99.00th=[ 94], 99.50th=[ 103], 99.90th=[ 161], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 7, max= 191, per=24.97%, avg=80.40, stdev=37.45, samples=154 iops : min= 1, max= 47, avg=19.66, stdev= 9.37, samples=154 lat (msec) : 10=13.09%, 20=38.02%, 50=39.78%, 100=8.72%, 250=0.36% lat (msec) : 1000=0.03% cpu : usr=0.09%, sys=7.38%, ctx=5390, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182545: Tue Nov 18 13:47:26 2025 read: IOPS=19, BW=77.9KiB/s (79.7kB/s)(6052KiB/77722msec) clat (msec): min=4, max=115, avg=20.55, stdev=16.38 lat (msec): min=4, max=115, avg=20.55, stdev=16.38 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 15], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 29], 90.00th=[ 45], 95.00th=[ 57], | 99.00th=[ 79], 99.50th=[ 89], 99.90th=[ 114], 99.95th=[ 116], | 99.99th=[ 116] bw ( KiB/s): min= 7, max= 199, per=25.35%, avg=78.09, stdev=38.90, samples=152 iops : min= 1, max= 49, avg=19.09, stdev= 9.75, samples=152 write: IOPS=20, BW=80.2KiB/s (82.2kB/s)(6236KiB/77722msec); 0 zone resets clat (msec): min=7, max=792, avg=29.73, stdev=26.04 lat (msec): min=7, max=792, avg=29.73, stdev=26.04 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 16], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 24], 60.00th=[ 28], | 70.00th=[ 34], 80.00th=[ 42], 90.00th=[ 54], 95.00th=[ 62], | 99.00th=[ 89], 99.50th=[ 110], 99.90th=[ 146], 99.95th=[ 793], | 99.99th=[ 793] bw ( KiB/s): min= 7, max= 199, per=24.89%, avg=80.14, stdev=36.71, samples=153 iops : min= 1, max= 49, avg=19.59, stdev= 9.18, samples=153 lat (msec) : 10=13.18%, 20=38.80%, 50=38.25%, 100=9.31%, 250=0.42% lat (msec) : 1000=0.03% cpu : usr=0.09%, sys=7.47%, ctx=5519, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=308KiB/s (315kB/s), 76.9KiB/s-77.9KiB/s (78.7kB/s-79.7kB/s), io=23.4MiB (24.6MB), run=77272-77950msec WRITE: bw=323KiB/s (330kB/s), 80.2KiB/s-82.2KiB/s (82.2kB/s-84.1kB/s), io=24.6MiB (25.7MB), run=77272-77950msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=182574: Tue Nov 18 13:47:49 2025 read: IOPS=22, BW=357KiB/s (366kB/s)(5920KiB/16581msec) clat (usec): min=153, max=79067, avg=22634.35, stdev=13085.48 lat (usec): min=154, max=79068, avg=22637.32, stdev=13085.85 clat percentiles (usec): | 1.00th=[ 258], 5.00th=[ 433], 10.00th=[ 9110], 20.00th=[12780], | 30.00th=[15139], 40.00th=[17433], 50.00th=[21103], 60.00th=[23987], | 70.00th=[27395], 80.00th=[31851], 90.00th=[39060], 95.00th=[47449], | 99.00th=[65799], 99.50th=[69731], 99.90th=[79168], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 189, max= 512, per=24.17%, avg=348.34, stdev=76.32, samples=32 iops : min= 11, max= 32, avg=21.28, stdev= 4.83, samples=32 write: IOPS=24, BW=384KiB/s (393kB/s)(6368KiB/16581msec); 0 zone resets clat (usec): min=1348, max=97678, avg=20395.28, stdev=15787.50 lat (usec): min=1350, max=97681, avg=20398.15, stdev=15786.76 clat percentiles (usec): | 1.00th=[ 1647], 5.00th=[ 3261], 10.00th=[ 4146], 20.00th=[ 5997], | 30.00th=[ 9765], 40.00th=[13566], 50.00th=[16712], 60.00th=[20841], | 70.00th=[24773], 80.00th=[30278], 90.00th=[42730], 95.00th=[52691], | 99.00th=[70779], 99.50th=[85459], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 159, max= 544, per=25.18%, avg=381.69, stdev=95.07, samples=32 iops : min= 9, max= 34, avg=23.38, stdev= 5.93, samples=32 lat (usec) : 250=0.39%, 500=2.21%, 750=0.52%, 1000=0.26% lat (msec) : 2=1.30%, 4=3.78%, 10=13.15%, 20=30.99%, 50=42.32% lat (msec) : 100=5.08% cpu : usr=0.10%, sys=21.45%, ctx=2329, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182575: Tue Nov 18 13:47:49 2025 read: IOPS=21, BW=346KiB/s (354kB/s)(5744KiB/16620msec) clat (usec): min=136, max=87988, avg=24082.08, stdev=13198.84 lat (usec): min=137, max=87988, avg=24083.00, stdev=13198.84 clat percentiles (usec): | 1.00th=[ 219], 5.00th=[ 6783], 10.00th=[10945], 20.00th=[13960], | 30.00th=[16712], 40.00th=[19792], 50.00th=[22676], 60.00th=[25560], | 70.00th=[28181], 80.00th=[32900], 90.00th=[39584], 95.00th=[44827], | 99.00th=[72877], 99.50th=[74974], 99.90th=[87557], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 192, max= 541, per=23.58%, avg=339.75, stdev=80.35, samples=32 iops : min= 12, max= 33, avg=20.75, stdev= 5.00, samples=32 write: IOPS=24, BW=394KiB/s (403kB/s)(6544KiB/16620msec); 0 zone resets clat (usec): min=1291, max=101431, avg=19233.94, stdev=17122.38 lat (usec): min=1292, max=101432, avg=19235.77, stdev=17122.37 clat percentiles (usec): | 1.00th=[ 1647], 5.00th=[ 2769], 10.00th=[ 3687], 20.00th=[ 4883], | 30.00th=[ 7308], 40.00th=[ 11076], 50.00th=[ 13960], 60.00th=[ 18220], | 70.00th=[ 23462], 80.00th=[ 30278], 90.00th=[ 44303], 95.00th=[ 54264], | 99.00th=[ 80217], 99.50th=[ 90702], 99.90th=[101188], 99.95th=[101188], | 99.99th=[101188] bw ( KiB/s): min= 191, max= 544, per=25.04%, avg=379.56, stdev=106.51, samples=32 iops : min= 11, max= 34, avg=23.25, stdev= 6.76, samples=32 lat (usec) : 250=0.65%, 500=1.30%, 1000=0.13% lat (msec) : 2=1.30%, 4=5.21%, 10=15.63%, 20=28.39%, 50=41.67% lat (msec) : 100=5.60%, 250=0.13% cpu : usr=0.05%, sys=20.85%, ctx=2253, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182576: Tue Nov 18 13:47:49 2025 read: IOPS=23, BW=382KiB/s (391kB/s)(6256KiB/16394msec) clat (usec): min=127, max=107621, avg=22738.88, stdev=14734.76 lat (usec): min=128, max=107623, avg=22739.90, stdev=14734.82 clat percentiles (usec): | 1.00th=[ 239], 5.00th=[ 437], 10.00th=[ 8979], 20.00th=[ 12518], | 30.00th=[ 15008], 40.00th=[ 17957], 50.00th=[ 20317], 60.00th=[ 23462], | 70.00th=[ 26870], 80.00th=[ 31065], 90.00th=[ 39584], 95.00th=[ 47973], | 99.00th=[ 78119], 99.50th=[ 98042], 99.90th=[107480], 99.95th=[107480], | 99.99th=[107480] bw ( KiB/s): min= 252, max= 733, per=26.36%, avg=379.78, stdev=101.92, samples=32 iops : min= 15, max= 45, avg=23.28, stdev= 6.28, samples=32 write: IOPS=22, BW=368KiB/s (377kB/s)(6032KiB/16394msec); 0 zone resets clat (usec): min=1643, max=104137, avg=19820.84, stdev=16123.28 lat (usec): min=1645, max=104139, avg=19826.21, stdev=16129.57 clat percentiles (usec): | 1.00th=[ 1762], 5.00th=[ 3130], 10.00th=[ 3687], 20.00th=[ 5145], | 30.00th=[ 8291], 40.00th=[ 11600], 50.00th=[ 15926], 60.00th=[ 19530], | 70.00th=[ 26084], 80.00th=[ 33162], 90.00th=[ 41681], 95.00th=[ 49021], | 99.00th=[ 73925], 99.50th=[ 80217], 99.90th=[104334], 99.95th=[104334], | 99.99th=[104334] bw ( KiB/s): min= 192, max= 608, per=23.84%, avg=361.41, stdev=103.95, samples=32 iops : min= 12, max= 38, avg=22.09, stdev= 6.64, samples=32 lat (usec) : 250=0.52%, 500=2.21%, 750=0.78%, 1000=0.26% lat (msec) : 2=0.78%, 4=5.21%, 10=14.06%, 20=30.99%, 50=40.89% lat (msec) : 100=4.04%, 250=0.26% cpu : usr=0.13%, sys=20.99%, ctx=2277, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182577: Tue Nov 18 13:47:49 2025 read: IOPS=23, BW=375KiB/s (384kB/s)(6032KiB/16106msec) clat (usec): min=151, max=107894, avg=22894.57, stdev=14603.56 lat (usec): min=152, max=107895, avg=22898.83, stdev=14606.20 clat percentiles (usec): | 1.00th=[ 180], 5.00th=[ 326], 10.00th=[ 693], 20.00th=[ 12780], | 30.00th=[ 15270], 40.00th=[ 18482], 50.00th=[ 21890], 60.00th=[ 24249], | 70.00th=[ 28181], 80.00th=[ 31589], 90.00th=[ 40109], 95.00th=[ 50594], | 99.00th=[ 71828], 99.50th=[ 78119], 99.90th=[107480], 99.95th=[107480], | 99.99th=[107480] bw ( KiB/s): min= 189, max= 539, per=25.62%, avg=369.13, stdev=105.59, samples=31 iops : min= 11, max= 33, avg=22.61, stdev= 6.60, samples=31 write: IOPS=24, BW=388KiB/s (398kB/s)(6256KiB/16106msec); 0 zone resets clat (usec): min=1055, max=79156, avg=18819.20, stdev=15032.99 lat (usec): min=1055, max=79158, avg=18820.83, stdev=15033.03 clat percentiles (usec): | 1.00th=[ 1516], 5.00th=[ 2540], 10.00th=[ 3916], 20.00th=[ 5473], | 30.00th=[ 7963], 40.00th=[12256], 50.00th=[15270], 60.00th=[19006], | 70.00th=[23200], 80.00th=[28443], 90.00th=[39584], 95.00th=[49546], | 99.00th=[70779], 99.50th=[77071], 99.90th=[79168], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 127, max= 544, per=25.48%, avg=386.32, stdev=85.48, samples=31 iops : min= 7, max= 34, avg=23.68, stdev= 5.49, samples=31 lat (usec) : 250=0.78%, 500=3.12%, 750=1.04%, 1000=0.13% lat (msec) : 2=2.08%, 4=4.04%, 10=12.76%, 20=29.43%, 50=41.67% lat (msec) : 100=4.82%, 250=0.13% cpu : usr=0.12%, sys=21.63%, ctx=2327, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1441KiB/s (1476kB/s), 346KiB/s-382KiB/s (354kB/s-391kB/s), io=23.4MiB (24.5MB), run=16106-16620msec WRITE: bw=1516KiB/s (1553kB/s), 368KiB/s-394KiB/s (377kB/s-403kB/s), io=24.6MiB (25.8MB), run=16106-16620msec rand-rw: (groupid=0, jobs=1): err= 0: pid=182578: Tue Nov 18 13:47:53 2025 read: IOPS=18, BW=296KiB/s (304kB/s)(5920KiB/19967msec) clat (msec): min=4, max=114, avg=20.48, stdev=16.37 lat (msec): min=4, max=114, avg=20.48, stdev=16.37 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 15], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 29], 90.00th=[ 45], 95.00th=[ 56], | 99.00th=[ 77], 99.50th=[ 82], 99.90th=[ 115], 99.95th=[ 115], | 99.99th=[ 115] bw ( KiB/s): min= 63, max= 796, per=24.39%, avg=289.49, stdev=174.86, samples=39 iops : min= 3, max= 49, avg=17.49, stdev=11.01, samples=39 write: IOPS=19, BW=319KiB/s (327kB/s)(6368KiB/19967msec); 0 zone resets clat (msec): min=7, max=121, avg=30.87, stdev=19.10 lat (msec): min=7, max=121, avg=30.88, stdev=19.10 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 12], 10.00th=[ 13], 20.00th=[ 16], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 26], 60.00th=[ 31], | 70.00th=[ 38], 80.00th=[ 45], 90.00th=[ 59], 95.00th=[ 68], | 99.00th=[ 99], 99.50th=[ 112], 99.90th=[ 122], 99.95th=[ 122], | 99.99th=[ 122] bw ( KiB/s): min= 63, max= 768, per=25.22%, avg=314.95, stdev=167.54, samples=39 iops : min= 3, max= 48, avg=19.08, stdev=10.59, samples=39 lat (msec) : 10=15.10%, 20=38.02%, 50=35.16%, 100=11.20%, 250=0.52% cpu : usr=0.10%, sys=8.42%, ctx=1434, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182579: Tue Nov 18 13:47:53 2025 read: IOPS=17, BW=285KiB/s (292kB/s)(5744KiB/20164msec) clat (msec): min=5, max=109, avg=22.17, stdev=18.48 lat (msec): min=5, max=109, avg=22.17, stdev=18.48 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 25], 80.00th=[ 32], 90.00th=[ 56], 95.00th=[ 65], | 99.00th=[ 83], 99.50th=[ 85], 99.90th=[ 110], 99.95th=[ 110], | 99.99th=[ 110] bw ( KiB/s): min= 57, max= 768, per=24.02%, avg=285.18, stdev=173.57, samples=40 iops : min= 3, max= 48, avg=17.23, stdev=10.95, samples=40 write: IOPS=20, BW=325KiB/s (332kB/s)(6544KiB/20164msec); 0 zone resets clat (msec): min=6, max=124, avg=29.37, stdev=19.00 lat (msec): min=6, max=124, avg=29.37, stdev=19.00 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 13], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 23], 60.00th=[ 28], | 70.00th=[ 36], 80.00th=[ 46], 90.00th=[ 54], 95.00th=[ 66], | 99.00th=[ 97], 99.50th=[ 108], 99.90th=[ 125], 99.95th=[ 125], | 99.99th=[ 125] bw ( KiB/s): min= 126, max= 896, per=25.91%, avg=323.57, stdev=185.27, samples=40 iops : min= 7, max= 56, avg=19.62, stdev=11.70, samples=40 lat (msec) : 10=14.84%, 20=37.11%, 50=35.03%, 100=12.50%, 250=0.52% cpu : usr=0.07%, sys=8.28%, ctx=1356, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182580: Tue Nov 18 13:47:53 2025 read: IOPS=19, BW=314KiB/s (322kB/s)(6256KiB/19892msec) clat (msec): min=4, max=111, avg=21.89, stdev=18.50 lat (msec): min=4, max=111, avg=21.89, stdev=18.50 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 33], 90.00th=[ 48], 95.00th=[ 64], | 99.00th=[ 92], 99.50th=[ 94], 99.90th=[ 112], 99.95th=[ 112], | 99.99th=[ 112] bw ( KiB/s): min= 95, max= 992, per=26.38%, avg=313.13, stdev=196.85, samples=39 iops : min= 5, max= 62, avg=18.97, stdev=12.42, samples=39 write: IOPS=18, BW=303KiB/s (311kB/s)(6032KiB/19892msec); 0 zone resets clat (usec): min=6672, max=97163, avg=29923.12, stdev=18281.21 lat (usec): min=6674, max=97164, avg=29925.29, stdev=18281.11 clat percentiles (usec): | 1.00th=[ 8717], 5.00th=[10421], 10.00th=[12125], 20.00th=[14484], | 30.00th=[16909], 40.00th=[20055], 50.00th=[23462], 60.00th=[29754], | 70.00th=[36963], 80.00th=[44827], 90.00th=[56886], 95.00th=[66847], | 99.00th=[86508], 99.50th=[89654], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 95, max= 764, per=23.81%, avg=297.44, stdev=173.53, samples=39 iops : min= 5, max= 47, avg=17.97, stdev=10.95, samples=39 lat (msec) : 10=16.15%, 20=35.94%, 50=36.20%, 100=11.59%, 250=0.13% cpu : usr=0.10%, sys=8.54%, ctx=1397, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182581: Tue Nov 18 13:47:53 2025 read: IOPS=18, BW=300KiB/s (307kB/s)(6032KiB/20099msec) clat (usec): min=4964, max=81827, avg=21337.91, stdev=17368.65 lat (usec): min=4965, max=81828, avg=21338.97, stdev=17368.67 clat percentiles (usec): | 1.00th=[ 5145], 5.00th=[ 6390], 10.00th=[ 7308], 20.00th=[ 8717], | 30.00th=[10159], 40.00th=[12125], 50.00th=[14484], 60.00th=[18220], | 70.00th=[22676], 80.00th=[30278], 90.00th=[47973], 95.00th=[64226], | 99.00th=[77071], 99.50th=[80217], 99.90th=[82314], 99.95th=[82314], | 99.99th=[82314] bw ( KiB/s): min= 127, max= 800, per=24.49%, avg=290.67, stdev=175.85, samples=39 iops : min= 7, max= 50, avg=17.56, stdev=11.14, samples=39 write: IOPS=19, BW=311KiB/s (319kB/s)(6256KiB/20099msec); 0 zone resets clat (msec): min=7, max=124, avg=30.37, stdev=20.80 lat (msec): min=7, max=124, avg=30.38, stdev=20.80 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 29], | 70.00th=[ 36], 80.00th=[ 45], 90.00th=[ 59], 95.00th=[ 70], | 99.00th=[ 112], 99.50th=[ 123], 99.90th=[ 125], 99.95th=[ 125], | 99.99th=[ 125] bw ( KiB/s): min= 127, max= 795, per=24.37%, avg=304.38, stdev=170.93, samples=39 iops : min= 7, max= 49, avg=18.41, stdev=10.83, samples=39 lat (msec) : 10=16.15%, 20=36.72%, 50=35.42%, 100=10.81%, 250=0.91% cpu : usr=0.11%, sys=8.53%, ctx=1477, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1188KiB/s (1216kB/s), 285KiB/s-314KiB/s (292kB/s-322kB/s), io=23.4MiB (24.5MB), run=19892-20164msec WRITE: bw=1250KiB/s (1280kB/s), 303KiB/s-325KiB/s (311kB/s-332kB/s), io=24.6MiB (25.8MB), run=19892-20164msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=182590: Tue Nov 18 13:48:03 2025 read: IOPS=1, BW=1957KiB/s (2004kB/s)(7168KiB/3662msec) clat (msec): min=57, max=1303, avg=290.56, stdev=450.77 lat (msec): min=57, max=1303, avg=290.56, stdev=450.77 clat percentiles (msec): | 1.00th=[ 57], 5.00th=[ 57], 10.00th=[ 57], 20.00th=[ 69], | 30.00th=[ 73], 40.00th=[ 73], 50.00th=[ 148], 60.00th=[ 169], | 70.00th=[ 169], 80.00th=[ 215], 90.00th=[ 1301], 95.00th=[ 1301], | 99.00th=[ 1301], 99.50th=[ 1301], 99.90th=[ 1301], 99.95th=[ 1301], | 99.99th=[ 1301] bw ( KiB/s): min= 2015, max= 5851, per=89.87%, avg=3987.33, stdev=1920.31, samples=3 iops : min= 1, max= 5, avg= 3.33, stdev= 2.08, samples=3 write: IOPS=1, BW=1398KiB/s (1432kB/s)(5120KiB/3662msec); 0 zone resets clat (msec): min=124, max=697, avg=323.86, stdev=221.85 lat (msec): min=124, max=697, avg=323.96, stdev=221.83 clat percentiles (msec): | 1.00th=[ 126], 5.00th=[ 126], 10.00th=[ 126], 20.00th=[ 126], | 30.00th=[ 194], 40.00th=[ 194], 50.00th=[ 292], 60.00th=[ 292], | 70.00th=[ 313], 80.00th=[ 313], 90.00th=[ 701], 95.00th=[ 701], | 99.00th=[ 701], 99.50th=[ 701], 99.90th=[ 701], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 2015, max= 4087, per=41.00%, avg=2547.00, stdev=1026.75, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 100=25.00%, 250=41.67%, 500=16.67%, 750=8.33%, 2000=8.33% cpu : usr=0.03%, sys=12.95%, ctx=92, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182592: Tue Nov 18 13:48:03 2025 read: IOPS=0, BW=459KiB/s (470kB/s)(2048KiB/4464msec) clat (msec): min=79, max=170, avg=124.99, stdev=64.25 lat (msec): min=79, max=170, avg=124.99, stdev=64.25 clat percentiles (msec): | 1.00th=[ 80], 5.00th=[ 80], 10.00th=[ 80], 20.00th=[ 80], | 30.00th=[ 80], 40.00th=[ 80], 50.00th=[ 80], 60.00th=[ 171], | 70.00th=[ 171], 80.00th=[ 171], 90.00th=[ 171], 95.00th=[ 171], | 99.00th=[ 171], 99.50th=[ 171], 99.90th=[ 171], 99.95th=[ 171], | 99.99th=[ 171] bw ( KiB/s): min= 1954, max= 2015, per=44.73%, avg=1984.50, stdev=43.13, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2294KiB/s (2349kB/s)(10.0MiB/4464msec); 0 zone resets clat (msec): min=54, max=1998, avg=414.81, stdev=581.97 lat (msec): min=54, max=1998, avg=414.89, stdev=581.98 clat percentiles (msec): | 1.00th=[ 55], 5.00th=[ 55], 10.00th=[ 55], 20.00th=[ 62], | 30.00th=[ 92], 40.00th=[ 157], 50.00th=[ 174], 60.00th=[ 209], | 70.00th=[ 418], 80.00th=[ 477], 90.00th=[ 510], 95.00th=[ 2005], | 99.00th=[ 2005], 99.50th=[ 2005], 99.90th=[ 2005], 99.95th=[ 2005], | 99.99th=[ 2005] bw ( KiB/s): min= 2015, max= 8175, per=65.30%, avg=4056.25, stdev=2897.59, samples=4 iops : min= 1, max= 7, avg= 3.00, stdev= 2.83, samples=4 lat (msec) : 100=33.33%, 250=33.33%, 500=16.67%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=13.82%, ctx=123, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182594: Tue Nov 18 13:48:03 2025 read: IOPS=1, BW=1331KiB/s (1363kB/s)(6144KiB/4615msec) clat (msec): min=68, max=453, avg=222.77, stdev=159.77 lat (msec): min=68, max=453, avg=222.77, stdev=159.77 clat percentiles (msec): | 1.00th=[ 69], 5.00th=[ 69], 10.00th=[ 69], 20.00th=[ 74], | 30.00th=[ 74], 40.00th=[ 124], 50.00th=[ 124], 60.00th=[ 264], | 70.00th=[ 355], 80.00th=[ 355], 90.00th=[ 456], 95.00th=[ 456], | 99.00th=[ 456], 99.50th=[ 456], 99.90th=[ 456], 99.95th=[ 456], | 99.99th=[ 456] bw ( KiB/s): min= 2015, max= 3992, per=56.87%, avg=2523.25, stdev=979.26, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=1, BW=1331KiB/s (1363kB/s)(6144KiB/4615msec); 0 zone resets clat (msec): min=111, max=1778, avg=538.45, stdev=616.46 lat (msec): min=111, max=1778, avg=538.54, stdev=616.46 clat percentiles (msec): | 1.00th=[ 111], 5.00th=[ 111], 10.00th=[ 111], 20.00th=[ 245], | 30.00th=[ 245], 40.00th=[ 321], 50.00th=[ 321], 60.00th=[ 376], | 70.00th=[ 397], 80.00th=[ 397], 90.00th=[ 1787], 95.00th=[ 1787], | 99.00th=[ 1787], 99.50th=[ 1787], 99.90th=[ 1787], 99.95th=[ 1787], | 99.99th=[ 1787] bw ( KiB/s): min= 1965, max= 4087, per=39.22%, avg=2436.20, stdev=923.44, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 lat (msec) : 100=16.67%, 250=25.00%, 500=50.00%, 2000=8.33% cpu : usr=0.00%, sys=13.11%, ctx=100, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182596: Tue Nov 18 13:48:03 2025 read: IOPS=1, BW=1233KiB/s (1263kB/s)(5120KiB/4152msec) clat (msec): min=76, max=462, avg=204.71, stdev=154.36 lat (msec): min=76, max=462, avg=204.71, stdev=154.36 clat percentiles (msec): | 1.00th=[ 77], 5.00th=[ 77], 10.00th=[ 77], 20.00th=[ 77], | 30.00th=[ 120], 40.00th=[ 120], 50.00th=[ 138], 60.00th=[ 138], | 70.00th=[ 230], 80.00th=[ 230], 90.00th=[ 464], 95.00th=[ 464], | 99.00th=[ 464], 99.50th=[ 464], 99.90th=[ 464], 99.95th=[ 464], | 99.99th=[ 464] bw ( KiB/s): min= 4031, max= 4087, per=91.48%, avg=4059.00, stdev=39.60, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1726KiB/s (1768kB/s)(7168KiB/4152msec); 0 zone resets clat (msec): min=58, max=1670, avg=440.29, stdev=568.32 lat (msec): min=58, max=1670, avg=440.39, stdev=568.34 clat percentiles (msec): | 1.00th=[ 59], 5.00th=[ 59], 10.00th=[ 59], 20.00th=[ 74], | 30.00th=[ 230], 40.00th=[ 230], 50.00th=[ 230], 60.00th=[ 245], | 70.00th=[ 245], 80.00th=[ 575], 90.00th=[ 1670], 95.00th=[ 1670], | 99.00th=[ 1670], 99.50th=[ 1670], 99.90th=[ 1670], 99.95th=[ 1670], | 99.99th=[ 1670] bw ( KiB/s): min= 1943, max= 4087, per=48.58%, avg=3017.50, stdev=1181.52, samples=4 iops : min= 1, max= 3, avg= 2.25, stdev= 0.96, samples=4 lat (msec) : 100=25.00%, 250=50.00%, 500=8.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=13.64%, ctx=146, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4438KiB/s (4544kB/s), 459KiB/s-1957KiB/s (470kB/s-2004kB/s), io=20.0MiB (20.0MB), run=3662-4615msec WRITE: bw=6213KiB/s (6362kB/s), 1331KiB/s-2294KiB/s (1363kB/s-2349kB/s), io=28.0MiB (29.4MB), run=3662-4615msec rand-rw: (groupid=0, jobs=1): err= 0: pid=182591: Tue Nov 18 13:48:03 2025 read: IOPS=1, BW=1897KiB/s (1942kB/s)(7168KiB/3779msec) clat (usec): min=822, max=1498.7k, avg=314824.22, stdev=525528.23 lat (usec): min=823, max=1498.7k, avg=314825.51, stdev=525528.34 clat percentiles (usec): | 1.00th=[ 824], 5.00th=[ 824], 10.00th=[ 824], | 20.00th=[ 104334], 30.00th=[ 119014], 40.00th=[ 119014], | 50.00th=[ 126354], 60.00th=[ 156238], 70.00th=[ 156238], | 80.00th=[ 198181], 90.00th=[1501561], 95.00th=[1501561], | 99.00th=[1501561], 99.50th=[1501561], 99.90th=[1501561], | 99.95th=[1501561], 99.99th=[1501561] bw ( KiB/s): min= 2048, max= 4087, per=68.47%, avg=3065.50, stdev=1174.91, samples=4 iops : min= 2, max= 3, avg= 2.50, stdev= 0.58, samples=4 write: IOPS=1, BW=1355KiB/s (1387kB/s)(5120KiB/3779msec); 0 zone resets clat (msec): min=84, max=678, avg=297.22, stdev=235.56 lat (msec): min=84, max=678, avg=297.32, stdev=235.56 clat percentiles (msec): | 1.00th=[ 85], 5.00th=[ 85], 10.00th=[ 85], 20.00th=[ 85], | 30.00th=[ 138], 40.00th=[ 138], 50.00th=[ 239], 60.00th=[ 239], | 70.00th=[ 347], 80.00th=[ 347], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 2048, max= 6083, per=64.86%, avg=4065.50, stdev=2853.18, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 lat (usec) : 1000=8.33% lat (msec) : 100=8.33%, 250=58.33%, 500=8.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=33.27%, ctx=1300, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182593: Tue Nov 18 13:48:03 2025 read: IOPS=0, BW=448KiB/s (458kB/s)(2048KiB/4574msec) clat (usec): min=1414, max=130501, avg=65957.95, stdev=91278.36 lat (usec): min=1416, max=130503, avg=65959.86, stdev=91278.76 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 1418], 10.00th=[ 1418], 20.00th=[ 1418], | 30.00th=[ 1418], 40.00th=[ 1418], 50.00th=[ 1418], 60.00th=[130548], | 70.00th=[130548], 80.00th=[130548], 90.00th=[130548], 95.00th=[130548], | 99.00th=[130548], 99.50th=[130548], 99.90th=[130548], 99.95th=[130548], | 99.99th=[130548] bw ( KiB/s): min= 2027, max= 2043, per=45.45%, avg=2035.00, stdev=11.31, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2239KiB/s (2292kB/s)(10.0MiB/4574msec); 0 zone resets clat (msec): min=64, max=2027, avg=439.56, stdev=621.38 lat (msec): min=65, max=2027, avg=439.64, stdev=621.36 clat percentiles (msec): | 1.00th=[ 65], 5.00th=[ 65], 10.00th=[ 65], 20.00th=[ 66], | 30.00th=[ 79], 40.00th=[ 80], 50.00th=[ 85], 60.00th=[ 178], | 70.00th=[ 359], 80.00th=[ 584], 90.00th=[ 877], 95.00th=[ 2022], | 99.00th=[ 2022], 99.50th=[ 2022], 99.90th=[ 2022], 99.95th=[ 2022], | 99.99th=[ 2022] bw ( KiB/s): min= 2027, max= 8192, per=72.92%, avg=4570.50, stdev=2592.95, samples=4 iops : min= 1, max= 8, avg= 3.75, stdev= 2.99, samples=4 lat (msec) : 2=8.33%, 100=41.67%, 250=16.67%, 500=8.33%, 750=8.33% lat (msec) : 1000=8.33%, >=2000=8.33% cpu : usr=0.02%, sys=25.21%, ctx=362, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182595: Tue Nov 18 13:48:03 2025 read: IOPS=1, BW=1606KiB/s (1645kB/s)(6144KiB/3825msec) clat (usec): min=670, max=448092, avg=75362.28, stdev=182599.94 lat (usec): min=671, max=448093, avg=75363.25, stdev=182599.72 clat percentiles (usec): | 1.00th=[ 668], 5.00th=[ 668], 10.00th=[ 668], 20.00th=[ 807], | 30.00th=[ 807], 40.00th=[ 848], 50.00th=[ 848], 60.00th=[ 873], | 70.00th=[ 881], 80.00th=[ 881], 90.00th=[446694], 95.00th=[446694], | 99.00th=[446694], 99.50th=[446694], 99.90th=[446694], 99.95th=[446694], | 99.99th=[446694] bw ( KiB/s): min= 2027, max= 4071, per=75.07%, avg=3360.67, stdev=1155.81, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=1, BW=1606KiB/s (1645kB/s)(6144KiB/3825msec); 0 zone resets clat (msec): min=184, max=1757, avg=541.64, stdev=598.94 lat (msec): min=184, max=1757, avg=541.69, stdev=598.94 clat percentiles (msec): | 1.00th=[ 186], 5.00th=[ 186], 10.00th=[ 186], 20.00th=[ 300], | 30.00th=[ 300], 40.00th=[ 309], 50.00th=[ 309], 60.00th=[ 338], | 70.00th=[ 363], 80.00th=[ 363], 90.00th=[ 1754], 95.00th=[ 1754], | 99.00th=[ 1754], 99.50th=[ 1754], 99.90th=[ 1754], 99.95th=[ 1754], | 99.99th=[ 1754] bw ( KiB/s): min= 1367, max= 2048, per=30.21%, avg=1893.80, stdev=295.22, samples=5 iops : min= 1, max= 2, avg= 1.20, stdev= 0.45, samples=5 lat (usec) : 750=8.33%, 1000=33.33% lat (msec) : 250=8.33%, 500=41.67%, 2000=8.33% cpu : usr=0.00%, sys=27.93%, ctx=203, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182597: Tue Nov 18 13:48:03 2025 read: IOPS=1, BW=1181KiB/s (1210kB/s)(5120KiB/4334msec) clat (usec): min=824, max=423444, avg=177259.90, stdev=192091.92 lat (usec): min=825, max=423445, avg=177260.99, stdev=192091.85 clat percentiles (usec): | 1.00th=[ 824], 5.00th=[ 824], 10.00th=[ 824], 20.00th=[ 824], | 30.00th=[ 865], 40.00th=[ 865], 50.00th=[132645], 60.00th=[132645], | 70.00th=[329253], 80.00th=[329253], 90.00th=[421528], 95.00th=[421528], | 99.00th=[421528], 99.50th=[421528], 99.90th=[421528], 99.95th=[421528], | 99.99th=[421528] bw ( KiB/s): min= 2027, max= 3984, per=60.00%, avg=2686.33, stdev=1123.86, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 write: IOPS=1, BW=1654KiB/s (1694kB/s)(7168KiB/4334msec); 0 zone resets clat (msec): min=120, max=1644, avg=473.30, stdev=526.95 lat (msec): min=120, max=1644, avg=473.38, stdev=526.94 clat percentiles (msec): | 1.00th=[ 121], 5.00th=[ 121], 10.00th=[ 121], 20.00th=[ 161], | 30.00th=[ 275], 40.00th=[ 275], 50.00th=[ 355], 60.00th=[ 368], | 70.00th=[ 368], 80.00th=[ 393], 90.00th=[ 1653], 95.00th=[ 1653], | 99.00th=[ 1653], 99.50th=[ 1653], 99.90th=[ 1653], 99.95th=[ 1653], | 99.99th=[ 1653] bw ( KiB/s): min= 2027, max= 4079, per=43.29%, avg=2713.67, stdev=1182.42, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (usec) : 1000=16.67% lat (msec) : 250=25.00%, 500=50.00%, 2000=8.33% cpu : usr=0.00%, sys=28.23%, ctx=274, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4477KiB/s (4585kB/s), 448KiB/s-1897KiB/s (458kB/s-1942kB/s), io=20.0MiB (20.0MB), run=3779-4574msec WRITE: bw=6268KiB/s (6419kB/s), 1355KiB/s-2239KiB/s (1387kB/s-2292kB/s), io=28.0MiB (29.4MB), run=3779-4574msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=182608: Tue Nov 18 13:48:12 2025 read: IOPS=1, BW=8132KiB/s (8328kB/s)(12.0MiB/1511msec) clat (msec): min=436, max=551, avg=503.06, stdev=59.54 lat (msec): min=436, max=551, avg=503.06, stdev=59.54 clat percentiles (msec): | 1.00th=[ 439], 5.00th=[ 439], 10.00th=[ 439], 20.00th=[ 439], | 30.00th=[ 439], 40.00th=[ 523], 50.00th=[ 523], 60.00th=[ 523], | 70.00th=[ 550], 80.00th=[ 550], 90.00th=[ 550], 95.00th=[ 550], | 99.00th=[ 550], 99.50th=[ 550], 99.90th=[ 550], 99.95th=[ 550], | 99.99th=[ 550] bw ( KiB/s): min= 6140, max= 8175, per=76.37%, avg=7157.50, stdev=1438.96, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.00%, sys=45.17%, ctx=1494, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182609: Tue Nov 18 13:48:12 2025 read: IOPS=0, BW=2343KiB/s (2399kB/s)(4096KiB/1748msec) clat (nsec): min=427938k, max=427938k, avg=427937511.00, stdev= 0.00 lat (nsec): min=427939k, max=427939k, avg=427939036.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 430], 5.00th=[ 430], 10.00th=[ 430], 20.00th=[ 430], | 30.00th=[ 430], 40.00th=[ 430], 50.00th=[ 430], 60.00th=[ 430], | 70.00th=[ 430], 80.00th=[ 430], 90.00th=[ 430], 95.00th=[ 430], | 99.00th=[ 430], 99.50th=[ 430], 99.90th=[ 430], 99.95th=[ 430], | 99.99th=[ 430] bw ( KiB/s): min= 6215, max= 6215, per=66.31%, avg=6215.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4686KiB/s (4799kB/s)(8192KiB/1748msec); 0 zone resets clat (usec): min=658782, max=659308, avg=659045.43, stdev=371.96 lat (usec): min=659273, max=659707, avg=659490.48, stdev=306.56 clat percentiles (msec): | 1.00th=[ 659], 5.00th=[ 659], 10.00th=[ 659], 20.00th=[ 659], | 30.00th=[ 659], 40.00th=[ 659], 50.00th=[ 659], 60.00th=[ 659], | 70.00th=[ 659], 80.00th=[ 659], 90.00th=[ 659], 95.00th=[ 659], | 99.00th=[ 659], 99.50th=[ 659], 99.90th=[ 659], 99.95th=[ 659], | 99.99th=[ 659] bw ( KiB/s): min= 8175, max= 8175, per=51.05%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.00%, sys=20.09%, ctx=753, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182610: Tue Nov 18 13:48:12 2025 write: IOPS=1, BW=6006KiB/s (6150kB/s)(12.0MiB/2046msec); 0 zone resets clat (msec): min=193, max=1314, avg=681.43, stdev=574.09 lat (msec): min=194, max=1314, avg=681.71, stdev=574.07 clat percentiles (msec): | 1.00th=[ 194], 5.00th=[ 194], 10.00th=[ 194], 20.00th=[ 194], | 30.00th=[ 194], 40.00th=[ 535], 50.00th=[ 535], 60.00th=[ 535], | 70.00th=[ 1318], 80.00th=[ 1318], 90.00th=[ 1318], 95.00th=[ 1318], | 99.00th=[ 1318], 99.50th=[ 1318], 99.90th=[ 1318], 99.95th=[ 1318], | 99.99th=[ 1318] bw ( KiB/s): min= 8159, max= 8159, per=50.95%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=16.09%, ctx=47, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182611: Tue Nov 18 13:48:12 2025 write: IOPS=1, BW=7078KiB/s (7248kB/s)(12.0MiB/1736msec); 0 zone resets clat (msec): min=167, max=1083, avg=577.91, stdev=465.52 lat (msec): min=167, max=1084, avg=578.23, stdev=465.50 clat percentiles (msec): | 1.00th=[ 167], 5.00th=[ 167], 10.00th=[ 167], 20.00th=[ 167], | 30.00th=[ 167], 40.00th=[ 481], 50.00th=[ 481], 60.00th=[ 481], | 70.00th=[ 1083], 80.00th=[ 1083], 90.00th=[ 1083], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 6311, max= 8159, per=45.18%, avg=7235.00, stdev=1306.73, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.12%, sys=13.37%, ctx=44, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=9373KiB/s (9598kB/s), 2343KiB/s-8132KiB/s (2399kB/s-8328kB/s), io=16.0MiB (16.8MB), run=1511-1748msec WRITE: bw=15.6MiB/s (16.4MB/s), 4686KiB/s-7078KiB/s (4799kB/s-7248kB/s), io=32.0MiB (33.6MB), run=1736-2046msec rand-rw: (groupid=0, jobs=1): err= 0: pid=182603: Tue Nov 18 13:48:12 2025 read: IOPS=1, BW=6915KiB/s (7081kB/s)(12.0MiB/1777msec) clat (msec): min=264, max=768, avg=561.02, stdev=263.62 lat (msec): min=264, max=768, avg=561.02, stdev=263.62 clat percentiles (msec): | 1.00th=[ 266], 5.00th=[ 266], 10.00th=[ 266], 20.00th=[ 266], | 30.00th=[ 266], 40.00th=[ 651], 50.00th=[ 651], 60.00th=[ 651], | 70.00th=[ 768], 80.00th=[ 768], 90.00th=[ 768], 95.00th=[ 768], | 99.00th=[ 768], 99.50th=[ 768], 99.90th=[ 768], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 8175, max= 8192, per=88.76%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=18.13%, ctx=80, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182604: Tue Nov 18 13:48:12 2025 read: IOPS=1, BW=4597KiB/s (4707kB/s)(4096KiB/891msec) clat (nsec): min=136232k, max=136232k, avg=136231732.00, stdev= 0.00 lat (nsec): min=136233k, max=136233k, avg=136232817.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 136], | 30.00th=[ 136], 40.00th=[ 136], 50.00th=[ 136], 60.00th=[ 136], | 70.00th=[ 136], 80.00th=[ 136], 90.00th=[ 136], 95.00th=[ 136], | 99.00th=[ 136], 99.50th=[ 136], 99.90th=[ 136], 99.95th=[ 136], | 99.99th=[ 136] bw ( KiB/s): min= 5542, max= 5542, per=60.11%, avg=5542.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=2, BW=9194KiB/s (9415kB/s)(8192KiB/891msec); 0 zone resets clat (msec): min=324, max=332, avg=328.37, stdev= 5.20 lat (msec): min=325, max=332, avg=328.74, stdev= 5.11 clat percentiles (msec): | 1.00th=[ 326], 5.00th=[ 326], 10.00th=[ 326], 20.00th=[ 326], | 30.00th=[ 326], 40.00th=[ 326], 50.00th=[ 326], 60.00th=[ 334], | 70.00th=[ 334], 80.00th=[ 334], 90.00th=[ 334], 95.00th=[ 334], | 99.00th=[ 334], 99.50th=[ 334], 99.90th=[ 334], 99.95th=[ 334], | 99.99th=[ 334] bw ( KiB/s): min= 5542, max= 5542, per=40.15%, avg=5542.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=66.67% cpu : usr=0.00%, sys=34.94%, ctx=66, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182605: Tue Nov 18 13:48:12 2025 write: IOPS=1, BW=5176KiB/s (5300kB/s)(12.0MiB/2374msec); 0 zone resets clat (msec): min=99, max=1698, avg=756.86, stdev=836.91 lat (msec): min=99, max=1699, avg=757.32, stdev=837.12 clat percentiles (msec): | 1.00th=[ 100], 5.00th=[ 100], 10.00th=[ 100], 20.00th=[ 100], | 30.00th=[ 100], 40.00th=[ 472], 50.00th=[ 472], 60.00th=[ 472], | 70.00th=[ 1703], 80.00th=[ 1703], 90.00th=[ 1703], 95.00th=[ 1703], | 99.00th=[ 1703], 99.50th=[ 1703], 99.90th=[ 1703], 99.95th=[ 1703], | 99.99th=[ 1703] bw ( KiB/s): min= 8159, max= 8159, per=59.11%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 100=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=15.59%, ctx=67, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182606: Tue Nov 18 13:48:12 2025 write: IOPS=1, BW=7128KiB/s (7299kB/s)(12.0MiB/1724msec); 0 zone resets clat (msec): min=151, max=1137, avg=544.86, stdev=521.95 lat (msec): min=152, max=1137, avg=545.27, stdev=521.78 clat percentiles (msec): | 1.00th=[ 153], 5.00th=[ 153], 10.00th=[ 153], 20.00th=[ 153], | 30.00th=[ 153], 40.00th=[ 347], 50.00th=[ 347], 60.00th=[ 347], | 70.00th=[ 1133], 80.00th=[ 1133], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min=11237, max=11237, per=81.42%, avg=11237.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=16.42%, ctx=61, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=9220KiB/s (9441kB/s), 4597KiB/s-6915KiB/s (4707kB/s-7081kB/s), io=16.0MiB (16.8MB), run=891-1777msec WRITE: bw=13.5MiB/s (14.1MB/s), 5176KiB/s-9194KiB/s (5300kB/s-9415kB/s), io=32.0MiB (33.6MB), run=891-2374msec