== sanity test 398b: DIO and buffer IO race ============== 17:05:40 (1764108340) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.18836 s, 15.8 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155347: Tue Nov 25 17:06:59 2025 read: IOPS=22, BW=88.8KiB/s (90.9kB/s)(5992KiB/67471msec) clat (usec): min=84, max=782295, avg=21476.88, stdev=24151.29 lat (usec): min=85, max=782296, avg=21478.10, stdev=24151.25 clat percentiles (usec): | 1.00th=[ 141], 5.00th=[ 277], 10.00th=[ 5932], 20.00th=[ 11469], | 30.00th=[ 13435], 40.00th=[ 15401], 50.00th=[ 18220], 60.00th=[ 21103], | 70.00th=[ 25560], 80.00th=[ 30802], 90.00th=[ 39060], 95.00th=[ 47449], | 99.00th=[ 67634], 99.50th=[ 76022], 99.90th=[112722], 99.95th=[784335], | 99.99th=[784335] bw ( KiB/s): min= 8, max= 158, per=25.23%, avg=88.54, stdev=27.95, samples=134 iops : min= 2, max= 39, avg=21.58, stdev= 6.99, samples=134 write: IOPS=23, BW=93.3KiB/s (95.6kB/s)(6296KiB/67471msec); 0 zone resets clat (usec): min=766, max=296095, avg=22193.42, stdev=22011.30 lat (usec): min=767, max=296096, avg=22194.80, stdev=22011.33 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 1221], 10.00th=[ 1713], 20.00th=[ 2769], | 30.00th=[ 5276], 40.00th=[ 12649], 50.00th=[ 17957], 60.00th=[ 23200], | 70.00th=[ 29230], 80.00th=[ 36439], 90.00th=[ 49021], 95.00th=[ 61080], | 99.00th=[ 88605], 99.50th=[101188], 99.90th=[242222], 99.95th=[295699], | 99.99th=[295699] bw ( KiB/s): min= 24, max= 190, per=25.71%, avg=94.34, stdev=30.22, samples=132 iops : min= 6, max= 47, avg=23.04, stdev= 7.61, samples=132 lat (usec) : 100=0.03%, 250=2.12%, 500=2.12%, 750=0.13%, 1000=0.46% lat (msec) : 2=6.54%, 4=6.45%, 10=8.53%, 20=28.26%, 50=38.83% lat (msec) : 100=6.18%, 250=0.29%, 500=0.03%, 1000=0.03% cpu : usr=0.11%, sys=13.00%, ctx=7002, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155349: Tue Nov 25 17:06:59 2025 read: IOPS=21, BW=86.0KiB/s (89.1kB/s)(5940KiB/68286msec) clat (usec): min=116, max=190607, avg=21384.53, stdev=14900.79 lat (usec): min=116, max=190608, avg=21385.67, stdev=14900.78 clat percentiles (usec): | 1.00th=[ 151], 5.00th=[ 269], 10.00th=[ 7504], 20.00th=[ 11076], | 30.00th=[ 13566], 40.00th=[ 15926], 50.00th=[ 18220], 60.00th=[ 21627], | 70.00th=[ 25822], 80.00th=[ 31589], 90.00th=[ 39584], 95.00th=[ 45876], | 99.00th=[ 66323], 99.50th=[ 73925], 99.90th=[152044], 99.95th=[189793], | 99.99th=[189793] bw ( KiB/s): min= 6, max= 166, per=24.68%, avg=86.62, stdev=27.22, samples=134 iops : min= 1, max= 41, avg=21.10, stdev= 6.87, samples=134 write: IOPS=23, BW=92.0KiB/s (95.2kB/s)(6348KiB/68286msec); 0 zone resets clat (usec): min=684, max=723971, avg=22778.60, stdev=27652.12 lat (usec): min=685, max=723974, avg=22784.51, stdev=27650.65 clat percentiles (usec): | 1.00th=[ 1012], 5.00th=[ 1172], 10.00th=[ 1696], 20.00th=[ 2638], | 30.00th=[ 5276], 40.00th=[ 12780], 50.00th=[ 18220], 60.00th=[ 23725], | 70.00th=[ 29492], 80.00th=[ 37487], 90.00th=[ 51643], 95.00th=[ 61604], | 99.00th=[ 91751], 99.50th=[113771], 99.90th=[200279], 99.95th=[725615], | 99.99th=[725615] bw ( KiB/s): min= 6, max= 248, per=24.93%, avg=91.48, stdev=33.96, samples=135 iops : min= 1, max= 62, avg=22.33, stdev= 8.57, samples=135 lat (usec) : 250=2.12%, 500=2.08%, 750=0.16%, 1000=0.46% lat (msec) : 2=6.41%, 4=6.84%, 10=7.85%, 20=28.09%, 50=38.57% lat (msec) : 100=6.93%, 250=0.46%, 750=0.03% cpu : usr=0.14%, sys=12.77%, ctx=7038, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155350: Tue Nov 25 17:06:59 2025 read: IOPS=22, BW=90.5KiB/s (92.7kB/s)(6028KiB/66586msec) clat (usec): min=95, max=220725, avg=20508.43, stdev=15405.53 lat (usec): min=95, max=220726, avg=20513.39, stdev=15408.30 clat percentiles (usec): | 1.00th=[ 135], 5.00th=[ 212], 10.00th=[ 537], 20.00th=[ 10683], | 30.00th=[ 12780], 40.00th=[ 14877], 50.00th=[ 17171], 60.00th=[ 20579], | 70.00th=[ 25035], 80.00th=[ 30278], 90.00th=[ 38536], 95.00th=[ 45876], | 99.00th=[ 71828], 99.50th=[ 88605], 99.90th=[121111], 99.95th=[221250], | 99.99th=[221250] bw ( KiB/s): min= 23, max= 168, per=26.08%, avg=91.53, stdev=28.62, samples=130 iops : min= 5, max= 42, avg=22.33, stdev= 7.24, samples=130 write: IOPS=23, BW=94.0KiB/s (96.3kB/s)(6260KiB/66586msec); 0 zone resets clat (usec): min=725, max=824191, avg=22566.04, stdev=29579.23 lat (usec): min=726, max=824192, avg=22567.56, stdev=29579.23 clat percentiles (usec): | 1.00th=[ 1012], 5.00th=[ 1205], 10.00th=[ 1614], 20.00th=[ 2802], | 30.00th=[ 5669], 40.00th=[ 12125], 50.00th=[ 17433], 60.00th=[ 22676], | 70.00th=[ 28967], 80.00th=[ 36439], 90.00th=[ 49021], 95.00th=[ 61604], | 99.00th=[ 96994], 99.50th=[111674], 99.90th=[200279], 99.95th=[826278], | 99.99th=[826278] bw ( KiB/s): min= 39, max= 176, per=25.91%, avg=95.09, stdev=29.00, samples=130 iops : min= 9, max= 44, avg=23.22, stdev= 7.29, samples=130 lat (usec) : 100=0.03%, 250=2.86%, 500=1.99%, 750=0.20%, 1000=0.39% lat (msec) : 2=6.58%, 4=6.61%, 10=9.02%, 20=29.39%, 50=36.23% lat (msec) : 100=6.12%, 250=0.55%, 1000=0.03% cpu : usr=0.12%, sys=12.72%, ctx=6910, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155351: Tue Nov 25 17:06:59 2025 read: IOPS=22, BW=88.6KiB/s (90.7kB/s)(6052KiB/68327msec) clat (usec): min=117, max=742264, avg=21477.85, stdev=23718.07 lat (usec): min=117, max=742266, avg=21480.29, stdev=23717.98 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 243], 10.00th=[ 1172], 20.00th=[ 10814], | 30.00th=[ 13435], 40.00th=[ 15533], 50.00th=[ 17957], 60.00th=[ 21890], | 70.00th=[ 26084], 80.00th=[ 31065], 90.00th=[ 38011], 95.00th=[ 46400], | 99.00th=[ 67634], 99.50th=[ 79168], 99.90th=[175113], 99.95th=[742392], | 99.99th=[742392] bw ( KiB/s): min= 6, max= 182, per=24.92%, avg=87.47, stdev=30.26, samples=135 iops : min= 1, max= 45, avg=21.32, stdev= 7.60, samples=135 write: IOPS=22, BW=91.3KiB/s (93.5kB/s)(6236KiB/68327msec); 0 zone resets clat (usec): min=900, max=288349, avg=22747.43, stdev=21583.52 lat (usec): min=901, max=288350, avg=22751.16, stdev=21585.63 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 1237], 10.00th=[ 1745], 20.00th=[ 2868], | 30.00th=[ 6128], 40.00th=[ 13173], 50.00th=[ 19006], 60.00th=[ 23987], | 70.00th=[ 30278], 80.00th=[ 39060], 90.00th=[ 49546], 95.00th=[ 62129], | 99.00th=[ 88605], 99.50th=[105382], 99.90th=[133694], 99.95th=[287310], | 99.99th=[287310] bw ( KiB/s): min= 26, max= 152, per=24.79%, avg=90.99, stdev=25.84, samples=134 iops : min= 6, max= 38, avg=22.20, stdev= 6.50, samples=134 lat (usec) : 250=2.47%, 500=2.05%, 750=0.29%, 1000=0.42% lat (msec) : 2=6.12%, 4=6.41%, 10=8.46%, 20=27.64%, 50=39.49% lat (msec) : 100=6.12%, 250=0.46%, 500=0.03%, 750=0.03% cpu : usr=0.13%, sys=12.53%, ctx=7049, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=351KiB/s (360kB/s), 86.0KiB/s-90.5KiB/s (89.1kB/s-92.7kB/s), io=23.4MiB (24.6MB), run=66586-68327msec WRITE: bw=368KiB/s (377kB/s), 91.3KiB/s-94.0KiB/s (93.5kB/s-96.3kB/s), io=24.6MiB (25.7MB), run=66586-68327msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155352: Tue Nov 25 17:08:19 2025 read: IOPS=10, BW=40.6KiB/s (41.6kB/s)(5992KiB/147410msec) clat (msec): min=4, max=792, avg=19.63, stdev=26.76 lat (msec): min=4, max=792, avg=19.63, stdev=26.76 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 13], 60.00th=[ 16], | 70.00th=[ 20], 80.00th=[ 27], 90.00th=[ 39], 95.00th=[ 53], | 99.00th=[ 93], 99.50th=[ 116], 99.90th=[ 209], 99.95th=[ 793], | 99.99th=[ 793] bw ( KiB/s): min= 7, max= 119, per=25.50%, avg=41.32, stdev=25.15, samples=285 iops : min= 1, max= 29, avg= 9.80, stdev= 6.34, samples=285 write: IOPS=10, BW=42.7KiB/s (43.7kB/s)(6296KiB/147410msec); 0 zone resets clat (msec): min=24, max=310, avg=74.76, stdev=34.26 lat (msec): min=24, max=310, avg=74.76, stdev=34.25 clat percentiles (msec): | 1.00th=[ 31], 5.00th=[ 37], 10.00th=[ 42], 20.00th=[ 49], | 30.00th=[ 54], 40.00th=[ 60], 50.00th=[ 67], 60.00th=[ 74], | 70.00th=[ 84], 80.00th=[ 97], 90.00th=[ 122], 95.00th=[ 142], | 99.00th=[ 190], 99.50th=[ 226], 99.90th=[ 284], 99.95th=[ 313], | 99.99th=[ 313] bw ( KiB/s): min= 7, max= 95, per=25.12%, avg=42.45, stdev=16.96, samples=291 iops : min= 1, max= 23, avg=10.08, stdev= 4.29, samples=291 lat (msec) : 10=16.18%, 20=18.20%, 50=23.40%, 100=32.65%, 250=9.41% lat (msec) : 500=0.13%, 1000=0.03% cpu : usr=0.08%, sys=2.71%, ctx=5522, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155353: Tue Nov 25 17:08:19 2025 read: IOPS=10, BW=40.1KiB/s (41.1kB/s)(5940KiB/148046msec) clat (msec): min=4, max=742, avg=19.80, stdev=25.94 lat (msec): min=4, max=742, avg=19.80, stdev=25.94 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 13], 60.00th=[ 16], | 70.00th=[ 21], 80.00th=[ 27], 90.00th=[ 37], 95.00th=[ 50], | 99.00th=[ 95], 99.50th=[ 136], 99.90th=[ 205], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 7, max= 127, per=25.24%, avg=40.89, stdev=25.29, samples=285 iops : min= 1, max= 31, avg= 9.69, stdev= 6.33, samples=285 write: IOPS=10, BW=42.9KiB/s (43.9kB/s)(6348KiB/148046msec); 0 zone resets clat (msec): min=23, max=311, avg=74.54, stdev=33.64 lat (msec): min=23, max=311, avg=74.55, stdev=33.64 clat percentiles (msec): | 1.00th=[ 31], 5.00th=[ 37], 10.00th=[ 42], 20.00th=[ 48], | 30.00th=[ 55], 40.00th=[ 61], 50.00th=[ 66], 60.00th=[ 73], | 70.00th=[ 83], 80.00th=[ 96], 90.00th=[ 122], 95.00th=[ 142], | 99.00th=[ 188], 99.50th=[ 201], 99.90th=[ 253], 99.95th=[ 313], | 99.99th=[ 313] bw ( KiB/s): min= 7, max= 96, per=25.07%, avg=42.38, stdev=17.09, samples=293 iops : min= 1, max= 24, avg=10.06, stdev= 4.34, samples=293 lat (msec) : 10=15.17%, 20=18.20%, 50=24.71%, 100=32.26%, 250=9.57% lat (msec) : 500=0.07%, 750=0.03% cpu : usr=0.08%, sys=2.73%, ctx=5678, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155354: Tue Nov 25 17:08:19 2025 read: IOPS=10, BW=40.9KiB/s (41.9kB/s)(6028KiB/147245msec) clat (msec): min=4, max=211, avg=19.92, stdev=18.70 lat (msec): min=4, max=211, avg=19.92, stdev=18.70 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 16], | 70.00th=[ 21], 80.00th=[ 28], 90.00th=[ 40], 95.00th=[ 53], | 99.00th=[ 99], 99.50th=[ 140], 99.90th=[ 161], 99.95th=[ 211], | 99.99th=[ 211] bw ( KiB/s): min= 7, max= 144, per=25.67%, avg=41.59, stdev=25.34, samples=285 iops : min= 1, max= 36, avg= 9.87, stdev= 6.39, samples=285 write: IOPS=10, BW=42.5KiB/s (43.5kB/s)(6260KiB/147245msec); 0 zone resets clat (msec): min=24, max=811, avg=74.69, stdev=38.37 lat (msec): min=24, max=811, avg=74.70, stdev=38.37 clat percentiles (msec): | 1.00th=[ 31], 5.00th=[ 38], 10.00th=[ 43], 20.00th=[ 48], | 30.00th=[ 54], 40.00th=[ 59], 50.00th=[ 65], 60.00th=[ 73], | 70.00th=[ 82], 80.00th=[ 96], 90.00th=[ 123], 95.00th=[ 146], | 99.00th=[ 184], 99.50th=[ 211], 99.90th=[ 266], 99.95th=[ 810], | 99.99th=[ 810] bw ( KiB/s): min= 7, max= 80, per=24.94%, avg=42.14, stdev=17.25, samples=292 iops : min= 1, max= 20, avg=10.00, stdev= 4.37, samples=292 lat (msec) : 10=15.14%, 20=18.75%, 50=24.64%, 100=31.84%, 250=9.54% lat (msec) : 500=0.07%, 1000=0.03% cpu : usr=0.07%, sys=2.81%, ctx=5671, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155355: Tue Nov 25 17:08:19 2025 read: IOPS=10, BW=41.1KiB/s (42.1kB/s)(6052KiB/147103msec) clat (msec): min=4, max=206, avg=19.85, stdev=18.96 lat (msec): min=4, max=206, avg=19.85, stdev=18.96 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 13], 60.00th=[ 16], | 70.00th=[ 21], 80.00th=[ 29], 90.00th=[ 42], 95.00th=[ 53], | 99.00th=[ 90], 99.50th=[ 136], 99.90th=[ 201], 99.95th=[ 207], | 99.99th=[ 207] bw ( KiB/s): min= 7, max= 151, per=25.81%, avg=41.81, stdev=26.18, samples=284 iops : min= 1, max= 37, avg= 9.93, stdev= 6.57, samples=284 write: IOPS=10, BW=42.4KiB/s (43.4kB/s)(6236KiB/147103msec); 0 zone resets clat (msec): min=24, max=802, avg=74.87, stdev=38.15 lat (msec): min=24, max=802, avg=74.88, stdev=38.15 clat percentiles (msec): | 1.00th=[ 32], 5.00th=[ 39], 10.00th=[ 43], 20.00th=[ 48], | 30.00th=[ 54], 40.00th=[ 59], 50.00th=[ 65], 60.00th=[ 74], | 70.00th=[ 83], 80.00th=[ 99], 90.00th=[ 123], 95.00th=[ 140], | 99.00th=[ 188], 99.50th=[ 209], 99.90th=[ 264], 99.95th=[ 802], | 99.99th=[ 802] bw ( KiB/s): min= 7, max= 79, per=24.84%, avg=41.99, stdev=16.87, samples=291 iops : min= 1, max= 19, avg= 9.97, stdev= 4.28, samples=291 lat (msec) : 10=15.98%, 20=17.94%, 50=24.51%, 100=31.48%, 250=10.03% lat (msec) : 500=0.03%, 1000=0.03% cpu : usr=0.08%, sys=2.77%, ctx=5693, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=162KiB/s (166kB/s), 40.1KiB/s-41.1KiB/s (41.1kB/s-42.1kB/s), io=23.4MiB (24.6MB), run=147103-148046msec WRITE: bw=170KiB/s (174kB/s), 42.4KiB/s-42.9KiB/s (43.4kB/s-43.9kB/s), io=24.6MiB (25.7MB), run=147103-148046msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155405: Tue Nov 25 17:08:39 2025 read: IOPS=26, BW=417KiB/s (427kB/s)(5920KiB/14197msec) clat (usec): min=105, max=91994, avg=18336.26, stdev=13560.68 lat (usec): min=106, max=91996, avg=18337.63, stdev=13560.67 clat percentiles (usec): | 1.00th=[ 157], 5.00th=[ 223], 10.00th=[ 318], 20.00th=[ 750], | 30.00th=[12256], 40.00th=[15008], 50.00th=[17695], 60.00th=[20579], | 70.00th=[23987], 80.00th=[28181], 90.00th=[34866], 95.00th=[42206], | 99.00th=[55837], 99.50th=[62129], 99.90th=[91751], 99.95th=[91751], | 99.99th=[91751] bw ( KiB/s): min= 223, max= 731, per=25.09%, avg=418.68, stdev=130.13, samples=28 iops : min= 13, max= 45, avg=25.43, stdev= 8.19, samples=28 write: IOPS=28, BW=449KiB/s (459kB/s)(6368KiB/14197msec); 0 zone resets clat (usec): min=1269, max=127990, avg=18492.34, stdev=18170.58 lat (usec): min=1271, max=127991, avg=18494.47, stdev=18170.65 clat percentiles (usec): | 1.00th=[ 1549], 5.00th=[ 2024], 10.00th=[ 3228], 20.00th=[ 4817], | 30.00th=[ 6390], 40.00th=[ 8717], 50.00th=[ 12387], 60.00th=[ 17433], | 70.00th=[ 22152], 80.00th=[ 30540], 90.00th=[ 41157], 95.00th=[ 53740], | 99.00th=[101188], 99.50th=[108528], 99.90th=[128451], 99.95th=[128451], | 99.99th=[128451] bw ( KiB/s): min= 159, max= 666, per=25.62%, avg=449.89, stdev=139.79, samples=28 iops : min= 9, max= 41, avg=27.39, stdev= 8.82, samples=28 lat (usec) : 250=2.86%, 500=4.43%, 750=2.34%, 1000=0.13% lat (msec) : 2=2.86%, 4=5.47%, 10=16.41%, 20=26.82%, 50=34.51% lat (msec) : 100=3.65%, 250=0.52% cpu : usr=0.13%, sys=21.09%, ctx=1827, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155406: Tue Nov 25 17:08:39 2025 read: IOPS=25, BW=403KiB/s (413kB/s)(5744KiB/14239msec) clat (usec): min=94, max=108315, avg=18223.01, stdev=14618.26 lat (usec): min=94, max=108316, avg=18224.29, stdev=14618.31 clat percentiles (usec): | 1.00th=[ 159], 5.00th=[ 229], 10.00th=[ 318], 20.00th=[ 988], | 30.00th=[ 11863], 40.00th=[ 14091], 50.00th=[ 17171], 60.00th=[ 20317], | 70.00th=[ 23200], 80.00th=[ 28705], 90.00th=[ 35390], 95.00th=[ 41681], | 99.00th=[ 72877], 99.50th=[ 81265], 99.90th=[108528], 99.95th=[108528], | 99.99th=[108528] bw ( KiB/s): min= 53, max= 667, per=25.24%, avg=421.26, stdev=143.70, samples=27 iops : min= 3, max= 41, avg=25.63, stdev= 9.00, samples=27 write: IOPS=28, BW=460KiB/s (471kB/s)(6544KiB/14239msec); 0 zone resets clat (usec): min=1406, max=183873, avg=18436.73, stdev=20730.05 lat (usec): min=1408, max=183877, avg=18440.60, stdev=20729.97 clat percentiles (usec): | 1.00th=[ 1614], 5.00th=[ 2409], 10.00th=[ 3097], 20.00th=[ 4015], | 30.00th=[ 5473], 40.00th=[ 8291], 50.00th=[ 12256], 60.00th=[ 16057], | 70.00th=[ 21365], 80.00th=[ 30016], 90.00th=[ 40109], 95.00th=[ 49546], | 99.00th=[101188], 99.50th=[106431], 99.90th=[183501], 99.95th=[183501], | 99.99th=[183501] bw ( KiB/s): min= 127, max= 736, per=26.28%, avg=461.46, stdev=155.96, samples=28 iops : min= 7, max= 46, avg=28.14, stdev= 9.74, samples=28 lat (usec) : 100=0.13%, 250=2.47%, 500=4.82%, 750=1.43%, 1000=0.52% lat (msec) : 2=2.99%, 4=8.59%, 10=14.58%, 20=27.86%, 50=32.68% lat (msec) : 100=3.12%, 250=0.78% cpu : usr=0.17%, sys=21.00%, ctx=1828, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155407: Tue Nov 25 17:08:39 2025 read: IOPS=27, BW=437KiB/s (447kB/s)(6256KiB/14320msec) clat (usec): min=104, max=126709, avg=18635.08, stdev=14993.20 lat (usec): min=104, max=126710, avg=18636.43, stdev=14993.27 clat percentiles (usec): | 1.00th=[ 126], 5.00th=[ 225], 10.00th=[ 314], 20.00th=[ 619], | 30.00th=[ 11469], 40.00th=[ 15008], 50.00th=[ 17957], 60.00th=[ 21103], | 70.00th=[ 24773], 80.00th=[ 29754], 90.00th=[ 36963], 95.00th=[ 44303], | 99.00th=[ 56361], 99.50th=[ 76022], 99.90th=[126354], 99.95th=[126354], | 99.99th=[126354] bw ( KiB/s): min= 159, max= 727, per=26.34%, avg=439.64, stdev=123.96, samples=28 iops : min= 9, max= 45, avg=26.75, stdev= 7.69, samples=28 write: IOPS=26, BW=421KiB/s (431kB/s)(6032KiB/14320msec); 0 zone resets clat (usec): min=1201, max=118652, avg=18340.11, stdev=17621.76 lat (usec): min=1202, max=118652, avg=18344.83, stdev=17620.04 clat percentiles (usec): | 1.00th=[ 1565], 5.00th=[ 2245], 10.00th=[ 3195], 20.00th=[ 4080], | 30.00th=[ 5735], 40.00th=[ 8455], 50.00th=[ 13566], 60.00th=[ 18220], | 70.00th=[ 23462], 80.00th=[ 30802], 90.00th=[ 39060], 95.00th=[ 47449], | 99.00th=[104334], 99.50th=[113771], 99.90th=[119014], 99.95th=[119014], | 99.99th=[119014] bw ( KiB/s): min= 159, max= 731, per=23.68%, avg=415.89, stdev=142.84, samples=28 iops : min= 9, max= 45, avg=25.25, stdev= 8.98, samples=28 lat (usec) : 250=3.52%, 500=5.99%, 750=1.43%, 1000=0.65% lat (msec) : 2=2.47%, 4=8.20%, 10=12.24%, 20=25.78%, 50=36.46% lat (msec) : 100=2.60%, 250=0.65% cpu : usr=0.17%, sys=20.21%, ctx=1810, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155408: Tue Nov 25 17:08:39 2025 read: IOPS=26, BW=420KiB/s (431kB/s)(6032KiB/14346msec) clat (usec): min=118, max=77833, avg=18993.13, stdev=14555.86 lat (usec): min=120, max=77834, avg=18994.44, stdev=14555.92 clat percentiles (usec): | 1.00th=[ 135], 5.00th=[ 186], 10.00th=[ 269], 20.00th=[ 594], | 30.00th=[12649], 40.00th=[15139], 50.00th=[17957], 60.00th=[21365], | 70.00th=[25560], 80.00th=[30802], 90.00th=[38536], 95.00th=[44303], | 99.00th=[60031], 99.50th=[70779], 99.90th=[78119], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 95, max= 763, per=25.07%, avg=418.46, stdev=161.53, samples=28 iops : min= 5, max= 47, avg=25.43, stdev=10.11, samples=28 write: IOPS=27, BW=436KiB/s (447kB/s)(6256KiB/14346msec); 0 zone resets clat (usec): min=1496, max=135544, avg=18090.98, stdev=19678.70 lat (usec): min=1497, max=135546, avg=18093.32, stdev=19678.75 clat percentiles (usec): | 1.00th=[ 1647], 5.00th=[ 2212], 10.00th=[ 2737], 20.00th=[ 3982], | 30.00th=[ 5145], 40.00th=[ 7635], 50.00th=[ 11076], 60.00th=[ 16057], | 70.00th=[ 21365], 80.00th=[ 28967], 90.00th=[ 41681], 95.00th=[ 52167], | 99.00th=[108528], 99.50th=[120062], 99.90th=[135267], 99.95th=[135267], | 99.99th=[135267] bw ( KiB/s): min= 127, max= 728, per=24.64%, avg=432.64, stdev=126.88, samples=28 iops : min= 7, max= 45, avg=26.32, stdev= 7.98, samples=28 lat (usec) : 250=4.56%, 500=4.69%, 750=1.17%, 1000=0.52% lat (msec) : 2=2.47%, 4=8.72%, 10=14.32%, 20=25.52%, 50=33.59% lat (msec) : 100=3.91%, 250=0.52% cpu : usr=0.18%, sys=20.32%, ctx=1810, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1670KiB/s (1710kB/s), 403KiB/s-437KiB/s (413kB/s-447kB/s), io=23.4MiB (24.5MB), run=14197-14346msec WRITE: bw=1757KiB/s (1799kB/s), 421KiB/s-460KiB/s (431kB/s-471kB/s), io=24.6MiB (25.8MB), run=14197-14346msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155400: Tue Nov 25 17:08:59 2025 read: IOPS=10, BW=173KiB/s (177kB/s)(5920KiB/34229msec) clat (msec): min=5, max=140, avg=18.37, stdev=18.19 lat (msec): min=5, max=140, avg=18.37, stdev=18.19 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 14], | 70.00th=[ 16], 80.00th=[ 23], 90.00th=[ 36], 95.00th=[ 54], | 99.00th=[ 105], 99.50th=[ 108], 99.90th=[ 142], 99.95th=[ 142], | 99.99th=[ 142] bw ( KiB/s): min= 31, max= 446, per=24.78%, avg=172.71, stdev=82.82, samples=68 iops : min= 1, max= 27, avg=10.19, stdev= 5.18, samples=68 write: IOPS=11, BW=186KiB/s (191kB/s)(6368KiB/34229msec); 0 zone resets clat (msec): min=28, max=198, avg=68.68, stdev=24.81 lat (msec): min=28, max=198, avg=68.69, stdev=24.81 clat percentiles (msec): | 1.00th=[ 33], 5.00th=[ 40], 10.00th=[ 44], 20.00th=[ 49], | 30.00th=[ 54], 40.00th=[ 59], 50.00th=[ 64], 60.00th=[ 68], | 70.00th=[ 77], 80.00th=[ 86], 90.00th=[ 104], 95.00th=[ 115], | 99.00th=[ 148], 99.50th=[ 188], 99.90th=[ 199], 99.95th=[ 199], | 99.99th=[ 199] bw ( KiB/s): min= 63, max= 320, per=25.32%, avg=185.57, stdev=61.63, samples=68 iops : min= 3, max= 20, avg=10.99, stdev= 3.94, samples=68 lat (msec) : 10=17.58%, 20=19.27%, 50=20.18%, 100=36.59%, 250=6.38% cpu : usr=0.08%, sys=3.88%, ctx=1543, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155401: Tue Nov 25 17:08:59 2025 read: IOPS=10, BW=167KiB/s (171kB/s)(5744KiB/34359msec) clat (msec): min=5, max=183, avg=17.13, stdev=17.62 lat (msec): min=5, max=183, avg=17.14, stdev=17.62 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 15], 80.00th=[ 23], 90.00th=[ 37], 95.00th=[ 49], | 99.00th=[ 86], 99.50th=[ 115], 99.90th=[ 184], 99.95th=[ 184], | 99.99th=[ 184] bw ( KiB/s): min= 31, max= 415, per=24.19%, avg=168.58, stdev=92.93, samples=67 iops : min= 1, max= 25, avg= 9.93, stdev= 5.82, samples=67 write: IOPS=11, BW=190KiB/s (195kB/s)(6544KiB/34359msec); 0 zone resets clat (msec): min=27, max=194, avg=68.69, stdev=27.85 lat (msec): min=27, max=194, avg=68.69, stdev=27.85 clat percentiles (msec): | 1.00th=[ 33], 5.00th=[ 37], 10.00th=[ 43], 20.00th=[ 47], | 30.00th=[ 53], 40.00th=[ 58], 50.00th=[ 63], 60.00th=[ 67], | 70.00th=[ 75], 80.00th=[ 85], 90.00th=[ 104], 95.00th=[ 126], | 99.00th=[ 169], 99.50th=[ 188], 99.90th=[ 194], 99.95th=[ 194], | 99.99th=[ 194] bw ( KiB/s): min= 63, max= 319, per=25.90%, avg=189.82, stdev=63.67, samples=68 iops : min= 3, max= 19, avg=11.25, stdev= 4.07, samples=68 lat (msec) : 10=19.92%, 20=16.41%, 50=21.48%, 100=35.55%, 250=6.64% cpu : usr=0.10%, sys=3.47%, ctx=1472, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155402: Tue Nov 25 17:08:59 2025 read: IOPS=11, BW=188KiB/s (192kB/s)(6256KiB/33302msec) clat (msec): min=4, max=184, avg=17.93, stdev=19.12 lat (msec): min=4, max=184, avg=17.93, stdev=19.12 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 13], | 70.00th=[ 15], 80.00th=[ 19], 90.00th=[ 40], 95.00th=[ 58], | 99.00th=[ 99], 99.50th=[ 131], 99.90th=[ 184], 99.95th=[ 184], | 99.99th=[ 184] bw ( KiB/s): min= 31, max= 477, per=27.02%, avg=188.36, stdev=116.47, samples=66 iops : min= 1, max= 29, avg=11.17, stdev= 7.26, samples=66 write: IOPS=11, BW=181KiB/s (185kB/s)(6032KiB/33302msec); 0 zone resets clat (msec): min=30, max=195, avg=69.45, stdev=27.22 lat (msec): min=30, max=195, avg=69.45, stdev=27.23 clat percentiles (msec): | 1.00th=[ 33], 5.00th=[ 38], 10.00th=[ 42], 20.00th=[ 48], | 30.00th=[ 54], 40.00th=[ 58], 50.00th=[ 63], 60.00th=[ 70], | 70.00th=[ 75], 80.00th=[ 88], 90.00th=[ 108], 95.00th=[ 125], | 99.00th=[ 163], 99.50th=[ 186], 99.90th=[ 197], 99.95th=[ 197], | 99.99th=[ 197] bw ( KiB/s): min= 64, max= 352, per=24.65%, avg=180.70, stdev=62.35, samples=66 iops : min= 4, max= 22, avg=10.68, stdev= 4.00, samples=66 lat (msec) : 10=20.31%, 20=20.70%, 50=17.97%, 100=34.24%, 250=6.77% cpu : usr=0.10%, sys=3.74%, ctx=1476, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155403: Tue Nov 25 17:08:59 2025 read: IOPS=11, BW=178KiB/s (183kB/s)(6032KiB/33809msec) clat (msec): min=4, max=151, avg=17.56, stdev=16.87 lat (msec): min=4, max=151, avg=17.56, stdev=16.87 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 16], 80.00th=[ 23], 90.00th=[ 37], 95.00th=[ 52], | 99.00th=[ 94], 99.50th=[ 117], 99.90th=[ 153], 99.95th=[ 153], | 99.99th=[ 153] bw ( KiB/s): min= 31, max= 448, per=26.18%, avg=182.46, stdev=92.96, samples=65 iops : min= 1, max= 28, avg=10.80, stdev= 5.93, samples=65 write: IOPS=11, BW=185KiB/s (189kB/s)(6256KiB/33809msec); 0 zone resets clat (msec): min=31, max=181, avg=69.41, stdev=26.17 lat (msec): min=31, max=181, avg=69.42, stdev=26.18 clat percentiles (msec): | 1.00th=[ 33], 5.00th=[ 41], 10.00th=[ 44], 20.00th=[ 48], | 30.00th=[ 54], 40.00th=[ 59], 50.00th=[ 65], 60.00th=[ 70], | 70.00th=[ 77], 80.00th=[ 87], 90.00th=[ 103], 95.00th=[ 124], | 99.00th=[ 163], 99.50th=[ 169], 99.90th=[ 182], 99.95th=[ 182], | 99.99th=[ 182] bw ( KiB/s): min= 63, max= 320, per=25.23%, avg=184.96, stdev=61.71, samples=67 iops : min= 3, max= 20, avg=10.96, stdev= 3.92, samples=67 lat (msec) : 10=17.32%, 20=19.66%, 50=21.35%, 100=36.07%, 250=5.60% cpu : usr=0.09%, sys=3.59%, ctx=1490, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=697KiB/s (714kB/s), 167KiB/s-188KiB/s (171kB/s-192kB/s), io=23.4MiB (24.5MB), run=33302-34359msec WRITE: bw=733KiB/s (751kB/s), 181KiB/s-190KiB/s (185kB/s-195kB/s), io=24.6MiB (25.8MB), run=33302-34359msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155418: Tue Nov 25 17:09:12 2025 read: IOPS=1, BW=1817KiB/s (1861kB/s)(7168KiB/3944msec) clat (msec): min=64, max=1889, avg=450.31, stdev=660.77 lat (msec): min=64, max=1889, avg=450.31, stdev=660.77 clat percentiles (msec): | 1.00th=[ 65], 5.00th=[ 65], 10.00th=[ 65], 20.00th=[ 71], | 30.00th=[ 80], 40.00th=[ 80], 50.00th=[ 109], 60.00th=[ 468], | 70.00th=[ 468], 80.00th=[ 472], 90.00th=[ 1888], 95.00th=[ 1888], | 99.00th=[ 1888], 99.50th=[ 1888], 99.90th=[ 1888], 99.95th=[ 1888], | 99.99th=[ 1888] bw ( KiB/s): min= 1809, max= 5515, per=100.00%, avg=3806.67, stdev=1869.86, samples=3 iops : min= 1, max= 5, avg= 3.33, stdev= 2.08, samples=3 write: IOPS=1, BW=1298KiB/s (1329kB/s)(5120KiB/3944msec); 0 zone resets clat (msec): min=103, max=227, avg=152.62, stdev=49.52 lat (msec): min=103, max=227, avg=152.74, stdev=49.54 clat percentiles (msec): | 1.00th=[ 104], 5.00th=[ 104], 10.00th=[ 104], 20.00th=[ 104], | 30.00th=[ 122], 40.00th=[ 122], 50.00th=[ 138], 60.00th=[ 138], | 70.00th=[ 176], 80.00th=[ 176], 90.00th=[ 228], 95.00th=[ 228], | 99.00th=[ 228], 99.50th=[ 228], 99.90th=[ 228], 99.95th=[ 228], | 99.99th=[ 228] bw ( KiB/s): min= 1809, max= 4087, per=49.50%, avg=2445.50, stdev=1099.50, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 100=25.00%, 250=50.00%, 500=16.67%, 2000=8.33% cpu : usr=0.00%, sys=5.81%, ctx=60, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155419: Tue Nov 25 17:09:12 2025 read: IOPS=0, BW=353KiB/s (361kB/s)(2048KiB/5804msec) clat (msec): min=81, max=114, avg=98.05, stdev=22.74 lat (msec): min=81, max=114, avg=98.05, stdev=22.74 clat percentiles (msec): | 1.00th=[ 83], 5.00th=[ 83], 10.00th=[ 83], 20.00th=[ 83], | 30.00th=[ 83], 40.00th=[ 83], 50.00th=[ 83], 60.00th=[ 114], | 70.00th=[ 114], 80.00th=[ 114], 90.00th=[ 114], 95.00th=[ 114], | 99.00th=[ 114], 99.50th=[ 114], 99.90th=[ 114], 99.95th=[ 114], | 99.99th=[ 114] bw ( KiB/s): min= 1865, max= 2003, per=54.82%, avg=1934.00, stdev=97.58, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1764KiB/s (1807kB/s)(10.0MiB/5804msec); 0 zone resets clat (msec): min=141, max=2248, avg=544.77, stdev=711.80 lat (msec): min=141, max=2248, avg=544.86, stdev=711.79 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 144], | 30.00th=[ 153], 40.00th=[ 184], 50.00th=[ 209], 60.00th=[ 220], | 70.00th=[ 271], 80.00th=[ 472], 90.00th=[ 1401], 95.00th=[ 2265], | 99.00th=[ 2265], 99.50th=[ 2265], 99.90th=[ 2265], 99.95th=[ 2265], | 99.99th=[ 2265] bw ( KiB/s): min= 2003, max= 6131, per=73.78%, avg=3644.80, stdev=2226.09, samples=5 iops : min= 1, max= 5, avg= 2.80, stdev= 2.05, samples=5 lat (msec) : 100=8.33%, 250=58.33%, 500=16.67%, 2000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=7.13%, ctx=255, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155420: Tue Nov 25 17:09:12 2025 read: IOPS=1, BW=1087KiB/s (1113kB/s)(6144KiB/5653msec) clat (msec): min=93, max=312, avg=192.60, stdev=82.06 lat (msec): min=93, max=312, avg=192.60, stdev=82.06 clat percentiles (msec): | 1.00th=[ 93], 5.00th=[ 93], 10.00th=[ 93], 20.00th=[ 108], | 30.00th=[ 108], 40.00th=[ 197], 50.00th=[ 197], 60.00th=[ 211], | 70.00th=[ 236], 80.00th=[ 236], 90.00th=[ 313], 95.00th=[ 313], | 99.00th=[ 313], 99.50th=[ 313], 99.90th=[ 313], 99.95th=[ 313], | 99.99th=[ 313] bw ( KiB/s): min= 1809, max= 6023, per=93.35%, avg=3293.33, stdev=2366.98, samples=3 iops : min= 1, max= 5, avg= 2.67, stdev= 2.08, samples=3 write: IOPS=1, BW=1087KiB/s (1113kB/s)(6144KiB/5653msec); 0 zone resets clat (msec): min=121, max=2761, avg=724.37, stdev=1011.51 lat (msec): min=121, max=2761, avg=724.47, stdev=1011.49 clat percentiles (msec): | 1.00th=[ 122], 5.00th=[ 122], 10.00th=[ 122], 20.00th=[ 264], | 30.00th=[ 264], 40.00th=[ 271], 50.00th=[ 271], 60.00th=[ 313], | 70.00th=[ 617], 80.00th=[ 617], 90.00th=[ 2769], 95.00th=[ 2769], | 99.00th=[ 2769], 99.50th=[ 2769], 99.90th=[ 2769], 99.95th=[ 2769], | 99.99th=[ 2769] bw ( KiB/s): min= 1809, max= 3737, per=48.36%, avg=2389.00, stdev=903.41, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 100=8.33%, 250=41.67%, 500=33.33%, 750=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=7.20%, ctx=100, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155421: Tue Nov 25 17:09:12 2025 read: IOPS=1, BW=1108KiB/s (1134kB/s)(5120KiB/4623msec) clat (msec): min=83, max=175, avg=133.01, stdev=42.01 lat (msec): min=83, max=175, avg=133.02, stdev=42.01 clat percentiles (msec): | 1.00th=[ 84], 5.00th=[ 84], 10.00th=[ 84], 20.00th=[ 84], | 30.00th=[ 92], 40.00th=[ 92], 50.00th=[ 157], 60.00th=[ 157], | 70.00th=[ 159], 80.00th=[ 159], 90.00th=[ 176], 95.00th=[ 176], | 99.00th=[ 176], 99.50th=[ 176], 99.90th=[ 176], 99.95th=[ 176], | 99.99th=[ 176] bw ( KiB/s): min= 7236, max= 7236, per=100.00%, avg=7236.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=1, BW=1551KiB/s (1588kB/s)(7168KiB/4623msec); 0 zone resets clat (msec): min=152, max=2275, avg=556.63, stdev=773.77 lat (msec): min=152, max=2275, avg=556.77, stdev=773.75 clat percentiles (msec): | 1.00th=[ 153], 5.00th=[ 153], 10.00th=[ 153], 20.00th=[ 165], | 30.00th=[ 165], 40.00th=[ 165], 50.00th=[ 220], 60.00th=[ 326], | 70.00th=[ 326], 80.00th=[ 592], 90.00th=[ 2265], 95.00th=[ 2265], | 99.00th=[ 2265], 99.50th=[ 2265], 99.90th=[ 2265], 99.95th=[ 2265], | 99.99th=[ 2265] bw ( KiB/s): min= 1950, max= 4087, per=57.25%, avg=2828.00, stdev=1113.87, samples=5 iops : min= 1, max= 3, avg= 2.20, stdev= 0.84, samples=5 lat (msec) : 100=16.67%, 250=58.33%, 500=8.33%, 750=8.33%, >=2000=8.33% cpu : usr=0.02%, sys=8.16%, ctx=238, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3529KiB/s (3613kB/s), 353KiB/s-1817KiB/s (361kB/s-1861kB/s), io=20.0MiB (20.0MB), run=3944-5804msec WRITE: bw=4940KiB/s (5059kB/s), 1087KiB/s-1764KiB/s (1113kB/s-1807kB/s), io=28.0MiB (29.4MB), run=3944-5804msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155423: Tue Nov 25 17:09:12 2025 read: IOPS=1, BW=1414KiB/s (1448kB/s)(7168KiB/5068msec) clat (usec): min=1548, max=2005.9k, avg=506876.87, stdev=686759.37 lat (usec): min=1550, max=2005.9k, avg=506878.65, stdev=686759.77 clat percentiles (usec): | 1.00th=[ 1549], 5.00th=[ 1549], 10.00th=[ 1549], | 20.00th=[ 141558], 30.00th=[ 212861], 40.00th=[ 212861], | 50.00th=[ 233833], 60.00th=[ 354419], 70.00th=[ 354419], | 80.00th=[ 599786], 90.00th=[2004878], 95.00th=[2004878], | 99.00th=[2004878], 99.50th=[2004878], 99.90th=[2004878], | 99.95th=[2004878], 99.99th=[2004878] bw ( KiB/s): min= 2019, max= 4096, per=94.65%, avg=3333.67, stdev=1143.37, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=0, BW=1010KiB/s (1035kB/s)(5120KiB/5068msec); 0 zone resets clat (msec): min=131, max=572, avg=279.93, stdev=178.17 lat (msec): min=131, max=572, avg=280.03, stdev=178.16 clat percentiles (msec): | 1.00th=[ 132], 5.00th=[ 132], 10.00th=[ 132], 20.00th=[ 132], | 30.00th=[ 142], 40.00th=[ 142], 50.00th=[ 271], 60.00th=[ 271], | 70.00th=[ 284], 80.00th=[ 284], 90.00th=[ 575], 95.00th=[ 575], | 99.00th=[ 575], 99.50th=[ 575], 99.90th=[ 575], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 1928, max= 5862, per=78.99%, avg=3895.00, stdev=2781.76, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 lat (msec) : 2=8.33%, 250=41.67%, 500=25.00%, 750=16.67%, >=2000=8.33% cpu : usr=0.02%, sys=21.71%, ctx=1000, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155424: Tue Nov 25 17:09:12 2025 read: IOPS=0, BW=352KiB/s (361kB/s)(2048KiB/5814msec) clat (msec): min=2, max=165, avg=84.38, stdev=115.24 lat (msec): min=2, max=165, avg=84.38, stdev=115.24 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 3], 20.00th=[ 3], | 30.00th=[ 3], 40.00th=[ 3], 50.00th=[ 3], 60.00th=[ 167], | 70.00th=[ 167], 80.00th=[ 167], 90.00th=[ 167], 95.00th=[ 167], | 99.00th=[ 167], 99.50th=[ 167], 99.90th=[ 167], 99.95th=[ 167], | 99.99th=[ 167] bw ( KiB/s): min= 1865, max= 1954, per=54.22%, avg=1909.50, stdev=62.93, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1761KiB/s (1804kB/s)(10.0MiB/5814msec); 0 zone resets clat (msec): min=133, max=2126, avg=537.68, stdev=614.79 lat (msec): min=133, max=2126, avg=538.34, stdev=614.45 clat percentiles (msec): | 1.00th=[ 134], 5.00th=[ 134], 10.00th=[ 134], 20.00th=[ 194], | 30.00th=[ 201], 40.00th=[ 207], 50.00th=[ 215], 60.00th=[ 249], | 70.00th=[ 435], 80.00th=[ 693], 90.00th=[ 927], 95.00th=[ 2123], | 99.00th=[ 2123], 99.50th=[ 2123], 99.90th=[ 2123], 99.95th=[ 2123], | 99.99th=[ 2123] bw ( KiB/s): min= 1943, max= 4071, per=54.38%, avg=2681.67, stdev=1064.65, samples=6 iops : min= 1, max= 3, avg= 1.83, stdev= 0.98, samples=6 lat (msec) : 4=8.33%, 250=58.33%, 500=8.33%, 750=8.33%, 1000=8.33% lat (msec) : >=2000=8.33% cpu : usr=0.00%, sys=17.93%, ctx=320, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155425: Tue Nov 25 17:09:12 2025 read: IOPS=1, BW=1061KiB/s (1086kB/s)(6144KiB/5792msec) clat (usec): min=1081, max=363177, avg=210877.09, stdev=121952.11 lat (usec): min=1083, max=363179, avg=210878.86, stdev=121952.18 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 1090], 10.00th=[ 1090], 20.00th=[177210], | 30.00th=[177210], 40.00th=[196084], 50.00th=[196084], 60.00th=[258999], | 70.00th=[270533], 80.00th=[270533], 90.00th=[362808], 95.00th=[362808], | 99.00th=[362808], 99.50th=[362808], 99.90th=[362808], 99.95th=[362808], | 99.99th=[362808] bw ( KiB/s): min= 1943, max= 4071, per=71.00%, avg=2500.75, stdev=1047.64, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=1, BW=1061KiB/s (1086kB/s)(6144KiB/5792msec); 0 zone resets clat (msec): min=134, max=3091, avg=726.09, stdev=1163.94 lat (msec): min=134, max=3091, avg=726.18, stdev=1163.94 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 186], | 30.00th=[ 186], 40.00th=[ 190], 50.00th=[ 190], 60.00th=[ 330], | 70.00th=[ 430], 80.00th=[ 430], 90.00th=[ 3104], 95.00th=[ 3104], | 99.00th=[ 3104], 99.50th=[ 3104], 99.90th=[ 3104], 99.95th=[ 3104], | 99.99th=[ 3104] bw ( KiB/s): min= 1943, max= 4071, per=59.14%, avg=2916.00, stdev=1127.62, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 lat (msec) : 2=8.33%, 250=41.67%, 500=41.67%, >=2000=8.33% cpu : usr=0.02%, sys=18.24%, ctx=381, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155426: Tue Nov 25 17:09:12 2025 read: IOPS=1, BW=1041KiB/s (1066kB/s)(5120KiB/4918msec) clat (usec): min=817, max=364931, avg=161728.14, stdev=159889.22 lat (usec): min=819, max=364933, avg=161729.69, stdev=159889.30 clat percentiles (usec): | 1.00th=[ 816], 5.00th=[ 816], 10.00th=[ 816], 20.00th=[ 816], | 30.00th=[ 963], 40.00th=[ 963], 50.00th=[187696], 60.00th=[187696], | 70.00th=[254804], 80.00th=[254804], 90.00th=[367002], 95.00th=[367002], | 99.00th=[367002], 99.50th=[367002], 99.90th=[367002], 99.95th=[367002], | 99.99th=[367002] bw ( KiB/s): min= 3886, max= 3908, per=100.00%, avg=3897.00, stdev=15.56, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1458KiB/s (1492kB/s)(7168KiB/4918msec); 0 zone resets clat (msec): min=119, max=2499, avg=581.23, stdev=849.87 lat (msec): min=119, max=2499, avg=581.31, stdev=849.87 clat percentiles (msec): | 1.00th=[ 120], 5.00th=[ 120], 10.00th=[ 120], 20.00th=[ 220], | 30.00th=[ 268], 40.00th=[ 268], 50.00th=[ 271], 60.00th=[ 309], | 70.00th=[ 309], 80.00th=[ 380], 90.00th=[ 2500], 95.00th=[ 2500], | 99.00th=[ 2500], 99.50th=[ 2500], 99.90th=[ 2500], 99.95th=[ 2500], | 99.99th=[ 2500] bw ( KiB/s): min= 1805, max= 3856, per=48.84%, avg=2408.50, stdev=969.15, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (usec) : 1000=16.67% lat (msec) : 250=25.00%, 500=50.00%, >=2000=8.33% cpu : usr=0.00%, sys=24.12%, ctx=253, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3523KiB/s (3607kB/s), 352KiB/s-1414KiB/s (361kB/s-1448kB/s), io=20.0MiB (20.0MB), run=4918-5814msec WRITE: bw=4932KiB/s (5050kB/s), 1010KiB/s-1761KiB/s (1035kB/s-1804kB/s), io=28.0MiB (29.4MB), run=4918-5814msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155433: Tue Nov 25 17:09:33 2025 read: IOPS=0, BW=3074KiB/s (3147kB/s)(12.0MiB/3998msec) clat (msec): min=655, max=1863, avg=1151.05, stdev=632.83 lat (msec): min=655, max=1863, avg=1151.05, stdev=632.83 clat percentiles (msec): | 1.00th=[ 659], 5.00th=[ 659], 10.00th=[ 659], 20.00th=[ 659], | 30.00th=[ 659], 40.00th=[ 936], 50.00th=[ 936], 60.00th=[ 936], | 70.00th=[ 1871], 80.00th=[ 1871], 90.00th=[ 1871], 95.00th=[ 1871], | 99.00th=[ 1871], 99.50th=[ 1871], 99.90th=[ 1871], 99.95th=[ 1871], | 99.99th=[ 1871] bw ( KiB/s): min= 6860, max= 8159, per=100.00%, avg=7509.50, stdev=918.53, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.30%, sys=6.83%, ctx=56, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155434: Tue Nov 25 17:09:33 2025 read: IOPS=0, BW=626KiB/s (641kB/s)(4096KiB/6540msec) clat (nsec): min=392785k, max=392785k, avg=392784714.00, stdev= 0.00 lat (nsec): min=392788k, max=392788k, avg=392787701.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 393], 5.00th=[ 393], 10.00th=[ 393], 20.00th=[ 393], | 30.00th=[ 393], 40.00th=[ 393], 50.00th=[ 393], 60.00th=[ 393], | 70.00th=[ 393], 80.00th=[ 393], 90.00th=[ 393], 95.00th=[ 393], | 99.00th=[ 393], 99.50th=[ 393], 99.90th=[ 393], 99.95th=[ 393], | 99.99th=[ 393] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1253KiB/s (1283kB/s)(8192KiB/6540msec); 0 zone resets clat (msec): min=1509, max=4201, avg=2855.39, stdev=1903.36 lat (msec): min=1509, max=4201, avg=2855.68, stdev=1903.52 clat percentiles (msec): | 1.00th=[ 1502], 5.00th=[ 1502], 10.00th=[ 1502], 20.00th=[ 1502], | 30.00th=[ 1502], 40.00th=[ 1502], 50.00th=[ 1502], 60.00th=[ 4212], | 70.00th=[ 4212], 80.00th=[ 4212], 90.00th=[ 4212], 95.00th=[ 4212], | 99.00th=[ 4212], 99.50th=[ 4212], 99.90th=[ 4212], 99.95th=[ 4212], | 99.99th=[ 4212] bw ( KiB/s): min= 6860, max= 6860, per=100.00%, avg=6860.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.06%, sys=2.28%, ctx=75, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155435: Tue Nov 25 17:09:33 2025 write: IOPS=0, BW=1931KiB/s (1977kB/s)(12.0MiB/6365msec); 0 zone resets clat (msec): min=1581, max=2659, avg=1971.17, stdev=597.70 lat (msec): min=1581, max=2659, avg=1971.54, stdev=597.71 clat percentiles (msec): | 1.00th=[ 1586], 5.00th=[ 1586], 10.00th=[ 1586], 20.00th=[ 1586], | 30.00th=[ 1586], 40.00th=[ 1670], 50.00th=[ 1670], 60.00th=[ 1670], | 70.00th=[ 2668], 80.00th=[ 2668], 90.00th=[ 2668], 95.00th=[ 2668], | 99.00th=[ 2668], 99.50th=[ 2668], 99.90th=[ 2668], 99.95th=[ 2668], | 99.99th=[ 2668] bw ( KiB/s): min= 6860, max= 7787, per=100.00%, avg=7323.50, stdev=655.49, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 2000=66.67%, >=2000=33.33% cpu : usr=0.13%, sys=3.87%, ctx=69, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155436: Tue Nov 25 17:09:33 2025 write: IOPS=0, BW=3250KiB/s (3328kB/s)(12.0MiB/3781msec); 0 zone resets clat (msec): min=669, max=1503, avg=1115.41, stdev=419.81 lat (msec): min=669, max=1503, avg=1122.15, stdev=421.38 clat percentiles (msec): | 1.00th=[ 667], 5.00th=[ 667], 10.00th=[ 667], 20.00th=[ 667], | 30.00th=[ 667], 40.00th=[ 1167], 50.00th=[ 1167], 60.00th=[ 1167], | 70.00th=[ 1502], 80.00th=[ 1502], 90.00th=[ 1502], 95.00th=[ 1502], | 99.00th=[ 1502], 99.50th=[ 1502], 99.90th=[ 1502], 99.95th=[ 1502], | 99.99th=[ 1502] bw ( KiB/s): min= 7907, max= 8159, per=100.00%, avg=8033.00, stdev=178.19, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.03%, sys=4.31%, ctx=68, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2505KiB/s (2565kB/s), 626KiB/s-3074KiB/s (641kB/s-3147kB/s), io=16.0MiB (16.8MB), run=3998-6540msec WRITE: bw=5010KiB/s (5131kB/s), 1253KiB/s-3250KiB/s (1283kB/s-3328kB/s), io=32.0MiB (33.6MB), run=3781-6540msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155438: Tue Nov 25 17:09:34 2025 read: IOPS=0, BW=3147KiB/s (3222kB/s)(12.0MiB/3905msec) clat (msec): min=1203, max=1402, avg=1301.12, stdev=99.20 lat (msec): min=1203, max=1402, avg=1301.12, stdev=99.20 clat percentiles (msec): | 1.00th=[ 1200], 5.00th=[ 1200], 10.00th=[ 1200], 20.00th=[ 1200], | 30.00th=[ 1200], 40.00th=[ 1301], 50.00th=[ 1301], 60.00th=[ 1301], | 70.00th=[ 1401], 80.00th=[ 1401], 90.00th=[ 1401], 95.00th=[ 1401], | 99.00th=[ 1401], 99.50th=[ 1401], 99.90th=[ 1401], 99.95th=[ 1401], | 99.99th=[ 1401] bw ( KiB/s): min= 7892, max= 8047, per=100.00%, avg=7969.50, stdev=109.60, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 2000=100.00% cpu : usr=0.08%, sys=10.02%, ctx=294, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155439: Tue Nov 25 17:09:34 2025 read: IOPS=0, BW=785KiB/s (804kB/s)(4096KiB/5215msec) clat (nsec): min=1541.9M, max=1541.9M, avg=1541867946.00, stdev= 0.00 lat (nsec): min=1541.9M, max=1541.9M, avg=1541869362.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1536], 5.00th=[ 1536], 10.00th=[ 1536], 20.00th=[ 1536], | 30.00th=[ 1536], 40.00th=[ 1536], 50.00th=[ 1536], 60.00th=[ 1536], | 70.00th=[ 1536], 80.00th=[ 1536], 90.00th=[ 1536], 95.00th=[ 1536], | 99.00th=[ 1536], 99.50th=[ 1536], 99.90th=[ 1536], 99.95th=[ 1536], | 99.99th=[ 1536] bw ( KiB/s): min= 8000, max= 8000, per=100.00%, avg=8000.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1571KiB/s (1609kB/s)(8192KiB/5215msec); 0 zone resets clat (msec): min=1438, max=2204, avg=1821.38, stdev=541.81 lat (msec): min=1466, max=2205, avg=1835.74, stdev=522.26 clat percentiles (msec): | 1.00th=[ 1435], 5.00th=[ 1435], 10.00th=[ 1435], 20.00th=[ 1435], | 30.00th=[ 1435], 40.00th=[ 1435], 50.00th=[ 1435], 60.00th=[ 2198], | 70.00th=[ 2198], 80.00th=[ 2198], 90.00th=[ 2198], 95.00th=[ 2198], | 99.00th=[ 2198], 99.50th=[ 2198], 99.90th=[ 2198], 99.95th=[ 2198], | 99.99th=[ 2198] bw ( KiB/s): min= 8126, max= 8126, per=100.00%, avg=8126.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 2000=66.67%, >=2000=33.33% cpu : usr=0.29%, sys=3.24%, ctx=97, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155440: Tue Nov 25 17:09:34 2025 write: IOPS=0, BW=1994KiB/s (2042kB/s)(12.0MiB/6162msec); 0 zone resets clat (msec): min=537, max=3608, avg=2053.23, stdev=1535.94 lat (msec): min=537, max=3608, avg=2053.50, stdev=1535.95 clat percentiles (msec): | 1.00th=[ 542], 5.00th=[ 542], 10.00th=[ 542], 20.00th=[ 542], | 30.00th=[ 542], 40.00th=[ 2022], 50.00th=[ 2022], 60.00th=[ 2022], | 70.00th=[ 3608], 80.00th=[ 3608], 90.00th=[ 3608], 95.00th=[ 3608], | 99.00th=[ 3608], 99.50th=[ 3608], 99.90th=[ 3608], 99.95th=[ 3608], | 99.99th=[ 3608] bw ( KiB/s): min= 7801, max= 7922, per=100.00%, avg=7861.50, stdev=85.56, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, >=2000=66.67% cpu : usr=0.21%, sys=2.40%, ctx=45, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155441: Tue Nov 25 17:09:34 2025 write: IOPS=0, BW=3137KiB/s (3212kB/s)(12.0MiB/3917msec); 0 zone resets clat (msec): min=686, max=1794, avg=1305.01, stdev=564.98 lat (msec): min=686, max=1794, avg=1305.29, stdev=564.91 clat percentiles (msec): | 1.00th=[ 684], 5.00th=[ 684], 10.00th=[ 684], 20.00th=[ 684], | 30.00th=[ 684], 40.00th=[ 1435], 50.00th=[ 1435], 60.00th=[ 1435], | 70.00th=[ 1787], 80.00th=[ 1787], 90.00th=[ 1787], 95.00th=[ 1787], | 99.00th=[ 1787], 99.50th=[ 1787], 99.90th=[ 1787], 99.95th=[ 1787], | 99.99th=[ 1787] bw ( KiB/s): min= 7937, max= 8143, per=100.00%, avg=8040.00, stdev=145.66, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.18%, sys=4.01%, ctx=45, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3142KiB/s (3217kB/s), 785KiB/s-3147KiB/s (804kB/s-3222kB/s), io=16.0MiB (16.8MB), run=3905-5215msec WRITE: bw=5318KiB/s (5445kB/s), 1571KiB/s-3137KiB/s (1609kB/s-3212kB/s), io=32.0MiB (33.6MB), run=3917-6162msec