== sanity test 398b: DIO and buffer IO race ============== 15:03:09 (1755802989) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.13426 s, 16.1 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=193270: Thu Aug 21 15:04:20 2025 read: IOPS=25, BW=102KiB/s (105kB/s)(5992KiB/58673msec) clat (usec): min=98, max=682948, avg=21913.42, stdev=19898.33 lat (usec): min=98, max=682949, avg=21913.82, stdev=19898.33 clat percentiles (usec): | 1.00th=[ 553], 5.00th=[ 7963], 10.00th=[ 9896], 20.00th=[ 13304], | 30.00th=[ 15664], 40.00th=[ 17957], 50.00th=[ 20055], 60.00th=[ 22414], | 70.00th=[ 25822], 80.00th=[ 29230], 90.00th=[ 34866], 95.00th=[ 39060], | 99.00th=[ 55313], 99.50th=[ 61080], 99.90th=[ 79168], 99.95th=[683672], | 99.99th=[683672] bw ( KiB/s): min= 13, max= 208, per=26.01%, avg=101.96, stdev=34.04, samples=116 iops : min= 3, max= 52, avg=25.08, stdev= 8.58, samples=116 write: IOPS=26, BW=107KiB/s (110kB/s)(6296KiB/58673msec); 0 zone resets clat (usec): min=510, max=68884, avg=16237.60, stdev=11255.54 lat (usec): min=510, max=68885, avg=16238.11, stdev=11255.55 clat percentiles (usec): | 1.00th=[ 1188], 5.00th=[ 2900], 10.00th=[ 4178], 20.00th=[ 5997], | 30.00th=[ 8455], 40.00th=[10945], 50.00th=[13829], 60.00th=[17171], | 70.00th=[20579], 80.00th=[25035], 90.00th=[31589], 95.00th=[38536], | 99.00th=[51119], 99.50th=[54789], 99.90th=[60556], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 32, max= 215, per=26.31%, avg=108.14, stdev=35.52, samples=115 iops : min= 8, max= 53, avg=26.63, stdev= 8.89, samples=115 lat (usec) : 100=0.03%, 500=0.33%, 750=0.59%, 1000=0.26% lat (msec) : 2=0.55%, 4=3.94%, 10=17.55%, 20=35.97%, 50=39.39% lat (msec) : 100=1.37%, 750=0.03% cpu : usr=0.03%, sys=29.15%, ctx=10090, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193271: Thu Aug 21 15:04:20 2025 read: IOPS=24, BW=99.2KiB/s (102kB/s)(5940KiB/59880msec) clat (usec): min=101, max=85033, avg=22010.11, stdev=10593.26 lat (usec): min=102, max=85034, avg=22010.53, stdev=10593.30 clat percentiles (usec): | 1.00th=[ 660], 5.00th=[ 8586], 10.00th=[10421], 20.00th=[13435], | 30.00th=[16057], 40.00th=[17957], 50.00th=[20317], 60.00th=[23200], | 70.00th=[26346], 80.00th=[29492], 90.00th=[34341], 95.00th=[39060], | 99.00th=[56886], 99.50th=[62129], 99.90th=[83362], 99.95th=[85459], | 99.99th=[85459] bw ( KiB/s): min= 6, max= 208, per=25.27%, avg=99.05, stdev=33.04, samples=119 iops : min= 1, max= 52, avg=24.35, stdev= 8.30, samples=119 write: IOPS=26, BW=106KiB/s (109kB/s)(6348KiB/59880msec); 0 zone resets clat (usec): min=631, max=634482, avg=16946.95, stdev=19074.63 lat (usec): min=632, max=634483, avg=16949.33, stdev=19074.37 clat percentiles (usec): | 1.00th=[ 963], 5.00th=[ 2966], 10.00th=[ 4228], 20.00th=[ 6390], | 30.00th=[ 8848], 40.00th=[ 11731], 50.00th=[ 14615], 60.00th=[ 17433], | 70.00th=[ 21103], 80.00th=[ 25560], 90.00th=[ 32375], 95.00th=[ 38011], | 99.00th=[ 49546], 99.50th=[ 52167], 99.90th=[ 71828], 99.95th=[633340], | 99.99th=[633340] bw ( KiB/s): min= 6, max= 207, per=25.76%, avg=105.87, stdev=36.82, samples=119 iops : min= 1, max= 51, avg=26.07, stdev= 9.25, samples=119 lat (usec) : 250=0.03%, 500=0.23%, 750=0.42%, 1000=0.52% lat (msec) : 2=0.52%, 4=3.81%, 10=16.28%, 20=36.00%, 50=40.59% lat (msec) : 100=1.56%, 750=0.03% cpu : usr=0.05%, sys=29.06%, ctx=10282, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193272: Thu Aug 21 15:04:20 2025 read: IOPS=24, BW=98.6KiB/s (101kB/s)(6028KiB/61162msec) clat (usec): min=291, max=72923, avg=21771.48, stdev=10370.32 lat (usec): min=292, max=72923, avg=21771.90, stdev=10370.29 clat percentiles (usec): | 1.00th=[ 676], 5.00th=[ 8160], 10.00th=[ 9896], 20.00th=[12911], | 30.00th=[15401], 40.00th=[17957], 50.00th=[20579], 60.00th=[22938], | 70.00th=[25822], 80.00th=[29492], 90.00th=[35390], 95.00th=[40633], | 99.00th=[54264], 99.50th=[60556], 99.90th=[68682], 99.95th=[72877], | 99.99th=[72877] bw ( KiB/s): min= 23, max= 224, per=25.20%, avg=98.78, stdev=31.95, samples=120 iops : min= 5, max= 56, avg=24.29, stdev= 8.06, samples=120 write: IOPS=25, BW=102KiB/s (105kB/s)(6260KiB/61162msec); 0 zone resets clat (usec): min=554, max=660664, avg=17927.36, stdev=20211.89 lat (usec): min=554, max=660665, avg=17927.89, stdev=20211.87 clat percentiles (usec): | 1.00th=[ 1205], 5.00th=[ 2900], 10.00th=[ 4293], 20.00th=[ 6390], | 30.00th=[ 9634], 40.00th=[ 12387], 50.00th=[ 15139], 60.00th=[ 18482], | 70.00th=[ 22414], 80.00th=[ 27395], 90.00th=[ 34341], 95.00th=[ 39584], | 99.00th=[ 53740], 99.50th=[ 61604], 99.90th=[ 73925], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 31, max= 224, per=24.91%, avg=102.38, stdev=37.15, samples=120 iops : min= 7, max= 56, avg=25.19, stdev= 9.31, samples=120 lat (usec) : 500=0.23%, 750=0.59%, 1000=0.29% lat (msec) : 2=0.65%, 4=3.84%, 10=15.23%, 20=34.99%, 50=42.71% lat (msec) : 100=1.43%, 750=0.03% cpu : usr=0.06%, sys=28.75%, ctx=10189, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193273: Thu Aug 21 15:04:20 2025 read: IOPS=25, BW=100KiB/s (102kB/s)(6052KiB/60495msec) clat (usec): min=256, max=91311, avg=21586.74, stdev=10509.99 lat (usec): min=257, max=91311, avg=21587.16, stdev=10509.99 clat percentiles (usec): | 1.00th=[ 979], 5.00th=[ 8586], 10.00th=[10159], 20.00th=[12911], | 30.00th=[15401], 40.00th=[17171], 50.00th=[19792], 60.00th=[22414], | 70.00th=[25560], 80.00th=[29492], 90.00th=[35390], 95.00th=[40633], | 99.00th=[53216], 99.50th=[61604], 99.90th=[72877], 99.95th=[91751], | 99.99th=[91751] bw ( KiB/s): min= 48, max= 216, per=25.90%, avg=101.53, stdev=32.72, samples=118 iops : min= 12, max= 54, avg=24.98, stdev= 8.21, samples=118 write: IOPS=25, BW=103KiB/s (106kB/s)(6236KiB/60495msec); 0 zone resets clat (usec): min=557, max=658414, avg=17666.97, stdev=19878.78 lat (usec): min=557, max=658416, avg=17667.49, stdev=19878.80 clat percentiles (usec): | 1.00th=[ 1909], 5.00th=[ 3359], 10.00th=[ 4817], 20.00th=[ 6915], | 30.00th=[ 9503], 40.00th=[ 12387], 50.00th=[ 14877], 60.00th=[ 17957], | 70.00th=[ 21365], 80.00th=[ 26084], 90.00th=[ 33817], 95.00th=[ 40109], | 99.00th=[ 51643], 99.50th=[ 55313], 99.90th=[ 69731], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 31, max= 208, per=25.20%, avg=103.58, stdev=32.33, samples=119 iops : min= 7, max= 52, avg=25.49, stdev= 8.11, samples=119 lat (usec) : 500=0.20%, 750=0.33%, 1000=0.29% lat (msec) : 2=0.46%, 4=2.90%, 10=16.63%, 20=38.12%, 50=39.75% lat (msec) : 100=1.30%, 750=0.03% cpu : usr=0.04%, sys=28.74%, ctx=10494, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=393KiB/s (402kB/s), 98.6KiB/s-102KiB/s (101kB/s-105kB/s), io=23.4MiB (24.6MB), run=58673-61162msec WRITE: bw=411KiB/s (421kB/s), 102KiB/s-107KiB/s (105kB/s-110kB/s), io=24.6MiB (25.7MB), run=58673-61162msec rand-rw: (groupid=0, jobs=1): err= 0: pid=193275: Thu Aug 21 15:04:26 2025 read: IOPS=22, BW=89.0KiB/s (91.1kB/s)(5992KiB/67320msec) clat (msec): min=2, max=702, avg=20.39, stdev=22.64 lat (msec): min=2, max=702, avg=20.39, stdev=22.64 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 30], 90.00th=[ 40], 95.00th=[ 51], | 99.00th=[ 73], 99.50th=[ 79], 99.90th=[ 94], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 23, max= 296, per=25.06%, avg=89.20, stdev=51.55, samples=133 iops : min= 5, max= 74, avg=21.86, stdev=13.03, samples=133 write: IOPS=23, BW=93.5KiB/s (95.8kB/s)(6296KiB/67320msec); 0 zone resets clat (msec): min=4, max=109, avg=23.34, stdev=14.85 lat (msec): min=4, max=109, avg=23.34, stdev=14.85 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 10], | 30.00th=[ 13], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 29], 80.00th=[ 36], 90.00th=[ 44], 95.00th=[ 51], | 99.00th=[ 69], 99.50th=[ 77], 99.90th=[ 103], 99.95th=[ 110], | 99.99th=[ 110] bw ( KiB/s): min= 16, max= 312, per=25.09%, avg=93.59, stdev=54.52, samples=133 iops : min= 4, max= 78, avg=22.95, stdev=13.75, samples=133 lat (msec) : 4=0.42%, 10=24.38%, 20=32.36%, 50=37.60%, 100=5.14% lat (msec) : 250=0.07%, 750=0.03% cpu : usr=0.04%, sys=16.13%, ctx=4939, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193276: Thu Aug 21 15:04:26 2025 read: IOPS=22, BW=88.5KiB/s (90.6kB/s)(5940KiB/67116msec) clat (msec): min=3, max=649, avg=19.94, stdev=21.38 lat (msec): min=3, max=649, avg=19.94, stdev=21.38 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 23], 80.00th=[ 29], 90.00th=[ 40], 95.00th=[ 48], | 99.00th=[ 70], 99.50th=[ 75], 99.90th=[ 93], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 15, max= 288, per=24.89%, avg=88.61, stdev=53.17, samples=132 iops : min= 3, max= 72, avg=21.71, stdev=13.43, samples=132 write: IOPS=23, BW=94.6KiB/s (96.9kB/s)(6348KiB/67116msec); 0 zone resets clat (usec): min=4466, max=94662, avg=23612.51, stdev=15428.48 lat (usec): min=4467, max=94663, avg=23613.04, stdev=15428.47 clat percentiles (usec): | 1.00th=[ 5997], 5.00th=[ 7177], 10.00th=[ 8094], 20.00th=[ 9896], | 30.00th=[12649], 40.00th=[15795], 50.00th=[19268], 60.00th=[23987], | 70.00th=[28967], 80.00th=[35914], 90.00th=[45876], 95.00th=[52691], | 99.00th=[74974], 99.50th=[83362], 99.90th=[93848], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 31, max= 320, per=25.30%, avg=94.36, stdev=53.09, samples=132 iops : min= 7, max= 80, avg=23.15, stdev=13.41, samples=132 lat (msec) : 4=0.29%, 10=24.22%, 20=33.50%, 50=36.59%, 100=5.37% lat (msec) : 750=0.03% cpu : usr=0.03%, sys=15.96%, ctx=4967, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193277: Thu Aug 21 15:04:26 2025 read: IOPS=22, BW=89.7KiB/s (91.9kB/s)(6028KiB/67202msec) clat (usec): min=2824, max=83460, avg=19720.18, stdev=13752.22 lat (usec): min=2825, max=83460, avg=19720.63, stdev=13752.21 clat percentiles (usec): | 1.00th=[ 4293], 5.00th=[ 6128], 10.00th=[ 6980], 20.00th=[ 8291], | 30.00th=[10290], 40.00th=[12911], 50.00th=[15533], 60.00th=[19006], | 70.00th=[23462], 80.00th=[29492], 90.00th=[39060], 95.00th=[47449], | 99.00th=[68682], 99.50th=[74974], 99.90th=[83362], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 23, max= 304, per=25.18%, avg=89.64, stdev=54.59, samples=132 iops : min= 5, max= 76, avg=21.97, stdev=13.78, samples=132 write: IOPS=23, BW=93.2KiB/s (95.4kB/s)(6260KiB/67202msec); 0 zone resets clat (msec): min=5, max=655, avg=23.93, stdev=22.09 lat (msec): min=5, max=655, avg=23.93, stdev=22.09 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 10], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 29], 80.00th=[ 35], 90.00th=[ 45], 95.00th=[ 55], | 99.00th=[ 74], 99.50th=[ 79], 99.90th=[ 91], 99.95th=[ 659], | 99.99th=[ 659] bw ( KiB/s): min= 31, max= 296, per=24.86%, avg=92.73, stdev=51.21, samples=132 iops : min= 7, max= 74, avg=22.74, stdev=12.94, samples=132 lat (msec) : 4=0.29%, 10=24.15%, 20=32.45%, 50=37.47%, 100=5.60% lat (msec) : 750=0.03% cpu : usr=0.05%, sys=16.25%, ctx=4996, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193278: Thu Aug 21 15:04:26 2025 read: IOPS=22, BW=90.1KiB/s (92.2kB/s)(6052KiB/67187msec) clat (usec): min=3347, max=96708, avg=20214.93, stdev=14084.61 lat (usec): min=3347, max=96709, avg=20215.36, stdev=14084.60 clat percentiles (usec): | 1.00th=[ 4293], 5.00th=[ 6325], 10.00th=[ 6915], 20.00th=[ 8455], | 30.00th=[10945], 40.00th=[13566], 50.00th=[15926], 60.00th=[19006], | 70.00th=[23200], 80.00th=[30278], 90.00th=[40633], 95.00th=[48497], | 99.00th=[66323], 99.50th=[73925], 99.90th=[91751], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 23, max= 320, per=25.23%, avg=89.83, stdev=50.40, samples=132 iops : min= 5, max= 80, avg=22.02, stdev=12.73, samples=132 write: IOPS=23, BW=92.8KiB/s (95.0kB/s)(6236KiB/67187msec); 0 zone resets clat (msec): min=4, max=651, avg=23.45, stdev=21.78 lat (msec): min=4, max=651, avg=23.45, stdev=21.78 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 28], 80.00th=[ 35], 90.00th=[ 44], 95.00th=[ 53], | 99.00th=[ 69], 99.50th=[ 78], 99.90th=[ 99], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 23, max= 312, per=24.90%, avg=92.86, stdev=54.66, samples=132 iops : min= 5, max= 78, avg=22.77, stdev=13.83, samples=132 lat (msec) : 4=0.33%, 10=23.14%, 20=34.15%, 50=37.30%, 100=5.05% lat (msec) : 750=0.03% cpu : usr=0.05%, sys=16.13%, ctx=4982, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=357KiB/s (365kB/s), 88.5KiB/s-90.1KiB/s (90.6kB/s-92.2kB/s), io=23.4MiB (24.6MB), run=67116-67320msec WRITE: bw=373KiB/s (382kB/s), 92.8KiB/s-94.6KiB/s (95.0kB/s-96.9kB/s), io=24.6MiB (25.7MB), run=67116-67320msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=193306: Thu Aug 21 15:04:53 2025 read: IOPS=19, BW=305KiB/s (312kB/s)(5920KiB/19439msec) clat (usec): min=139, max=76151, avg=26125.18, stdev=12964.37 lat (usec): min=139, max=76151, avg=26125.82, stdev=12964.26 clat percentiles (usec): | 1.00th=[ 433], 5.00th=[ 791], 10.00th=[13173], 20.00th=[16909], | 30.00th=[19792], 40.00th=[21627], 50.00th=[24773], 60.00th=[27657], | 70.00th=[31327], 80.00th=[35390], 90.00th=[40633], 95.00th=[50594], | 99.00th=[63701], 99.50th=[72877], 99.90th=[76022], 99.95th=[76022], | 99.99th=[76022] bw ( KiB/s): min= 128, max= 475, per=25.51%, avg=302.84, stdev=80.52, samples=38 iops : min= 8, max= 29, avg=18.42, stdev= 4.96, samples=38 write: IOPS=20, BW=328KiB/s (335kB/s)(6368KiB/19439msec); 0 zone resets clat (usec): min=1047, max=78337, avg=24334.83, stdev=15023.34 lat (usec): min=1048, max=78337, avg=24335.57, stdev=15023.37 clat percentiles (usec): | 1.00th=[ 3982], 5.00th=[ 7242], 10.00th=[ 8586], 20.00th=[10945], | 30.00th=[14484], 40.00th=[17433], 50.00th=[20055], 60.00th=[23725], | 70.00th=[30016], 80.00th=[36963], 90.00th=[46924], 95.00th=[52691], | 99.00th=[70779], 99.50th=[77071], 99.90th=[78119], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 63, max= 606, per=26.25%, avg=327.87, stdev=100.02, samples=38 iops : min= 3, max= 37, avg=19.97, stdev= 6.26, samples=38 lat (usec) : 250=0.26%, 500=0.26%, 750=1.56%, 1000=1.04% lat (msec) : 2=0.39%, 4=0.39%, 10=7.81%, 20=29.17%, 50=53.26% lat (msec) : 100=5.86% cpu : usr=0.06%, sys=31.25%, ctx=3231, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193307: Thu Aug 21 15:04:53 2025 read: IOPS=17, BW=288KiB/s (295kB/s)(5744KiB/19965msec) clat (usec): min=281, max=82050, avg=28290.12, stdev=13945.27 lat (usec): min=282, max=82051, avg=28290.56, stdev=13945.27 clat percentiles (usec): | 1.00th=[ 449], 5.00th=[ 9896], 10.00th=[14484], 20.00th=[17695], | 30.00th=[21365], 40.00th=[23987], 50.00th=[26346], 60.00th=[29492], | 70.00th=[33817], 80.00th=[36963], 90.00th=[43254], 95.00th=[56886], | 99.00th=[77071], 99.50th=[81265], 99.90th=[82314], 99.95th=[82314], | 99.99th=[82314] bw ( KiB/s): min= 128, max= 542, per=24.08%, avg=285.79, stdev=91.05, samples=39 iops : min= 8, max= 33, avg=17.36, stdev= 5.62, samples=39 write: IOPS=20, BW=328KiB/s (336kB/s)(6544KiB/19965msec); 0 zone resets clat (usec): min=949, max=86994, avg=23928.48, stdev=14000.46 lat (usec): min=950, max=86995, avg=23929.37, stdev=14000.48 clat percentiles (usec): | 1.00th=[ 4686], 5.00th=[ 7701], 10.00th=[ 8586], 20.00th=[10945], | 30.00th=[14877], 40.00th=[17957], 50.00th=[21627], 60.00th=[24773], | 70.00th=[29230], 80.00th=[34341], 90.00th=[43779], 95.00th=[52167], | 99.00th=[64226], 99.50th=[69731], 99.90th=[86508], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 158, max= 561, per=25.81%, avg=322.41, stdev=103.67, samples=39 iops : min= 9, max= 35, avg=19.67, stdev= 6.57, samples=39 lat (usec) : 500=0.52%, 750=0.65%, 1000=1.04% lat (msec) : 4=0.13%, 10=8.33%, 20=25.13%, 50=57.42%, 100=6.77% cpu : usr=0.02%, sys=31.70%, ctx=3257, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193308: Thu Aug 21 15:04:53 2025 read: IOPS=19, BW=310KiB/s (318kB/s)(6256KiB/20169msec) clat (usec): min=612, max=80354, avg=28100.27, stdev=12173.03 lat (usec): min=613, max=80354, avg=28100.73, stdev=12173.04 clat percentiles (usec): | 1.00th=[ 742], 5.00th=[11338], 10.00th=[15139], 20.00th=[18220], | 30.00th=[21103], 40.00th=[23987], 50.00th=[26870], 60.00th=[29492], | 70.00th=[32375], 80.00th=[37487], 90.00th=[44827], 95.00th=[49546], | 99.00th=[65274], 99.50th=[70779], 99.90th=[80217], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 160, max= 512, per=25.95%, avg=308.05, stdev=76.38, samples=39 iops : min= 10, max= 32, avg=18.77, stdev= 4.82, samples=39 write: IOPS=18, BW=299KiB/s (306kB/s)(6032KiB/20169msec); 0 zone resets clat (usec): min=815, max=98328, avg=24180.45, stdev=15487.26 lat (usec): min=816, max=98329, avg=24181.20, stdev=15487.23 clat percentiles (usec): | 1.00th=[ 2278], 5.00th=[ 6980], 10.00th=[ 8160], 20.00th=[10552], | 30.00th=[13566], 40.00th=[16581], 50.00th=[20055], 60.00th=[25035], | 70.00th=[30016], 80.00th=[35914], 90.00th=[46400], 95.00th=[52691], | 99.00th=[74974], 99.50th=[80217], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 95, max= 589, per=23.12%, avg=288.82, stdev=97.44, samples=39 iops : min= 5, max= 36, avg=17.54, stdev= 6.13, samples=39 lat (usec) : 750=0.52%, 1000=0.52% lat (msec) : 2=0.26%, 4=0.52%, 10=8.46%, 20=26.82%, 50=56.77% lat (msec) : 100=6.12% cpu : usr=0.02%, sys=30.40%, ctx=3209, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193309: Thu Aug 21 15:04:53 2025 read: IOPS=19, BW=315KiB/s (323kB/s)(6032KiB/19150msec) clat (usec): min=284, max=94013, avg=27277.71, stdev=14005.63 lat (usec): min=284, max=94013, avg=27278.15, stdev=14005.63 clat percentiles (usec): | 1.00th=[ 379], 5.00th=[ 807], 10.00th=[11863], 20.00th=[16450], | 30.00th=[19792], 40.00th=[22938], 50.00th=[26608], 60.00th=[30278], | 70.00th=[33162], 80.00th=[38011], 90.00th=[44303], 95.00th=[51643], | 99.00th=[67634], 99.50th=[81265], 99.90th=[93848], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 191, max= 574, per=26.27%, avg=311.81, stdev=82.91, samples=37 iops : min= 11, max= 35, avg=18.97, stdev= 5.15, samples=37 write: IOPS=20, BW=327KiB/s (335kB/s)(6256KiB/19150msec); 0 zone resets clat (usec): min=1127, max=78859, avg=22544.92, stdev=13810.39 lat (usec): min=1128, max=78860, avg=22545.70, stdev=13810.39 clat percentiles (usec): | 1.00th=[ 3654], 5.00th=[ 7373], 10.00th=[ 7898], 20.00th=[ 9896], | 30.00th=[13304], 40.00th=[16581], 50.00th=[19792], 60.00th=[23200], | 70.00th=[27395], 80.00th=[32375], 90.00th=[41681], 95.00th=[50594], | 99.00th=[65274], 99.50th=[70779], 99.90th=[79168], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 96, max= 574, per=26.06%, avg=325.43, stdev=101.69, samples=37 iops : min= 6, max= 35, avg=19.81, stdev= 6.35, samples=37 lat (usec) : 500=0.52%, 750=1.56%, 1000=0.52% lat (msec) : 2=0.26%, 4=1.04%, 10=10.16%, 20=27.47%, 50=52.86% lat (msec) : 100=5.60% cpu : usr=0.04%, sys=31.54%, ctx=3127, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1188KiB/s (1216kB/s), 288KiB/s-315KiB/s (295kB/s-323kB/s), io=23.4MiB (24.5MB), run=19150-20169msec WRITE: bw=1249KiB/s (1279kB/s), 299KiB/s-328KiB/s (306kB/s-336kB/s), io=24.6MiB (25.8MB), run=19150-20169msec rand-rw: (groupid=0, jobs=1): err= 0: pid=193311: Thu Aug 21 15:04:54 2025 read: IOPS=17, BW=277KiB/s (283kB/s)(5920KiB/21393msec) clat (msec): min=4, max=104, avg=25.56, stdev=18.03 lat (msec): min=4, max=104, avg=25.56, stdev=18.03 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 32], 80.00th=[ 39], 90.00th=[ 53], 95.00th=[ 64], | 99.00th=[ 87], 99.50th=[ 88], 99.90th=[ 105], 99.95th=[ 105], | 99.99th=[ 105] bw ( KiB/s): min= 127, max= 832, per=24.53%, avg=273.50, stdev=147.22, samples=42 iops : min= 7, max= 52, avg=16.50, stdev= 9.27, samples=42 write: IOPS=18, BW=298KiB/s (305kB/s)(6368KiB/21393msec); 0 zone resets clat (msec): min=5, max=120, avg=29.84, stdev=20.42 lat (msec): min=5, max=120, avg=29.84, stdev=20.42 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 13], | 30.00th=[ 17], 40.00th=[ 21], 50.00th=[ 25], 60.00th=[ 29], | 70.00th=[ 36], 80.00th=[ 43], 90.00th=[ 59], 95.00th=[ 75], | 99.00th=[ 100], 99.50th=[ 110], 99.90th=[ 121], 99.95th=[ 121], | 99.99th=[ 121] bw ( KiB/s): min= 95, max= 864, per=25.31%, avg=296.88, stdev=168.03, samples=42 iops : min= 5, max= 54, avg=17.95, stdev=10.63, samples=42 lat (msec) : 10=13.28%, 20=32.03%, 50=41.28%, 100=12.89%, 250=0.52% cpu : usr=0.03%, sys=15.41%, ctx=1361, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193312: Thu Aug 21 15:04:54 2025 read: IOPS=16, BW=268KiB/s (274kB/s)(5744KiB/21469msec) clat (msec): min=2, max=110, avg=25.97, stdev=18.59 lat (msec): min=2, max=110, avg=25.97, stdev=18.59 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 31], 80.00th=[ 41], 90.00th=[ 51], 95.00th=[ 65], | 99.00th=[ 88], 99.50th=[ 104], 99.90th=[ 111], 99.95th=[ 111], | 99.99th=[ 111] bw ( KiB/s): min= 47, max= 832, per=23.77%, avg=265.02, stdev=140.08, samples=42 iops : min= 2, max= 52, avg=15.95, stdev= 8.85, samples=42 write: IOPS=19, BW=305KiB/s (312kB/s)(6544KiB/21469msec); 0 zone resets clat (msec): min=3, max=115, avg=29.38, stdev=19.80 lat (msec): min=3, max=115, avg=29.38, stdev=19.80 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 13], | 30.00th=[ 16], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 31], | 70.00th=[ 36], 80.00th=[ 47], 90.00th=[ 56], 95.00th=[ 69], | 99.00th=[ 95], 99.50th=[ 103], 99.90th=[ 115], 99.95th=[ 115], | 99.99th=[ 115] bw ( KiB/s): min= 128, max= 1024, per=25.40%, avg=297.90, stdev=175.62, samples=42 iops : min= 8, max= 64, avg=18.02, stdev=11.06, samples=42 lat (msec) : 4=1.04%, 10=11.72%, 20=33.20%, 50=40.23%, 100=13.15% lat (msec) : 250=0.65% cpu : usr=0.03%, sys=15.33%, ctx=1452, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193313: Thu Aug 21 15:04:54 2025 read: IOPS=18, BW=295KiB/s (302kB/s)(6256KiB/21224msec) clat (msec): min=3, max=103, avg=24.50, stdev=17.73 lat (msec): min=3, max=103, avg=24.50, stdev=17.73 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 19], 60.00th=[ 22], | 70.00th=[ 28], 80.00th=[ 37], 90.00th=[ 51], 95.00th=[ 62], | 99.00th=[ 89], 99.50th=[ 99], 99.90th=[ 104], 99.95th=[ 104], | 99.99th=[ 104] bw ( KiB/s): min= 95, max= 1024, per=26.29%, avg=293.10, stdev=194.08, samples=41 iops : min= 5, max= 64, avg=17.73, stdev=12.26, samples=41 write: IOPS=17, BW=284KiB/s (291kB/s)(6032KiB/21224msec); 0 zone resets clat (msec): min=5, max=108, avg=30.49, stdev=18.88 lat (msec): min=5, max=108, avg=30.49, stdev=18.88 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 21], 50.00th=[ 26], 60.00th=[ 32], | 70.00th=[ 38], 80.00th=[ 46], 90.00th=[ 57], 95.00th=[ 66], | 99.00th=[ 92], 99.50th=[ 97], 99.90th=[ 109], 99.95th=[ 109], | 99.99th=[ 109] bw ( KiB/s): min= 126, max= 736, per=23.41%, avg=274.61, stdev=116.48, samples=41 iops : min= 7, max= 46, avg=16.56, stdev= 7.39, samples=41 lat (msec) : 4=0.13%, 10=10.16%, 20=36.72%, 50=39.97%, 100=12.76% lat (msec) : 250=0.26% cpu : usr=0.02%, sys=15.69%, ctx=1444, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193314: Thu Aug 21 15:04:54 2025 read: IOPS=17, BW=281KiB/s (288kB/s)(6032KiB/21438msec) clat (usec): min=3175, max=98007, avg=25084.22, stdev=16916.60 lat (usec): min=3175, max=98008, avg=25085.01, stdev=16916.34 clat percentiles (usec): | 1.00th=[ 4178], 5.00th=[ 6915], 10.00th=[ 8586], 20.00th=[11338], | 30.00th=[14222], 40.00th=[16581], 50.00th=[19006], 60.00th=[23725], | 70.00th=[28705], 80.00th=[39584], 90.00th=[51119], 95.00th=[59507], | 99.00th=[73925], 99.50th=[79168], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 63, max= 960, per=24.84%, avg=276.93, stdev=160.99, samples=42 iops : min= 3, max= 60, avg=16.69, stdev=10.19, samples=42 write: IOPS=18, BW=292KiB/s (299kB/s)(6256KiB/21438msec); 0 zone resets clat (usec): min=4307, max=93369, avg=30257.87, stdev=18907.07 lat (usec): min=4308, max=93370, avg=30258.91, stdev=18907.04 clat percentiles (usec): | 1.00th=[ 5932], 5.00th=[ 8848], 10.00th=[10552], 20.00th=[14091], | 30.00th=[16909], 40.00th=[20055], 50.00th=[23200], 60.00th=[30278], | 70.00th=[39584], 80.00th=[47449], 90.00th=[57410], 95.00th=[65799], | 99.00th=[86508], 99.50th=[90702], 99.90th=[93848], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 158, max= 768, per=24.58%, avg=288.38, stdev=127.06, samples=42 iops : min= 9, max= 48, avg=17.43, stdev= 8.03, samples=42 lat (msec) : 4=0.26%, 10=10.81%, 20=34.90%, 50=40.23%, 100=13.80% cpu : usr=0.03%, sys=15.25%, ctx=1434, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1116KiB/s (1142kB/s), 268KiB/s-295KiB/s (274kB/s-302kB/s), io=23.4MiB (24.5MB), run=21224-21469msec WRITE: bw=1174KiB/s (1202kB/s), 284KiB/s-305KiB/s (291kB/s-312kB/s), io=24.6MiB (25.8MB), run=21224-21469msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=193328: Thu Aug 21 15:05:11 2025 read: IOPS=1, BW=1350KiB/s (1383kB/s)(7168KiB/5309msec) clat (usec): min=1278, max=553133, avg=297239.93, stdev=189239.65 lat (usec): min=1279, max=553134, avg=297240.70, stdev=189239.63 clat percentiles (usec): | 1.00th=[ 1287], 5.00th=[ 1287], 10.00th=[ 1287], 20.00th=[175113], | 30.00th=[219153], 40.00th=[219153], 50.00th=[312476], 60.00th=[325059], | 70.00th=[325059], 80.00th=[497026], 90.00th=[549454], 95.00th=[549454], | 99.00th=[549454], 99.50th=[549454], 99.90th=[549454], 99.95th=[549454], | 99.99th=[549454] bw ( KiB/s): min= 2027, max= 2972, per=69.89%, avg=2226.80, stdev=416.67, samples=5 iops : min= 1, max= 2, avg= 1.60, stdev= 0.55, samples=5 write: IOPS=0, BW=964KiB/s (988kB/s)(5120KiB/5309msec); 0 zone resets clat (msec): min=347, max=862, avg=634.98, stdev=221.81 lat (msec): min=347, max=862, avg=635.04, stdev=221.80 clat percentiles (msec): | 1.00th=[ 347], 5.00th=[ 347], 10.00th=[ 347], 20.00th=[ 347], | 30.00th=[ 523], 40.00th=[ 523], 50.00th=[ 592], 60.00th=[ 592], | 70.00th=[ 852], 80.00th=[ 852], 90.00th=[ 860], 95.00th=[ 860], | 99.00th=[ 860], 99.50th=[ 860], 99.90th=[ 860], 99.95th=[ 860], | 99.99th=[ 860] bw ( KiB/s): min= 1969, max= 3792, per=58.00%, avg=2587.00, stdev=1043.68, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 2=8.33%, 250=16.67%, 500=33.33%, 750=25.00%, 1000=16.67% cpu : usr=0.00%, sys=47.04%, ctx=722, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193329: Thu Aug 21 15:05:11 2025 read: IOPS=0, BW=319KiB/s (326kB/s)(2048KiB/6428msec) clat (msec): min=471, max=692, avg=581.99, stdev=156.12 lat (msec): min=471, max=692, avg=581.99, stdev=156.12 clat percentiles (msec): | 1.00th=[ 472], 5.00th=[ 472], 10.00th=[ 472], 20.00th=[ 472], | 30.00th=[ 472], 40.00th=[ 472], 50.00th=[ 472], 60.00th=[ 693], | 70.00th=[ 693], 80.00th=[ 693], 90.00th=[ 693], 95.00th=[ 693], | 99.00th=[ 693], 99.50th=[ 693], 99.90th=[ 693], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 1992, max= 2039, per=63.26%, avg=2015.50, stdev=33.23, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1593KiB/s (1631kB/s)(10.0MiB/6428msec); 0 zone resets clat (msec): min=103, max=1074, avg=519.62, stdev=349.94 lat (msec): min=103, max=1074, avg=519.69, stdev=349.93 clat percentiles (msec): | 1.00th=[ 104], 5.00th=[ 104], 10.00th=[ 104], 20.00th=[ 174], | 30.00th=[ 180], 40.00th=[ 234], 50.00th=[ 439], 60.00th=[ 558], | 70.00th=[ 676], 80.00th=[ 827], 90.00th=[ 936], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 1896, max= 4096, per=51.19%, avg=2283.00, stdev=801.54, samples=7 iops : min= 1, max= 4, avg= 1.71, stdev= 1.11, samples=7 lat (msec) : 250=33.33%, 500=16.67%, 750=25.00%, 1000=16.67%, 2000=8.33% cpu : usr=0.05%, sys=46.01%, ctx=336, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193330: Thu Aug 21 15:05:11 2025 read: IOPS=1, BW=1066KiB/s (1092kB/s)(6144KiB/5761msec) clat (usec): min=1306, max=620843, avg=235778.48, stdev=232200.40 lat (usec): min=1308, max=620844, avg=235779.32, stdev=232200.21 clat percentiles (usec): | 1.00th=[ 1303], 5.00th=[ 1303], 10.00th=[ 1303], 20.00th=[ 70779], | 30.00th=[ 70779], 40.00th=[ 79168], 50.00th=[ 79168], 60.00th=[316670], | 70.00th=[325059], 80.00th=[325059], 90.00th=[624952], 95.00th=[624952], | 99.00th=[624952], 99.50th=[624952], 99.90th=[624952], 99.95th=[624952], | 99.99th=[624952] bw ( KiB/s): min= 1992, max= 4096, per=100.00%, avg=3293.33, stdev=1137.19, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=1, BW=1066KiB/s (1092kB/s)(6144KiB/5761msec); 0 zone resets clat (msec): min=298, max=1089, avg=708.65, stdev=262.08 lat (msec): min=298, max=1089, avg=708.71, stdev=262.09 clat percentiles (msec): | 1.00th=[ 300], 5.00th=[ 300], 10.00th=[ 300], 20.00th=[ 609], | 30.00th=[ 609], 40.00th=[ 651], 50.00th=[ 651], 60.00th=[ 785], | 70.00th=[ 818], 80.00th=[ 818], 90.00th=[ 1083], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 1992, max= 4079, per=56.94%, avg=2539.50, stdev=1026.63, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 2=8.33%, 100=16.67%, 500=25.00%, 750=25.00%, 1000=16.67% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=38.59%, ctx=366, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193331: Thu Aug 21 15:05:11 2025 read: IOPS=0, BW=1001KiB/s (1025kB/s)(5120KiB/5116msec) clat (usec): min=1267, max=390824, avg=253598.52, stdev=148922.39 lat (usec): min=1268, max=390825, avg=253599.62, stdev=148922.24 clat percentiles (usec): | 1.00th=[ 1270], 5.00th=[ 1270], 10.00th=[ 1270], 20.00th=[ 1270], | 30.00th=[258999], 40.00th=[258999], 50.00th=[299893], 60.00th=[299893], | 70.00th=[316670], 80.00th=[316670], 90.00th=[392168], 95.00th=[392168], | 99.00th=[392168], 99.50th=[392168], 99.90th=[392168], 99.95th=[392168], | 99.99th=[392168] bw ( KiB/s): min= 2000, max= 4079, per=85.03%, avg=2709.00, stdev=1186.70, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 write: IOPS=1, BW=1401KiB/s (1435kB/s)(7168KiB/5116msec); 0 zone resets clat (msec): min=202, max=1077, avg=546.92, stdev=271.84 lat (msec): min=202, max=1077, avg=546.99, stdev=271.85 clat percentiles (msec): | 1.00th=[ 203], 5.00th=[ 203], 10.00th=[ 203], 20.00th=[ 393], | 30.00th=[ 472], 40.00th=[ 472], 50.00th=[ 485], 60.00th=[ 558], | 70.00th=[ 558], 80.00th=[ 642], 90.00th=[ 1083], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 1458, max= 2048, per=42.77%, avg=1907.67, stdev=227.91, samples=6 iops : min= 1, max= 2, avg= 1.33, stdev= 0.52, samples=6 lat (msec) : 2=8.33%, 250=8.33%, 500=58.33%, 750=16.67%, 2000=8.33% cpu : usr=0.02%, sys=47.41%, ctx=373, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3186KiB/s (3263kB/s), 319KiB/s-1350KiB/s (326kB/s-1383kB/s), io=20.0MiB (20.0MB), run=5116-6428msec WRITE: bw=4460KiB/s (4568kB/s), 964KiB/s-1593KiB/s (988kB/s-1631kB/s), io=28.0MiB (29.4MB), run=5116-6428msec rand-rw: (groupid=0, jobs=1): err= 0: pid=193324: Thu Aug 21 15:05:11 2025 read: IOPS=1, BW=1390KiB/s (1424kB/s)(7168KiB/5156msec) clat (msec): min=135, max=621, avg=299.12, stdev=183.07 lat (msec): min=135, max=621, avg=299.12, stdev=183.07 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 144], | 30.00th=[ 146], 40.00th=[ 146], 50.00th=[ 253], 60.00th=[ 363], | 70.00th=[ 363], 80.00th=[ 430], 90.00th=[ 625], 95.00th=[ 625], | 99.00th=[ 625], 99.50th=[ 625], 99.90th=[ 625], 99.95th=[ 625], | 99.99th=[ 625] bw ( KiB/s): min= 1896, max= 4338, per=80.43%, avg=2547.25, stdev=1195.13, samples=4 iops : min= 1, max= 4, avg= 1.75, stdev= 1.50, samples=4 write: IOPS=0, BW=993KiB/s (1017kB/s)(5120KiB/5156msec); 0 zone resets clat (msec): min=281, max=1135, avg=603.00, stdev=347.09 lat (msec): min=281, max=1135, avg=603.05, stdev=347.08 clat percentiles (msec): | 1.00th=[ 284], 5.00th=[ 284], 10.00th=[ 284], 20.00th=[ 284], | 30.00th=[ 300], 40.00th=[ 300], 50.00th=[ 634], 60.00th=[ 634], | 70.00th=[ 659], 80.00th=[ 659], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 2043, max= 4079, per=61.42%, avg=2723.33, stdev=1174.04, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 lat (msec) : 250=25.00%, 500=41.67%, 750=25.00%, 2000=8.33% cpu : usr=0.00%, sys=20.48%, ctx=233, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193325: Thu Aug 21 15:05:11 2025 read: IOPS=0, BW=340KiB/s (348kB/s)(2048KiB/6027msec) clat (msec): min=425, max=588, avg=506.71, stdev=115.44 lat (msec): min=425, max=588, avg=506.71, stdev=115.44 clat percentiles (msec): | 1.00th=[ 426], 5.00th=[ 426], 10.00th=[ 426], 20.00th=[ 426], | 30.00th=[ 426], 40.00th=[ 426], 50.00th=[ 426], 60.00th=[ 592], | 70.00th=[ 592], 80.00th=[ 592], 90.00th=[ 592], 95.00th=[ 592], | 99.00th=[ 592], 99.50th=[ 592], 99.90th=[ 592], 99.95th=[ 592], | 99.99th=[ 592] bw ( KiB/s): min= 1442, max= 2039, per=54.96%, avg=1740.50, stdev=422.14, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1699KiB/s (1740kB/s)(10.0MiB/6027msec); 0 zone resets clat (msec): min=150, max=1094, avg=497.02, stdev=334.34 lat (msec): min=150, max=1094, avg=497.10, stdev=334.32 clat percentiles (msec): | 1.00th=[ 150], 5.00th=[ 150], 10.00th=[ 150], 20.00th=[ 218], | 30.00th=[ 284], 40.00th=[ 300], 50.00th=[ 330], 60.00th=[ 388], | 70.00th=[ 558], 80.00th=[ 600], 90.00th=[ 1053], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 1992, max= 4096, per=57.77%, avg=2561.71, stdev=927.20, samples=7 iops : min= 1, max= 4, avg= 1.86, stdev= 1.21, samples=7 lat (msec) : 250=16.67%, 500=41.67%, 750=25.00%, 2000=16.67% cpu : usr=0.02%, sys=16.93%, ctx=269, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193326: Thu Aug 21 15:05:11 2025 read: IOPS=0, BW=950KiB/s (973kB/s)(6144KiB/6465msec) clat (msec): min=68, max=1389, avg=477.26, stdev=500.22 lat (msec): min=68, max=1389, avg=477.26, stdev=500.22 clat percentiles (msec): | 1.00th=[ 69], 5.00th=[ 69], 10.00th=[ 69], 20.00th=[ 81], | 30.00th=[ 81], 40.00th=[ 300], 50.00th=[ 300], 60.00th=[ 338], | 70.00th=[ 684], 80.00th=[ 684], 90.00th=[ 1385], 95.00th=[ 1385], | 99.00th=[ 1385], 99.50th=[ 1385], 99.90th=[ 1385], 99.95th=[ 1385], | 99.99th=[ 1385] bw ( KiB/s): min= 1861, max= 4096, per=78.16%, avg=2475.25, stdev=1083.54, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 write: IOPS=0, BW=950KiB/s (973kB/s)(6144KiB/6465msec); 0 zone resets clat (msec): min=92, max=1374, avg=598.63, stdev=552.78 lat (msec): min=92, max=1375, avg=598.69, stdev=552.76 clat percentiles (msec): | 1.00th=[ 93], 5.00th=[ 93], 10.00th=[ 93], 20.00th=[ 142], | 30.00th=[ 142], 40.00th=[ 275], 50.00th=[ 275], 60.00th=[ 518], | 70.00th=[ 1183], 80.00th=[ 1183], 90.00th=[ 1368], 95.00th=[ 1368], | 99.00th=[ 1368], 99.50th=[ 1368], 99.90th=[ 1368], 99.95th=[ 1368], | 99.99th=[ 1368] bw ( KiB/s): min= 1992, max= 2892, per=50.51%, avg=2239.75, stdev=435.45, samples=4 iops : min= 1, max= 2, avg= 1.50, stdev= 0.58, samples=4 lat (msec) : 100=25.00%, 250=8.33%, 500=25.00%, 750=16.67%, 2000=25.00% cpu : usr=0.00%, sys=12.79%, ctx=143, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193327: Thu Aug 21 15:05:11 2025 read: IOPS=0, BW=876KiB/s (897kB/s)(5120KiB/5843msec) clat (msec): min=130, max=515, avg=308.28, stdev=175.20 lat (msec): min=130, max=515, avg=308.28, stdev=175.20 clat percentiles (msec): | 1.00th=[ 131], 5.00th=[ 131], 10.00th=[ 131], 20.00th=[ 131], | 30.00th=[ 192], 40.00th=[ 192], 50.00th=[ 228], 60.00th=[ 228], | 70.00th=[ 477], 80.00th=[ 477], 90.00th=[ 518], 95.00th=[ 518], | 99.00th=[ 518], 99.50th=[ 518], 99.90th=[ 518], 99.95th=[ 518], | 99.99th=[ 518] bw ( KiB/s): min= 3792, max= 4079, per=100.00%, avg=3935.50, stdev=202.94, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1227KiB/s (1256kB/s)(7168KiB/5843msec); 0 zone resets clat (msec): min=421, max=1138, avg=607.52, stdev=250.96 lat (msec): min=421, max=1138, avg=607.59, stdev=250.96 clat percentiles (msec): | 1.00th=[ 422], 5.00th=[ 422], 10.00th=[ 422], 20.00th=[ 447], | 30.00th=[ 464], 40.00th=[ 464], 50.00th=[ 523], 60.00th=[ 584], | 70.00th=[ 584], 80.00th=[ 684], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 1499, max= 2048, per=43.04%, avg=1908.29, stdev=191.37, samples=7 iops : min= 1, max= 2, avg= 1.14, stdev= 0.38, samples=7 lat (msec) : 250=25.00%, 500=33.33%, 750=33.33%, 2000=8.33% cpu : usr=0.00%, sys=22.75%, ctx=185, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3168KiB/s (3244kB/s), 340KiB/s-1390KiB/s (348kB/s-1424kB/s), io=20.0MiB (20.0MB), run=5156-6465msec WRITE: bw=4435KiB/s (4541kB/s), 950KiB/s-1699KiB/s (973kB/s-1740kB/s), io=28.0MiB (29.4MB), run=5156-6465msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=193342: Thu Aug 21 15:05:25 2025 read: IOPS=2, BW=9143KiB/s (9362kB/s)(12.0MiB/1344msec) clat (msec): min=2, max=823, avg=424.95, stdev=411.12 lat (msec): min=2, max=823, avg=424.95, stdev=411.12 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 3], 20.00th=[ 3], | 30.00th=[ 3], 40.00th=[ 451], 50.00th=[ 451], 60.00th=[ 451], | 70.00th=[ 827], 80.00th=[ 827], 90.00th=[ 827], 95.00th=[ 827], | 99.00th=[ 827], 99.50th=[ 827], 99.90th=[ 827], 99.95th=[ 827], | 99.99th=[ 827] bw ( KiB/s): min= 5712, max= 5712, per=100.00%, avg=5712.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 4=33.33%, 500=33.33%, 1000=33.33% cpu : usr=0.00%, sys=43.86%, ctx=1004, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193343: Thu Aug 21 15:05:25 2025 read: IOPS=0, BW=1330KiB/s (1362kB/s)(4096KiB/3080msec) clat (nsec): min=487156k, max=487156k, avg=487156159.00, stdev= 0.00 lat (nsec): min=487157k, max=487157k, avg=487157069.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 489], 5.00th=[ 489], 10.00th=[ 489], 20.00th=[ 489], | 30.00th=[ 489], 40.00th=[ 489], 50.00th=[ 489], 60.00th=[ 489], | 70.00th=[ 489], 80.00th=[ 489], 90.00th=[ 489], 95.00th=[ 489], | 99.00th=[ 489], 99.50th=[ 489], 99.90th=[ 489], 99.95th=[ 489], | 99.99th=[ 489] bw ( KiB/s): min= 5595, max= 5595, per=100.00%, avg=5595.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2660KiB/s (2724kB/s)(8192KiB/3080msec); 0 zone resets clat (msec): min=1142, max=1401, avg=1271.74, stdev=183.19 lat (msec): min=1142, max=1401, avg=1272.01, stdev=183.27 clat percentiles (msec): | 1.00th=[ 1150], 5.00th=[ 1150], 10.00th=[ 1150], 20.00th=[ 1150], | 30.00th=[ 1150], 40.00th=[ 1150], 50.00th=[ 1150], 60.00th=[ 1401], | 70.00th=[ 1401], 80.00th=[ 1401], 90.00th=[ 1401], 95.00th=[ 1401], | 99.00th=[ 1401], 99.50th=[ 1401], 99.90th=[ 1401], 99.95th=[ 1401], | 99.99th=[ 1401] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.00%, sys=64.11%, ctx=487, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193344: Thu Aug 21 15:05:25 2025 write: IOPS=0, BW=2981KiB/s (3053kB/s)(12.0MiB/4122msec); 0 zone resets clat (msec): min=701, max=1745, avg=1364.23, stdev=576.09 lat (msec): min=701, max=1746, avg=1364.48, stdev=576.13 clat percentiles (msec): | 1.00th=[ 701], 5.00th=[ 701], 10.00th=[ 701], 20.00th=[ 701], | 30.00th=[ 701], 40.00th=[ 1653], 50.00th=[ 1653], 60.00th=[ 1653], | 70.00th=[ 1754], 80.00th=[ 1754], 90.00th=[ 1754], 95.00th=[ 1754], | 99.00th=[ 1754], 99.50th=[ 1754], 99.90th=[ 1754], 99.95th=[ 1754], | 99.99th=[ 1754] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=46.32%, ctx=25, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193345: Thu Aug 21 15:05:25 2025 write: IOPS=0, BW=3606KiB/s (3692kB/s)(12.0MiB/3408msec); 0 zone resets clat (msec): min=704, max=1434, avg=1119.62, stdev=374.98 lat (msec): min=708, max=1434, avg=1121.00, stdev=373.14 clat percentiles (msec): | 1.00th=[ 709], 5.00th=[ 709], 10.00th=[ 709], 20.00th=[ 709], | 30.00th=[ 709], 40.00th=[ 1217], 50.00th=[ 1217], 60.00th=[ 1217], | 70.00th=[ 1435], 80.00th=[ 1435], 90.00th=[ 1435], 95.00th=[ 1435], | 99.00th=[ 1435], 99.50th=[ 1435], 99.90th=[ 1435], 99.95th=[ 1435], | 99.99th=[ 1435] bw ( KiB/s): min= 8000, max= 8192, per=100.00%, avg=8096.00, stdev=135.76, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=57.35%, ctx=82, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5319KiB/s (5447kB/s), 1330KiB/s-9143KiB/s (1362kB/s-9362kB/s), io=16.0MiB (16.8MB), run=1344-3080msec WRITE: bw=7950KiB/s (8140kB/s), 2660KiB/s-3606KiB/s (2724kB/s-3692kB/s), io=32.0MiB (33.6MB), run=3080-4122msec rand-rw: (groupid=0, jobs=1): err= 0: pid=193338: Thu Aug 21 15:05:26 2025 read: IOPS=1, BW=4160KiB/s (4260kB/s)(12.0MiB/2954msec) clat (msec): min=272, max=2283, avg=960.35, stdev=1145.78 lat (msec): min=272, max=2283, avg=960.35, stdev=1145.78 clat percentiles (msec): | 1.00th=[ 275], 5.00th=[ 275], 10.00th=[ 275], 20.00th=[ 275], | 30.00th=[ 275], 40.00th=[ 326], 50.00th=[ 326], 60.00th=[ 326], | 70.00th=[ 2299], 80.00th=[ 2299], 90.00th=[ 2299], 95.00th=[ 2299], | 99.00th=[ 2299], 99.50th=[ 2299], 99.90th=[ 2299], 99.95th=[ 2299], | 99.99th=[ 2299] bw ( KiB/s): min= 7613, max= 8159, per=100.00%, avg=7886.00, stdev=386.08, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=12.43%, ctx=72, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193339: Thu Aug 21 15:05:26 2025 read: IOPS=0, BW=953KiB/s (976kB/s)(4096KiB/4299msec) clat (nsec): min=583429k, max=583429k, avg=583429480.00, stdev= 0.00 lat (nsec): min=583430k, max=583430k, avg=583430436.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 584], 5.00th=[ 584], 10.00th=[ 584], 20.00th=[ 584], | 30.00th=[ 584], 40.00th=[ 584], 50.00th=[ 584], 60.00th=[ 584], | 70.00th=[ 584], 80.00th=[ 584], 90.00th=[ 584], 95.00th=[ 584], | 99.00th=[ 584], 99.50th=[ 584], 99.90th=[ 584], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1906KiB/s (1951kB/s)(8192KiB/4299msec); 0 zone resets clat (msec): min=1275, max=2393, avg=1834.23, stdev=790.35 lat (msec): min=1275, max=2393, avg=1834.39, stdev=790.27 clat percentiles (msec): | 1.00th=[ 1284], 5.00th=[ 1284], 10.00th=[ 1284], 20.00th=[ 1284], | 30.00th=[ 1284], 40.00th=[ 1284], 50.00th=[ 1284], 60.00th=[ 2400], | 70.00th=[ 2400], 80.00th=[ 2400], 90.00th=[ 2400], 95.00th=[ 2400], | 99.00th=[ 2400], 99.50th=[ 2400], 99.90th=[ 2400], 99.95th=[ 2400], | 99.99th=[ 2400] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=20.01%, ctx=120, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193340: Thu Aug 21 15:05:26 2025 write: IOPS=0, BW=2668KiB/s (2732kB/s)(12.0MiB/4606msec); 0 zone resets clat (msec): min=494, max=2812, avg=1522.54, stdev=1180.65 lat (msec): min=495, max=2812, avg=1522.82, stdev=1180.63 clat percentiles (msec): | 1.00th=[ 493], 5.00th=[ 493], 10.00th=[ 493], 20.00th=[ 493], | 30.00th=[ 493], 40.00th=[ 1267], 50.00th=[ 1267], 60.00th=[ 1267], | 70.00th=[ 2802], 80.00th=[ 2802], 90.00th=[ 2802], 95.00th=[ 2802], | 99.00th=[ 2802], 99.50th=[ 2802], 99.90th=[ 2802], 99.95th=[ 2802], | 99.99th=[ 2802] bw ( KiB/s): min= 8159, max= 8192, per=100.00%, avg=8175.50, stdev=23.33, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=21.63%, ctx=116, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193341: Thu Aug 21 15:05:26 2025 write: IOPS=1, BW=4669KiB/s (4781kB/s)(12.0MiB/2632msec); 0 zone resets clat (msec): min=123, max=2250, avg=868.41, stdev=1198.12 lat (msec): min=123, max=2250, avg=868.55, stdev=1198.08 clat percentiles (msec): | 1.00th=[ 125], 5.00th=[ 125], 10.00th=[ 125], 20.00th=[ 125], | 30.00th=[ 125], 40.00th=[ 232], 50.00th=[ 232], 60.00th=[ 232], | 70.00th=[ 2265], 80.00th=[ 2265], 90.00th=[ 2265], 95.00th=[ 2265], | 99.00th=[ 2265], 99.50th=[ 2265], 99.90th=[ 2265], 99.95th=[ 2265], | 99.99th=[ 2265] bw ( KiB/s): min=11040, max=11040, per=100.00%, avg=11040.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=21.21%, ctx=128, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3811KiB/s (3903kB/s), 953KiB/s-4160KiB/s (976kB/s-4260kB/s), io=16.0MiB (16.8MB), run=2954-4299msec WRITE: bw=7114KiB/s (7285kB/s), 1906KiB/s-4669KiB/s (1951kB/s-4781kB/s), io=32.0MiB (33.6MB), run=2632-4606msec