== sanity test 398b: DIO and buffer IO race ============== 11:23:37 (1754407417) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.35486 s, 15.0 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147471: Tue Aug 5 11:24:53 2025 read: IOPS=22, BW=91.3KiB/s (93.4kB/s)(5992KiB/65661msec) clat (usec): min=105, max=908878, avg=23238.27, stdev=26325.22 lat (usec): min=106, max=908880, avg=23243.94, stdev=26327.09 clat percentiles (usec): | 1.00th=[ 498], 5.00th=[ 2868], 10.00th=[ 10290], 20.00th=[ 13698], | 30.00th=[ 15795], 40.00th=[ 17957], 50.00th=[ 20579], 60.00th=[ 23725], | 70.00th=[ 26870], 80.00th=[ 30802], 90.00th=[ 36963], 95.00th=[ 43779], | 99.00th=[ 74974], 99.50th=[ 84411], 99.90th=[120062], 99.95th=[910164], | 99.99th=[910164] bw ( KiB/s): min= 15, max= 176, per=24.97%, avg=91.16, stdev=24.67, samples=129 iops : min= 3, max= 44, avg=22.35, stdev= 6.22, samples=129 write: IOPS=23, BW=95.9KiB/s (98.2kB/s)(6296KiB/65661msec); 0 zone resets clat (usec): min=856, max=163968, avg=19398.34, stdev=15493.08 lat (usec): min=857, max=163970, avg=19400.04, stdev=15493.18 clat percentiles (usec): | 1.00th=[ 1254], 5.00th=[ 3032], 10.00th=[ 4228], 20.00th=[ 5866], | 30.00th=[ 8717], 40.00th=[ 13042], 50.00th=[ 16450], 60.00th=[ 20055], | 70.00th=[ 23987], 80.00th=[ 29754], 90.00th=[ 39060], 95.00th=[ 46924], | 99.00th=[ 70779], 99.50th=[ 86508], 99.90th=[116917], 99.95th=[164627], | 99.99th=[164627] bw ( KiB/s): min= 39, max= 152, per=25.19%, avg=96.23, stdev=25.77, samples=128 iops : min= 9, max= 38, avg=23.63, stdev= 6.55, samples=128 lat (usec) : 250=0.13%, 500=0.39%, 750=1.01%, 1000=0.75% lat (msec) : 2=1.14%, 4=3.61%, 10=14.36%, 20=32.65%, 50=42.58% lat (msec) : 100=3.09%, 250=0.26%, 1000=0.03% cpu : usr=0.13%, sys=28.84%, ctx=9217, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147473: Tue Aug 5 11:24:53 2025 read: IOPS=22, BW=90.4KiB/s (92.6kB/s)(5940KiB/65697msec) clat (usec): min=96, max=86542, avg=22806.36, stdev=11976.96 lat (usec): min=97, max=86543, avg=22807.82, stdev=11977.04 clat percentiles (usec): | 1.00th=[ 429], 5.00th=[ 7439], 10.00th=[10683], 20.00th=[13829], | 30.00th=[16188], 40.00th=[18220], 50.00th=[21103], 60.00th=[23987], | 70.00th=[26870], 80.00th=[31065], 90.00th=[37487], 95.00th=[42730], | 99.00th=[67634], 99.50th=[72877], 99.90th=[84411], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 7, max= 167, per=24.84%, avg=90.67, stdev=26.19, samples=129 iops : min= 1, max= 41, avg=22.22, stdev= 6.59, samples=129 write: IOPS=24, BW=96.6KiB/s (98.9kB/s)(6348KiB/65697msec); 0 zone resets clat (usec): min=734, max=903763, avg=19854.50, stdev=26993.97 lat (usec): min=736, max=903764, avg=19857.11, stdev=26993.60 clat percentiles (usec): | 1.00th=[ 1205], 5.00th=[ 2966], 10.00th=[ 4228], 20.00th=[ 5800], | 30.00th=[ 8717], 40.00th=[ 12911], 50.00th=[ 16188], 60.00th=[ 19530], | 70.00th=[ 23987], 80.00th=[ 30540], 90.00th=[ 39060], 95.00th=[ 47449], | 99.00th=[ 72877], 99.50th=[ 88605], 99.90th=[149947], 99.95th=[901776], | 99.99th=[901776] bw ( KiB/s): min= 7, max= 159, per=25.29%, avg=96.60, stdev=25.16, samples=129 iops : min= 1, max= 39, avg=23.71, stdev= 6.31, samples=129 lat (usec) : 100=0.03%, 250=0.07%, 500=0.55%, 750=0.68%, 1000=0.62% lat (msec) : 2=1.24%, 4=3.45%, 10=14.42%, 20=33.11%, 50=42.48% lat (msec) : 100=3.19%, 250=0.13%, 1000=0.03% cpu : usr=0.16%, sys=28.93%, ctx=9328, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147476: Tue Aug 5 11:24:53 2025 read: IOPS=23, BW=92.4KiB/s (94.6kB/s)(6028KiB/65230msec) clat (usec): min=109, max=140830, avg=22534.25, stdev=12805.74 lat (usec): min=109, max=140833, avg=22535.76, stdev=12805.78 clat percentiles (usec): | 1.00th=[ 285], 5.00th=[ 1188], 10.00th=[ 9896], 20.00th=[ 13435], | 30.00th=[ 15926], 40.00th=[ 18220], 50.00th=[ 20841], 60.00th=[ 23725], | 70.00th=[ 26870], 80.00th=[ 31065], 90.00th=[ 36963], 95.00th=[ 43254], | 99.00th=[ 63701], 99.50th=[ 74974], 99.90th=[108528], 99.95th=[141558], | 99.99th=[141558] bw ( KiB/s): min= 31, max= 160, per=25.57%, avg=93.35, stdev=26.24, samples=127 iops : min= 7, max= 40, avg=22.91, stdev= 6.67, samples=127 write: IOPS=23, BW=95.0KiB/s (98.3kB/s)(6260KiB/65230msec); 0 zone resets clat (usec): min=697, max=932178, avg=19785.66, stdev=27804.14 lat (usec): min=697, max=932183, avg=19791.22, stdev=27803.15 clat percentiles (usec): | 1.00th=[ 1156], 5.00th=[ 2769], 10.00th=[ 4228], 20.00th=[ 5669], | 30.00th=[ 7963], 40.00th=[ 12387], 50.00th=[ 16188], 60.00th=[ 19530], | 70.00th=[ 23987], 80.00th=[ 30016], 90.00th=[ 39060], 95.00th=[ 46924], | 99.00th=[ 70779], 99.50th=[ 82314], 99.90th=[143655], 99.95th=[935330], | 99.99th=[935330] bw ( KiB/s): min= 31, max= 160, per=25.31%, avg=96.69, stdev=27.63, samples=127 iops : min= 7, max= 40, avg=23.74, stdev= 6.98, samples=127 lat (usec) : 250=0.36%, 500=0.75%, 750=0.88%, 1000=0.65% lat (msec) : 2=1.27%, 4=3.16%, 10=16.05%, 20=30.70%, 50=43.03% lat (msec) : 100=2.90%, 250=0.23%, 1000=0.03% cpu : usr=0.11%, sys=28.72%, ctx=9226, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147477: Tue Aug 5 11:24:53 2025 read: IOPS=23, BW=92.0KiB/s (94.2kB/s)(6052KiB/65767msec) clat (usec): min=170, max=171701, avg=22615.96, stdev=13197.04 lat (usec): min=172, max=171702, avg=22617.41, stdev=13197.11 clat percentiles (usec): | 1.00th=[ 474], 5.00th=[ 5604], 10.00th=[ 10028], 20.00th=[ 12911], | 30.00th=[ 15533], 40.00th=[ 17957], 50.00th=[ 20579], 60.00th=[ 23462], | 70.00th=[ 27132], 80.00th=[ 31327], 90.00th=[ 37487], 95.00th=[ 43779], | 99.00th=[ 63701], 99.50th=[ 76022], 99.90th=[135267], 99.95th=[170918], | 99.99th=[170918] bw ( KiB/s): min= 39, max= 175, per=25.52%, avg=93.15, stdev=23.84, samples=129 iops : min= 9, max= 43, avg=22.84, stdev= 6.02, samples=129 write: IOPS=23, BW=94.8KiB/s (97.1kB/s)(6236KiB/65767msec); 0 zone resets clat (usec): min=749, max=923991, avg=20023.87, stdev=27191.46 lat (usec): min=750, max=923992, avg=20032.03, stdev=27188.97 clat percentiles (usec): | 1.00th=[ 1188], 5.00th=[ 3097], 10.00th=[ 4555], 20.00th=[ 6128], | 30.00th=[ 8848], 40.00th=[ 13173], 50.00th=[ 16450], 60.00th=[ 20317], | 70.00th=[ 24773], 80.00th=[ 30802], 90.00th=[ 39060], 95.00th=[ 46400], | 99.00th=[ 64226], 99.50th=[ 76022], 99.90th=[156238], 99.95th=[926942], | 99.99th=[926942] bw ( KiB/s): min= 40, max= 191, per=25.11%, avg=95.92, stdev=26.61, samples=129 iops : min= 10, max= 47, avg=23.54, stdev= 6.70, samples=129 lat (usec) : 250=0.03%, 500=0.52%, 750=0.94%, 1000=0.59% lat (msec) : 2=1.27%, 4=2.67%, 10=15.72%, 20=31.74%, 50=43.29% lat (msec) : 100=3.06%, 250=0.13%, 1000=0.03% cpu : usr=0.14%, sys=28.75%, ctx=9340, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=365KiB/s (374kB/s), 90.4KiB/s-92.4KiB/s (92.6kB/s-94.6kB/s), io=23.4MiB (24.6MB), run=65230-65767msec WRITE: bw=382KiB/s (391kB/s), 94.8KiB/s-96.6KiB/s (97.1kB/s-98.9kB/s), io=24.6MiB (25.7MB), run=65230-65767msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147470: Tue Aug 5 11:25:25 2025 read: IOPS=15, BW=61.2KiB/s (62.7kB/s)(5992KiB/97828msec) clat (msec): min=3, max=914, avg=20.28, stdev=28.32 lat (msec): min=3, max=914, avg=20.28, stdev=28.32 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 21], 80.00th=[ 28], 90.00th=[ 41], 95.00th=[ 55], | 99.00th=[ 86], 99.50th=[ 101], 99.90th=[ 130], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 7, max= 184, per=25.14%, avg=61.59, stdev=35.16, samples=193 iops : min= 1, max= 46, avg=15.02, stdev= 8.92, samples=193 write: IOPS=16, BW=64.4KiB/s (65.9kB/s)(6296KiB/97828msec); 0 zone resets clat (msec): min=15, max=214, avg=42.66, stdev=21.29 lat (msec): min=15, max=214, avg=42.66, stdev=21.29 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 22], 10.00th=[ 24], 20.00th=[ 27], | 30.00th=[ 29], 40.00th=[ 33], 50.00th=[ 36], 60.00th=[ 42], | 70.00th=[ 49], 80.00th=[ 58], 90.00th=[ 71], 95.00th=[ 82], | 99.00th=[ 120], 99.50th=[ 140], 99.90th=[ 188], 99.95th=[ 215], | 99.99th=[ 215] bw ( KiB/s): min= 15, max= 136, per=25.26%, avg=64.66, stdev=29.03, samples=193 iops : min= 3, max= 34, avg=15.79, stdev= 7.41, samples=193 lat (msec) : 4=0.62%, 10=12.99%, 20=21.32%, 50=47.30%, 100=16.57% lat (msec) : 250=1.17%, 1000=0.03% cpu : usr=0.10%, sys=9.88%, ctx=4902, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147472: Tue Aug 5 11:25:25 2025 read: IOPS=15, BW=60.8KiB/s (62.2kB/s)(5940KiB/97763msec) clat (msec): min=3, max=914, avg=19.91, stdev=28.06 lat (msec): min=3, max=914, avg=19.91, stdev=28.06 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 21], 80.00th=[ 27], 90.00th=[ 39], 95.00th=[ 51], | 99.00th=[ 83], 99.50th=[ 99], 99.90th=[ 163], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 7, max= 168, per=25.03%, avg=61.32, stdev=33.42, samples=191 iops : min= 1, max= 42, avg=14.95, stdev= 8.48, samples=191 write: IOPS=16, BW=64.9KiB/s (66.5kB/s)(6348KiB/97763msec); 0 zone resets clat (msec): min=14, max=194, avg=42.78, stdev=21.14 lat (msec): min=14, max=194, avg=42.79, stdev=21.14 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 27], | 30.00th=[ 30], 40.00th=[ 33], 50.00th=[ 36], 60.00th=[ 42], | 70.00th=[ 50], 80.00th=[ 58], 90.00th=[ 70], 95.00th=[ 82], | 99.00th=[ 116], 99.50th=[ 136], 99.90th=[ 159], 99.95th=[ 194], | 99.99th=[ 194] bw ( KiB/s): min= 23, max= 136, per=25.44%, avg=65.12, stdev=29.21, samples=192 iops : min= 5, max= 34, avg=15.90, stdev= 7.46, samples=192 lat (msec) : 4=0.91%, 10=12.86%, 20=21.16%, 50=47.56%, 100=16.15% lat (msec) : 250=1.33%, 1000=0.03% cpu : usr=0.10%, sys=9.84%, ctx=4842, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147474: Tue Aug 5 11:25:25 2025 read: IOPS=15, BW=61.5KiB/s (63.0kB/s)(6028KiB/97960msec) clat (msec): min=2, max=121, avg=19.42, stdev=15.51 lat (msec): min=2, max=121, avg=19.42, stdev=15.51 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 27], 90.00th=[ 39], 95.00th=[ 52], | 99.00th=[ 80], 99.50th=[ 87], 99.90th=[ 116], 99.95th=[ 122], | 99.99th=[ 122] bw ( KiB/s): min= 7, max= 183, per=25.19%, avg=61.72, stdev=33.10, samples=193 iops : min= 1, max= 45, avg=15.05, stdev= 8.39, samples=193 write: IOPS=15, BW=63.9KiB/s (65.4kB/s)(6260KiB/97960msec); 0 zone resets clat (msec): min=12, max=916, avg=43.70, stdev=31.87 lat (msec): min=12, max=916, avg=43.71, stdev=31.87 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 27], | 30.00th=[ 30], 40.00th=[ 33], 50.00th=[ 36], 60.00th=[ 41], | 70.00th=[ 49], 80.00th=[ 59], 90.00th=[ 72], 95.00th=[ 86], | 99.00th=[ 131], 99.50th=[ 167], 99.90th=[ 209], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 8, max= 128, per=25.00%, avg=63.99, stdev=29.31, samples=193 iops : min= 2, max= 32, avg=15.62, stdev= 7.48, samples=193 lat (msec) : 4=0.55%, 10=13.54%, 20=20.80%, 50=48.11%, 100=15.59% lat (msec) : 250=1.37%, 1000=0.03% cpu : usr=0.09%, sys=9.87%, ctx=4939, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147475: Tue Aug 5 11:25:25 2025 read: IOPS=15, BW=62.0KiB/s (63.5kB/s)(6052KiB/97587msec) clat (msec): min=3, max=134, avg=20.15, stdev=17.17 lat (msec): min=3, max=134, avg=20.15, stdev=17.17 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 12], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 27], 90.00th=[ 40], 95.00th=[ 55], | 99.00th=[ 90], 99.50th=[ 109], 99.90th=[ 130], 99.95th=[ 136], | 99.99th=[ 136] bw ( KiB/s): min= 7, max= 199, per=25.44%, avg=62.32, stdev=34.55, samples=192 iops : min= 1, max= 49, avg=15.20, stdev= 8.76, samples=192 write: IOPS=15, BW=63.9KiB/s (65.4kB/s)(6236KiB/97587msec); 0 zone resets clat (msec): min=15, max=917, avg=42.85, stdev=31.14 lat (msec): min=15, max=917, avg=42.85, stdev=31.14 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 26], | 30.00th=[ 29], 40.00th=[ 32], 50.00th=[ 36], 60.00th=[ 41], | 70.00th=[ 48], 80.00th=[ 58], 90.00th=[ 70], 95.00th=[ 83], | 99.00th=[ 113], 99.50th=[ 163], 99.90th=[ 215], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 15, max= 128, per=25.05%, avg=64.12, stdev=29.39, samples=192 iops : min= 3, max= 32, avg=15.65, stdev= 7.51, samples=192 lat (msec) : 4=0.62%, 10=12.70%, 20=21.52%, 50=48.24%, 100=15.36% lat (msec) : 250=1.53%, 1000=0.03% cpu : usr=0.11%, sys=9.97%, ctx=4908, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=245KiB/s (251kB/s), 60.8KiB/s-62.0KiB/s (62.2kB/s-63.5kB/s), io=23.4MiB (24.6MB), run=97587-97960msec WRITE: bw=257KiB/s (263kB/s), 63.9KiB/s-64.9KiB/s (65.4kB/s-66.5kB/s), io=24.6MiB (25.7MB), run=97587-97960msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147508: Tue Aug 5 11:25:43 2025 read: IOPS=30, BW=496KiB/s (508kB/s)(5920KiB/11938msec) clat (usec): min=185, max=68617, avg=16001.12, stdev=9865.36 lat (usec): min=186, max=68618, avg=16002.10, stdev=9865.38 clat percentiles (usec): | 1.00th=[ 219], 5.00th=[ 375], 10.00th=[ 578], 20.00th=[ 9241], | 30.00th=[11207], 40.00th=[13173], 50.00th=[14877], 60.00th=[17433], | 70.00th=[20055], 80.00th=[23200], 90.00th=[28181], 95.00th=[33162], | 99.00th=[43779], 99.50th=[47973], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 318, max= 704, per=26.13%, avg=497.83, stdev=113.49, samples=23 iops : min= 19, max= 44, avg=30.65, stdev= 7.25, samples=23 write: IOPS=33, BW=533KiB/s (546kB/s)(6368KiB/11938msec); 0 zone resets clat (usec): min=1250, max=123198, avg=15048.23, stdev=14045.26 lat (usec): min=1252, max=123200, avg=15051.70, stdev=14045.08 clat percentiles (usec): | 1.00th=[ 1369], 5.00th=[ 3064], 10.00th=[ 3916], 20.00th=[ 4555], | 30.00th=[ 5604], 40.00th=[ 8029], 50.00th=[ 11076], 60.00th=[ 13698], | 70.00th=[ 17695], 80.00th=[ 23200], 90.00th=[ 32375], 95.00th=[ 42206], | 99.00th=[ 71828], 99.50th=[ 84411], 99.90th=[123208], 99.95th=[123208], | 99.99th=[123208] bw ( KiB/s): min= 319, max= 736, per=26.97%, avg=540.52, stdev=121.37, samples=23 iops : min= 19, max= 46, avg=33.30, stdev= 7.61, samples=23 lat (usec) : 250=0.52%, 500=3.39%, 750=1.30%, 1000=0.91% lat (msec) : 2=1.69%, 4=4.43%, 10=23.31%, 20=37.24%, 50=25.52% lat (msec) : 100=1.56%, 250=0.13% cpu : usr=0.17%, sys=32.14%, ctx=2194, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147510: Tue Aug 5 11:25:43 2025 read: IOPS=28, BW=458KiB/s (469kB/s)(5744KiB/12530msec) clat (usec): min=101, max=73217, avg=17060.94, stdev=11371.27 lat (usec): min=101, max=73218, avg=17061.86, stdev=11371.30 clat percentiles (usec): | 1.00th=[ 176], 5.00th=[ 334], 10.00th=[ 486], 20.00th=[ 9110], | 30.00th=[11469], 40.00th=[13566], 50.00th=[15795], 60.00th=[18220], | 70.00th=[21627], 80.00th=[25035], 90.00th=[31065], 95.00th=[35390], | 99.00th=[54264], 99.50th=[56886], 99.90th=[72877], 99.95th=[72877], | 99.99th=[72877] bw ( KiB/s): min= 58, max= 704, per=24.01%, avg=457.38, stdev=127.09, samples=24 iops : min= 3, max= 44, avg=28.12, stdev= 8.05, samples=24 write: IOPS=32, BW=522KiB/s (535kB/s)(6544KiB/12530msec); 0 zone resets clat (usec): min=1007, max=113048, avg=15491.19, stdev=14681.85 lat (usec): min=1009, max=113051, avg=15492.87, stdev=14681.88 clat percentiles (usec): | 1.00th=[ 1565], 5.00th=[ 2835], 10.00th=[ 3556], 20.00th=[ 4490], | 30.00th=[ 5538], 40.00th=[ 8160], 50.00th=[ 11076], 60.00th=[ 13960], | 70.00th=[ 18220], 80.00th=[ 24511], 90.00th=[ 32900], 95.00th=[ 43779], | 99.00th=[ 76022], 99.50th=[ 86508], 99.90th=[112722], 99.95th=[112722], | 99.99th=[112722] bw ( KiB/s): min= 286, max= 830, per=25.50%, avg=511.04, stdev=150.88, samples=24 iops : min= 17, max= 51, avg=31.50, stdev= 9.49, samples=24 lat (usec) : 250=1.04%, 500=3.65%, 750=1.30%, 1000=0.26% lat (msec) : 2=1.69%, 4=6.51%, 10=21.61%, 20=34.24%, 50=27.60% lat (msec) : 100=1.95%, 250=0.13% cpu : usr=0.15%, sys=31.47%, ctx=2195, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147512: Tue Aug 5 11:25:43 2025 read: IOPS=31, BW=498KiB/s (510kB/s)(6256KiB/12570msec) clat (usec): min=73, max=71479, avg=16347.92, stdev=11576.57 lat (usec): min=74, max=71479, avg=16349.20, stdev=11576.32 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 359], 10.00th=[ 441], 20.00th=[ 8094], | 30.00th=[10683], 40.00th=[13042], 50.00th=[14877], 60.00th=[17433], | 70.00th=[20841], 80.00th=[24773], 90.00th=[31851], 95.00th=[36439], | 99.00th=[54789], 99.50th=[64226], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min= 349, max= 768, per=26.04%, avg=496.04, stdev=116.83, samples=24 iops : min= 21, max= 48, avg=30.54, stdev= 7.32, samples=24 write: IOPS=29, BW=480KiB/s (491kB/s)(6032KiB/12570msec); 0 zone resets clat (usec): min=996, max=133076, avg=16164.19, stdev=14871.02 lat (usec): min=998, max=133079, avg=16166.31, stdev=14870.89 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 2671], 10.00th=[ 3752], 20.00th=[ 4686], | 30.00th=[ 5866], 40.00th=[ 8586], 50.00th=[ 12256], 60.00th=[ 15664], | 70.00th=[ 20841], 80.00th=[ 25297], 90.00th=[ 33817], 95.00th=[ 38011], | 99.00th=[ 78119], 99.50th=[ 99091], 99.90th=[132645], 99.95th=[132645], | 99.99th=[132645] bw ( KiB/s): min= 159, max= 768, per=23.15%, avg=463.88, stdev=139.28, samples=24 iops : min= 9, max= 48, avg=28.54, stdev= 8.89, samples=24 lat (usec) : 100=0.13%, 250=0.65%, 500=5.21%, 750=1.69%, 1000=0.78% lat (msec) : 2=1.95%, 4=4.17%, 10=20.44%, 20=32.81%, 50=30.34% lat (msec) : 100=1.69%, 250=0.13% cpu : usr=0.11%, sys=30.68%, ctx=2175, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147514: Tue Aug 5 11:25:43 2025 read: IOPS=30, BW=486KiB/s (498kB/s)(6032KiB/12402msec) clat (usec): min=107, max=75023, avg=16330.15, stdev=12199.74 lat (usec): min=108, max=75025, avg=16331.24, stdev=12199.76 clat percentiles (usec): | 1.00th=[ 157], 5.00th=[ 314], 10.00th=[ 392], 20.00th=[ 7373], | 30.00th=[10421], 40.00th=[12911], 50.00th=[15139], 60.00th=[18482], | 70.00th=[20579], 80.00th=[24511], 90.00th=[31327], 95.00th=[37487], | 99.00th=[66323], 99.50th=[69731], 99.90th=[74974], 99.95th=[74974], | 99.99th=[74974] bw ( KiB/s): min= 233, max= 736, per=25.16%, avg=479.33, stdev=148.05, samples=24 iops : min= 14, max= 46, avg=29.50, stdev= 9.33, samples=24 write: IOPS=31, BW=504KiB/s (517kB/s)(6256KiB/12402msec); 0 zone resets clat (usec): min=1001, max=93498, avg=15735.21, stdev=14055.90 lat (usec): min=1003, max=93500, avg=15736.98, stdev=14055.99 clat percentiles (usec): | 1.00th=[ 1647], 5.00th=[ 2606], 10.00th=[ 3982], 20.00th=[ 4752], | 30.00th=[ 5866], 40.00th=[ 8455], 50.00th=[11076], 60.00th=[13960], | 70.00th=[18482], 80.00th=[26084], 90.00th=[34341], 95.00th=[45351], | 99.00th=[63177], 99.50th=[86508], 99.90th=[93848], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 320, max= 701, per=25.17%, avg=504.33, stdev=102.03, samples=24 iops : min= 20, max= 43, avg=31.08, stdev= 6.37, samples=24 lat (usec) : 250=0.78%, 500=6.51%, 750=1.30%, 1000=0.52% lat (msec) : 2=1.43%, 4=3.91%, 10=22.66%, 20=32.94%, 50=27.47% lat (msec) : 100=2.47% cpu : usr=0.16%, sys=31.41%, ctx=2152, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1905KiB/s (1951kB/s), 458KiB/s-498KiB/s (469kB/s-510kB/s), io=23.4MiB (24.5MB), run=11938-12570msec WRITE: bw=2005KiB/s (2053kB/s), 480KiB/s-533KiB/s (491kB/s-546kB/s), io=24.6MiB (25.8MB), run=11938-12570msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147509: Tue Aug 5 11:25:53 2025 read: IOPS=16, BW=264KiB/s (270kB/s)(5920KiB/22458msec) clat (msec): min=3, max=148, avg=18.12, stdev=16.66 lat (msec): min=3, max=148, avg=18.12, stdev=16.66 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 20], 80.00th=[ 24], 90.00th=[ 35], 95.00th=[ 49], | 99.00th=[ 92], 99.50th=[ 101], 99.90th=[ 148], 99.95th=[ 148], | 99.99th=[ 148] bw ( KiB/s): min= 95, max= 544, per=24.23%, avg=256.32, stdev=104.93, samples=44 iops : min= 5, max= 34, avg=15.70, stdev= 6.72, samples=44 write: IOPS=17, BW=284KiB/s (290kB/s)(6368KiB/22458msec); 0 zone resets clat (msec): min=15, max=132, avg=39.36, stdev=17.52 lat (msec): min=15, max=132, avg=39.37, stdev=17.52 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 22], 10.00th=[ 23], 20.00th=[ 26], | 30.00th=[ 30], 40.00th=[ 33], 50.00th=[ 35], 60.00th=[ 39], | 70.00th=[ 43], 80.00th=[ 51], 90.00th=[ 64], 95.00th=[ 73], | 99.00th=[ 110], 99.50th=[ 120], 99.90th=[ 133], 99.95th=[ 133], | 99.99th=[ 133] bw ( KiB/s): min= 95, max= 480, per=25.13%, avg=279.73, stdev=108.95, samples=44 iops : min= 5, max= 30, avg=17.18, stdev= 7.06, samples=44 lat (msec) : 4=1.56%, 10=14.97%, 20=19.79%, 50=50.65%, 100=12.11% lat (msec) : 250=0.91% cpu : usr=0.13%, sys=9.77%, ctx=1242, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147511: Tue Aug 5 11:25:53 2025 read: IOPS=15, BW=254KiB/s (260kB/s)(5744KiB/22624msec) clat (msec): min=3, max=130, avg=17.50, stdev=16.51 lat (msec): min=3, max=130, avg=17.51, stdev=16.51 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 19], 80.00th=[ 22], 90.00th=[ 31], 95.00th=[ 54], | 99.00th=[ 97], 99.50th=[ 123], 99.90th=[ 131], 99.95th=[ 131], | 99.99th=[ 131] bw ( KiB/s): min= 58, max= 608, per=24.06%, avg=254.58, stdev=134.47, samples=45 iops : min= 3, max= 38, avg=15.58, stdev= 8.56, samples=45 write: IOPS=18, BW=289KiB/s (296kB/s)(6544KiB/22624msec); 0 zone resets clat (msec): min=13, max=157, avg=39.86, stdev=20.04 lat (msec): min=13, max=157, avg=39.86, stdev=20.04 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 20], 10.00th=[ 23], 20.00th=[ 26], | 30.00th=[ 29], 40.00th=[ 32], 50.00th=[ 35], 60.00th=[ 38], | 70.00th=[ 43], 80.00th=[ 51], 90.00th=[ 65], 95.00th=[ 86], | 99.00th=[ 113], 99.50th=[ 115], 99.90th=[ 159], 99.95th=[ 159], | 99.99th=[ 159] bw ( KiB/s): min= 126, max= 734, per=25.95%, avg=288.87, stdev=121.74, samples=45 iops : min= 7, max= 45, avg=17.73, stdev= 7.75, samples=45 lat (msec) : 4=1.17%, 10=14.97%, 20=22.14%, 50=48.44%, 100=11.72% lat (msec) : 250=1.56% cpu : usr=0.14%, sys=9.40%, ctx=1237, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147513: Tue Aug 5 11:25:53 2025 read: IOPS=17, BW=282KiB/s (289kB/s)(6256KiB/22204msec) clat (usec): min=3245, max=89840, avg=17676.03, stdev=14202.09 lat (usec): min=3245, max=89841, avg=17677.17, stdev=14202.02 clat percentiles (usec): | 1.00th=[ 3752], 5.00th=[ 4293], 10.00th=[ 5538], 20.00th=[ 6849], | 30.00th=[ 8848], 40.00th=[11076], 50.00th=[13698], 60.00th=[16581], | 70.00th=[19268], 80.00th=[25035], 90.00th=[33817], 95.00th=[49021], | 99.00th=[78119], 99.50th=[81265], 99.90th=[89654], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 95, max= 832, per=26.78%, avg=283.36, stdev=147.81, samples=44 iops : min= 5, max= 52, avg=17.41, stdev= 9.38, samples=44 write: IOPS=16, BW=272KiB/s (278kB/s)(6032KiB/22204msec); 0 zone resets clat (msec): min=19, max=162, avg=40.31, stdev=17.33 lat (msec): min=19, max=162, avg=40.31, stdev=17.33 clat percentiles (msec): | 1.00th=[ 21], 5.00th=[ 23], 10.00th=[ 24], 20.00th=[ 27], | 30.00th=[ 30], 40.00th=[ 34], 50.00th=[ 36], 60.00th=[ 39], | 70.00th=[ 45], 80.00th=[ 52], 90.00th=[ 61], 95.00th=[ 74], | 99.00th=[ 107], 99.50th=[ 111], 99.90th=[ 163], 99.95th=[ 163], | 99.99th=[ 163] bw ( KiB/s): min= 126, max= 448, per=24.28%, avg=270.27, stdev=99.24, samples=44 iops : min= 7, max= 28, avg=16.59, stdev= 6.44, samples=44 lat (msec) : 4=1.30%, 10=16.67%, 20=18.62%, 50=50.00%, 100=12.89% lat (msec) : 250=0.52% cpu : usr=0.13%, sys=9.53%, ctx=1183, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147515: Tue Aug 5 11:25:53 2025 read: IOPS=16, BW=272KiB/s (278kB/s)(6032KiB/22208msec) clat (msec): min=3, max=145, avg=17.07, stdev=15.24 lat (msec): min=3, max=145, avg=17.07, stdev=15.24 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 16], | 70.00th=[ 18], 80.00th=[ 23], 90.00th=[ 36], 95.00th=[ 45], | 99.00th=[ 79], 99.50th=[ 100], 99.90th=[ 146], 99.95th=[ 146], | 99.99th=[ 146] bw ( KiB/s): min= 63, max= 512, per=25.42%, avg=268.93, stdev=118.46, samples=44 iops : min= 3, max= 32, avg=16.50, stdev= 7.58, samples=44 write: IOPS=17, BW=282KiB/s (288kB/s)(6256KiB/22208msec); 0 zone resets clat (msec): min=19, max=149, avg=40.14, stdev=17.76 lat (msec): min=19, max=149, avg=40.14, stdev=17.76 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 22], 10.00th=[ 24], 20.00th=[ 27], | 30.00th=[ 30], 40.00th=[ 32], 50.00th=[ 35], 60.00th=[ 39], | 70.00th=[ 44], 80.00th=[ 53], 90.00th=[ 66], 95.00th=[ 78], | 99.00th=[ 101], 99.50th=[ 111], 99.90th=[ 150], 99.95th=[ 150], | 99.99th=[ 150] bw ( KiB/s): min= 95, max= 544, per=25.26%, avg=281.14, stdev=110.31, samples=44 iops : min= 5, max= 34, avg=17.27, stdev= 7.08, samples=44 lat (msec) : 4=0.91%, 10=16.93%, 20=20.31%, 50=48.57%, 100=12.76% lat (msec) : 250=0.52% cpu : usr=0.13%, sys=9.74%, ctx=1229, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1059KiB/s (1084kB/s), 254KiB/s-282KiB/s (260kB/s-289kB/s), io=23.4MiB (24.5MB), run=22204-22624msec WRITE: bw=1114KiB/s (1141kB/s), 272KiB/s-289KiB/s (278kB/s-296kB/s), io=24.6MiB (25.8MB), run=22204-22624msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147523: Tue Aug 5 11:26:04 2025 read: IOPS=2, BW=2081KiB/s (2131kB/s)(7168KiB/3445msec) clat (msec): min=50, max=328, avg=168.94, stdev=108.81 lat (msec): min=50, max=328, avg=168.94, stdev=108.81 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 51], 10.00th=[ 51], 20.00th=[ 51], | 30.00th=[ 117], 40.00th=[ 117], 50.00th=[ 153], 60.00th=[ 197], | 70.00th=[ 197], 80.00th=[ 288], 90.00th=[ 330], 95.00th=[ 330], | 99.00th=[ 330], 99.50th=[ 330], 99.90th=[ 330], 99.95th=[ 330], | 99.99th=[ 330] bw ( KiB/s): min= 2048, max= 5154, per=53.17%, avg=2824.50, stdev=1553.00, samples=4 iops : min= 2, max= 5, avg= 2.75, stdev= 1.50, samples=4 write: IOPS=1, BW=1486KiB/s (1522kB/s)(5120KiB/3445msec); 0 zone resets clat (msec): min=153, max=1355, avg=449.81, stdev=508.22 lat (msec): min=153, max=1356, avg=449.87, stdev=508.22 clat percentiles (msec): | 1.00th=[ 153], 5.00th=[ 153], 10.00th=[ 153], 20.00th=[ 153], | 30.00th=[ 234], 40.00th=[ 234], 50.00th=[ 251], 60.00th=[ 251], | 70.00th=[ 255], 80.00th=[ 255], 90.00th=[ 1351], 95.00th=[ 1351], | 99.00th=[ 1351], 99.50th=[ 1351], 99.90th=[ 1351], 99.95th=[ 1351], | 99.99th=[ 1351] bw ( KiB/s): min= 2048, max= 4096, per=36.72%, avg=2730.67, stdev=1182.41, samples=3 iops : min= 2, max= 4, avg= 2.67, stdev= 1.15, samples=3 lat (msec) : 100=16.67%, 250=41.67%, 500=33.33%, 2000=8.33% cpu : usr=0.00%, sys=15.24%, ctx=320, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147525: Tue Aug 5 11:26:04 2025 read: IOPS=0, BW=531KiB/s (544kB/s)(2048KiB/3855msec) clat (usec): min=80789, max=99296, avg=90043.01, stdev=13087.01 lat (usec): min=80790, max=99299, avg=90044.76, stdev=13087.77 clat percentiles (usec): | 1.00th=[81265], 5.00th=[81265], 10.00th=[81265], 20.00th=[81265], | 30.00th=[81265], 40.00th=[81265], 50.00th=[81265], 60.00th=[99091], | 70.00th=[99091], 80.00th=[99091], 90.00th=[99091], 95.00th=[99091], | 99.00th=[99091], 99.50th=[99091], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 1712, max= 2048, per=35.39%, avg=1880.00, stdev=237.59, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2656KiB/s (2720kB/s)(10.0MiB/3855msec); 0 zone resets clat (msec): min=102, max=1043, avg=363.33, stdev=287.35 lat (msec): min=102, max=1043, avg=363.42, stdev=287.35 clat percentiles (msec): | 1.00th=[ 103], 5.00th=[ 103], 10.00th=[ 103], 20.00th=[ 125], | 30.00th=[ 194], 40.00th=[ 209], 50.00th=[ 255], 60.00th=[ 271], | 70.00th=[ 342], 80.00th=[ 468], 90.00th=[ 625], 95.00th=[ 1045], | 99.00th=[ 1045], 99.50th=[ 1045], 99.90th=[ 1045], 99.95th=[ 1045], | 99.99th=[ 1045] bw ( KiB/s): min= 2007, max= 6144, per=43.95%, avg=3268.60, stdev=1838.74, samples=5 iops : min= 1, max= 6, avg= 3.00, stdev= 2.00, samples=5 lat (msec) : 100=16.67%, 250=33.33%, 500=33.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=18.86%, ctx=407, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147526: Tue Aug 5 11:26:04 2025 read: IOPS=1, BW=1824KiB/s (1868kB/s)(6144KiB/3368msec) clat (msec): min=64, max=707, avg=262.08, stdev=239.99 lat (msec): min=64, max=707, avg=262.08, stdev=239.99 clat percentiles (msec): | 1.00th=[ 65], 5.00th=[ 65], 10.00th=[ 65], 20.00th=[ 73], | 30.00th=[ 73], 40.00th=[ 159], 50.00th=[ 159], 60.00th=[ 245], | 70.00th=[ 326], 80.00th=[ 326], 90.00th=[ 709], 95.00th=[ 709], | 99.00th=[ 709], 99.50th=[ 709], 99.90th=[ 709], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 2048, max= 4096, per=64.26%, avg=3413.33, stdev=1182.41, samples=3 iops : min= 2, max= 4, avg= 3.33, stdev= 1.15, samples=3 write: IOPS=1, BW=1824KiB/s (1868kB/s)(6144KiB/3368msec); 0 zone resets clat (msec): min=174, max=507, avg=291.74, stdev=154.26 lat (msec): min=174, max=508, avg=291.83, stdev=154.26 clat percentiles (msec): | 1.00th=[ 176], 5.00th=[ 176], 10.00th=[ 176], 20.00th=[ 186], | 30.00th=[ 186], 40.00th=[ 192], 50.00th=[ 192], 60.00th=[ 222], | 70.00th=[ 472], 80.00th=[ 472], 90.00th=[ 510], 95.00th=[ 510], | 99.00th=[ 510], 99.50th=[ 510], 99.90th=[ 510], 99.95th=[ 510], | 99.99th=[ 510] bw ( KiB/s): min= 1723, max= 2048, per=26.81%, avg=1993.83, stdev=132.68, samples=6 iops : min= 1, max= 2, avg= 1.83, stdev= 0.41, samples=6 lat (msec) : 100=16.67%, 250=50.00%, 500=16.67%, 750=16.67% cpu : usr=0.03%, sys=14.40%, ctx=118, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147527: Tue Aug 5 11:26:04 2025 read: IOPS=1, BW=1408KiB/s (1442kB/s)(5120KiB/3636msec) clat (msec): min=57, max=407, avg=157.20, stdev=148.63 lat (msec): min=57, max=407, avg=157.20, stdev=148.63 clat percentiles (msec): | 1.00th=[ 57], 5.00th=[ 57], 10.00th=[ 57], 20.00th=[ 57], | 30.00th=[ 64], 40.00th=[ 64], 50.00th=[ 79], 60.00th=[ 79], | 70.00th=[ 182], 80.00th=[ 182], 90.00th=[ 409], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 4096, max= 4096, per=77.11%, avg=4096.00, stdev= 0.00, samples=2 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1971KiB/s (2019kB/s)(7168KiB/3636msec); 0 zone resets clat (msec): min=107, max=898, avg=399.22, stdev=281.86 lat (msec): min=107, max=898, avg=399.30, stdev=281.88 clat percentiles (msec): | 1.00th=[ 108], 5.00th=[ 108], 10.00th=[ 108], 20.00th=[ 142], | 30.00th=[ 199], 40.00th=[ 199], 50.00th=[ 414], 60.00th=[ 456], | 70.00th=[ 456], 80.00th=[ 584], 90.00th=[ 902], 95.00th=[ 902], | 99.00th=[ 902], 99.50th=[ 902], 99.90th=[ 902], 99.95th=[ 902], | 99.99th=[ 902] bw ( KiB/s): min= 1723, max= 4015, per=31.22%, avg=2321.67, stdev=839.68, samples=6 iops : min= 1, max= 3, avg= 2.00, stdev= 0.63, samples=6 lat (msec) : 100=25.00%, 250=33.33%, 500=25.00%, 750=8.33%, 1000=8.33% cpu : usr=0.00%, sys=14.25%, ctx=330, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5313KiB/s (5440kB/s), 531KiB/s-2081KiB/s (544kB/s-2131kB/s), io=20.0MiB (20.0MB), run=3368-3855msec WRITE: bw=7438KiB/s (7616kB/s), 1486KiB/s-2656KiB/s (1522kB/s-2720kB/s), io=28.0MiB (29.4MB), run=3368-3855msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147528: Tue Aug 5 11:26:04 2025 read: IOPS=1, BW=1960KiB/s (2007kB/s)(7168KiB/3657msec) clat (msec): min=116, max=278, avg=194.37, stdev=53.22 lat (msec): min=116, max=278, avg=194.37, stdev=53.22 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 148], | 30.00th=[ 186], 40.00th=[ 186], 50.00th=[ 188], 60.00th=[ 215], | 70.00th=[ 215], 80.00th=[ 230], 90.00th=[ 279], 95.00th=[ 279], | 99.00th=[ 279], 99.50th=[ 279], 99.90th=[ 279], 99.95th=[ 279], | 99.99th=[ 279] bw ( KiB/s): min= 2035, max= 3871, per=54.79%, avg=2866.50, stdev=963.85, samples=4 iops : min= 1, max= 3, avg= 2.25, stdev= 0.96, samples=4 write: IOPS=1, BW=1400KiB/s (1434kB/s)(5120KiB/3657msec); 0 zone resets clat (msec): min=155, max=1188, avg=448.70, stdev=434.70 lat (msec): min=155, max=1188, avg=448.78, stdev=434.70 clat percentiles (msec): | 1.00th=[ 157], 5.00th=[ 157], 10.00th=[ 157], 20.00th=[ 157], | 30.00th=[ 184], 40.00th=[ 184], 50.00th=[ 224], 60.00th=[ 224], | 70.00th=[ 493], 80.00th=[ 493], 90.00th=[ 1183], 95.00th=[ 1183], | 99.00th=[ 1183], 99.50th=[ 1183], 99.90th=[ 1183], 99.95th=[ 1183], | 99.99th=[ 1183] bw ( KiB/s): min= 2035, max= 4087, per=34.84%, avg=2552.25, stdev=1023.18, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 250=75.00%, 500=16.67%, 2000=8.33% cpu : usr=0.00%, sys=38.68%, ctx=1203, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147529: Tue Aug 5 11:26:04 2025 read: IOPS=0, BW=523KiB/s (536kB/s)(2048KiB/3914msec) clat (usec): min=779, max=191830, avg=96305.43, stdev=135093.54 lat (usec): min=780, max=191833, avg=96307.02, stdev=135094.49 clat percentiles (usec): | 1.00th=[ 783], 5.00th=[ 783], 10.00th=[ 783], 20.00th=[ 783], | 30.00th=[ 783], 40.00th=[ 783], 50.00th=[ 783], 60.00th=[191890], | 70.00th=[191890], 80.00th=[191890], 90.00th=[191890], 95.00th=[191890], | 99.00th=[191890], 99.50th=[191890], 99.90th=[191890], 99.95th=[191890], | 99.99th=[191890] bw ( KiB/s): min= 1759, max= 2048, per=36.38%, avg=1903.50, stdev=204.35, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2616KiB/s (2679kB/s)(10.0MiB/3914msec); 0 zone resets clat (msec): min=161, max=763, avg=360.48, stdev=218.39 lat (msec): min=161, max=763, avg=360.59, stdev=218.39 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 171], | 30.00th=[ 188], 40.00th=[ 234], 50.00th=[ 284], 60.00th=[ 326], | 70.00th=[ 351], 80.00th=[ 397], 90.00th=[ 735], 95.00th=[ 768], | 99.00th=[ 768], 99.50th=[ 768], 99.90th=[ 768], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 1935, max= 4096, per=36.95%, avg=2706.67, stdev=1073.48, samples=6 iops : min= 1, max= 4, avg= 2.00, stdev= 1.26, samples=6 lat (usec) : 1000=8.33% lat (msec) : 250=41.67%, 500=33.33%, 750=8.33%, 1000=8.33% cpu : usr=0.03%, sys=37.90%, ctx=178, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147530: Tue Aug 5 11:26:04 2025 read: IOPS=2, BW=2476KiB/s (2536kB/s)(6144KiB/2481msec) clat (usec): min=761, max=182421, avg=70270.46, stdev=76987.09 lat (usec): min=762, max=182423, avg=70271.39, stdev=76987.10 clat percentiles (usec): | 1.00th=[ 758], 5.00th=[ 758], 10.00th=[ 758], 20.00th=[ 848], | 30.00th=[ 848], 40.00th=[ 16909], 50.00th=[ 16909], 60.00th=[ 84411], | 70.00th=[135267], 80.00th=[135267], 90.00th=[181404], 95.00th=[181404], | 99.00th=[181404], 99.50th=[181404], 99.90th=[181404], 99.95th=[181404], | 99.99th=[181404] bw ( KiB/s): min= 2043, max= 3871, per=56.52%, avg=2957.00, stdev=1292.59, samples=2 iops : min= 1, max= 3, avg= 2.00, stdev= 1.41, samples=2 write: IOPS=2, BW=2476KiB/s (2536kB/s)(6144KiB/2481msec); 0 zone resets clat (msec): min=117, max=519, avg=333.56, stdev=146.19 lat (msec): min=117, max=519, avg=333.64, stdev=146.20 clat percentiles (msec): | 1.00th=[ 117], 5.00th=[ 117], 10.00th=[ 117], 20.00th=[ 218], | 30.00th=[ 218], 40.00th=[ 334], 50.00th=[ 334], 60.00th=[ 380], | 70.00th=[ 430], 80.00th=[ 430], 90.00th=[ 518], 95.00th=[ 518], | 99.00th=[ 518], 99.50th=[ 518], 99.90th=[ 518], 99.95th=[ 518], | 99.99th=[ 518] bw ( KiB/s): min= 1726, max= 2043, per=26.41%, avg=1934.75, stdev=147.59, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 lat (usec) : 1000=16.67% lat (msec) : 20=8.33%, 100=8.33%, 250=33.33%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=40.28%, ctx=585, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147531: Tue Aug 5 11:26:04 2025 read: IOPS=1, BW=1311KiB/s (1343kB/s)(5120KiB/3904msec) clat (usec): min=1122, max=1124.0k, avg=275762.54, stdev=478531.46 lat (usec): min=1122, max=1124.0k, avg=275763.44, stdev=478531.46 clat percentiles (usec): | 1.00th=[ 1123], 5.00th=[ 1123], 10.00th=[ 1123], | 20.00th=[ 1123], 30.00th=[ 1450], 40.00th=[ 1450], | 50.00th=[ 104334], 60.00th=[ 104334], 70.00th=[ 147850], | 80.00th=[ 147850], 90.00th=[1115685], 95.00th=[1115685], | 99.00th=[1115685], 99.50th=[1115685], 99.90th=[1115685], | 99.95th=[1115685], 99.99th=[1115685] bw ( KiB/s): min= 2048, max= 6119, per=78.05%, avg=4083.50, stdev=2878.63, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 write: IOPS=1, BW=1836KiB/s (1880kB/s)(7168KiB/3904msec); 0 zone resets clat (msec): min=91, max=620, avg=355.90, stdev=197.38 lat (msec): min=91, max=620, avg=355.98, stdev=197.37 clat percentiles (msec): | 1.00th=[ 92], 5.00th=[ 92], 10.00th=[ 92], 20.00th=[ 109], | 30.00th=[ 330], 40.00th=[ 330], 50.00th=[ 405], 60.00th=[ 426], | 70.00th=[ 426], 80.00th=[ 514], 90.00th=[ 617], 95.00th=[ 617], | 99.00th=[ 617], 99.50th=[ 617], 99.90th=[ 617], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 1732, max= 2043, per=26.71%, avg=1956.80, stdev=133.52, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (msec) : 2=16.67%, 100=8.33%, 250=25.00%, 500=25.00%, 750=16.67% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=32.59%, ctx=672, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5232KiB/s (5358kB/s), 523KiB/s-2476KiB/s (536kB/s-2536kB/s), io=20.0MiB (20.0MB), run=2481-3914msec WRITE: bw=7325KiB/s (7501kB/s), 1400KiB/s-2616KiB/s (1434kB/s-2679kB/s), io=28.0MiB (29.4MB), run=2481-3914msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147537: Tue Aug 5 11:26:14 2025 read: IOPS=1, BW=5148KiB/s (5271kB/s)(12.0MiB/2387msec) clat (msec): min=691, max=844, avg=749.57, stdev=82.63 lat (msec): min=691, max=844, avg=749.57, stdev=82.63 clat percentiles (msec): | 1.00th=[ 693], 5.00th=[ 693], 10.00th=[ 693], 20.00th=[ 693], | 30.00th=[ 693], 40.00th=[ 718], 50.00th=[ 718], 60.00th=[ 718], | 70.00th=[ 844], 80.00th=[ 844], 90.00th=[ 844], 95.00th=[ 844], | 99.00th=[ 844], 99.50th=[ 844], 99.90th=[ 844], 99.95th=[ 844], | 99.99th=[ 844] bw ( KiB/s): min= 8175, max= 8192, per=100.00%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 1000=33.33% cpu : usr=0.00%, sys=44.26%, ctx=1847, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147538: Tue Aug 5 11:26:14 2025 read: IOPS=0, BW=887KiB/s (909kB/s)(4096KiB/4616msec) clat (nsec): min=691208k, max=691208k, avg=691208022.00, stdev= 0.00 lat (nsec): min=691209k, max=691209k, avg=691209406.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 693], 5.00th=[ 693], 10.00th=[ 693], 20.00th=[ 693], | 30.00th=[ 693], 40.00th=[ 693], 50.00th=[ 693], 60.00th=[ 693], | 70.00th=[ 693], 80.00th=[ 693], 90.00th=[ 693], 95.00th=[ 693], | 99.00th=[ 693], 99.50th=[ 693], 99.90th=[ 693], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1775KiB/s (1817kB/s)(8192KiB/4616msec); 0 zone resets clat (msec): min=1306, max=2513, avg=1909.89, stdev=853.14 lat (msec): min=1306, max=2513, avg=1910.21, stdev=853.26 clat percentiles (msec): | 1.00th=[ 1301], 5.00th=[ 1301], 10.00th=[ 1301], 20.00th=[ 1301], | 30.00th=[ 1301], 40.00th=[ 1301], 50.00th=[ 1301], 60.00th=[ 2500], | 70.00th=[ 2500], 80.00th=[ 2500], 90.00th=[ 2500], 95.00th=[ 2500], | 99.00th=[ 2500], 99.50th=[ 2500], 99.90th=[ 2500], 99.95th=[ 2500], | 99.99th=[ 2500] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.02%, sys=27.95%, ctx=976, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147540: Tue Aug 5 11:26:14 2025 write: IOPS=0, BW=3051KiB/s (3124kB/s)(12.0MiB/4028msec); 0 zone resets clat (msec): min=928, max=1518, avg=1277.38, stdev=309.62 lat (msec): min=928, max=1519, avg=1277.93, stdev=309.75 clat percentiles (msec): | 1.00th=[ 927], 5.00th=[ 927], 10.00th=[ 927], 20.00th=[ 927], | 30.00th=[ 927], 40.00th=[ 1385], 50.00th=[ 1385], 60.00th=[ 1385], | 70.00th=[ 1519], 80.00th=[ 1519], 90.00th=[ 1519], 95.00th=[ 1519], | 99.00th=[ 1519], 99.50th=[ 1519], 99.90th=[ 1519], 99.95th=[ 1519], | 99.99th=[ 1519] bw ( KiB/s): min= 8175, max= 8192, per=100.00%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=48.75%, ctx=192, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147542: Tue Aug 5 11:26:14 2025 write: IOPS=0, BW=3160KiB/s (3236kB/s)(12.0MiB/3889msec); 0 zone resets clat (msec): min=965, max=1420, avg=1243.94, stdev=244.29 lat (msec): min=965, max=1420, avg=1244.40, stdev=244.26 clat percentiles (msec): | 1.00th=[ 969], 5.00th=[ 969], 10.00th=[ 969], 20.00th=[ 969], | 30.00th=[ 969], 40.00th=[ 1351], 50.00th=[ 1351], 60.00th=[ 1351], | 70.00th=[ 1418], 80.00th=[ 1418], 90.00th=[ 1418], 95.00th=[ 1418], | 99.00th=[ 1418], 99.50th=[ 1418], 99.90th=[ 1418], 99.95th=[ 1418], | 99.99th=[ 1418] bw ( KiB/s): min= 8175, max= 8192, per=100.00%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=54.45%, ctx=270, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3549KiB/s (3635kB/s), 887KiB/s-5148KiB/s (909kB/s-5271kB/s), io=16.0MiB (16.8MB), run=2387-4616msec WRITE: bw=7099KiB/s (7269kB/s), 1775KiB/s-3160KiB/s (1817kB/s-3236kB/s), io=32.0MiB (33.6MB), run=3889-4616msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147539: Tue Aug 5 11:26:14 2025 read: IOPS=0, BW=2903KiB/s (2973kB/s)(12.0MiB/4233msec) clat (msec): min=523, max=2424, avg=1368.76, stdev=968.14 lat (msec): min=523, max=2424, avg=1368.76, stdev=968.14 clat percentiles (msec): | 1.00th=[ 523], 5.00th=[ 523], 10.00th=[ 523], 20.00th=[ 523], | 30.00th=[ 523], 40.00th=[ 1167], 50.00th=[ 1167], 60.00th=[ 1167], | 70.00th=[ 2433], 80.00th=[ 2433], 90.00th=[ 2433], 95.00th=[ 2433], | 99.00th=[ 2433], 99.50th=[ 2433], 99.90th=[ 2433], 99.95th=[ 2433], | 99.99th=[ 2433] bw ( KiB/s): min= 8159, max= 8175, per=100.00%, avg=8167.00, stdev=11.31, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=12.38%, ctx=338, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147541: Tue Aug 5 11:26:14 2025 read: IOPS=0, BW=1031KiB/s (1056kB/s)(4096KiB/3971msec) clat (nsec): min=387762k, max=387762k, avg=387761944.00, stdev= 0.00 lat (nsec): min=387765k, max=387765k, avg=387765044.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 388], 5.00th=[ 388], 10.00th=[ 388], 20.00th=[ 388], | 30.00th=[ 388], 40.00th=[ 388], 50.00th=[ 388], 60.00th=[ 388], | 70.00th=[ 388], 80.00th=[ 388], 90.00th=[ 388], 95.00th=[ 388], | 99.00th=[ 388], 99.50th=[ 388], 99.90th=[ 388], 99.95th=[ 388], | 99.99th=[ 388] bw ( KiB/s): min= 6262, max= 6262, per=100.00%, avg=6262.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2063KiB/s (2112kB/s)(8192KiB/3971msec); 0 zone resets clat (msec): min=1408, max=2073, avg=1741.11, stdev=470.12 lat (msec): min=1409, max=2073, avg=1741.44, stdev=470.07 clat percentiles (msec): | 1.00th=[ 1401], 5.00th=[ 1401], 10.00th=[ 1401], 20.00th=[ 1401], | 30.00th=[ 1401], 40.00th=[ 1401], 50.00th=[ 1401], 60.00th=[ 2072], | 70.00th=[ 2072], 80.00th=[ 2072], 90.00th=[ 2072], 95.00th=[ 2072], | 99.00th=[ 2072], 99.50th=[ 2072], 99.90th=[ 2072], 99.95th=[ 2072], | 99.99th=[ 2072] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=18.99%, ctx=250, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147543: Tue Aug 5 11:26:14 2025 write: IOPS=0, BW=2756KiB/s (2822kB/s)(12.0MiB/4459msec); 0 zone resets clat (msec): min=237, max=2180, avg=1441.20, stdev=1051.24 lat (msec): min=238, max=2181, avg=1441.70, stdev=1051.46 clat percentiles (msec): | 1.00th=[ 239], 5.00th=[ 239], 10.00th=[ 239], 20.00th=[ 239], | 30.00th=[ 239], 40.00th=[ 1905], 50.00th=[ 1905], 60.00th=[ 1905], | 70.00th=[ 2165], 80.00th=[ 2165], 90.00th=[ 2165], 95.00th=[ 2165], | 99.00th=[ 2165], 99.50th=[ 2165], 99.90th=[ 2165], 99.95th=[ 2165], | 99.99th=[ 2165] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.02%, sys=19.31%, ctx=129, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147544: Tue Aug 5 11:26:14 2025 write: IOPS=1, BW=4452KiB/s (4559kB/s)(12.0MiB/2760msec); 0 zone resets clat (msec): min=332, max=1948, avg=874.05, stdev=930.89 lat (msec): min=333, max=1949, avg=874.46, stdev=930.94 clat percentiles (msec): | 1.00th=[ 334], 5.00th=[ 334], 10.00th=[ 334], 20.00th=[ 334], | 30.00th=[ 334], 40.00th=[ 342], 50.00th=[ 342], 60.00th=[ 342], | 70.00th=[ 1955], 80.00th=[ 1955], 90.00th=[ 1955], 95.00th=[ 1955], | 99.00th=[ 1955], 99.50th=[ 1955], 99.90th=[ 1955], 99.95th=[ 1955], | 99.99th=[ 1955] bw ( KiB/s): min= 5970, max= 8175, per=96.25%, avg=7072.50, stdev=1559.17, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=18.38%, ctx=248, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3871KiB/s (3963kB/s), 1031KiB/s-2903KiB/s (1056kB/s-2973kB/s), io=16.0MiB (16.8MB), run=3971-4233msec WRITE: bw=7349KiB/s (7525kB/s), 2063KiB/s-4452KiB/s (2112kB/s-4559kB/s), io=32.0MiB (33.6MB), run=2760-4459msec