== sanity test 398b: DIO and buffer IO race ============== 18:01:51 (1756677711) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.42941 s, 14.7 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159701: Sun Aug 31 18:02:50 2025 read: IOPS=30, BW=122KiB/s (125kB/s)(5992KiB/49143msec) clat (usec): min=84, max=948351, avg=16906.56, stdev=26255.55 lat (usec): min=85, max=948351, avg=16907.17, stdev=26255.58 clat percentiles (usec): | 1.00th=[ 314], 5.00th=[ 3752], 10.00th=[ 7111], 20.00th=[ 8848], | 30.00th=[ 10552], 40.00th=[ 12387], 50.00th=[ 14484], 60.00th=[ 16450], | 70.00th=[ 19268], 80.00th=[ 22676], 90.00th=[ 28967], 95.00th=[ 33162], | 99.00th=[ 51119], 99.50th=[ 65799], 99.90th=[147850], 99.95th=[952108], | 99.99th=[952108] bw ( KiB/s): min= 7, max= 208, per=24.99%, avg=121.96, stdev=39.17, samples=96 iops : min= 1, max= 52, avg=30.16, stdev= 9.85, samples=96 write: IOPS=32, BW=128KiB/s (131kB/s)(6296KiB/49143msec); 0 zone resets clat (usec): min=626, max=116526, avg=14948.63, stdev=13331.22 lat (usec): min=626, max=116527, avg=14949.32, stdev=13331.25 clat percentiles (usec): | 1.00th=[ 717], 5.00th=[ 2147], 10.00th=[ 2835], 20.00th=[ 3785], | 30.00th=[ 5997], 40.00th=[ 8717], 50.00th=[ 11731], 60.00th=[ 15139], | 70.00th=[ 18482], 80.00th=[ 23462], 90.00th=[ 31327], 95.00th=[ 39060], | 99.00th=[ 66847], 99.50th=[ 78119], 99.90th=[113771], 99.95th=[116917], | 99.99th=[116917] bw ( KiB/s): min= 55, max= 223, per=25.27%, avg=129.14, stdev=38.74, samples=95 iops : min= 13, max= 55, avg=31.96, stdev= 9.72, samples=95 lat (usec) : 100=0.10%, 250=0.13%, 500=1.46%, 750=1.14%, 1000=0.52% lat (msec) : 2=1.30%, 4=8.79%, 10=22.69%, 20=36.98%, 50=25.42% lat (msec) : 100=1.30%, 250=0.13%, 1000=0.03% cpu : usr=0.08%, sys=28.42%, ctx=8436, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159703: Sun Aug 31 18:02:50 2025 read: IOPS=30, BW=122KiB/s (125kB/s)(5940KiB/48522msec) clat (usec): min=97, max=153997, avg=16856.79, stdev=11089.86 lat (usec): min=97, max=153998, avg=16857.42, stdev=11089.90 clat percentiles (usec): | 1.00th=[ 351], 5.00th=[ 5735], 10.00th=[ 7635], 20.00th=[ 9503], | 30.00th=[ 11076], 40.00th=[ 12649], 50.00th=[ 14746], 60.00th=[ 16909], | 70.00th=[ 19006], 80.00th=[ 22414], 90.00th=[ 28181], 95.00th=[ 34866], | 99.00th=[ 63177], 99.50th=[ 68682], 99.90th=[ 93848], 99.95th=[154141], | 99.99th=[154141] bw ( KiB/s): min= 7, max= 215, per=25.36%, avg=123.75, stdev=39.67, samples=95 iops : min= 1, max= 53, avg=30.60, stdev= 9.97, samples=95 write: IOPS=32, BW=131KiB/s (134kB/s)(6348KiB/48522msec); 0 zone resets clat (usec): min=687, max=944977, avg=14620.47, stdev=26116.75 lat (usec): min=688, max=944978, avg=14621.77, stdev=26116.87 clat percentiles (usec): | 1.00th=[ 750], 5.00th=[ 2245], 10.00th=[ 2900], 20.00th=[ 3916], | 30.00th=[ 6128], 40.00th=[ 8979], 50.00th=[ 11207], 60.00th=[ 13960], | 70.00th=[ 17433], 80.00th=[ 21890], 90.00th=[ 28967], 95.00th=[ 35914], | 99.00th=[ 54264], 99.50th=[ 71828], 99.90th=[ 90702], 99.95th=[943719], | 99.99th=[943719] bw ( KiB/s): min= 47, max= 231, per=26.09%, avg=133.31, stdev=41.45, samples=94 iops : min= 11, max= 57, avg=33.00, stdev=10.37, samples=94 lat (usec) : 100=0.03%, 250=0.20%, 500=1.07%, 750=0.78%, 1000=0.52% lat (msec) : 2=1.24%, 4=8.76%, 10=21.29%, 20=40.69%, 50=23.96% lat (msec) : 100=1.40%, 250=0.03%, 1000=0.03% cpu : usr=0.07%, sys=28.91%, ctx=8452, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159705: Sun Aug 31 18:02:50 2025 read: IOPS=30, BW=123KiB/s (126kB/s)(6028KiB/48905msec) clat (usec): min=81, max=116996, avg=16697.06, stdev=10976.48 lat (usec): min=81, max=116996, avg=16698.49, stdev=10976.43 clat percentiles (usec): | 1.00th=[ 326], 5.00th=[ 5080], 10.00th=[ 7504], 20.00th=[ 9372], | 30.00th=[ 11076], 40.00th=[ 12780], 50.00th=[ 14484], 60.00th=[ 16450], | 70.00th=[ 19006], 80.00th=[ 22414], 90.00th=[ 28705], 95.00th=[ 34866], | 99.00th=[ 60556], 99.50th=[ 72877], 99.90th=[ 92799], 99.95th=[116917], | 99.99th=[116917] bw ( KiB/s): min= 55, max= 199, per=25.72%, avg=125.52, stdev=34.54, samples=95 iops : min= 13, max= 49, avg=31.05, stdev= 8.69, samples=95 write: IOPS=32, BW=128KiB/s (131kB/s)(6260KiB/48905msec); 0 zone resets clat (usec): min=684, max=949323, avg=14988.01, stdev=26350.35 lat (usec): min=684, max=949324, avg=14988.81, stdev=26350.36 clat percentiles (usec): | 1.00th=[ 947], 5.00th=[ 2376], 10.00th=[ 3064], 20.00th=[ 4293], | 30.00th=[ 6521], 40.00th=[ 9110], 50.00th=[ 11731], 60.00th=[ 14746], | 70.00th=[ 18482], 80.00th=[ 22152], 90.00th=[ 29230], 95.00th=[ 35914], | 99.00th=[ 54264], 99.50th=[ 66847], 99.90th=[105382], 99.95th=[952108], | 99.99th=[952108] bw ( KiB/s): min= 15, max= 208, per=25.22%, avg=128.85, stdev=39.71, samples=96 iops : min= 3, max= 52, avg=31.89, stdev= 9.96, samples=96 lat (usec) : 100=0.07%, 250=0.07%, 500=1.27%, 750=0.81%, 1000=0.52% lat (msec) : 2=1.27%, 4=7.88%, 10=21.32%, 20=41.28%, 50=23.86% lat (msec) : 100=1.56%, 250=0.07%, 1000=0.03% cpu : usr=0.08%, sys=28.91%, ctx=8555, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159707: Sun Aug 31 18:02:50 2025 read: IOPS=30, BW=124KiB/s (127kB/s)(6052KiB/48901msec) clat (usec): min=79, max=98737, avg=16537.64, stdev=10618.46 lat (usec): min=79, max=98738, avg=16538.38, stdev=10618.44 clat percentiles (usec): | 1.00th=[ 334], 5.00th=[ 5145], 10.00th=[ 7504], 20.00th=[ 9110], | 30.00th=[10683], 40.00th=[12518], 50.00th=[14615], 60.00th=[16450], | 70.00th=[19006], 80.00th=[22414], 90.00th=[28443], 95.00th=[33817], | 99.00th=[56886], 99.50th=[73925], 99.90th=[91751], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 39, max= 200, per=25.88%, avg=126.27, stdev=37.44, samples=95 iops : min= 9, max= 50, avg=31.24, stdev= 9.39, samples=95 write: IOPS=31, BW=128KiB/s (131kB/s)(6236KiB/48901msec); 0 zone resets clat (usec): min=654, max=1024.4k, avg=15135.47, stdev=28269.84 lat (usec): min=655, max=1024.4k, avg=15136.28, stdev=28269.85 clat percentiles (usec): | 1.00th=[ 775], 5.00th=[ 2376], 10.00th=[ 2933], | 20.00th=[ 3916], 30.00th=[ 6259], 40.00th=[ 9372], | 50.00th=[ 11994], 60.00th=[ 15008], 70.00th=[ 18482], | 80.00th=[ 22414], 90.00th=[ 29754], 95.00th=[ 36439], | 99.00th=[ 53216], 99.50th=[ 59507], 99.90th=[ 166724], | 99.95th=[1027605], 99.99th=[1027605] bw ( KiB/s): min= 55, max= 216, per=25.44%, avg=130.00, stdev=36.69, samples=95 iops : min= 13, max= 54, avg=32.17, stdev= 9.19, samples=95 lat (usec) : 100=0.07%, 250=0.26%, 500=0.94%, 750=0.85%, 1000=0.75% lat (msec) : 2=1.01%, 4=8.82%, 10=21.09%, 20=39.94%, 50=24.87% lat (msec) : 100=1.33%, 250=0.03%, 2000=0.03% cpu : usr=0.08%, sys=28.76%, ctx=8457, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=489KiB/s (500kB/s), 122KiB/s-124KiB/s (125kB/s-127kB/s), io=23.4MiB (24.6MB), run=48522-49143msec WRITE: bw=512KiB/s (524kB/s), 128KiB/s-131KiB/s (131kB/s-134kB/s), io=24.6MiB (25.7MB), run=48522-49143msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159700: Sun Aug 31 18:03:15 2025 read: IOPS=20, BW=80.5KiB/s (82.5kB/s)(5992KiB/74397msec) clat (msec): min=2, max=964, avg=14.85, stdev=28.35 lat (msec): min=2, max=964, avg=14.86, stdev=28.35 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 16], 80.00th=[ 20], 90.00th=[ 31], 95.00th=[ 41], | 99.00th=[ 73], 99.50th=[ 97], 99.90th=[ 163], 99.95th=[ 961], | 99.99th=[ 961] bw ( KiB/s): min= 15, max= 216, per=25.49%, avg=82.08, stdev=45.04, samples=145 iops : min= 3, max= 54, avg=20.30, stdev=11.38, samples=145 write: IOPS=21, BW=84.6KiB/s (86.7kB/s)(6296KiB/74397msec); 0 zone resets clat (msec): min=12, max=171, avg=32.95, stdev=17.96 lat (msec): min=12, max=171, avg=32.95, stdev=17.96 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 20], | 30.00th=[ 22], 40.00th=[ 24], 50.00th=[ 27], 60.00th=[ 32], | 70.00th=[ 37], 80.00th=[ 45], 90.00th=[ 58], 95.00th=[ 68], | 99.00th=[ 92], 99.50th=[ 106], 99.90th=[ 167], 99.95th=[ 171], | 99.99th=[ 171] bw ( KiB/s): min= 15, max= 176, per=25.37%, avg=85.51, stdev=41.68, samples=146 iops : min= 3, max= 44, avg=21.16, stdev=10.55, samples=146 lat (msec) : 4=6.25%, 10=16.67%, 20=27.57%, 50=40.89%, 100=8.07% lat (msec) : 250=0.52%, 1000=0.03% cpu : usr=0.07%, sys=9.81%, ctx=4314, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159702: Sun Aug 31 18:03:15 2025 read: IOPS=19, BW=79.8KiB/s (81.7kB/s)(5940KiB/74432msec) clat (msec): min=2, max=998, avg=14.69, stdev=28.65 lat (msec): min=2, max=998, avg=14.69, stdev=28.65 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 16], 80.00th=[ 21], 90.00th=[ 31], 95.00th=[ 41], | 99.00th=[ 63], 99.50th=[ 73], 99.90th=[ 157], 99.95th=[ 995], | 99.99th=[ 995] bw ( KiB/s): min= 15, max= 216, per=25.06%, avg=80.69, stdev=44.29, samples=146 iops : min= 3, max= 54, avg=19.95, stdev=11.18, samples=146 write: IOPS=21, BW=85.3KiB/s (87.3kB/s)(6348KiB/74432msec); 0 zone resets clat (msec): min=12, max=169, avg=32.97, stdev=17.50 lat (msec): min=12, max=169, avg=32.98, stdev=17.50 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 21], | 30.00th=[ 23], 40.00th=[ 25], 50.00th=[ 28], 60.00th=[ 32], | 70.00th=[ 37], 80.00th=[ 44], 90.00th=[ 56], 95.00th=[ 66], | 99.00th=[ 97], 99.50th=[ 118], 99.90th=[ 144], 99.95th=[ 169], | 99.99th=[ 169] bw ( KiB/s): min= 23, max= 176, per=25.54%, avg=86.06, stdev=41.05, samples=146 iops : min= 5, max= 44, avg=21.29, stdev=10.40, samples=146 lat (msec) : 4=7.55%, 10=15.85%, 20=25.00%, 50=43.16%, 100=7.94% lat (msec) : 250=0.46%, 1000=0.03% cpu : usr=0.06%, sys=9.72%, ctx=4142, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159704: Sun Aug 31 18:03:15 2025 read: IOPS=20, BW=81.1KiB/s (82.0kB/s)(6028KiB/74372msec) clat (msec): min=2, max=127, avg=14.64, stdev=13.21 lat (msec): min=2, max=127, avg=14.65, stdev=13.21 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 7], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 21], 90.00th=[ 31], 95.00th=[ 41], | 99.00th=[ 64], 99.50th=[ 79], 99.90th=[ 117], 99.95th=[ 128], | 99.99th=[ 128] bw ( KiB/s): min= 16, max= 224, per=25.47%, avg=82.01, stdev=43.58, samples=146 iops : min= 4, max= 56, avg=20.28, stdev=11.02, samples=146 write: IOPS=21, BW=84.2KiB/s (86.2kB/s)(6260KiB/74372msec); 0 zone resets clat (msec): min=13, max=1006, avg=33.24, stdev=30.08 lat (msec): min=13, max=1006, avg=33.24, stdev=30.08 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 20], | 30.00th=[ 23], 40.00th=[ 25], 50.00th=[ 27], 60.00th=[ 31], | 70.00th=[ 36], 80.00th=[ 44], 90.00th=[ 55], 95.00th=[ 67], | 99.00th=[ 91], 99.50th=[ 109], 99.90th=[ 174], 99.95th=[ 1003], | 99.99th=[ 1003] bw ( KiB/s): min= 23, max= 176, per=25.26%, avg=85.13, stdev=40.68, samples=146 iops : min= 5, max= 44, avg=21.06, stdev=10.29, samples=146 lat (msec) : 4=6.02%, 10=15.63%, 20=27.90%, 50=42.15%, 100=7.81% lat (msec) : 250=0.46%, 2000=0.03% cpu : usr=0.07%, sys=9.99%, ctx=4303, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159706: Sun Aug 31 18:03:15 2025 read: IOPS=20, BW=82.0KiB/s (83.0kB/s)(6052KiB/73787msec) clat (msec): min=2, max=152, avg=14.43, stdev=13.77 lat (msec): min=2, max=152, avg=14.43, stdev=13.77 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 16], 80.00th=[ 21], 90.00th=[ 33], 95.00th=[ 42], | 99.00th=[ 71], 99.50th=[ 79], 99.90th=[ 108], 99.95th=[ 153], | 99.99th=[ 153] bw ( KiB/s): min= 15, max= 256, per=25.80%, avg=83.06, stdev=47.49, samples=145 iops : min= 3, max= 64, avg=20.54, stdev=11.99, samples=145 write: IOPS=21, BW=84.5KiB/s (86.5kB/s)(6236KiB/73787msec); 0 zone resets clat (msec): min=12, max=1011, avg=33.15, stdev=29.85 lat (msec): min=12, max=1011, avg=33.15, stdev=29.85 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 20], | 30.00th=[ 22], 40.00th=[ 25], 50.00th=[ 28], 60.00th=[ 31], | 70.00th=[ 37], 80.00th=[ 44], 90.00th=[ 55], 95.00th=[ 64], | 99.00th=[ 96], 99.50th=[ 115], 99.90th=[ 153], 99.95th=[ 1011], | 99.99th=[ 1011] bw ( KiB/s): min= 31, max= 176, per=25.37%, avg=85.50, stdev=40.38, samples=145 iops : min= 7, max= 44, avg=21.15, stdev=10.22, samples=145 lat (msec) : 4=6.51%, 10=16.80%, 20=26.69%, 50=41.60%, 100=7.85% lat (msec) : 250=0.52%, 2000=0.03% cpu : usr=0.06%, sys=9.87%, ctx=4253, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=323KiB/s (330kB/s), 79.8KiB/s-82.0KiB/s (81.7kB/s-83.0kB/s), io=23.4MiB (24.6MB), run=73787-74432msec WRITE: bw=338KiB/s (346kB/s), 84.2KiB/s-85.3KiB/s (86.2kB/s-87.3kB/s), io=24.6MiB (25.7MB), run=73787-74432msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159733: Sun Aug 31 18:03:36 2025 read: IOPS=25, BW=404KiB/s (414kB/s)(5920KiB/14655msec) clat (usec): min=87, max=79505, avg=19963.01, stdev=11982.81 lat (usec): min=87, max=79506, avg=19963.55, stdev=11982.83 clat percentiles (usec): | 1.00th=[ 338], 5.00th=[ 619], 10.00th=[ 7504], 20.00th=[11207], | 30.00th=[13829], 40.00th=[15533], 50.00th=[18220], 60.00th=[20579], | 70.00th=[24511], 80.00th=[27395], 90.00th=[34866], 95.00th=[41681], | 99.00th=[54264], 99.50th=[70779], 99.90th=[79168], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 191, max= 672, per=25.23%, avg=402.24, stdev=144.17, samples=29 iops : min= 11, max= 42, avg=24.48, stdev= 9.05, samples=29 write: IOPS=27, BW=435KiB/s (445kB/s)(6368KiB/14655msec); 0 zone resets clat (usec): min=1337, max=97503, avg=18065.68, stdev=12936.11 lat (usec): min=1338, max=97504, avg=18066.62, stdev=12936.16 clat percentiles (usec): | 1.00th=[ 3294], 5.00th=[ 4424], 10.00th=[ 5014], 20.00th=[ 7177], | 30.00th=[ 9372], 40.00th=[11863], 50.00th=[15401], 60.00th=[19006], | 70.00th=[22152], 80.00th=[26346], 90.00th=[34866], 95.00th=[41157], | 99.00th=[64226], 99.50th=[89654], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 159, max= 798, per=25.86%, avg=434.00, stdev=154.95, samples=29 iops : min= 9, max= 49, avg=26.45, stdev= 9.70, samples=29 lat (usec) : 100=0.13%, 250=0.13%, 500=1.56%, 750=1.69%, 1000=0.65% lat (msec) : 2=0.26%, 4=1.17%, 10=18.49%, 20=36.72%, 50=37.11% lat (msec) : 100=2.08% cpu : usr=0.10%, sys=32.77%, ctx=2918, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159734: Sun Aug 31 18:03:36 2025 read: IOPS=23, BW=384KiB/s (393kB/s)(5744KiB/14964msec) clat (usec): min=158, max=74149, avg=21633.50, stdev=11946.65 lat (usec): min=159, max=74150, avg=21634.07, stdev=11946.68 clat percentiles (usec): | 1.00th=[ 420], 5.00th=[ 5866], 10.00th=[ 8979], 20.00th=[12387], | 30.00th=[14746], 40.00th=[17433], 50.00th=[20055], 60.00th=[22676], | 70.00th=[25035], 80.00th=[29230], 90.00th=[37487], 95.00th=[45351], | 99.00th=[62653], 99.50th=[64750], 99.90th=[73925], 99.95th=[73925], | 99.99th=[73925] bw ( KiB/s): min= 191, max= 606, per=23.68%, avg=377.52, stdev=115.69, samples=29 iops : min= 11, max= 37, avg=22.93, stdev= 7.28, samples=29 write: IOPS=27, BW=437KiB/s (448kB/s)(6544KiB/14964msec); 0 zone resets clat (usec): min=1554, max=73771, avg=17463.74, stdev=12031.09 lat (usec): min=1554, max=73772, avg=17464.86, stdev=12031.18 clat percentiles (usec): | 1.00th=[ 2868], 5.00th=[ 4228], 10.00th=[ 4883], 20.00th=[ 7504], | 30.00th=[ 9372], 40.00th=[11994], 50.00th=[15139], 60.00th=[18482], | 70.00th=[21627], 80.00th=[25035], 90.00th=[31589], 95.00th=[42206], | 99.00th=[62129], 99.50th=[68682], 99.90th=[73925], 99.95th=[73925], | 99.99th=[73925] bw ( KiB/s): min= 191, max= 798, per=25.32%, avg=424.83, stdev=160.78, samples=29 iops : min= 11, max= 49, avg=25.90, stdev=10.02, samples=29 lat (usec) : 250=0.13%, 500=0.65%, 750=1.04% lat (msec) : 2=0.13%, 4=2.21%, 10=18.62%, 20=34.77%, 50=39.71% lat (msec) : 100=2.73% cpu : usr=0.07%, sys=32.89%, ctx=2915, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159735: Sun Aug 31 18:03:36 2025 read: IOPS=26, BW=417KiB/s (427kB/s)(6256KiB/15017msec) clat (usec): min=201, max=81783, avg=21109.27, stdev=12670.01 lat (usec): min=201, max=81784, avg=21109.86, stdev=12670.05 clat percentiles (usec): | 1.00th=[ 355], 5.00th=[ 709], 10.00th=[ 8291], 20.00th=[11469], | 30.00th=[13960], 40.00th=[16450], 50.00th=[19006], 60.00th=[22152], | 70.00th=[25822], 80.00th=[30016], 90.00th=[36963], 95.00th=[43254], | 99.00th=[58459], 99.50th=[73925], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 255, max= 768, per=25.37%, avg=404.34, stdev=127.73, samples=29 iops : min= 15, max= 48, avg=24.59, stdev= 8.01, samples=29 write: IOPS=25, BW=402KiB/s (411kB/s)(6032KiB/15017msec); 0 zone resets clat (usec): min=1302, max=81583, avg=17879.58, stdev=12274.80 lat (usec): min=1303, max=81585, avg=17880.79, stdev=12274.88 clat percentiles (usec): | 1.00th=[ 1631], 5.00th=[ 4113], 10.00th=[ 4817], 20.00th=[ 7570], | 30.00th=[ 9896], 40.00th=[12256], 50.00th=[14746], 60.00th=[18482], | 70.00th=[22676], 80.00th=[25822], 90.00th=[33424], 95.00th=[44827], | 99.00th=[55313], 99.50th=[59507], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 160, max= 640, per=22.56%, avg=378.52, stdev=151.67, samples=29 iops : min= 10, max= 40, avg=23.00, stdev= 9.50, samples=29 lat (usec) : 250=0.13%, 500=1.69%, 750=0.91%, 1000=0.65% lat (msec) : 2=0.78%, 4=1.30%, 10=16.67%, 20=36.33%, 50=38.54% lat (msec) : 100=2.99% cpu : usr=0.07%, sys=31.84%, ctx=2894, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159736: Sun Aug 31 18:03:36 2025 read: IOPS=25, BW=415KiB/s (425kB/s)(6032KiB/14548msec) clat (usec): min=212, max=60640, avg=19373.81, stdev=11294.69 lat (usec): min=213, max=60641, avg=19374.67, stdev=11294.91 clat percentiles (usec): | 1.00th=[ 310], 5.00th=[ 545], 10.00th=[ 3654], 20.00th=[10945], | 30.00th=[13042], 40.00th=[15664], 50.00th=[18482], 60.00th=[21103], | 70.00th=[24249], 80.00th=[27395], 90.00th=[33162], 95.00th=[41157], | 99.00th=[52691], 99.50th=[56886], 99.90th=[60556], 99.95th=[60556], | 99.99th=[60556] bw ( KiB/s): min= 94, max= 766, per=25.68%, avg=409.39, stdev=154.25, samples=28 iops : min= 5, max= 47, avg=24.89, stdev= 9.64, samples=28 write: IOPS=26, BW=430KiB/s (440kB/s)(6256KiB/14548msec); 0 zone resets clat (usec): min=1079, max=98010, avg=18245.32, stdev=13210.83 lat (usec): min=1080, max=98011, avg=18246.27, stdev=13210.87 clat percentiles (usec): | 1.00th=[ 1713], 5.00th=[ 4113], 10.00th=[ 4948], 20.00th=[ 7242], | 30.00th=[ 9241], 40.00th=[11994], 50.00th=[15401], 60.00th=[18220], | 70.00th=[22414], 80.00th=[27657], 90.00th=[35914], 95.00th=[43779], | 99.00th=[58983], 99.50th=[74974], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 223, max= 670, per=25.45%, avg=427.11, stdev=117.64, samples=28 iops : min= 13, max= 41, avg=26.00, stdev= 7.30, samples=28 lat (usec) : 250=0.13%, 500=1.82%, 750=1.43%, 1000=0.91% lat (msec) : 2=0.91%, 4=1.95%, 10=18.23%, 20=35.68%, 50=36.98% lat (msec) : 100=1.95% cpu : usr=0.04%, sys=32.93%, ctx=2615, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1595KiB/s (1633kB/s), 384KiB/s-417KiB/s (393kB/s-427kB/s), io=23.4MiB (24.5MB), run=14548-15017msec WRITE: bw=1678KiB/s (1718kB/s), 402KiB/s-437KiB/s (411kB/s-448kB/s), io=24.6MiB (25.8MB), run=14548-15017msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159737: Sun Aug 31 18:03:42 2025 read: IOPS=17, BW=285KiB/s (292kB/s)(5920KiB/20796msec) clat (usec): min=2754, max=82876, avg=18429.88, stdev=15685.83 lat (usec): min=2754, max=82877, avg=18430.53, stdev=15685.80 clat percentiles (usec): | 1.00th=[ 3163], 5.00th=[ 4293], 10.00th=[ 5080], 20.00th=[ 6980], | 30.00th=[ 8455], 40.00th=[ 9896], 50.00th=[12387], 60.00th=[15401], | 70.00th=[20579], 80.00th=[28181], 90.00th=[44303], 95.00th=[53216], | 99.00th=[69731], 99.50th=[78119], 99.90th=[83362], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 95, max= 704, per=24.58%, avg=281.68, stdev=131.13, samples=41 iops : min= 5, max= 44, avg=17.24, stdev= 8.39, samples=41 write: IOPS=19, BW=306KiB/s (314kB/s)(6368KiB/20796msec); 0 zone resets clat (msec): min=13, max=137, avg=34.83, stdev=17.58 lat (msec): min=13, max=137, avg=34.83, stdev=17.58 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 18], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 25], 40.00th=[ 27], 50.00th=[ 29], 60.00th=[ 33], | 70.00th=[ 38], 80.00th=[ 44], 90.00th=[ 58], 95.00th=[ 69], | 99.00th=[ 107], 99.50th=[ 132], 99.90th=[ 138], 99.95th=[ 138], | 99.99th=[ 138] bw ( KiB/s): min= 95, max= 640, per=25.28%, avg=304.66, stdev=136.96, samples=41 iops : min= 5, max= 40, avg=18.68, stdev= 8.76, samples=41 lat (msec) : 4=1.95%, 10=17.71%, 20=18.75%, 50=50.26%, 100=10.81% lat (msec) : 250=0.52% cpu : usr=0.07%, sys=12.25%, ctx=1198, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159738: Sun Aug 31 18:03:42 2025 read: IOPS=17, BW=275KiB/s (281kB/s)(5744KiB/20897msec) clat (usec): min=2868, max=94428, avg=17707.25, stdev=15947.76 lat (usec): min=2868, max=94428, avg=17707.83, stdev=15947.73 clat percentiles (usec): | 1.00th=[ 3261], 5.00th=[ 3851], 10.00th=[ 4948], 20.00th=[ 7308], | 30.00th=[ 8717], 40.00th=[ 9896], 50.00th=[11207], 60.00th=[13566], | 70.00th=[17433], 80.00th=[26084], 90.00th=[39584], 95.00th=[54264], | 99.00th=[76022], 99.50th=[77071], 99.90th=[94897], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 32, max= 608, per=23.87%, avg=273.54, stdev=146.32, samples=41 iops : min= 2, max= 38, avg=16.73, stdev= 9.30, samples=41 write: IOPS=19, BW=313KiB/s (321kB/s)(6544KiB/20897msec); 0 zone resets clat (msec): min=13, max=150, avg=35.41, stdev=19.57 lat (msec): min=13, max=150, avg=35.42, stdev=19.57 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 19], 10.00th=[ 21], 20.00th=[ 22], | 30.00th=[ 24], 40.00th=[ 27], 50.00th=[ 29], 60.00th=[ 32], | 70.00th=[ 38], 80.00th=[ 47], 90.00th=[ 61], 95.00th=[ 78], | 99.00th=[ 108], 99.50th=[ 121], 99.90th=[ 150], 99.95th=[ 150], | 99.99th=[ 150] bw ( KiB/s): min= 96, max= 640, per=25.69%, avg=309.51, stdev=138.61, samples=41 iops : min= 6, max= 40, avg=18.98, stdev= 8.87, samples=41 lat (msec) : 4=2.47%, 10=16.54%, 20=20.44%, 50=48.31%, 100=11.33% lat (msec) : 250=0.91% cpu : usr=0.04%, sys=12.25%, ctx=1300, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159739: Sun Aug 31 18:03:42 2025 read: IOPS=19, BW=306KiB/s (313kB/s)(6256KiB/20437msec) clat (msec): min=2, max=116, avg=18.82, stdev=17.78 lat (msec): min=2, max=116, avg=18.82, stdev=17.78 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 15], | 70.00th=[ 19], 80.00th=[ 28], 90.00th=[ 40], 95.00th=[ 58], | 99.00th=[ 87], 99.50th=[ 104], 99.90th=[ 116], 99.95th=[ 116], | 99.99th=[ 116] bw ( KiB/s): min= 95, max= 832, per=26.74%, avg=306.45, stdev=181.10, samples=40 iops : min= 5, max= 52, avg=18.78, stdev=11.50, samples=40 write: IOPS=18, BW=295KiB/s (302kB/s)(6032KiB/20437msec); 0 zone resets clat (msec): min=13, max=154, avg=34.60, stdev=17.19 lat (msec): min=13, max=154, avg=34.60, stdev=17.19 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 19], 10.00th=[ 21], 20.00th=[ 23], | 30.00th=[ 25], 40.00th=[ 27], 50.00th=[ 29], 60.00th=[ 33], | 70.00th=[ 38], 80.00th=[ 45], 90.00th=[ 56], 95.00th=[ 66], | 99.00th=[ 103], 99.50th=[ 110], 99.90th=[ 155], 99.95th=[ 155], | 99.99th=[ 155] bw ( KiB/s): min= 95, max= 608, per=24.16%, avg=291.10, stdev=126.26, samples=40 iops : min= 5, max= 38, avg=17.83, stdev= 8.09, samples=40 lat (msec) : 4=1.69%, 10=17.97%, 20=22.27%, 50=47.66%, 100=9.64% lat (msec) : 250=0.78% cpu : usr=0.07%, sys=12.55%, ctx=1295, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159740: Sun Aug 31 18:03:42 2025 read: IOPS=18, BW=290KiB/s (297kB/s)(6032KiB/20789msec) clat (usec): min=2998, max=90730, avg=18256.46, stdev=16134.99 lat (usec): min=2998, max=90731, avg=18257.10, stdev=16135.03 clat percentiles (usec): | 1.00th=[ 3195], 5.00th=[ 3916], 10.00th=[ 4817], 20.00th=[ 6652], | 30.00th=[ 8717], 40.00th=[10290], 50.00th=[12518], 60.00th=[15270], | 70.00th=[19792], 80.00th=[27657], 90.00th=[40633], 95.00th=[54789], | 99.00th=[84411], 99.50th=[87557], 99.90th=[90702], 99.95th=[90702], | 99.99th=[90702] bw ( KiB/s): min= 64, max= 640, per=25.09%, avg=287.54, stdev=137.98, samples=41 iops : min= 4, max= 40, avg=17.61, stdev= 8.82, samples=41 write: IOPS=18, BW=301KiB/s (308kB/s)(6256KiB/20789msec); 0 zone resets clat (msec): min=14, max=190, avg=35.37, stdev=19.39 lat (msec): min=14, max=190, avg=35.37, stdev=19.39 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 18], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 25], 40.00th=[ 27], 50.00th=[ 31], 60.00th=[ 34], | 70.00th=[ 39], 80.00th=[ 45], 90.00th=[ 56], 95.00th=[ 73], | 99.00th=[ 110], 99.50th=[ 132], 99.90th=[ 190], 99.95th=[ 190], | 99.99th=[ 190] bw ( KiB/s): min= 64, max= 672, per=24.89%, avg=299.90, stdev=146.14, samples=41 iops : min= 4, max= 42, avg=18.37, stdev= 9.33, samples=41 lat (msec) : 4=2.60%, 10=16.02%, 20=21.35%, 50=50.00%, 100=9.24% lat (msec) : 250=0.78% cpu : usr=0.05%, sys=12.17%, ctx=1212, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1146KiB/s (1174kB/s), 275KiB/s-306KiB/s (281kB/s-313kB/s), io=23.4MiB (24.5MB), run=20437-20897msec WRITE: bw=1206KiB/s (1235kB/s), 295KiB/s-313KiB/s (302kB/s-321kB/s), io=24.6MiB (25.8MB), run=20437-20897msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159749: Sun Aug 31 18:03:51 2025 read: IOPS=4, BW=4583KiB/s (4693kB/s)(7168KiB/1564msec) clat (msec): min=41, max=330, avg=93.43, stdev=104.72 lat (msec): min=41, max=330, avg=93.43, stdev=104.72 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 42], 10.00th=[ 42], 20.00th=[ 47], | 30.00th=[ 52], 40.00th=[ 52], 50.00th=[ 56], 60.00th=[ 64], | 70.00th=[ 64], 80.00th=[ 67], 90.00th=[ 330], 95.00th=[ 330], | 99.00th=[ 330], 99.50th=[ 330], 99.90th=[ 330], 99.95th=[ 330], | 99.99th=[ 330] bw ( KiB/s): min= 8722, max= 8722, per=100.00%, avg=8722.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=3, BW=3274KiB/s (3352kB/s)(5120KiB/1564msec); 0 zone resets clat (msec): min=70, max=322, avg=175.57, stdev=94.33 lat (msec): min=70, max=322, avg=175.63, stdev=94.33 clat percentiles (msec): | 1.00th=[ 71], 5.00th=[ 71], 10.00th=[ 71], 20.00th=[ 71], | 30.00th=[ 133], 40.00th=[ 133], 50.00th=[ 153], 60.00th=[ 153], | 70.00th=[ 199], 80.00th=[ 199], 90.00th=[ 321], 95.00th=[ 321], | 99.00th=[ 321], 99.50th=[ 321], 99.90th=[ 321], 99.95th=[ 321], | 99.99th=[ 321] bw ( KiB/s): min= 1744, max= 6119, per=48.57%, avg=3931.50, stdev=3093.59, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 lat (msec) : 50=16.67%, 100=41.67%, 250=25.00%, 500=16.67% cpu : usr=0.13%, sys=20.92%, ctx=90, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159750: Sun Aug 31 18:03:51 2025 read: IOPS=0, BW=942KiB/s (964kB/s)(2048KiB/2175msec) clat (msec): min=61, max=489, avg=275.68, stdev=302.58 lat (msec): min=61, max=489, avg=275.68, stdev=302.58 clat percentiles (msec): | 1.00th=[ 62], 5.00th=[ 62], 10.00th=[ 62], 20.00th=[ 62], | 30.00th=[ 62], 40.00th=[ 62], 50.00th=[ 62], 60.00th=[ 489], | 70.00th=[ 489], 80.00th=[ 489], 90.00th=[ 489], 95.00th=[ 489], | 99.00th=[ 489], 99.50th=[ 489], 99.90th=[ 489], 99.95th=[ 489], | 99.99th=[ 489] bw ( KiB/s): min= 1712, max= 1851, per=30.81%, avg=1781.50, stdev=98.29, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=4, BW=4708KiB/s (4821kB/s)(10.0MiB/2175msec); 0 zone resets clat (msec): min=55, max=363, avg=159.42, stdev=109.38 lat (msec): min=55, max=363, avg=159.50, stdev=109.38 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 56], 10.00th=[ 56], 20.00th=[ 66], | 30.00th=[ 72], 40.00th=[ 73], 50.00th=[ 101], 60.00th=[ 133], | 70.00th=[ 192], 80.00th=[ 247], 90.00th=[ 296], 95.00th=[ 363], | 99.00th=[ 363], 99.50th=[ 363], 99.90th=[ 363], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 1851, max= 6849, per=51.66%, avg=4181.75, stdev=2607.72, samples=4 iops : min= 1, max= 6, avg= 3.25, stdev= 2.63, samples=4 lat (msec) : 100=50.00%, 250=25.00%, 500=25.00% cpu : usr=0.05%, sys=16.88%, ctx=112, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159751: Sun Aug 31 18:03:51 2025 read: IOPS=3, BW=3432KiB/s (3515kB/s)(6144KiB/1790msec) clat (msec): min=50, max=363, avg=164.35, stdev=115.13 lat (msec): min=50, max=363, avg=164.35, stdev=115.13 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 51], 10.00th=[ 51], 20.00th=[ 56], | 30.00th=[ 56], 40.00th=[ 142], 50.00th=[ 142], 60.00th=[ 184], | 70.00th=[ 192], 80.00th=[ 192], 90.00th=[ 363], 95.00th=[ 363], | 99.00th=[ 363], 99.50th=[ 363], 99.90th=[ 363], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 1851, max= 4079, per=53.90%, avg=3116.33, stdev=1144.42, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=3, BW=3432KiB/s (3515kB/s)(6144KiB/1790msec); 0 zone resets clat (msec): min=59, max=317, avg=132.27, stdev=99.47 lat (msec): min=59, max=317, avg=132.33, stdev=99.47 clat percentiles (msec): | 1.00th=[ 60], 5.00th=[ 60], 10.00th=[ 60], 20.00th=[ 64], | 30.00th=[ 64], 40.00th=[ 65], 50.00th=[ 65], 60.00th=[ 131], | 70.00th=[ 159], 80.00th=[ 159], 90.00th=[ 317], 95.00th=[ 317], | 99.00th=[ 317], 99.50th=[ 317], 99.90th=[ 317], 99.95th=[ 317], | 99.99th=[ 317] bw ( KiB/s): min= 3703, max= 6838, per=65.12%, avg=5270.50, stdev=2216.78, samples=2 iops : min= 3, max= 6, avg= 4.50, stdev= 2.12, samples=2 lat (msec) : 100=41.67%, 250=41.67%, 500=16.67% cpu : usr=0.06%, sys=17.10%, ctx=81, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159752: Sun Aug 31 18:03:51 2025 read: IOPS=1, BW=1446KiB/s (1480kB/s)(5120KiB/3542msec) clat (msec): min=62, max=377, avg=170.75, stdev=125.31 lat (msec): min=62, max=377, avg=170.75, stdev=125.31 clat percentiles (msec): | 1.00th=[ 63], 5.00th=[ 63], 10.00th=[ 63], 20.00th=[ 63], | 30.00th=[ 81], 40.00th=[ 81], 50.00th=[ 165], 60.00th=[ 165], | 70.00th=[ 167], 80.00th=[ 167], 90.00th=[ 380], 95.00th=[ 380], | 99.00th=[ 380], 99.50th=[ 380], 99.90th=[ 380], 99.95th=[ 380], | 99.99th=[ 380] bw ( KiB/s): min= 1851, max= 6119, per=68.92%, avg=3985.00, stdev=3017.93, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=1, BW=2024KiB/s (2072kB/s)(7168KiB/3542msec); 0 zone resets clat (msec): min=82, max=1900, avg=382.14, stdev=671.80 lat (msec): min=82, max=1900, avg=382.20, stdev=671.78 clat percentiles (msec): | 1.00th=[ 83], 5.00th=[ 83], 10.00th=[ 83], 20.00th=[ 85], | 30.00th=[ 97], 40.00th=[ 97], 50.00th=[ 129], 60.00th=[ 134], | 70.00th=[ 134], 80.00th=[ 249], 90.00th=[ 1905], 95.00th=[ 1905], | 99.00th=[ 1905], 99.50th=[ 1905], 99.90th=[ 1905], 99.95th=[ 1905], | 99.99th=[ 1905] bw ( KiB/s): min= 3703, max= 6954, per=65.83%, avg=5328.50, stdev=2298.80, samples=2 iops : min= 3, max= 6, avg= 4.50, stdev= 2.12, samples=2 lat (msec) : 100=41.67%, 250=41.67%, 500=8.33%, 2000=8.33% cpu : usr=0.00%, sys=10.65%, ctx=166, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5782KiB/s (5921kB/s), 942KiB/s-4583KiB/s (964kB/s-4693kB/s), io=20.0MiB (20.0MB), run=1564-3542msec WRITE: bw=8095KiB/s (8289kB/s), 2024KiB/s-4708KiB/s (2072kB/s-4821kB/s), io=28.0MiB (29.4MB), run=1564-3542msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159754: Sun Aug 31 18:03:51 2025 read: IOPS=3, BW=3436KiB/s (3519kB/s)(7168KiB/2086msec) clat (usec): min=857, max=242296, avg=167762.44, stdev=80857.56 lat (usec): min=858, max=242297, avg=167763.42, stdev=80857.69 clat percentiles (usec): | 1.00th=[ 857], 5.00th=[ 857], 10.00th=[ 857], 20.00th=[156238], | 30.00th=[166724], 40.00th=[166724], 50.00th=[170918], 60.00th=[202376], | 70.00th=[202376], 80.00th=[235930], 90.00th=[242222], 95.00th=[242222], | 99.00th=[242222], 99.50th=[242222], 99.90th=[242222], 99.95th=[242222], | 99.99th=[242222] bw ( KiB/s): min= 2039, max= 4079, per=47.19%, avg=3008.00, stdev=1119.82, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 write: IOPS=2, BW=2454KiB/s (2513kB/s)(5120KiB/2086msec); 0 zone resets clat (msec): min=116, max=282, avg=182.05, stdev=66.88 lat (msec): min=116, max=282, avg=182.10, stdev=66.87 clat percentiles (msec): | 1.00th=[ 117], 5.00th=[ 117], 10.00th=[ 117], 20.00th=[ 117], | 30.00th=[ 144], 40.00th=[ 144], 50.00th=[ 153], 60.00th=[ 153], | 70.00th=[ 218], 80.00th=[ 218], 90.00th=[ 284], 95.00th=[ 284], | 99.00th=[ 284], 99.50th=[ 284], 99.90th=[ 284], 99.95th=[ 284], | 99.99th=[ 284] bw ( KiB/s): min= 2039, max= 4087, per=30.50%, avg=2721.67, stdev=1182.41, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (usec) : 1000=8.33% lat (msec) : 250=83.33%, 500=8.33% cpu : usr=0.00%, sys=57.12%, ctx=940, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159755: Sun Aug 31 18:03:51 2025 read: IOPS=0, BW=898KiB/s (920kB/s)(2048KiB/2280msec) clat (msec): min=161, max=185, avg=173.28, stdev=16.64 lat (msec): min=161, max=185, avg=173.28, stdev=16.64 clat percentiles (msec): | 1.00th=[ 163], 5.00th=[ 163], 10.00th=[ 163], 20.00th=[ 163], | 30.00th=[ 163], 40.00th=[ 163], 50.00th=[ 163], 60.00th=[ 186], | 70.00th=[ 186], 80.00th=[ 186], 90.00th=[ 186], 95.00th=[ 186], | 99.00th=[ 186], 99.50th=[ 186], 99.90th=[ 186], 99.95th=[ 186], | 99.99th=[ 186] bw ( KiB/s): min= 1957, max= 2039, per=31.35%, avg=1998.00, stdev=57.98, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=4, BW=4491KiB/s (4599kB/s)(10.0MiB/2280msec); 0 zone resets clat (msec): min=95, max=293, avg=190.68, stdev=59.56 lat (msec): min=95, max=293, avg=190.73, stdev=59.57 clat percentiles (msec): | 1.00th=[ 95], 5.00th=[ 95], 10.00th=[ 95], 20.00th=[ 136], | 30.00th=[ 146], 40.00th=[ 186], 50.00th=[ 190], 60.00th=[ 192], | 70.00th=[ 194], 80.00th=[ 203], 90.00th=[ 275], 95.00th=[ 296], | 99.00th=[ 296], 99.50th=[ 296], 99.90th=[ 296], 99.95th=[ 296], | 99.99th=[ 296] bw ( KiB/s): min= 1957, max= 6131, per=45.52%, avg=4061.50, stdev=1704.15, samples=4 iops : min= 1, max= 5, avg= 3.00, stdev= 1.63, samples=4 lat (msec) : 100=8.33%, 250=75.00%, 500=16.67% cpu : usr=0.00%, sys=62.26%, ctx=495, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159756: Sun Aug 31 18:03:51 2025 read: IOPS=1, BW=1912KiB/s (1958kB/s)(6144KiB/3213msec) clat (usec): min=463, max=1745.5k, avg=318081.54, stdev=702214.45 lat (usec): min=463, max=1745.5k, avg=318082.26, stdev=702214.29 clat percentiles (usec): | 1.00th=[ 465], 5.00th=[ 465], 10.00th=[ 465], | 20.00th=[ 627], 30.00th=[ 627], 40.00th=[ 693], | 50.00th=[ 693], 60.00th=[ 742], 70.00th=[ 160433], | 80.00th=[ 160433], 90.00th=[1753220], 95.00th=[1753220], | 99.00th=[1753220], 99.50th=[1753220], 99.90th=[1753220], | 99.95th=[1753220], 99.99th=[1753220] bw ( KiB/s): min= 2039, max= 2048, per=32.06%, avg=2043.50, stdev= 6.36, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1912KiB/s (1958kB/s)(6144KiB/3213msec); 0 zone resets clat (msec): min=72, max=447, avg=217.16, stdev=177.63 lat (msec): min=72, max=447, avg=217.22, stdev=177.63 clat percentiles (msec): | 1.00th=[ 73], 5.00th=[ 73], 10.00th=[ 73], 20.00th=[ 82], | 30.00th=[ 82], 40.00th=[ 89], 50.00th=[ 89], 60.00th=[ 178], | 70.00th=[ 435], 80.00th=[ 435], 90.00th=[ 447], 95.00th=[ 447], | 99.00th=[ 447], 99.50th=[ 447], 99.90th=[ 447], 99.95th=[ 447], | 99.99th=[ 447] bw ( KiB/s): min= 1932, max= 2039, per=22.45%, avg=2003.33, stdev=61.78, samples=3 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=3 lat (usec) : 500=8.33%, 750=25.00% lat (msec) : 100=25.00%, 250=16.67%, 500=16.67%, 2000=8.33% cpu : usr=0.03%, sys=27.96%, ctx=343, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159757: Sun Aug 31 18:03:51 2025 read: IOPS=1, BW=1610KiB/s (1648kB/s)(5120KiB/3181msec) clat (usec): min=699, max=146405, avg=83598.94, stdev=75872.10 lat (usec): min=700, max=146406, avg=83599.69, stdev=75872.09 clat percentiles (usec): | 1.00th=[ 701], 5.00th=[ 701], 10.00th=[ 701], 20.00th=[ 701], | 30.00th=[ 742], 40.00th=[ 742], 50.00th=[130548], 60.00th=[130548], | 70.00th=[139461], 80.00th=[139461], 90.00th=[145753], 95.00th=[145753], | 99.00th=[145753], 99.50th=[145753], 99.90th=[145753], 99.95th=[145753], | 99.99th=[145753] bw ( KiB/s): min= 2048, max= 6131, per=64.16%, avg=4089.50, stdev=2887.12, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 write: IOPS=2, BW=2253KiB/s (2307kB/s)(7168KiB/3181msec); 0 zone resets clat (msec): min=96, max=894, avg=394.44, stdev=308.38 lat (msec): min=96, max=895, avg=394.49, stdev=308.39 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 96], 10.00th=[ 96], 20.00th=[ 115], | 30.00th=[ 169], 40.00th=[ 169], 50.00th=[ 368], 60.00th=[ 409], | 70.00th=[ 409], 80.00th=[ 709], 90.00th=[ 894], 95.00th=[ 894], | 99.00th=[ 894], 99.50th=[ 894], 99.90th=[ 894], 99.95th=[ 894], | 99.99th=[ 894] bw ( KiB/s): min= 2039, max= 4096, per=27.48%, avg=2452.00, stdev=919.03, samples=5 iops : min= 1, max= 4, avg= 1.60, stdev= 1.34, samples=5 lat (usec) : 750=16.67% lat (msec) : 100=8.33%, 250=41.67%, 500=16.67%, 750=8.33%, 1000=8.33% cpu : usr=0.00%, sys=33.62%, ctx=175, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6374KiB/s (6527kB/s), 898KiB/s-3436KiB/s (920kB/s-3519kB/s), io=20.0MiB (20.0MB), run=2086-3213msec WRITE: bw=8924KiB/s (9138kB/s), 1912KiB/s-4491KiB/s (1958kB/s-4599kB/s), io=28.0MiB (29.4MB), run=2086-3213msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159762: Sun Aug 31 18:04:00 2025 read: IOPS=1, BW=4683KiB/s (4795kB/s)(12.0MiB/2624msec) clat (msec): min=658, max=1068, avg=820.54, stdev=218.29 lat (msec): min=658, max=1068, avg=820.54, stdev=218.29 clat percentiles (msec): | 1.00th=[ 659], 5.00th=[ 659], 10.00th=[ 659], 20.00th=[ 659], | 30.00th=[ 659], 40.00th=[ 735], 50.00th=[ 735], 60.00th=[ 735], | 70.00th=[ 1070], 80.00th=[ 1070], 90.00th=[ 1070], 95.00th=[ 1070], | 99.00th=[ 1070], 99.50th=[ 1070], 99.90th=[ 1070], 99.95th=[ 1070], | 99.99th=[ 1070] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=15.90%, ctx=65, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159763: Sun Aug 31 18:04:00 2025 read: IOPS=1, BW=4649KiB/s (4761kB/s)(4096KiB/881msec) clat (nsec): min=127072k, max=127072k, avg=127071636.00, stdev= 0.00 lat (nsec): min=127074k, max=127074k, avg=127073980.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 128], 5.00th=[ 128], 10.00th=[ 128], 20.00th=[ 128], | 30.00th=[ 128], 40.00th=[ 128], 50.00th=[ 128], 60.00th=[ 128], | 70.00th=[ 128], 80.00th=[ 128], 90.00th=[ 128], 95.00th=[ 128], | 99.00th=[ 128], 99.50th=[ 128], 99.90th=[ 128], 99.95th=[ 128], | 99.99th=[ 128] bw ( KiB/s): min= 5468, max= 5468, per=87.59%, avg=5468.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=2, BW=9299KiB/s (9522kB/s)(8192KiB/881msec); 0 zone resets clat (msec): min=281, max=311, avg=296.40, stdev=20.81 lat (msec): min=281, max=311, avg=296.65, stdev=20.80 clat percentiles (msec): | 1.00th=[ 284], 5.00th=[ 284], 10.00th=[ 284], 20.00th=[ 284], | 30.00th=[ 284], 40.00th=[ 284], 50.00th=[ 284], 60.00th=[ 313], | 70.00th=[ 313], 80.00th=[ 313], 90.00th=[ 313], 95.00th=[ 313], | 99.00th=[ 313], 99.50th=[ 313], 99.90th=[ 313], 99.95th=[ 313], | 99.99th=[ 313] bw ( KiB/s): min= 5468, max= 5468, per=46.76%, avg=5468.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=66.67% cpu : usr=0.00%, sys=43.07%, ctx=322, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159764: Sun Aug 31 18:04:00 2025 write: IOPS=1, BW=4385KiB/s (4491kB/s)(12.0MiB/2802msec); 0 zone resets clat (msec): min=441, max=1164, avg=882.48, stdev=386.49 lat (msec): min=442, max=1164, avg=882.79, stdev=386.48 clat percentiles (msec): | 1.00th=[ 443], 5.00th=[ 443], 10.00th=[ 443], 20.00th=[ 443], | 30.00th=[ 443], 40.00th=[ 1045], 50.00th=[ 1045], 60.00th=[ 1045], | 70.00th=[ 1167], 80.00th=[ 1167], 90.00th=[ 1167], 95.00th=[ 1167], | 99.00th=[ 1167], 99.50th=[ 1167], 99.90th=[ 1167], 99.95th=[ 1167], | 99.99th=[ 1167] bw ( KiB/s): min= 5512, max= 8192, per=58.59%, avg=6852.00, stdev=1895.05, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.00%, sys=19.31%, ctx=130, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159765: Sun Aug 31 18:04:00 2025 write: IOPS=1, BW=5829KiB/s (5969kB/s)(12.0MiB/2108msec); 0 zone resets clat (msec): min=148, max=1531, avg=648.39, stdev=767.36 lat (msec): min=149, max=1532, avg=648.72, stdev=767.33 clat percentiles (msec): | 1.00th=[ 148], 5.00th=[ 148], 10.00th=[ 148], 20.00th=[ 148], | 30.00th=[ 148], 40.00th=[ 266], 50.00th=[ 266], 60.00th=[ 266], | 70.00th=[ 1536], 80.00th=[ 1536], 90.00th=[ 1536], 95.00th=[ 1536], | 99.00th=[ 1536], 99.50th=[ 1536], 99.90th=[ 1536], 99.95th=[ 1536], | 99.99th=[ 1536] bw ( KiB/s): min= 8175, max= 8192, per=69.98%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=20.03%, ctx=80, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6244KiB/s (6394kB/s), 4649KiB/s-4683KiB/s (4761kB/s-4795kB/s), io=16.0MiB (16.8MB), run=881-2624msec WRITE: bw=11.4MiB/s (11.0MB/s), 4385KiB/s-9299KiB/s (4491kB/s-9522kB/s), io=32.0MiB (33.6MB), run=881-2802msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159767: Sun Aug 31 18:04:01 2025 read: IOPS=1, BW=7470KiB/s (7649kB/s)(12.0MiB/1645msec) clat (msec): min=377, max=812, avg=547.71, stdev=232.71 lat (msec): min=377, max=812, avg=547.71, stdev=232.71 clat percentiles (msec): | 1.00th=[ 380], 5.00th=[ 380], 10.00th=[ 380], 20.00th=[ 380], | 30.00th=[ 380], 40.00th=[ 451], 50.00th=[ 451], 60.00th=[ 451], | 70.00th=[ 810], 80.00th=[ 810], 90.00th=[ 810], 95.00th=[ 810], | 99.00th=[ 810], 99.50th=[ 810], 99.90th=[ 810], 99.95th=[ 810], | 99.99th=[ 810] bw ( KiB/s): min= 6930, max= 8192, per=100.00%, avg=7561.00, stdev=892.37, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=66.67%, 1000=33.33% cpu : usr=0.06%, sys=39.96%, ctx=2027, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159768: Sun Aug 31 18:04:01 2025 read: IOPS=0, BW=1352KiB/s (1385kB/s)(4096KiB/3029msec) clat (nsec): min=492762k, max=492762k, avg=492762450.00, stdev= 0.00 lat (nsec): min=492764k, max=492764k, avg=492763809.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 493], 5.00th=[ 493], 10.00th=[ 493], 20.00th=[ 493], | 30.00th=[ 493], 40.00th=[ 493], 50.00th=[ 493], 60.00th=[ 493], | 70.00th=[ 493], 80.00th=[ 493], 90.00th=[ 493], 95.00th=[ 493], | 99.00th=[ 493], 99.50th=[ 493], 99.90th=[ 493], 99.95th=[ 493], | 99.99th=[ 493] bw ( KiB/s): min= 6491, max= 6491, per=100.00%, avg=6491.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2705KiB/s (2769kB/s)(8192KiB/3029msec); 0 zone resets clat (msec): min=1099, max=1435, avg=1267.44, stdev=238.16 lat (msec): min=1099, max=1436, avg=1267.62, stdev=238.22 clat percentiles (msec): | 1.00th=[ 1099], 5.00th=[ 1099], 10.00th=[ 1099], 20.00th=[ 1099], | 30.00th=[ 1099], 40.00th=[ 1099], 50.00th=[ 1099], 60.00th=[ 1435], | 70.00th=[ 1435], 80.00th=[ 1435], 90.00th=[ 1435], 95.00th=[ 1435], | 99.00th=[ 1435], 99.50th=[ 1435], 99.90th=[ 1435], 99.95th=[ 1435], | 99.99th=[ 1435] bw ( KiB/s): min= 8047, max= 8047, per=76.06%, avg=8047.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.00%, sys=32.40%, ctx=953, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159769: Sun Aug 31 18:04:01 2025 write: IOPS=0, BW=3968KiB/s (4063kB/s)(12.0MiB/3097msec); 0 zone resets clat (msec): min=510, max=1334, avg=1031.71, stdev=453.70 lat (msec): min=510, max=1334, avg=1031.97, stdev=453.71 clat percentiles (msec): | 1.00th=[ 510], 5.00th=[ 510], 10.00th=[ 510], 20.00th=[ 510], | 30.00th=[ 510], 40.00th=[ 1250], 50.00th=[ 1250], 60.00th=[ 1250], | 70.00th=[ 1334], 80.00th=[ 1334], 90.00th=[ 1334], 95.00th=[ 1334], | 99.00th=[ 1334], 99.50th=[ 1334], 99.90th=[ 1334], 99.95th=[ 1334], | 99.99th=[ 1334] bw ( KiB/s): min= 8159, max= 8192, per=77.27%, avg=8175.50, stdev=23.33, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.03%, sys=41.51%, ctx=89, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159770: Sun Aug 31 18:04:01 2025 write: IOPS=1, BW=4345KiB/s (4449kB/s)(12.0MiB/2828msec); 0 zone resets clat (msec): min=623, max=1113, avg=941.68, stdev=275.78 lat (msec): min=623, max=1113, avg=941.94, stdev=275.88 clat percentiles (msec): | 1.00th=[ 625], 5.00th=[ 625], 10.00th=[ 625], 20.00th=[ 625], | 30.00th=[ 625], 40.00th=[ 1083], 50.00th=[ 1083], 60.00th=[ 1083], | 70.00th=[ 1116], 80.00th=[ 1116], 90.00th=[ 1116], 95.00th=[ 1116], | 99.00th=[ 1116], 99.50th=[ 1116], 99.90th=[ 1116], 99.95th=[ 1116], | 99.99th=[ 1116] bw ( KiB/s): min= 8159, max= 8192, per=77.27%, avg=8175.50, stdev=23.33, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=51.19%, ctx=108, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5409KiB/s (5539kB/s), 1352KiB/s-7470KiB/s (1385kB/s-7649kB/s), io=16.0MiB (16.8MB), run=1645-3029msec WRITE: bw=10.3MiB/s (10.8MB/s), 2705KiB/s-4345KiB/s (2769kB/s-4449kB/s), io=32.0MiB (33.6MB), run=2828-3097msec