== sanity test 398b: DIO and buffer IO race ============== 14:00:15 (1763406015) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 4.88558 s, 10.3 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147589: Mon Nov 17 14:02:12 2025 read: IOPS=15, BW=60.0KiB/s (61.5kB/s)(5992KiB/99794msec) clat (usec): min=100, max=294322, avg=32179.71, stdev=25021.33 lat (usec): min=100, max=294324, avg=32181.21, stdev=25021.31 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 184], 10.00th=[ 371], 20.00th=[ 15533], | 30.00th=[ 19530], 40.00th=[ 23725], 50.00th=[ 27919], 60.00th=[ 32900], | 70.00th=[ 39584], 80.00th=[ 47449], 90.00th=[ 60031], 95.00th=[ 71828], | 99.00th=[114820], 99.50th=[149947], 99.90th=[267387], 99.95th=[295699], | 99.99th=[295699] bw ( KiB/s): min= 7, max= 119, per=25.31%, avg=60.23, stdev=21.98, samples=195 iops : min= 1, max= 29, avg=14.44, stdev= 5.52, samples=195 write: IOPS=15, BW=63.1KiB/s (64.6kB/s)(6296KiB/99794msec); 0 zone resets clat (usec): min=892, max=661622, avg=32558.43, stdev=38004.36 lat (usec): min=892, max=661624, avg=32560.22, stdev=38004.30 clat percentiles (usec): | 1.00th=[ 1012], 5.00th=[ 1172], 10.00th=[ 1319], 20.00th=[ 2802], | 30.00th=[ 8225], 40.00th=[ 16909], 50.00th=[ 24511], 60.00th=[ 31851], | 70.00th=[ 41157], 80.00th=[ 54264], 90.00th=[ 72877], 95.00th=[ 93848], | 99.00th=[158335], 99.50th=[198181], 99.90th=[497026], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 7, max= 151, per=25.12%, avg=62.55, stdev=24.23, samples=197 iops : min= 1, max= 37, avg=15.01, stdev= 6.05, samples=197 lat (usec) : 250=3.84%, 500=1.24%, 750=0.16%, 1000=0.46% lat (msec) : 2=8.14%, 4=3.68%, 10=5.31%, 20=14.71%, 50=41.96% lat (msec) : 100=18.03%, 250=2.34%, 500=0.10%, 750=0.03% cpu : usr=0.13%, sys=7.87%, ctx=6552, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147590: Mon Nov 17 14:02:12 2025 read: IOPS=14, BW=58.0KiB/s (60.4kB/s)(5940KiB/100730msec) clat (usec): min=100, max=821499, avg=33233.85, stdev=35428.96 lat (usec): min=100, max=821500, avg=33235.79, stdev=35428.97 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 176], 10.00th=[ 343], 20.00th=[ 15795], | 30.00th=[ 20317], 40.00th=[ 23987], 50.00th=[ 28705], 60.00th=[ 33817], | 70.00th=[ 39584], 80.00th=[ 47449], 90.00th=[ 59507], 95.00th=[ 72877], | 99.00th=[114820], 99.50th=[135267], 99.90th=[509608], 99.95th=[817890], | 99.99th=[817890] bw ( KiB/s): min= 7, max= 127, per=24.69%, avg=58.77, stdev=24.83, samples=196 iops : min= 1, max= 31, avg=14.07, stdev= 6.24, samples=196 write: IOPS=15, BW=63.0KiB/s (64.5kB/s)(6348KiB/100730msec); 0 zone resets clat (usec): min=949, max=387029, avg=32144.32, stdev=34859.17 lat (usec): min=951, max=387031, avg=32147.92, stdev=34859.24 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 1205], 10.00th=[ 1319], 20.00th=[ 2343], | 30.00th=[ 6652], 40.00th=[ 14353], 50.00th=[ 23200], 60.00th=[ 32375], | 70.00th=[ 42206], 80.00th=[ 57934], 90.00th=[ 74974], 95.00th=[ 90702], | 99.00th=[152044], 99.50th=[173016], 99.90th=[354419], 99.95th=[387974], | 99.99th=[387974] bw ( KiB/s): min= 7, max= 134, per=25.00%, avg=62.25, stdev=24.66, samples=197 iops : min= 1, max= 33, avg=14.94, stdev= 6.17, samples=197 lat (usec) : 250=4.23%, 500=1.04%, 750=0.10%, 1000=0.52% lat (msec) : 2=9.05%, 4=3.03%, 10=6.90%, 20=13.41%, 50=40.69% lat (msec) : 100=18.33%, 250=2.44%, 500=0.20%, 750=0.03%, 1000=0.03% cpu : usr=0.12%, sys=7.63%, ctx=6353, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147591: Mon Nov 17 14:02:12 2025 read: IOPS=15, BW=60.1KiB/s (61.6kB/s)(6028KiB/100226msec) clat (usec): min=110, max=774326, avg=32143.45, stdev=30837.21 lat (usec): min=110, max=774328, avg=32146.08, stdev=30837.09 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 180], 10.00th=[ 322], 20.00th=[ 15533], | 30.00th=[ 19792], 40.00th=[ 23725], 50.00th=[ 27395], 60.00th=[ 32900], | 70.00th=[ 39060], 80.00th=[ 46924], 90.00th=[ 59507], 95.00th=[ 70779], | 99.00th=[116917], 99.50th=[156238], 99.90th=[229639], 99.95th=[775947], | 99.99th=[775947] bw ( KiB/s): min= 7, max= 126, per=25.13%, avg=59.82, stdev=24.40, samples=196 iops : min= 1, max= 31, avg=14.33, stdev= 6.12, samples=196 write: IOPS=15, BW=62.5KiB/s (63.0kB/s)(6260KiB/100226msec); 0 zone resets clat (usec): min=921, max=354823, avg=32871.75, stdev=34251.11 lat (usec): min=922, max=354824, avg=32873.93, stdev=34251.30 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 1188], 10.00th=[ 1385], 20.00th=[ 3359], | 30.00th=[ 7832], 40.00th=[ 16581], 50.00th=[ 24773], 60.00th=[ 33817], | 70.00th=[ 43779], 80.00th=[ 55837], 90.00th=[ 74974], 95.00th=[ 93848], | 99.00th=[147850], 99.50th=[185598], 99.90th=[320865], 99.95th=[354419], | 99.99th=[354419] bw ( KiB/s): min= 7, max= 125, per=24.95%, avg=62.13, stdev=24.69, samples=196 iops : min= 1, max= 31, avg=14.91, stdev= 6.19, samples=196 lat (usec) : 250=4.43%, 500=0.98%, 750=0.16%, 1000=0.36% lat (msec) : 2=7.42%, 4=3.87%, 10=6.15%, 20=14.52%, 50=41.54% lat (msec) : 100=17.68%, 250=2.80%, 500=0.07%, 1000=0.03% cpu : usr=0.10%, sys=7.47%, ctx=6401, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147592: Mon Nov 17 14:02:12 2025 read: IOPS=15, BW=61.8KiB/s (63.3kB/s)(6052KiB/97974msec) clat (usec): min=106, max=482327, avg=31706.85, stdev=26153.44 lat (usec): min=106, max=482329, avg=31710.83, stdev=26154.20 clat percentiles (usec): | 1.00th=[ 141], 5.00th=[ 178], 10.00th=[ 322], 20.00th=[ 15401], | 30.00th=[ 19530], 40.00th=[ 23462], 50.00th=[ 27657], 60.00th=[ 32637], | 70.00th=[ 39060], 80.00th=[ 46400], 90.00th=[ 59507], 95.00th=[ 70779], | 99.00th=[109577], 99.50th=[137364], 99.90th=[258999], 99.95th=[480248], | 99.99th=[480248] bw ( KiB/s): min= 15, max= 119, per=25.90%, avg=61.65, stdev=23.66, samples=192 iops : min= 3, max= 29, avg=14.79, stdev= 5.94, samples=192 write: IOPS=15, BW=63.6KiB/s (65.2kB/s)(6236KiB/97974msec); 0 zone resets clat (usec): min=968, max=658477, avg=31842.26, stdev=35786.34 lat (usec): min=970, max=658478, avg=31844.92, stdev=35786.79 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 1188], 10.00th=[ 1303], 20.00th=[ 2835], | 30.00th=[ 6980], 40.00th=[ 15270], 50.00th=[ 24249], 60.00th=[ 32637], | 70.00th=[ 41157], 80.00th=[ 53740], 90.00th=[ 73925], 95.00th=[ 92799], | 99.00th=[139461], 99.50th=[173016], 99.90th=[316670], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 7, max= 135, per=25.37%, avg=63.17, stdev=23.00, samples=193 iops : min= 1, max= 33, avg=15.18, stdev= 5.74, samples=193 lat (usec) : 250=4.20%, 500=1.07%, 750=0.10%, 1000=0.16% lat (msec) : 2=9.05%, 4=3.16%, 10=6.54%, 20=13.83%, 50=42.22% lat (msec) : 100=17.19%, 250=2.34%, 500=0.10%, 750=0.03% cpu : usr=0.11%, sys=7.77%, ctx=6566, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=238KiB/s (244kB/s), 58.0KiB/s-61.8KiB/s (60.4kB/s-63.3kB/s), io=23.4MiB (24.6MB), run=97974-100730msec WRITE: bw=250KiB/s (256kB/s), 62.5KiB/s-63.6KiB/s (63.0kB/s-65.2kB/s), io=24.6MiB (25.7MB), run=97974-100730msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147594: Mon Nov 17 14:04:31 2025 read: IOPS=6, BW=25.1KiB/s (25.7kB/s)(5992KiB/239045msec) clat (msec): min=6, max=801, avg=32.60, stdev=31.93 lat (msec): min=6, max=801, avg=32.60, stdev=31.93 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 31], | 70.00th=[ 39], 80.00th=[ 48], 90.00th=[ 62], 95.00th=[ 77], | 99.00th=[ 131], 99.50th=[ 161], 99.90th=[ 262], 99.95th=[ 802], | 99.99th=[ 802] bw ( KiB/s): min= 7, max= 110, per=27.40%, avg=27.40, stdev=17.22, samples=424 iops : min= 1, max= 27, avg= 6.24, stdev= 4.35, samples=424 write: IOPS=6, BW=26.3KiB/s (26.0kB/s)(6296KiB/239045msec); 0 zone resets clat (msec): min=35, max=526, avg=120.75, stdev=53.35 lat (msec): min=35, max=526, avg=120.76, stdev=53.35 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 60], 10.00th=[ 68], 20.00th=[ 80], | 30.00th=[ 89], 40.00th=[ 100], 50.00th=[ 109], 60.00th=[ 122], | 70.00th=[ 136], 80.00th=[ 155], 90.00th=[ 188], 95.00th=[ 222], | 99.00th=[ 292], 99.50th=[ 347], 99.90th=[ 498], 99.95th=[ 527], | 99.99th=[ 527] bw ( KiB/s): min= 7, max= 56, per=24.91%, avg=25.90, stdev=10.27, samples=470 iops : min= 1, max= 14, avg= 5.85, stdev= 2.63, samples=470 lat (msec) : 10=2.77%, 20=17.77%, 50=20.25%, 100=27.86%, 250=29.88% lat (msec) : 500=1.40%, 750=0.03%, 1000=0.03% cpu : usr=0.05%, sys=1.62%, ctx=5588, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147595: Mon Nov 17 14:04:31 2025 read: IOPS=6, BW=24.9KiB/s (25.5kB/s)(5940KiB/238397msec) clat (msec): min=5, max=700, avg=32.52, stdev=32.52 lat (msec): min=5, max=700, avg=32.52, stdev=32.52 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 19], 50.00th=[ 23], 60.00th=[ 29], | 70.00th=[ 39], 80.00th=[ 49], 90.00th=[ 62], 95.00th=[ 77], | 99.00th=[ 126], 99.50th=[ 178], 99.90th=[ 363], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 7, max= 102, per=27.27%, avg=27.27, stdev=17.73, samples=422 iops : min= 1, max= 25, avg= 6.20, stdev= 4.47, samples=422 write: IOPS=6, BW=26.6KiB/s (27.3kB/s)(6348KiB/238397msec); 0 zone resets clat (msec): min=39, max=573, avg=119.69, stdev=52.75 lat (msec): min=39, max=573, avg=119.70, stdev=52.75 clat percentiles (msec): | 1.00th=[ 47], 5.00th=[ 59], 10.00th=[ 67], 20.00th=[ 79], | 30.00th=[ 90], 40.00th=[ 100], 50.00th=[ 109], 60.00th=[ 121], | 70.00th=[ 134], 80.00th=[ 153], 90.00th=[ 184], 95.00th=[ 220], | 99.00th=[ 309], 99.50th=[ 342], 99.90th=[ 435], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 7, max= 56, per=25.24%, avg=26.25, stdev=10.38, samples=468 iops : min= 1, max= 14, avg= 5.94, stdev= 2.66, samples=468 lat (msec) : 10=3.52%, 20=17.45%, 50=19.17%, 100=28.55%, 250=29.88% lat (msec) : 500=1.37%, 750=0.07% cpu : usr=0.05%, sys=1.60%, ctx=5736, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147596: Mon Nov 17 14:04:31 2025 read: IOPS=6, BW=25.1KiB/s (25.7kB/s)(6028KiB/239794msec) clat (msec): min=4, max=375, avg=33.17, stdev=27.98 lat (msec): min=4, max=375, avg=33.17, stdev=27.98 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 31], | 70.00th=[ 40], 80.00th=[ 50], 90.00th=[ 65], 95.00th=[ 79], | 99.00th=[ 133], 99.50th=[ 178], 99.90th=[ 266], 99.95th=[ 376], | 99.99th=[ 376] bw ( KiB/s): min= 7, max= 135, per=26.76%, avg=26.76, stdev=17.54, samples=436 iops : min= 1, max= 33, avg= 6.08, stdev= 4.44, samples=436 write: IOPS=6, BW=26.1KiB/s (26.7kB/s)(6260KiB/239794msec); 0 zone resets clat (msec): min=37, max=788, avg=121.18, stdev=57.19 lat (msec): min=37, max=788, avg=121.18, stdev=57.19 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 61], 10.00th=[ 68], 20.00th=[ 80], | 30.00th=[ 88], 40.00th=[ 99], 50.00th=[ 108], 60.00th=[ 120], | 70.00th=[ 134], 80.00th=[ 155], 90.00th=[ 192], 95.00th=[ 228], | 99.00th=[ 326], 99.50th=[ 359], 99.90th=[ 575], 99.95th=[ 793], | 99.99th=[ 793] bw ( KiB/s): min= 7, max= 55, per=24.76%, avg=25.75, stdev=10.73, samples=469 iops : min= 1, max= 13, avg= 5.81, stdev= 2.75, samples=469 lat (msec) : 10=3.52%, 20=16.37%, 50=20.18%, 100=29.46%, 250=28.74% lat (msec) : 500=1.66%, 750=0.03%, 1000=0.03% cpu : usr=0.07%, sys=1.60%, ctx=5680, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147597: Mon Nov 17 14:04:31 2025 read: IOPS=6, BW=25.5KiB/s (26.2kB/s)(6052KiB/236912msec) clat (msec): min=5, max=374, avg=32.33, stdev=28.76 lat (msec): min=5, max=374, avg=32.34, stdev=28.76 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 16], 40.00th=[ 20], 50.00th=[ 23], 60.00th=[ 30], | 70.00th=[ 38], 80.00th=[ 47], 90.00th=[ 63], 95.00th=[ 82], | 99.00th=[ 131], 99.50th=[ 171], 99.90th=[ 355], 99.95th=[ 376], | 99.99th=[ 376] bw ( KiB/s): min= 7, max= 93, per=27.35%, avg=27.35, stdev=17.82, samples=429 iops : min= 1, max= 23, avg= 6.22, stdev= 4.49, samples=429 write: IOPS=6, BW=26.3KiB/s (26.0kB/s)(6236KiB/236912msec); 0 zone resets clat (msec): min=29, max=784, avg=120.51, stdev=56.88 lat (msec): min=29, max=784, avg=120.51, stdev=56.88 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 58], 10.00th=[ 67], 20.00th=[ 80], | 30.00th=[ 88], 40.00th=[ 99], 50.00th=[ 107], 60.00th=[ 120], | 70.00th=[ 136], 80.00th=[ 155], 90.00th=[ 190], 95.00th=[ 224], | 99.00th=[ 330], 99.50th=[ 368], 99.90th=[ 502], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 7, max= 63, per=24.95%, avg=25.95, stdev=10.79, samples=465 iops : min= 1, max= 15, avg= 5.86, stdev= 2.77, samples=465 lat (msec) : 10=3.81%, 20=17.51%, 50=20.15%, 100=28.22%, 250=28.52% lat (msec) : 500=1.73%, 750=0.03%, 1000=0.03% cpu : usr=0.05%, sys=1.69%, ctx=5832, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=100KiB/s (103kB/s), 24.9KiB/s-25.5KiB/s (25.5kB/s-26.2kB/s), io=23.4MiB (24.6MB), run=236912-239794msec WRITE: bw=105KiB/s (107kB/s), 26.1KiB/s-26.6KiB/s (26.7kB/s-27.3kB/s), io=24.6MiB (25.7MB), run=236912-239794msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147663: Mon Nov 17 14:05:09 2025 read: IOPS=13, BW=216KiB/s (221kB/s)(5920KiB/27408msec) clat (usec): min=115, max=355374, avg=37950.04, stdev=34860.52 lat (usec): min=115, max=355376, avg=37951.76, stdev=34860.50 clat percentiles (usec): | 1.00th=[ 123], 5.00th=[ 163], 10.00th=[ 182], 20.00th=[ 457], | 30.00th=[ 21365], 40.00th=[ 28443], 50.00th=[ 35914], 60.00th=[ 41681], | 70.00th=[ 49021], 80.00th=[ 57410], 90.00th=[ 71828], 95.00th=[ 89654], | 99.00th=[149947], 99.50th=[223347], 99.90th=[354419], 99.95th=[354419], | 99.99th=[354419] bw ( KiB/s): min= 63, max= 383, per=25.62%, avg=215.17, stdev=78.08, samples=53 iops : min= 3, max= 23, avg=12.77, stdev= 4.95, samples=53 write: IOPS=14, BW=232KiB/s (238kB/s)(6368KiB/27408msec); 0 zone resets clat (usec): min=1339, max=268504, avg=33145.53, stdev=36174.63 lat (usec): min=1340, max=268507, avg=33203.79, stdev=36257.02 clat percentiles (usec): | 1.00th=[ 1450], 5.00th=[ 1631], 10.00th=[ 1745], 20.00th=[ 3195], | 30.00th=[ 6915], 40.00th=[ 13698], 50.00th=[ 20055], 60.00th=[ 30278], | 70.00th=[ 44303], 80.00th=[ 59507], 90.00th=[ 82314], 95.00th=[105382], | 99.00th=[160433], 99.50th=[170918], 99.90th=[270533], 99.95th=[270533], | 99.99th=[270533] bw ( KiB/s): min= 31, max= 503, per=26.32%, avg=232.40, stdev=93.56, samples=53 iops : min= 1, max= 31, avg=13.85, stdev= 5.84, samples=53 lat (usec) : 250=8.85%, 500=1.04%, 750=0.26% lat (msec) : 2=7.94%, 4=4.04%, 10=7.81%, 20=9.24%, 50=33.46% lat (msec) : 100=22.53%, 250=4.56%, 500=0.26% cpu : usr=0.12%, sys=7.03%, ctx=1384, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147664: Mon Nov 17 14:05:09 2025 read: IOPS=12, BW=201KiB/s (206kB/s)(5744KiB/28508msec) clat (usec): min=108, max=198812, avg=36788.58, stdev=30701.90 lat (usec): min=108, max=198814, avg=36790.06, stdev=30701.88 clat percentiles (usec): | 1.00th=[ 121], 5.00th=[ 153], 10.00th=[ 182], 20.00th=[ 545], | 30.00th=[ 21365], 40.00th=[ 27395], 50.00th=[ 33162], 60.00th=[ 40109], | 70.00th=[ 47973], 80.00th=[ 57410], 90.00th=[ 73925], 95.00th=[ 89654], | 99.00th=[128451], 99.50th=[196084], 99.90th=[198181], 99.95th=[198181], | 99.99th=[198181] bw ( KiB/s): min= 31, max= 383, per=24.31%, avg=204.22, stdev=96.36, samples=54 iops : min= 1, max= 23, avg=12.04, stdev= 6.07, samples=54 write: IOPS=14, BW=230KiB/s (235kB/s)(6544KiB/28508msec); 0 zone resets clat (usec): min=1347, max=264313, avg=36436.39, stdev=40504.05 lat (usec): min=1348, max=264315, avg=36438.74, stdev=40503.99 clat percentiles (usec): | 1.00th=[ 1434], 5.00th=[ 1614], 10.00th=[ 1795], 20.00th=[ 5014], | 30.00th=[ 10028], 40.00th=[ 15795], 50.00th=[ 24511], 60.00th=[ 30540], | 70.00th=[ 45351], 80.00th=[ 63177], 90.00th=[ 84411], 95.00th=[108528], | 99.00th=[198181], 99.50th=[223347], 99.90th=[265290], 99.95th=[265290], | 99.99th=[265290] bw ( KiB/s): min= 31, max= 417, per=25.59%, avg=225.95, stdev=84.20, samples=55 iops : min= 1, max= 26, avg=13.44, stdev= 5.35, samples=55 lat (usec) : 250=8.07%, 500=1.17%, 750=0.39% lat (msec) : 2=7.94%, 4=2.21%, 10=6.64%, 20=10.29%, 50=36.46% lat (msec) : 100=21.88%, 250=4.82%, 500=0.13% cpu : usr=0.08%, sys=7.15%, ctx=1415, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147665: Mon Nov 17 14:05:09 2025 read: IOPS=14, BW=228KiB/s (234kB/s)(6256KiB/27402msec) clat (usec): min=100, max=205268, avg=38992.63, stdev=29297.25 lat (usec): min=100, max=205269, avg=38994.15, stdev=29297.27 clat percentiles (usec): | 1.00th=[ 113], 5.00th=[ 161], 10.00th=[ 186], 20.00th=[ 14222], | 30.00th=[ 25297], 40.00th=[ 30540], 50.00th=[ 37487], 60.00th=[ 43254], | 70.00th=[ 50594], 80.00th=[ 60556], 90.00th=[ 72877], 95.00th=[ 90702], | 99.00th=[122160], 99.50th=[158335], 99.90th=[204473], 99.95th=[204473], | 99.99th=[204473] bw ( KiB/s): min= 93, max= 536, per=27.85%, avg=233.90, stdev=98.64, samples=52 iops : min= 5, max= 33, avg=13.94, stdev= 6.29, samples=52 write: IOPS=13, BW=220KiB/s (225kB/s)(6032KiB/27402msec); 0 zone resets clat (usec): min=1321, max=187963, avg=31214.81, stdev=32166.91 lat (usec): min=1323, max=187965, avg=31217.24, stdev=32166.91 clat percentiles (usec): | 1.00th=[ 1450], 5.00th=[ 1631], 10.00th=[ 1762], 20.00th=[ 2802], | 30.00th=[ 8455], 40.00th=[ 13960], 50.00th=[ 21103], 60.00th=[ 29492], | 70.00th=[ 41157], 80.00th=[ 52167], 90.00th=[ 80217], 95.00th=[ 96994], | 99.00th=[129500], 99.50th=[162530], 99.90th=[187696], 99.95th=[187696], | 99.99th=[187696] bw ( KiB/s): min= 60, max= 383, per=24.62%, avg=217.40, stdev=76.47, samples=53 iops : min= 3, max= 23, avg=12.89, stdev= 4.77, samples=53 lat (usec) : 250=7.55%, 500=1.30%, 750=0.13%, 1000=0.13% lat (msec) : 2=7.68%, 4=4.17%, 10=4.04%, 20=11.72%, 50=36.59% lat (msec) : 100=23.18%, 250=3.52% cpu : usr=0.10%, sys=7.39%, ctx=1414, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147666: Mon Nov 17 14:05:09 2025 read: IOPS=13, BW=221KiB/s (226kB/s)(6032KiB/27305msec) clat (usec): min=90, max=285951, avg=37775.25, stdev=33749.45 lat (usec): min=90, max=285953, avg=37776.76, stdev=33749.48 clat percentiles (usec): | 1.00th=[ 110], 5.00th=[ 153], 10.00th=[ 174], 20.00th=[ 553], | 30.00th=[ 20841], 40.00th=[ 28443], 50.00th=[ 34866], 60.00th=[ 42730], | 70.00th=[ 49546], 80.00th=[ 60556], 90.00th=[ 71828], 95.00th=[ 88605], | 99.00th=[170918], 99.50th=[265290], 99.90th=[287310], 99.95th=[287310], | 99.99th=[287310] bw ( KiB/s): min= 29, max= 477, per=26.99%, avg=226.69, stdev=83.96, samples=52 iops : min= 1, max= 29, avg=13.46, stdev= 5.31, samples=52 write: IOPS=14, BW=229KiB/s (235kB/s)(6256KiB/27305msec); 0 zone resets clat (usec): min=1246, max=369011, avg=32128.98, stdev=39178.11 lat (usec): min=1248, max=369014, avg=32131.40, stdev=39178.19 clat percentiles (usec): | 1.00th=[ 1483], 5.00th=[ 1663], 10.00th=[ 1860], 20.00th=[ 3720], | 30.00th=[ 7635], 40.00th=[ 13566], 50.00th=[ 19268], 60.00th=[ 28181], | 70.00th=[ 36963], 80.00th=[ 53740], 90.00th=[ 77071], 95.00th=[ 98042], | 99.00th=[189793], 99.50th=[202376], 99.90th=[367002], 99.95th=[367002], | 99.99th=[367002] bw ( KiB/s): min= 42, max= 479, per=26.15%, avg=230.87, stdev=85.20, samples=53 iops : min= 2, max= 29, avg=13.72, stdev= 5.32, samples=53 lat (usec) : 100=0.13%, 250=9.11%, 500=0.52%, 750=0.13% lat (msec) : 2=7.68%, 4=3.39%, 10=7.16%, 20=12.24%, 50=34.11% lat (msec) : 100=21.88%, 250=3.26%, 500=0.39% cpu : usr=0.07%, sys=7.30%, ctx=1370, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=840KiB/s (860kB/s), 201KiB/s-228KiB/s (206kB/s-234kB/s), io=23.4MiB (24.5MB), run=27305-28508msec WRITE: bw=884KiB/s (905kB/s), 220KiB/s-232KiB/s (225kB/s-238kB/s), io=24.6MiB (25.8MB), run=27305-28508msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147659: Mon Nov 17 14:05:48 2025 read: IOPS=5, BW=87.4KiB/s (89.5kB/s)(5920KiB/67754msec) clat (msec): min=7, max=348, avg=35.89, stdev=41.81 lat (msec): min=7, max=348, avg=35.89, stdev=41.81 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 19], 50.00th=[ 22], 60.00th=[ 26], | 70.00th=[ 31], 80.00th=[ 44], 90.00th=[ 77], 95.00th=[ 123], | 99.00th=[ 203], 99.50th=[ 326], 99.90th=[ 351], 99.95th=[ 351], | 99.99th=[ 351] bw ( KiB/s): min= 28, max= 380, per=26.84%, avg=94.21, stdev=65.47, samples=123 iops : min= 1, max= 23, avg= 5.17, stdev= 4.14, samples=123 write: IOPS=5, BW=93.0KiB/s (96.2kB/s)(6368KiB/67754msec); 0 zone resets clat (msec): min=53, max=468, avg=135.64, stdev=63.99 lat (msec): min=53, max=468, avg=135.65, stdev=63.99 clat percentiles (msec): | 1.00th=[ 55], 5.00th=[ 68], 10.00th=[ 77], 20.00th=[ 89], | 30.00th=[ 99], 40.00th=[ 109], 50.00th=[ 123], 60.00th=[ 133], | 70.00th=[ 146], 80.00th=[ 167], 90.00th=[ 218], 95.00th=[ 275], | 99.00th=[ 393], 99.50th=[ 451], 99.90th=[ 468], 99.95th=[ 468], | 99.99th=[ 468] bw ( KiB/s): min= 30, max= 191, per=25.92%, avg=95.65, stdev=42.06, samples=131 iops : min= 1, max= 11, avg= 5.25, stdev= 2.71, samples=131 lat (msec) : 10=2.08%, 20=20.31%, 50=17.19%, 100=22.53%, 250=34.11% lat (msec) : 500=3.78% cpu : usr=0.04%, sys=1.36%, ctx=1343, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147660: Mon Nov 17 14:05:48 2025 read: IOPS=5, BW=84.3KiB/s (86.3kB/s)(5744KiB/68116msec) clat (msec): min=6, max=250, avg=34.40, stdev=34.44 lat (msec): min=6, max=250, avg=34.40, stdev=34.44 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 32], 80.00th=[ 47], 90.00th=[ 82], 95.00th=[ 108], | 99.00th=[ 169], 99.50th=[ 182], 99.90th=[ 251], 99.95th=[ 251], | 99.99th=[ 251] bw ( KiB/s): min= 21, max= 380, per=28.51%, avg=100.07, stdev=67.21, samples=113 iops : min= 1, max= 23, avg= 5.55, stdev= 4.28, samples=113 write: IOPS=6, BW=96.1KiB/s (98.4kB/s)(6544KiB/68116msec); 0 zone resets clat (msec): min=47, max=535, avg=135.42, stdev=68.20 lat (msec): min=47, max=535, avg=135.42, stdev=68.20 clat percentiles (msec): | 1.00th=[ 55], 5.00th=[ 67], 10.00th=[ 74], 20.00th=[ 88], | 30.00th=[ 96], 40.00th=[ 108], 50.00th=[ 116], 60.00th=[ 130], | 70.00th=[ 144], 80.00th=[ 174], 90.00th=[ 226], 95.00th=[ 271], | 99.00th=[ 372], 99.50th=[ 443], 99.90th=[ 535], 99.95th=[ 535], | 99.99th=[ 535] bw ( KiB/s): min= 28, max= 223, per=26.15%, avg=96.51, stdev=43.48, samples=133 iops : min= 1, max= 13, avg= 5.30, stdev= 2.80, samples=133 lat (msec) : 10=2.08%, 20=18.23%, 50=18.23%, 100=22.92%, 250=34.64% lat (msec) : 500=3.78%, 750=0.13% cpu : usr=0.04%, sys=1.36%, ctx=1321, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147661: Mon Nov 17 14:05:48 2025 read: IOPS=5, BW=94.6KiB/s (96.9kB/s)(6256KiB/66098msec) clat (msec): min=5, max=336, avg=32.96, stdev=37.50 lat (msec): min=5, max=336, avg=32.96, stdev=37.50 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 19], 50.00th=[ 22], 60.00th=[ 25], | 70.00th=[ 29], 80.00th=[ 39], 90.00th=[ 65], 95.00th=[ 102], | 99.00th=[ 209], 99.50th=[ 288], 99.90th=[ 338], 99.95th=[ 338], | 99.99th=[ 338] bw ( KiB/s): min= 30, max= 351, per=30.56%, avg=107.25, stdev=71.05, samples=115 iops : min= 1, max= 21, avg= 5.94, stdev= 4.45, samples=115 write: IOPS=5, BW=91.3KiB/s (93.4kB/s)(6032KiB/66098msec); 0 zone resets clat (msec): min=47, max=585, avg=140.87, stdev=70.23 lat (msec): min=47, max=585, avg=140.88, stdev=70.23 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 66], 10.00th=[ 78], 20.00th=[ 92], | 30.00th=[ 102], 40.00th=[ 110], 50.00th=[ 122], 60.00th=[ 134], | 70.00th=[ 155], 80.00th=[ 180], 90.00th=[ 234], 95.00th=[ 279], | 99.00th=[ 397], 99.50th=[ 447], 99.90th=[ 584], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 28, max= 190, per=24.95%, avg=92.08, stdev=39.60, samples=128 iops : min= 1, max= 11, avg= 4.98, stdev= 2.51, samples=128 lat (msec) : 10=2.86%, 20=19.79%, 50=21.61%, 100=17.97%, 250=33.85% lat (msec) : 500=3.78%, 750=0.13% cpu : usr=0.05%, sys=1.45%, ctx=1409, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147662: Mon Nov 17 14:05:48 2025 read: IOPS=5, BW=90.3KiB/s (92.5kB/s)(6032KiB/66792msec) clat (msec): min=6, max=222, avg=31.91, stdev=30.33 lat (msec): min=6, max=222, avg=31.91, stdev=30.33 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 22], 60.00th=[ 25], | 70.00th=[ 32], 80.00th=[ 42], 90.00th=[ 66], 95.00th=[ 90], | 99.00th=[ 167], 99.50th=[ 188], 99.90th=[ 222], 99.95th=[ 222], | 99.99th=[ 222] bw ( KiB/s): min= 28, max= 349, per=28.61%, avg=100.43, stdev=64.05, samples=117 iops : min= 1, max= 21, avg= 5.56, stdev= 4.08, samples=117 write: IOPS=5, BW=93.7KiB/s (95.9kB/s)(6256KiB/66792msec); 0 zone resets clat (msec): min=51, max=409, avg=139.06, stdev=63.13 lat (msec): min=51, max=409, avg=139.06, stdev=63.13 clat percentiles (msec): | 1.00th=[ 55], 5.00th=[ 66], 10.00th=[ 75], 20.00th=[ 91], | 30.00th=[ 101], 40.00th=[ 111], 50.00th=[ 124], 60.00th=[ 138], | 70.00th=[ 159], 80.00th=[ 184], 90.00th=[ 222], 95.00th=[ 264], | 99.00th=[ 355], 99.50th=[ 401], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 30, max= 191, per=25.52%, avg=94.18, stdev=39.66, samples=130 iops : min= 1, max= 11, avg= 5.16, stdev= 2.54, samples=130 lat (msec) : 10=1.82%, 20=19.92%, 50=19.40%, 100=21.35%, 250=33.98% lat (msec) : 500=3.52% cpu : usr=0.04%, sys=1.51%, ctx=1371, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=352KiB/s (360kB/s), 84.3KiB/s-94.6KiB/s (86.3kB/s-96.9kB/s), io=23.4MiB (24.5MB), run=66098-68116msec WRITE: bw=370KiB/s (379kB/s), 91.3KiB/s-96.1KiB/s (93.4kB/s-98.4kB/s), io=24.6MiB (25.8MB), run=66098-68116msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147686: Mon Nov 17 14:06:08 2025 read: IOPS=0, BW=745KiB/s (762kB/s)(7168KiB/9627msec) clat (usec): min=988, max=3908.6k, avg=1237475.35, stdev=1663204.33 lat (usec): min=989, max=3908.6k, avg=1237476.65, stdev=1663204.53 clat percentiles (usec): | 1.00th=[ 988], 5.00th=[ 988], 10.00th=[ 988], | 20.00th=[ 1762], 30.00th=[ 2933], 40.00th=[ 2933], | 50.00th=[ 6587], 60.00th=[1736442], 70.00th=[1736442], | 80.00th=[3003122], 90.00th=[3909092], 95.00th=[3909092], | 99.00th=[3909092], 99.50th=[3909092], 99.90th=[3909092], | 99.95th=[3909092], 99.99th=[3909092] bw ( KiB/s): min= 2003, max= 6131, per=100.00%, avg=3392.33, stdev=2371.84, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 write: IOPS=0, BW=532KiB/s (545kB/s)(5120KiB/9627msec); 0 zone resets clat (msec): min=120, max=306, avg=166.00, stdev=78.70 lat (msec): min=120, max=306, avg=166.13, stdev=78.64 clat percentiles (msec): | 1.00th=[ 121], 5.00th=[ 121], 10.00th=[ 121], 20.00th=[ 121], | 30.00th=[ 133], 40.00th=[ 133], 50.00th=[ 134], 60.00th=[ 134], | 70.00th=[ 138], 80.00th=[ 138], 90.00th=[ 309], 95.00th=[ 309], | 99.00th=[ 309], 99.50th=[ 309], 99.90th=[ 309], 99.95th=[ 309], | 99.99th=[ 309] bw ( KiB/s): min= 4087, max= 4087, per=100.00%, avg=4087.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (usec) : 1000=8.33% lat (msec) : 2=8.33%, 4=8.33%, 10=8.33%, 250=33.33%, 500=8.33% lat (msec) : 2000=8.33%, >=2000=16.67% cpu : usr=0.01%, sys=7.40%, ctx=1311, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147687: Mon Nov 17 14:06:08 2025 read: IOPS=0, BW=198KiB/s (203kB/s)(2048KiB/10327msec) clat (msec): min=3, max=414, avg=208.87, stdev=290.71 lat (msec): min=3, max=414, avg=208.87, stdev=290.72 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 4], | 30.00th=[ 4], 40.00th=[ 4], 50.00th=[ 4], 60.00th=[ 414], | 70.00th=[ 414], 80.00th=[ 414], 90.00th=[ 414], 95.00th=[ 414], | 99.00th=[ 414], 99.50th=[ 414], 99.90th=[ 414], 99.95th=[ 414], | 99.99th=[ 414] bw ( KiB/s): min= 2007, max= 2043, per=100.00%, avg=2025.00, stdev=25.46, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=0, BW=992KiB/s (1015kB/s)(10.0MiB/10327msec); 0 zone resets clat (msec): min=141, max=4704, avg=966.44, stdev=1518.03 lat (msec): min=141, max=4704, avg=966.65, stdev=1517.97 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 165], | 30.00th=[ 184], 40.00th=[ 190], 50.00th=[ 220], 60.00th=[ 388], | 70.00th=[ 485], 80.00th=[ 518], 90.00th=[ 2668], 95.00th=[ 4732], | 99.00th=[ 4732], 99.50th=[ 4732], 99.90th=[ 4732], 99.95th=[ 4732], | 99.99th=[ 4732] bw ( KiB/s): min= 2019, max= 5907, per=100.00%, avg=3150.00, stdev=1711.57, samples=5 iops : min= 1, max= 5, avg= 2.20, stdev= 1.79, samples=5 lat (msec) : 4=8.33%, 250=41.67%, 500=25.00%, 750=8.33%, >=2000=16.67% cpu : usr=0.01%, sys=6.46%, ctx=138, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147688: Mon Nov 17 14:06:08 2025 read: IOPS=0, BW=666KiB/s (682kB/s)(6144KiB/9228msec) clat (usec): min=423, max=326598, avg=133384.22, stdev=124517.82 lat (usec): min=424, max=326600, avg=133385.66, stdev=124517.87 clat percentiles (usec): | 1.00th=[ 424], 5.00th=[ 424], 10.00th=[ 424], 20.00th=[ 848], | 30.00th=[ 848], 40.00th=[111674], 50.00th=[111674], 60.00th=[177210], | 70.00th=[183501], 80.00th=[183501], 90.00th=[325059], 95.00th=[325059], | 99.00th=[325059], 99.50th=[325059], 99.90th=[325059], 99.95th=[325059], | 99.99th=[325059] bw ( KiB/s): min= 4031, max= 4039, per=100.00%, avg=4035.00, stdev= 5.66, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=0, BW=666KiB/s (682kB/s)(6144KiB/9228msec); 0 zone resets clat (msec): min=104, max=5283, avg=1375.84, stdev=2107.41 lat (msec): min=104, max=5283, avg=1375.91, stdev=2107.42 clat percentiles (msec): | 1.00th=[ 105], 5.00th=[ 105], 10.00th=[ 105], 20.00th=[ 144], | 30.00th=[ 144], 40.00th=[ 153], 50.00th=[ 153], 60.00th=[ 215], | 70.00th=[ 2366], 80.00th=[ 2366], 90.00th=[ 5269], 95.00th=[ 5269], | 99.00th=[ 5269], 99.50th=[ 5269], 99.90th=[ 5269], 99.95th=[ 5269], | 99.99th=[ 5269] bw ( KiB/s): min= 2828, max= 4031, per=100.00%, avg=3429.50, stdev=850.65, samples=2 iops : min= 2, max= 3, avg= 2.50, stdev= 0.71, samples=2 lat (usec) : 500=8.33%, 1000=8.33% lat (msec) : 250=58.33%, 500=8.33%, >=2000=16.67% cpu : usr=0.00%, sys=5.12%, ctx=121, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147689: Mon Nov 17 14:06:08 2025 read: IOPS=0, BW=510KiB/s (522kB/s)(5120KiB/10041msec) clat (usec): min=649, max=221570, avg=125529.56, stdev=112735.36 lat (usec): min=650, max=221571, avg=125531.05, stdev=112735.15 clat percentiles (usec): | 1.00th=[ 652], 5.00th=[ 652], 10.00th=[ 652], 20.00th=[ 652], | 30.00th=[ 4424], 40.00th=[ 4424], 50.00th=[191890], 60.00th=[191890], | 70.00th=[208667], 80.00th=[208667], 90.00th=[221250], 95.00th=[221250], | 99.00th=[221250], 99.50th=[221250], 99.90th=[221250], 99.95th=[221250], | 99.99th=[221250] bw ( KiB/s): min= 2019, max= 6131, per=100.00%, avg=4075.00, stdev=2907.62, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=0, BW=714KiB/s (731kB/s)(7168KiB/10041msec); 0 zone resets clat (msec): min=116, max=5177, avg=1299.89, stdev=1930.35 lat (msec): min=116, max=5177, avg=1299.98, stdev=1930.34 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 239], | 30.00th=[ 292], 40.00th=[ 292], 50.00th=[ 309], 60.00th=[ 317], | 70.00th=[ 317], 80.00th=[ 2635], 90.00th=[ 5201], 95.00th=[ 5201], | 99.00th=[ 5201], 99.50th=[ 5201], 99.90th=[ 5201], 99.95th=[ 5201], | 99.99th=[ 5201] bw ( KiB/s): min= 1996, max= 3750, per=88.37%, avg=2453.25, stdev=864.76, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (usec) : 750=8.33% lat (msec) : 10=8.33%, 250=41.67%, 500=25.00%, >=2000=16.67% cpu : usr=0.00%, sys=6.12%, ctx=199, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1983KiB/s (2031kB/s), 198KiB/s-745KiB/s (203kB/s-762kB/s), io=20.0MiB (20.0MB), run=9228-10327msec WRITE: bw=2776KiB/s (2843kB/s), 532KiB/s-992KiB/s (545kB/s-1015kB/s), io=28.0MiB (29.4MB), run=9228-10327msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147682: Mon Nov 17 14:06:08 2025 read: IOPS=1, BW=1093KiB/s (1119kB/s)(7168KiB/6559msec) clat (msec): min=74, max=2964, avg=749.70, stdev=1111.43 lat (msec): min=74, max=2964, avg=749.70, stdev=1111.43 clat percentiles (msec): | 1.00th=[ 75], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 121], | 30.00th=[ 122], 40.00th=[ 122], 50.00th=[ 136], 60.00th=[ 275], | 70.00th=[ 275], 80.00th=[ 1552], 90.00th=[ 2970], 95.00th=[ 2970], | 99.00th=[ 2970], 99.50th=[ 2970], 99.90th=[ 2970], 99.95th=[ 2970], | 99.99th=[ 2970] bw ( KiB/s): min= 2000, max= 4079, per=100.00%, avg=3249.67, stdev=1101.39, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=0, BW=781KiB/s (799kB/s)(5120KiB/6559msec); 0 zone resets clat (msec): min=186, max=341, avg=256.31, stdev=62.24 lat (msec): min=186, max=343, avg=256.95, stdev=63.16 clat percentiles (msec): | 1.00th=[ 188], 5.00th=[ 188], 10.00th=[ 188], 20.00th=[ 188], | 30.00th=[ 205], 40.00th=[ 205], 50.00th=[ 271], 60.00th=[ 271], | 70.00th=[ 279], 80.00th=[ 279], 90.00th=[ 342], 95.00th=[ 342], | 99.00th=[ 342], 99.50th=[ 342], 99.90th=[ 342], 99.95th=[ 342], | 99.99th=[ 342] bw ( KiB/s): min= 2000, max= 3778, per=94.22%, avg=2597.67, stdev=1022.23, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 100=8.33%, 250=41.67%, 500=33.33%, 2000=8.33%, >=2000=8.33% cpu : usr=0.02%, sys=1.72%, ctx=52, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147683: Mon Nov 17 14:06:08 2025 read: IOPS=0, BW=199KiB/s (204kB/s)(2048KiB/10266msec) clat (msec): min=73, max=112, avg=93.12, stdev=27.93 lat (msec): min=73, max=112, avg=93.12, stdev=27.93 clat percentiles (msec): | 1.00th=[ 73], 5.00th=[ 73], 10.00th=[ 73], 20.00th=[ 73], | 30.00th=[ 73], 40.00th=[ 73], 50.00th=[ 73], 60.00th=[ 113], | 70.00th=[ 113], 80.00th=[ 113], 90.00th=[ 113], 95.00th=[ 113], | 99.00th=[ 113], 99.50th=[ 113], 99.90th=[ 113], 99.95th=[ 113], | 99.99th=[ 113] bw ( KiB/s): min= 1825, max= 2027, per=97.82%, avg=1926.00, stdev=142.84, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=0, BW=997KiB/s (1021kB/s)(10.0MiB/10266msec); 0 zone resets clat (msec): min=142, max=5026, avg=996.98, stdev=1590.86 lat (msec): min=142, max=5026, avg=997.09, stdev=1590.85 clat percentiles (msec): | 1.00th=[ 144], 5.00th=[ 144], 10.00th=[ 144], 20.00th=[ 171], | 30.00th=[ 199], 40.00th=[ 239], 50.00th=[ 253], 60.00th=[ 313], | 70.00th=[ 376], 80.00th=[ 701], 90.00th=[ 2567], 95.00th=[ 5000], | 99.00th=[ 5000], 99.50th=[ 5000], 99.90th=[ 5000], 99.95th=[ 5000], | 99.99th=[ 5000] bw ( KiB/s): min= 1889, max= 4087, per=100.00%, avg=3009.00, stdev=1149.73, samples=6 iops : min= 1, max= 3, avg= 2.00, stdev= 1.10, samples=6 lat (msec) : 100=8.33%, 250=41.67%, 500=25.00%, 750=8.33%, >=2000=16.67% cpu : usr=0.01%, sys=2.75%, ctx=130, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147684: Mon Nov 17 14:06:08 2025 read: IOPS=0, BW=591KiB/s (605kB/s)(6144KiB/10399msec) clat (msec): min=53, max=720, avg=234.50, stdev=251.44 lat (msec): min=53, max=720, avg=234.51, stdev=251.44 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 54], 10.00th=[ 54], 20.00th=[ 84], | 30.00th=[ 84], 40.00th=[ 89], 50.00th=[ 89], 60.00th=[ 192], | 70.00th=[ 271], 80.00th=[ 271], 90.00th=[ 718], 95.00th=[ 718], | 99.00th=[ 718], 99.50th=[ 718], 99.90th=[ 718], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 1889, max= 4055, per=100.00%, avg=2481.00, stdev=1050.61, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=0, BW=591KiB/s (605kB/s)(6144KiB/10399msec); 0 zone resets clat (msec): min=181, max=5042, avg=1489.60, stdev=2025.34 lat (msec): min=181, max=5042, avg=1489.69, stdev=2025.34 clat percentiles (msec): | 1.00th=[ 182], 5.00th=[ 182], 10.00th=[ 182], 20.00th=[ 211], | 30.00th=[ 211], 40.00th=[ 275], 50.00th=[ 275], 60.00th=[ 380], | 70.00th=[ 2836], 80.00th=[ 2836], 90.00th=[ 5067], 95.00th=[ 5067], | 99.00th=[ 5067], 99.50th=[ 5067], 99.90th=[ 5067], 99.95th=[ 5067], | 99.99th=[ 5067] bw ( KiB/s): min= 2015, max= 3930, per=100.00%, avg=2955.25, stdev=1079.30, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 lat (msec) : 100=25.00%, 250=25.00%, 500=25.00%, 750=8.33%, >=2000=16.67% cpu : usr=0.02%, sys=1.84%, ctx=71, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147685: Mon Nov 17 14:06:08 2025 read: IOPS=0, BW=516KiB/s (528kB/s)(5120KiB/9922msec) clat (msec): min=80, max=439, avg=231.92, stdev=178.82 lat (msec): min=80, max=439, avg=231.92, stdev=178.82 clat percentiles (msec): | 1.00th=[ 81], 5.00th=[ 81], 10.00th=[ 81], 20.00th=[ 81], | 30.00th=[ 90], 40.00th=[ 90], 50.00th=[ 138], 60.00th=[ 138], | 70.00th=[ 414], 80.00th=[ 414], 90.00th=[ 439], 95.00th=[ 439], | 99.00th=[ 439], 99.50th=[ 439], 99.90th=[ 439], 99.95th=[ 439], | 99.99th=[ 439] bw ( KiB/s): min= 1889, max= 4055, per=100.00%, avg=2657.00, stdev=1212.67, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=0, BW=722KiB/s (740kB/s)(7168KiB/9922msec); 0 zone resets clat (msec): min=159, max=4965, avg=1237.94, stdev=1862.49 lat (msec): min=174, max=4965, avg=1240.13, stdev=1861.07 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 182], | 30.00th=[ 218], 40.00th=[ 218], 50.00th=[ 232], 60.00th=[ 338], | 70.00th=[ 338], 80.00th=[ 2567], 90.00th=[ 4933], 95.00th=[ 4933], | 99.00th=[ 4933], 99.50th=[ 4933], 99.90th=[ 4933], 99.95th=[ 4933], | 99.99th=[ 4933] bw ( KiB/s): min= 1889, max= 4031, per=90.23%, avg=2487.75, stdev=1030.80, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 100=16.67%, 250=41.67%, 500=25.00%, >=2000=16.67% cpu : usr=0.01%, sys=2.34%, ctx=72, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1969KiB/s (2017kB/s), 199KiB/s-1093KiB/s (204kB/s-1119kB/s), io=20.0MiB (20.0MB), run=6559-10399msec WRITE: bw=2757KiB/s (2823kB/s), 591KiB/s-997KiB/s (605kB/s-1021kB/s), io=28.0MiB (29.4MB), run=6559-10399msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147701: Mon Nov 17 14:06:20 2025 read: IOPS=0, BW=3814KiB/s (3905kB/s)(12.0MiB/3222msec) clat (msec): min=856, max=1349, avg=1050.60, stdev=262.85 lat (msec): min=856, max=1349, avg=1050.60, stdev=262.85 clat percentiles (msec): | 1.00th=[ 860], 5.00th=[ 860], 10.00th=[ 860], 20.00th=[ 860], | 30.00th=[ 860], 40.00th=[ 944], 50.00th=[ 944], 60.00th=[ 944], | 70.00th=[ 1351], 80.00th=[ 1351], 90.00th=[ 1351], 95.00th=[ 1351], | 99.00th=[ 1351], 99.50th=[ 1351], 99.90th=[ 1351], 99.95th=[ 1351], | 99.99th=[ 1351] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=66.67%, 2000=33.33% cpu : usr=0.00%, sys=3.45%, ctx=43, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147702: Mon Nov 17 14:06:20 2025 read: IOPS=0, BW=1448KiB/s (1483kB/s)(4096KiB/2829msec) clat (nsec): min=1609.6M, max=1609.6M, avg=1609588498.00, stdev= 0.00 lat (nsec): min=1609.6M, max=1609.6M, avg=1609590585.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1603], 5.00th=[ 1603], 10.00th=[ 1603], 20.00th=[ 1603], | 30.00th=[ 1603], 40.00th=[ 1603], 50.00th=[ 1603], 60.00th=[ 1603], | 70.00th=[ 1603], 80.00th=[ 1603], 90.00th=[ 1603], 95.00th=[ 1603], | 99.00th=[ 1603], 99.50th=[ 1603], 99.90th=[ 1603], 99.95th=[ 1603], | 99.99th=[ 1603] bw ( KiB/s): min= 8126, max= 8126, per=100.00%, avg=8126.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2896KiB/s (2965kB/s)(8192KiB/2829msec); 0 zone resets clat (msec): min=436, max=743, avg=589.98, stdev=216.75 lat (msec): min=437, max=743, avg=590.47, stdev=216.63 clat percentiles (msec): | 1.00th=[ 439], 5.00th=[ 439], 10.00th=[ 439], 20.00th=[ 439], | 30.00th=[ 439], 40.00th=[ 439], 50.00th=[ 439], 60.00th=[ 743], | 70.00th=[ 743], 80.00th=[ 743], 90.00th=[ 743], 95.00th=[ 743], | 99.00th=[ 743], 99.50th=[ 743], 99.90th=[ 743], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 8159, max= 8159, per=72.88%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=4.49%, ctx=38, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147703: Mon Nov 17 14:06:20 2025 write: IOPS=1, BW=4198KiB/s (4299kB/s)(12.0MiB/2927msec); 0 zone resets clat (msec): min=762, max=1167, avg=947.59, stdev=205.08 lat (msec): min=762, max=1168, avg=947.91, stdev=205.11 clat percentiles (msec): | 1.00th=[ 760], 5.00th=[ 760], 10.00th=[ 760], 20.00th=[ 760], | 30.00th=[ 760], 40.00th=[ 911], 50.00th=[ 911], 60.00th=[ 911], | 70.00th=[ 1167], 80.00th=[ 1167], 90.00th=[ 1167], 95.00th=[ 1167], | 99.00th=[ 1167], 99.50th=[ 1167], 99.90th=[ 1167], 99.95th=[ 1167], | 99.99th=[ 1167] bw ( KiB/s): min= 8159, max= 8159, per=72.88%, avg=8159.00, stdev= 0.00, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=66.67%, 2000=33.33% cpu : usr=0.03%, sys=9.54%, ctx=89, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147704: Mon Nov 17 14:06:20 2025 write: IOPS=1, BW=5124KiB/s (5247kB/s)(12.0MiB/2398msec); 0 zone resets clat (msec): min=360, max=1461, avg=784.44, stdev=592.59 lat (msec): min=360, max=1461, avg=784.84, stdev=592.54 clat percentiles (msec): | 1.00th=[ 359], 5.00th=[ 359], 10.00th=[ 359], 20.00th=[ 359], | 30.00th=[ 359], 40.00th=[ 531], 50.00th=[ 531], 60.00th=[ 531], | 70.00th=[ 1469], 80.00th=[ 1469], 90.00th=[ 1469], 95.00th=[ 1469], | 99.00th=[ 1469], 99.50th=[ 1469], 99.90th=[ 1469], 99.95th=[ 1469], | 99.99th=[ 1469] bw ( KiB/s): min=16318, max=16318, per=100.00%, avg=16318.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.04%, sys=7.59%, ctx=49, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5085KiB/s (5207kB/s), 1448KiB/s-3814KiB/s (1483kB/s-3905kB/s), io=16.0MiB (16.8MB), run=2829-3222msec WRITE: bw=10.9MiB/s (11.5MB/s), 2896KiB/s-5124KiB/s (2965kB/s-5247kB/s), io=32.0MiB (33.6MB), run=2398-2927msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147697: Mon Nov 17 14:06:21 2025 read: IOPS=1, BW=4577KiB/s (4686kB/s)(12.0MiB/2685msec) clat (msec): min=673, max=1141, avg=838.63, stdev=262.80 lat (msec): min=673, max=1141, avg=838.63, stdev=262.80 clat percentiles (msec): | 1.00th=[ 676], 5.00th=[ 676], 10.00th=[ 676], 20.00th=[ 676], | 30.00th=[ 676], 40.00th=[ 701], 50.00th=[ 701], 60.00th=[ 701], | 70.00th=[ 1150], 80.00th=[ 1150], 90.00th=[ 1150], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min= 8078, max= 8159, per=100.00%, avg=8118.50, stdev=57.28, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=22.88%, ctx=426, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147698: Mon Nov 17 14:06:21 2025 read: IOPS=0, BW=2351KiB/s (2408kB/s)(4096KiB/1742msec) clat (nsec): min=516529k, max=516529k, avg=516528626.00, stdev= 0.00 lat (nsec): min=516530k, max=516530k, avg=516529804.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 518], 5.00th=[ 518], 10.00th=[ 518], 20.00th=[ 518], | 30.00th=[ 518], 40.00th=[ 518], 50.00th=[ 518], 60.00th=[ 518], | 70.00th=[ 518], 80.00th=[ 518], 90.00th=[ 518], 95.00th=[ 518], | 99.00th=[ 518], 99.50th=[ 518], 99.90th=[ 518], 99.95th=[ 518], | 99.99th=[ 518] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4703KiB/s (4816kB/s)(8192KiB/1742msec); 0 zone resets clat (msec): min=325, max=763, avg=544.63, stdev=309.87 lat (msec): min=325, max=766, avg=546.14, stdev=311.50 clat percentiles (msec): | 1.00th=[ 326], 5.00th=[ 326], 10.00th=[ 326], 20.00th=[ 326], | 30.00th=[ 326], 40.00th=[ 326], 50.00th=[ 326], 60.00th=[ 768], | 70.00th=[ 768], 80.00th=[ 768], 90.00th=[ 768], 95.00th=[ 768], | 99.00th=[ 768], 99.50th=[ 768], 99.90th=[ 768], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 8192, max= 8192, per=92.79%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=20.91%, ctx=124, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147699: Mon Nov 17 14:06:21 2025 write: IOPS=0, BW=3311KiB/s (3391kB/s)(12.0MiB/3711msec); 0 zone resets clat (msec): min=177, max=3140, avg=1177.78, stdev=1699.56 lat (msec): min=181, max=3140, avg=1179.30, stdev=1698.45 clat percentiles (msec): | 1.00th=[ 178], 5.00th=[ 178], 10.00th=[ 178], 20.00th=[ 178], | 30.00th=[ 178], 40.00th=[ 215], 50.00th=[ 215], 60.00th=[ 215], | 70.00th=[ 3138], 80.00th=[ 3138], 90.00th=[ 3138], 95.00th=[ 3138], | 99.00th=[ 3138], 99.50th=[ 3138], 99.90th=[ 3138], 99.95th=[ 3138], | 99.99th=[ 3138] bw ( KiB/s): min=16094, max=16094, per=100.00%, avg=16094.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, >=2000=33.33% cpu : usr=0.03%, sys=4.26%, ctx=51, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147700: Mon Nov 17 14:06:21 2025 write: IOPS=1, BW=5392KiB/s (5521kB/s)(12.0MiB/2279msec); 0 zone resets clat (msec): min=200, max=1531, avg=696.65, stdev=727.05 lat (msec): min=201, max=1531, avg=697.14, stdev=727.14 clat percentiles (msec): | 1.00th=[ 201], 5.00th=[ 201], 10.00th=[ 201], 20.00th=[ 201], | 30.00th=[ 201], 40.00th=[ 359], 50.00th=[ 359], 60.00th=[ 359], | 70.00th=[ 1536], 80.00th=[ 1536], 90.00th=[ 1536], 95.00th=[ 1536], | 99.00th=[ 1536], 99.50th=[ 1536], 99.90th=[ 1536], 99.95th=[ 1536], | 99.99th=[ 1536] bw ( KiB/s): min= 6360, max= 8192, per=82.41%, avg=7276.00, stdev=1295.42, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=9.83%, ctx=64, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6102KiB/s (6248kB/s), 2351KiB/s-4577KiB/s (2408kB/s-4686kB/s), io=16.0MiB (16.8MB), run=1742-2685msec WRITE: bw=8830KiB/s (9042kB/s), 3311KiB/s-5392KiB/s (3391kB/s-5521kB/s), io=32.0MiB (33.6MB), run=1742-3711msec