== sanity test 398b: DIO and buffer IO race ============== 04:42:11 (1789548131) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 4.87404 s, 10.3 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146703: Wed Sep 16 04:45:02 2026 read: IOPS=9, BW=39.1KiB/s (40.1kB/s)(5992KiB/153182msec) clat (usec): min=21, max=1584.9k, avg=50816.19, stdev=65045.16 lat (usec): min=21, max=1584.9k, avg=50818.14, stdev=65045.36 clat percentiles (usec): | 1.00th=[ 50], 5.00th=[ 61], 10.00th=[ 8848], | 20.00th=[ 18744], 30.00th=[ 24773], 40.00th=[ 31065], | 50.00th=[ 38011], 60.00th=[ 46400], 70.00th=[ 57410], | 80.00th=[ 72877], 90.00th=[ 98042], 95.00th=[ 131597], | 99.00th=[ 261096], 99.50th=[ 295699], 99.90th=[1019216], | 99.95th=[1585447], 99.99th=[1585447] bw ( KiB/s): min= 7, max= 110, per=25.84%, avg=40.05, stdev=21.81, samples=289 iops : min= 1, max= 27, avg= 9.39, stdev= 5.48, samples=289 write: IOPS=10, BW=41.1KiB/s (42.1kB/s)(6296KiB/153182msec); 0 zone resets clat (usec): min=706, max=613257, avg=48684.85, stdev=58156.80 lat (usec): min=707, max=613258, avg=48687.23, stdev=58156.23 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 1172], 10.00th=[ 1401], 20.00th=[ 4146], | 30.00th=[ 9503], 40.00th=[ 21627], 50.00th=[ 32113], 60.00th=[ 43779], | 70.00th=[ 58983], 80.00th=[ 80217], 90.00th=[117965], 95.00th=[156238], | 99.00th=[261096], 99.50th=[341836], 99.90th=[513803], 99.95th=[616563], | 99.99th=[616563] bw ( KiB/s): min= 7, max= 141, per=25.74%, avg=41.69, stdev=23.42, samples=292 iops : min= 1, max= 35, avg= 9.79, stdev= 5.90, samples=292 lat (usec) : 50=0.52%, 100=3.52%, 250=0.23%, 500=0.20%, 750=0.13% lat (usec) : 1000=0.29% lat (msec) : 2=6.71%, 4=3.03%, 10=6.15%, 20=9.51%, 50=33.85% lat (msec) : 100=24.41%, 250=10.29%, 500=1.04%, 750=0.07%, 2000=0.07% cpu : usr=0.07%, sys=4.58%, ctx=6404, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146704: Wed Sep 16 04:45:02 2026 read: IOPS=9, BW=39.0KiB/s (39.0kB/s)(5940KiB/152163msec) clat (usec): min=33, max=1638.6k, avg=49535.54, stdev=60134.00 lat (usec): min=33, max=1638.6k, avg=49539.80, stdev=60134.74 clat percentiles (usec): | 1.00th=[ 49], 5.00th=[ 60], 10.00th=[ 3130], | 20.00th=[ 17957], 30.00th=[ 23725], 40.00th=[ 30802], | 50.00th=[ 39060], 60.00th=[ 47449], 70.00th=[ 57410], | 80.00th=[ 70779], 90.00th=[ 102237], 95.00th=[ 125305], | 99.00th=[ 212861], 99.50th=[ 252707], 99.90th=[ 476054], | 99.95th=[1635779], 99.99th=[1635779] bw ( KiB/s): min= 2, max= 120, per=25.54%, avg=39.58, stdev=22.28, samples=291 iops : min= 0, max= 30, avg= 9.26, stdev= 5.62, samples=291 write: IOPS=10, BW=41.7KiB/s (42.7kB/s)(6348KiB/152163msec); 0 zone resets clat (usec): min=811, max=1043.3k, avg=49266.09, stdev=61482.89 lat (usec): min=812, max=1043.3k, avg=49267.80, stdev=61483.09 clat percentiles (usec): | 1.00th=[ 1012], 5.00th=[ 1172], 10.00th=[ 1385], | 20.00th=[ 3785], 30.00th=[ 10552], 40.00th=[ 21890], | 50.00th=[ 33817], 60.00th=[ 45876], 70.00th=[ 59507], | 80.00th=[ 80217], 90.00th=[ 116917], 95.00th=[ 152044], | 99.00th=[ 265290], 99.50th=[ 291505], 99.90th=[ 650118], | 99.95th=[1044382], 99.99th=[1044382] bw ( KiB/s): min= 2, max= 144, per=26.25%, avg=42.52, stdev=23.34, samples=290 iops : min= 0, max= 36, avg=10.00, stdev= 5.88, samples=290 lat (usec) : 50=0.59%, 100=3.48%, 250=0.49%, 500=0.07%, 750=0.03% lat (usec) : 1000=0.46% lat (msec) : 2=6.77%, 4=3.39%, 10=5.44%, 20=10.42%, 50=31.90% lat (msec) : 100=24.74%, 250=11.33%, 500=0.81%, 750=0.03%, 2000=0.07% cpu : usr=0.08%, sys=4.59%, ctx=6286, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146705: Wed Sep 16 04:45:02 2026 read: IOPS=9, BW=38.0KiB/s (39.9kB/s)(6028KiB/154643msec) clat (usec): min=30, max=1046.6k, avg=49504.52, stdev=52815.51 lat (usec): min=31, max=1046.6k, avg=49510.97, stdev=52816.34 clat percentiles (usec): | 1.00th=[ 44], 5.00th=[ 58], 10.00th=[ 293], | 20.00th=[ 17695], 30.00th=[ 24249], 40.00th=[ 31065], | 50.00th=[ 37487], 60.00th=[ 45876], 70.00th=[ 56361], | 80.00th=[ 72877], 90.00th=[ 95945], 95.00th=[ 131597], | 99.00th=[ 227541], 99.50th=[ 270533], 99.90th=[ 434111], | 99.95th=[1044382], 99.99th=[1044382] bw ( KiB/s): min= 4, max= 136, per=25.20%, avg=39.06, stdev=23.25, samples=297 iops : min= 1, max= 34, avg= 9.13, stdev= 5.85, samples=297 write: IOPS=10, BW=40.5KiB/s (41.5kB/s)(6260KiB/154643msec); 0 zone resets clat (usec): min=726, max=1591.1k, avg=50863.42, stdev=69988.82 lat (usec): min=727, max=1591.1k, avg=50870.29, stdev=69986.09 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1156], 10.00th=[ 1369], | 20.00th=[ 3916], 30.00th=[ 9896], 40.00th=[ 23725], | 50.00th=[ 34341], 60.00th=[ 46400], 70.00th=[ 61604], | 80.00th=[ 81265], 90.00th=[ 119014], 95.00th=[ 156238], | 99.00th=[ 299893], 99.50th=[ 346031], 99.90th=[ 501220], | 99.95th=[1585447], 99.99th=[1585447] bw ( KiB/s): min= 6, max= 128, per=25.54%, avg=41.38, stdev=21.69, samples=290 iops : min= 1, max= 32, avg= 9.71, stdev= 5.45, samples=290 lat (usec) : 50=0.75%, 100=3.65%, 250=0.46%, 500=0.07%, 750=0.10% lat (usec) : 1000=0.33% lat (msec) : 2=6.51%, 4=3.78%, 10=5.47%, 20=9.28%, 50=32.55% lat (msec) : 100=25.39%, 250=10.74%, 500=0.88%, 2000=0.07% cpu : usr=0.07%, sys=4.55%, ctx=6263, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146706: Wed Sep 16 04:45:02 2026 read: IOPS=9, BW=39.3KiB/s (40.2kB/s)(6052KiB/154056msec) clat (usec): min=31, max=630149, avg=48871.89, stdev=45323.14 lat (usec): min=32, max=630150, avg=48874.93, stdev=45324.57 clat percentiles (usec): | 1.00th=[ 49], 5.00th=[ 60], 10.00th=[ 594], 20.00th=[ 18220], | 30.00th=[ 24511], 40.00th=[ 31065], 50.00th=[ 38536], 60.00th=[ 47449], | 70.00th=[ 56886], 80.00th=[ 70779], 90.00th=[ 99091], 95.00th=[129500], | 99.00th=[204473], 99.50th=[256902], 99.90th=[358613], 99.95th=[633340], | 99.99th=[633340] bw ( KiB/s): min= 4, max= 103, per=25.90%, avg=40.15, stdev=21.82, samples=290 iops : min= 1, max= 25, avg= 9.41, stdev= 5.49, samples=290 write: IOPS=10, BW=40.5KiB/s (41.5kB/s)(6236KiB/154056msec); 0 zone resets clat (usec): min=798, max=1680.5k, avg=51081.57, stdev=78232.02 lat (usec): min=799, max=1680.5k, avg=51094.44, stdev=78227.93 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 1172], 10.00th=[ 1303], | 20.00th=[ 4080], 30.00th=[ 10552], 40.00th=[ 22414], | 50.00th=[ 34866], 60.00th=[ 46924], 70.00th=[ 61604], | 80.00th=[ 80217], 90.00th=[ 113771], 95.00th=[ 147850], | 99.00th=[ 287310], 99.50th=[ 358613], 99.90th=[1417675], | 99.95th=[1686111], 99.99th=[1686111] bw ( KiB/s): min= 6, max= 119, per=25.68%, avg=41.60, stdev=21.56, samples=289 iops : min= 1, max= 29, avg= 9.77, stdev= 5.43, samples=289 lat (usec) : 50=0.65%, 100=3.91%, 250=0.33%, 500=0.03%, 750=0.07% lat (usec) : 1000=0.26% lat (msec) : 2=6.80%, 4=3.09%, 10=5.14%, 20=10.12%, 50=32.26% lat (msec) : 100=25.75%, 250=10.68%, 500=0.78%, 750=0.07%, 2000=0.07% cpu : usr=0.07%, sys=4.54%, ctx=6424, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=155KiB/s (159kB/s), 38.0KiB/s-39.3KiB/s (39.9kB/s-40.2kB/s), io=23.4MiB (24.6MB), run=152163-154643msec WRITE: bw=163KiB/s (166kB/s), 40.5KiB/s-41.7KiB/s (41.5kB/s-42.7kB/s), io=24.6MiB (25.7MB), run=152163-154643msec fio: pid=146699, err=5/file:backend.c:481, func=full resid, error=Input/output error rand-rw: (groupid=0, jobs=1): err= 0: pid=146698: Wed Sep 16 04:47:08 2026 read: IOPS=5, BW=21.4KiB/s (21.9kB/s)(5992KiB/280488msec) clat (msec): min=5, max=1059, avg=40.70, stdev=59.90 lat (msec): min=5, max=1059, avg=40.70, stdev=59.91 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 29], | 70.00th=[ 35], 80.00th=[ 51], 90.00th=[ 88], 95.00th=[ 127], | 99.00th=[ 300], 99.50th=[ 397], 99.90th=[ 617], 99.95th=[ 1062], | 99.99th=[ 1062] bw ( KiB/s): min= 6, max= 88, per=32.38%, avg=25.25, stdev=17.03, samples=459 iops : min= 1, max= 22, avg= 5.71, stdev= 4.31, samples=459 write: IOPS=5, BW=22.4KiB/s (22.0kB/s)(6296KiB/280488msec); 0 zone resets clat (msec): min=31, max=1935, avg=139.18, stdev=117.24 lat (msec): min=31, max=1935, avg=139.18, stdev=117.24 clat percentiles (msec): | 1.00th=[ 43], 5.00th=[ 56], 10.00th=[ 63], 20.00th=[ 74], | 30.00th=[ 85], 40.00th=[ 97], 50.00th=[ 110], 60.00th=[ 123], | 70.00th=[ 142], 80.00th=[ 169], 90.00th=[ 234], 95.00th=[ 317], | 99.00th=[ 617], 99.50th=[ 718], 99.90th=[ 1301], 99.95th=[ 1938], | 99.99th=[ 1938] bw ( KiB/s): min= 6, max= 63, per=29.11%, avg=23.58, stdev=12.60, samples=515 iops : min= 1, max= 15, avg= 5.29, stdev= 3.22, samples=515 lat (msec) : 10=4.65%, 20=15.53%, 50=20.15%, 100=26.43%, 250=28.13% lat (msec) : 500=4.00%, 750=0.85%, 1000=0.13%, 2000=0.13% cpu : usr=0.04%, sys=1.09%, ctx=5382, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 5 (file:backend.c:481, func=full resid, error=Input/output error): pid=146699: Wed Sep 16 04:47:08 2026 read: IOPS=4, BW=17.8KiB/s (18.2kB/s)(3972KiB/223136msec) clat (msec): min=5, max=1219, avg=46.56, stdev=66.52 lat (msec): min=5, max=1219, avg=46.57, stdev=66.52 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 15], | 30.00th=[ 18], 40.00th=[ 23], 50.00th=[ 28], 60.00th=[ 35], | 70.00th=[ 48], 80.00th=[ 65], 90.00th=[ 95], 95.00th=[ 131], | 99.00th=[ 288], 99.50th=[ 468], 99.90th=[ 1217], 99.95th=[ 1217], | 99.99th=[ 1217] bw ( KiB/s): min= 6, max= 103, per=28.54%, avg=22.26, stdev=17.04, samples=343 iops : min= 1, max= 25, avg= 4.94, stdev= 4.28, samples=343 write: IOPS=4, BW=19.1KiB/s (19.6kB/s)(4268KiB/223136msec); 0 zone resets clat (msec): min=28, max=1760, avg=165.37, stdev=133.23 lat (msec): min=28, max=1760, avg=165.37, stdev=133.23 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 63], 10.00th=[ 72], 20.00th=[ 86], | 30.00th=[ 100], 40.00th=[ 112], 50.00th=[ 129], 60.00th=[ 144], | 70.00th=[ 169], 80.00th=[ 211], 90.00th=[ 305], 95.00th=[ 409], | 99.00th=[ 642], 99.50th=[ 735], 99.90th=[ 1653], 99.95th=[ 1754], | 99.99th=[ 1754] bw ( KiB/s): min= 6, max= 63, per=24.70%, avg=20.00, stdev=11.46, samples=408 iops : min= 1, max= 15, avg= 4.36, stdev= 2.94, samples=408 lat (msec) : 10=3.83%, 20=13.39%, 50=17.86%, 100=24.99%, 250=31.73% lat (msec) : 500=6.36%, 750=1.50%, 1000=0.15%, 2000=0.15% cpu : usr=0.03%, sys=0.95%, ctx=3230, majf=0, minf=19 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=994,1067,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146700: Wed Sep 16 04:47:08 2026 read: IOPS=5, BW=21.5KiB/s (22.0kB/s)(6028KiB/280264msec) clat (msec): min=5, max=558, avg=41.07, stdev=53.49 lat (msec): min=5, max=558, avg=41.07, stdev=53.49 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 21], 50.00th=[ 25], 60.00th=[ 31], | 70.00th=[ 38], 80.00th=[ 52], 90.00th=[ 87], 95.00th=[ 122], | 99.00th=[ 284], 99.50th=[ 397], 99.90th=[ 550], 99.95th=[ 558], | 99.99th=[ 558] bw ( KiB/s): min= 6, max= 110, per=31.88%, avg=24.87, stdev=17.47, samples=468 iops : min= 1, max= 27, avg= 5.61, stdev= 4.42, samples=468 write: IOPS=5, BW=22.3KiB/s (22.9kB/s)(6260KiB/280264msec); 0 zone resets clat (msec): min=32, max=2109, avg=139.24, stdev=124.27 lat (msec): min=32, max=2109, avg=139.24, stdev=124.27 clat percentiles (msec): | 1.00th=[ 44], 5.00th=[ 55], 10.00th=[ 63], 20.00th=[ 73], | 30.00th=[ 85], 40.00th=[ 94], 50.00th=[ 108], 60.00th=[ 124], | 70.00th=[ 144], 80.00th=[ 176], 90.00th=[ 239], 95.00th=[ 321], | 99.00th=[ 575], 99.50th=[ 667], 99.90th=[ 1921], 99.95th=[ 2106], | 99.99th=[ 2106] bw ( KiB/s): min= 6, max= 63, per=29.14%, avg=23.60, stdev=12.77, samples=511 iops : min= 1, max= 15, avg= 5.29, stdev= 3.27, samples=511 lat (msec) : 10=4.33%, 20=14.68%, 50=21.42%, 100=27.73%, 250=26.79% lat (msec) : 500=4.17%, 750=0.72%, 2000=0.13%, >=2000=0.03% cpu : usr=0.03%, sys=1.16%, ctx=5476, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146701: Wed Sep 16 04:47:08 2026 read: IOPS=5, BW=21.5KiB/s (22.0kB/s)(6052KiB/281275msec) clat (msec): min=4, max=935, avg=40.76, stdev=57.38 lat (msec): min=4, max=935, avg=40.77, stdev=57.38 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 21], 50.00th=[ 25], 60.00th=[ 29], | 70.00th=[ 36], 80.00th=[ 51], 90.00th=[ 85], 95.00th=[ 127], | 99.00th=[ 309], 99.50th=[ 388], 99.90th=[ 600], 99.95th=[ 936], | 99.99th=[ 936] bw ( KiB/s): min= 6, max= 111, per=32.65%, avg=25.47, stdev=18.05, samples=459 iops : min= 1, max= 27, avg= 5.77, stdev= 4.58, samples=459 write: IOPS=5, BW=22.2KiB/s (22.7kB/s)(6236KiB/281275msec); 0 zone resets clat (msec): min=29, max=1717, avg=140.56, stdev=116.99 lat (msec): min=29, max=1717, avg=140.56, stdev=116.99 clat percentiles (msec): | 1.00th=[ 45], 5.00th=[ 56], 10.00th=[ 65], 20.00th=[ 75], | 30.00th=[ 86], 40.00th=[ 96], 50.00th=[ 108], 60.00th=[ 123], | 70.00th=[ 144], 80.00th=[ 176], 90.00th=[ 245], 95.00th=[ 330], | 99.00th=[ 592], 99.50th=[ 709], 99.90th=[ 1385], 99.95th=[ 1720], | 99.99th=[ 1720] bw ( KiB/s): min= 6, max= 55, per=29.30%, avg=23.73, stdev=12.07, samples=506 iops : min= 1, max= 13, avg= 5.32, stdev= 3.09, samples=506 lat (msec) : 10=4.36%, 20=15.20%, 50=21.00%, 100=26.69%, 250=27.21% lat (msec) : 500=4.46%, 750=0.85%, 1000=0.10%, 2000=0.13% cpu : usr=0.04%, sys=1.12%, ctx=5602, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=78.4KiB/s (80.3kB/s), 17.8KiB/s-21.5KiB/s (18.2kB/s-22.0kB/s), io=21.5MiB (22.6MB), run=223136-281275msec WRITE: bw=81.0KiB/s (83.0kB/s), 19.1KiB/s-22.4KiB/s (19.6kB/s-22.0kB/s), io=22.5MiB (23.6MB), run=223136-281275msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146780: Wed Sep 16 04:47:50 2026 read: IOPS=13, BW=219KiB/s (224kB/s)(5920KiB/27085msec) clat (usec): min=34, max=288511, avg=38559.23, stdev=34326.31 lat (usec): min=35, max=288512, avg=38566.72, stdev=34329.27 clat percentiles (usec): | 1.00th=[ 43], 5.00th=[ 53], 10.00th=[ 60], 20.00th=[ 120], | 30.00th=[ 21890], 40.00th=[ 28967], 50.00th=[ 34341], 60.00th=[ 41681], | 70.00th=[ 50594], 80.00th=[ 60031], 90.00th=[ 73925], 95.00th=[ 92799], | 99.00th=[147850], 99.50th=[274727], 99.90th=[287310], 99.95th=[287310], | 99.99th=[287310] bw ( KiB/s): min= 63, max= 447, per=24.75%, avg=218.77, stdev=88.75, samples=53 iops : min= 3, max= 27, avg=12.89, stdev= 5.58, samples=53 write: IOPS=14, BW=235KiB/s (241kB/s)(6368KiB/27085msec); 0 zone resets clat (usec): min=1367, max=382334, avg=32035.70, stdev=38612.66 lat (usec): min=1368, max=382336, avg=32073.74, stdev=38615.36 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 1680], 10.00th=[ 2180], 20.00th=[ 5014], | 30.00th=[ 7177], 40.00th=[ 10552], 50.00th=[ 17695], 60.00th=[ 27657], | 70.00th=[ 36963], 80.00th=[ 54789], 90.00th=[ 83362], 95.00th=[105382], | 99.00th=[147850], 99.50th=[217056], 99.90th=[383779], 99.95th=[383779], | 99.99th=[383779] bw ( KiB/s): min= 31, max= 510, per=25.32%, avg=235.43, stdev=116.14, samples=53 iops : min= 1, max= 31, avg=13.94, stdev= 7.31, samples=53 lat (usec) : 50=1.43%, 100=7.81%, 250=0.91%, 750=0.13%, 1000=0.13% lat (msec) : 2=4.69%, 4=4.69%, 10=11.07%, 20=9.38%, 50=34.24% lat (msec) : 100=20.57%, 250=4.56%, 500=0.39% cpu : usr=0.06%, sys=5.70%, ctx=1396, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146781: Wed Sep 16 04:47:50 2026 read: IOPS=13, BW=215KiB/s (220kB/s)(5744KiB/26750msec) clat (usec): min=35, max=158878, avg=35469.07, stdev=29198.49 lat (usec): min=35, max=158880, avg=35474.17, stdev=29199.65 clat percentiles (usec): | 1.00th=[ 41], 5.00th=[ 56], 10.00th=[ 65], 20.00th=[ 88], | 30.00th=[ 20317], 40.00th=[ 26084], 50.00th=[ 31065], 60.00th=[ 39060], | 70.00th=[ 46924], 80.00th=[ 57934], 90.00th=[ 72877], 95.00th=[ 88605], | 99.00th=[132645], 99.50th=[149947], 99.90th=[158335], 99.95th=[158335], | 99.99th=[158335] bw ( KiB/s): min= 48, max= 506, per=24.67%, avg=218.08, stdev=95.37, samples=51 iops : min= 3, max= 31, avg=12.84, stdev= 5.96, samples=51 write: IOPS=15, BW=245KiB/s (251kB/s)(6544KiB/26750msec); 0 zone resets clat (usec): min=1299, max=295392, avg=34119.83, stdev=40832.72 lat (usec): min=1301, max=295392, avg=34129.18, stdev=40835.75 clat percentiles (usec): | 1.00th=[ 1516], 5.00th=[ 1745], 10.00th=[ 2114], 20.00th=[ 3982], | 30.00th=[ 7111], 40.00th=[ 11600], 50.00th=[ 18744], 60.00th=[ 28181], | 70.00th=[ 43254], 80.00th=[ 58983], 90.00th=[ 85459], 95.00th=[119014], | 99.00th=[166724], 99.50th=[231736], 99.90th=[295699], 99.95th=[295699], | 99.99th=[295699] bw ( KiB/s): min= 63, max= 601, per=26.21%, avg=243.79, stdev=104.86, samples=52 iops : min= 3, max= 37, avg=14.46, stdev= 6.62, samples=52 lat (usec) : 50=1.30%, 100=8.33%, 250=0.26%, 500=0.26% lat (msec) : 2=5.08%, 4=5.86%, 10=8.98%, 20=10.94%, 50=33.20% lat (msec) : 100=21.09%, 250=4.56%, 500=0.13% cpu : usr=0.06%, sys=5.95%, ctx=1394, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146782: Wed Sep 16 04:47:50 2026 read: IOPS=14, BW=239KiB/s (245kB/s)(6256KiB/26145msec) clat (usec): min=25, max=282387, avg=35609.71, stdev=31386.58 lat (usec): min=25, max=282388, avg=35610.81, stdev=31386.61 clat percentiles (usec): | 1.00th=[ 40], 5.00th=[ 56], 10.00th=[ 62], 20.00th=[ 83], | 30.00th=[ 20055], 40.00th=[ 26084], 50.00th=[ 31065], 60.00th=[ 39060], | 70.00th=[ 48497], 80.00th=[ 57410], 90.00th=[ 70779], 95.00th=[ 89654], | 99.00th=[122160], 99.50th=[179307], 99.90th=[283116], 99.95th=[283116], | 99.99th=[283116] bw ( KiB/s): min= 31, max= 483, per=27.28%, avg=241.14, stdev=102.65, samples=51 iops : min= 1, max= 30, avg=14.27, stdev= 6.46, samples=51 write: IOPS=14, BW=231KiB/s (236kB/s)(6032KiB/26145msec); 0 zone resets clat (usec): min=1405, max=234379, avg=32328.75, stdev=34200.78 lat (usec): min=1405, max=234381, avg=32338.34, stdev=34209.28 clat percentiles (usec): | 1.00th=[ 1467], 5.00th=[ 1680], 10.00th=[ 2073], 20.00th=[ 5342], | 30.00th=[ 8455], 40.00th=[ 14353], 50.00th=[ 22152], 60.00th=[ 30802], | 70.00th=[ 40633], 80.00th=[ 53216], 90.00th=[ 76022], 95.00th=[ 99091], | 99.00th=[166724], 99.50th=[202376], 99.90th=[233833], 99.95th=[233833], | 99.99th=[233833] bw ( KiB/s): min= 31, max= 470, per=24.69%, avg=229.59, stdev=97.79, samples=51 iops : min= 1, max= 29, avg=13.55, stdev= 6.18, samples=51 lat (usec) : 50=1.82%, 100=9.24%, 250=0.52%, 1000=0.13% lat (msec) : 2=4.82%, 4=3.26%, 10=8.59%, 20=9.90%, 50=36.72% lat (msec) : 100=21.35%, 250=3.52%, 500=0.13% cpu : usr=0.07%, sys=5.80%, ctx=1435, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146783: Wed Sep 16 04:47:50 2026 read: IOPS=13, BW=224KiB/s (229kB/s)(6032KiB/26975msec) clat (usec): min=36, max=270806, avg=38841.88, stdev=36420.47 lat (usec): min=36, max=270806, avg=38846.57, stdev=36421.60 clat percentiles (usec): | 1.00th=[ 40], 5.00th=[ 55], 10.00th=[ 60], 20.00th=[ 79], | 30.00th=[ 20055], 40.00th=[ 27395], 50.00th=[ 35390], 60.00th=[ 42730], | 70.00th=[ 49546], 80.00th=[ 61604], 90.00th=[ 81265], 95.00th=[ 94897], | 99.00th=[168821], 99.50th=[198181], 99.90th=[270533], 99.95th=[270533], | 99.99th=[270533] bw ( KiB/s): min= 91, max= 506, per=25.26%, avg=223.34, stdev=96.38, samples=53 iops : min= 5, max= 31, avg=13.17, stdev= 6.08, samples=53 write: IOPS=14, BW=232KiB/s (237kB/s)(6256KiB/26975msec); 0 zone resets clat (usec): min=1442, max=220748, avg=31432.23, stdev=34160.13 lat (usec): min=1443, max=220750, avg=31434.38, stdev=34160.30 clat percentiles (usec): | 1.00th=[ 1483], 5.00th=[ 1647], 10.00th=[ 1844], 20.00th=[ 4621], | 30.00th=[ 8094], 40.00th=[ 11338], 50.00th=[ 17695], 60.00th=[ 24511], | 70.00th=[ 42206], 80.00th=[ 56886], 90.00th=[ 80217], 95.00th=[105382], | 99.00th=[135267], 99.50th=[158335], 99.90th=[221250], 99.95th=[221250], | 99.99th=[221250] bw ( KiB/s): min= 62, max= 574, per=24.96%, avg=232.17, stdev=105.22, samples=53 iops : min= 3, max= 35, avg=13.74, stdev= 6.63, samples=53 lat (usec) : 50=2.08%, 100=9.11%, 250=0.65%, 500=0.26%, 750=0.26% lat (msec) : 2=5.99%, 4=3.78%, 10=9.90%, 20=9.38%, 50=31.12% lat (msec) : 100=22.40%, 250=4.95%, 500=0.13% cpu : usr=0.09%, sys=5.78%, ctx=1374, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=884KiB/s (906kB/s), 215KiB/s-239KiB/s (220kB/s-245kB/s), io=23.4MiB (24.5MB), run=26145-27085msec WRITE: bw=930KiB/s (953kB/s), 231KiB/s-245KiB/s (236kB/s-251kB/s), io=24.6MiB (25.8MB), run=26145-27085msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146775: Wed Sep 16 04:48:26 2026 read: IOPS=5, BW=92.5KiB/s (94.7kB/s)(5920KiB/64015msec) clat (msec): min=5, max=300, avg=35.59, stdev=36.91 lat (msec): min=5, max=300, avg=35.59, stdev=36.91 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 20], 50.00th=[ 23], 60.00th=[ 28], | 70.00th=[ 33], 80.00th=[ 50], 90.00th=[ 79], 95.00th=[ 103], | 99.00th=[ 188], 99.50th=[ 275], 99.90th=[ 300], 99.95th=[ 300], | 99.99th=[ 300] bw ( KiB/s): min= 30, max= 317, per=26.52%, avg=98.92, stdev=64.10, samples=117 iops : min= 1, max= 19, avg= 5.39, stdev= 4.04, samples=117 write: IOPS=6, BW=99.5KiB/s (102kB/s)(6368KiB/64015msec); 0 zone resets clat (msec): min=42, max=414, avg=127.38, stdev=62.29 lat (msec): min=42, max=414, avg=127.38, stdev=62.29 clat percentiles (msec): | 1.00th=[ 47], 5.00th=[ 58], 10.00th=[ 66], 20.00th=[ 80], | 30.00th=[ 88], 40.00th=[ 99], 50.00th=[ 111], 60.00th=[ 123], | 70.00th=[ 146], 80.00th=[ 169], 90.00th=[ 220], 95.00th=[ 249], | 99.00th=[ 351], 99.50th=[ 393], 99.90th=[ 414], 99.95th=[ 414], | 99.99th=[ 414] bw ( KiB/s): min= 31, max= 224, per=25.22%, avg=99.10, stdev=47.82, samples=126 iops : min= 1, max= 14, avg= 5.39, stdev= 3.06, samples=126 lat (msec) : 10=3.78%, 20=16.02%, 50=19.79%, 100=27.73%, 250=29.95% lat (msec) : 500=2.73% cpu : usr=0.05%, sys=1.28%, ctx=1302, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146776: Wed Sep 16 04:48:26 2026 read: IOPS=5, BW=90.8KiB/s (92.9kB/s)(5744KiB/63289msec) clat (msec): min=5, max=214, avg=32.82, stdev=30.51 lat (msec): min=5, max=214, avg=32.82, stdev=30.51 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 16], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 24], 60.00th=[ 27], | 70.00th=[ 34], 80.00th=[ 41], 90.00th=[ 62], 95.00th=[ 89], | 99.00th=[ 176], 99.50th=[ 180], 99.90th=[ 215], 99.95th=[ 215], | 99.99th=[ 215] bw ( KiB/s): min= 22, max= 318, per=27.81%, avg=103.72, stdev=64.09, samples=108 iops : min= 1, max= 19, avg= 5.69, stdev= 4.06, samples=108 write: IOPS=6, BW=103KiB/s (106kB/s)(6544KiB/63289msec); 0 zone resets clat (msec): min=35, max=408, avg=125.45, stdev=61.31 lat (msec): min=35, max=408, avg=125.46, stdev=61.31 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 55], 10.00th=[ 64], 20.00th=[ 80], | 30.00th=[ 87], 40.00th=[ 99], 50.00th=[ 113], 60.00th=[ 125], | 70.00th=[ 144], 80.00th=[ 163], 90.00th=[ 203], 95.00th=[ 249], | 99.00th=[ 342], 99.50th=[ 359], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 30, max= 224, per=26.34%, avg=103.51, stdev=46.31, samples=123 iops : min= 1, max= 14, avg= 5.67, stdev= 2.92, samples=123 lat (msec) : 10=3.12%, 20=14.32%, 50=24.87%, 100=24.35%, 250=30.99% lat (msec) : 500=2.34% cpu : usr=0.04%, sys=1.34%, ctx=1273, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146777: Wed Sep 16 04:48:26 2026 read: IOPS=6, BW=98.6KiB/s (101kB/s)(6256KiB/63463msec) clat (msec): min=6, max=392, avg=34.59, stdev=41.17 lat (msec): min=6, max=392, avg=34.60, stdev=41.17 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 23], 60.00th=[ 28], | 70.00th=[ 32], 80.00th=[ 40], 90.00th=[ 66], 95.00th=[ 102], | 99.00th=[ 275], 99.50th=[ 305], 99.90th=[ 393], 99.95th=[ 393], | 99.99th=[ 393] bw ( KiB/s): min= 30, max= 318, per=28.24%, avg=105.32, stdev=73.66, samples=117 iops : min= 1, max= 19, avg= 5.79, stdev= 4.67, samples=117 write: IOPS=5, BW=95.0KiB/s (97.3kB/s)(6032KiB/63463msec); 0 zone resets clat (msec): min=30, max=471, avg=131.99, stdev=66.97 lat (msec): min=30, max=471, avg=132.01, stdev=66.96 clat percentiles (msec): | 1.00th=[ 43], 5.00th=[ 62], 10.00th=[ 69], 20.00th=[ 84], | 30.00th=[ 95], 40.00th=[ 106], 50.00th=[ 113], 60.00th=[ 128], | 70.00th=[ 144], 80.00th=[ 171], 90.00th=[ 220], 95.00th=[ 268], | 99.00th=[ 414], 99.50th=[ 451], 99.90th=[ 472], 99.95th=[ 472], | 99.99th=[ 472] bw ( KiB/s): min= 30, max= 223, per=24.23%, avg=95.23, stdev=44.49, samples=124 iops : min= 1, max= 13, avg= 5.15, stdev= 2.81, samples=124 lat (msec) : 10=2.86%, 20=17.97%, 50=23.57%, 100=20.96%, 250=30.73% lat (msec) : 500=3.91% cpu : usr=0.03%, sys=1.29%, ctx=1348, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146778: Wed Sep 16 04:48:26 2026 read: IOPS=5, BW=94.1KiB/s (96.3kB/s)(6032KiB/64122msec) clat (msec): min=6, max=274, avg=33.60, stdev=33.62 lat (msec): min=6, max=274, avg=33.60, stdev=33.62 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 18], 40.00th=[ 20], 50.00th=[ 23], 60.00th=[ 27], | 70.00th=[ 33], 80.00th=[ 43], 90.00th=[ 69], 95.00th=[ 93], | 99.00th=[ 184], 99.50th=[ 220], 99.90th=[ 275], 99.95th=[ 275], | 99.99th=[ 275] bw ( KiB/s): min= 30, max= 318, per=27.54%, avg=102.73, stdev=68.56, samples=114 iops : min= 1, max= 19, avg= 5.61, stdev= 4.33, samples=114 write: IOPS=6, BW=97.6KiB/s (99.9kB/s)(6256KiB/64122msec); 0 zone resets clat (msec): min=36, max=462, avg=131.49, stdev=66.34 lat (msec): min=36, max=462, avg=131.50, stdev=66.35 clat percentiles (msec): | 1.00th=[ 45], 5.00th=[ 60], 10.00th=[ 68], 20.00th=[ 81], | 30.00th=[ 95], 40.00th=[ 106], 50.00th=[ 118], 60.00th=[ 128], | 70.00th=[ 142], 80.00th=[ 169], 90.00th=[ 222], 95.00th=[ 271], | 99.00th=[ 380], 99.50th=[ 405], 99.90th=[ 464], 99.95th=[ 464], | 99.99th=[ 464] bw ( KiB/s): min= 30, max= 256, per=24.79%, avg=97.44, stdev=45.21, samples=125 iops : min= 1, max= 16, avg= 5.29, stdev= 2.91, samples=125 lat (msec) : 10=2.86%, 20=17.58%, 50=22.14%, 100=22.53%, 250=31.90% lat (msec) : 500=2.99% cpu : usr=0.04%, sys=1.21%, ctx=1264, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=374KiB/s (383kB/s), 90.8KiB/s-98.6KiB/s (92.9kB/s-101kB/s), io=23.4MiB (24.5MB), run=63289-64122msec WRITE: bw=393KiB/s (402kB/s), 95.0KiB/s-103KiB/s (97.3kB/s-106kB/s), io=24.6MiB (25.8MB), run=63289-64122msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146802: Wed Sep 16 04:48:41 2026 read: IOPS=2, BW=2050KiB/s (2099kB/s)(7168KiB/3497msec) clat (usec): min=1421, max=1051.0k, avg=309476.81, stdev=342970.13 lat (usec): min=1422, max=1051.0k, avg=309478.24, stdev=342970.07 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 1418], 10.00th=[ 1418], | 20.00th=[ 164627], 30.00th=[ 175113], 40.00th=[ 175113], | 50.00th=[ 189793], 60.00th=[ 238027], 70.00th=[ 238027], | 80.00th=[ 346031], 90.00th=[1052771], 95.00th=[1052771], | 99.00th=[1052771], 99.50th=[1052771], 99.90th=[1052771], | 99.95th=[1052771], 99.99th=[1052771] bw ( KiB/s): min= 1954, max= 4039, per=55.29%, avg=2798.00, stdev=993.23, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 write: IOPS=1, BW=1464KiB/s (1499kB/s)(5120KiB/3497msec); 0 zone resets clat (msec): min=73, max=551, avg=265.73, stdev=195.57 lat (msec): min=73, max=551, avg=265.82, stdev=195.56 clat percentiles (msec): | 1.00th=[ 73], 5.00th=[ 73], 10.00th=[ 73], 20.00th=[ 73], | 30.00th=[ 155], 40.00th=[ 155], 50.00th=[ 171], 60.00th=[ 171], | 70.00th=[ 380], 80.00th=[ 380], 90.00th=[ 550], 95.00th=[ 550], | 99.00th=[ 550], 99.50th=[ 550], 99.90th=[ 550], 99.95th=[ 550], | 99.99th=[ 550] bw ( KiB/s): min= 2019, max= 6119, per=57.42%, avg=4069.00, stdev=2899.14, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 lat (msec) : 2=8.33%, 100=8.33%, 250=50.00%, 500=16.67%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=13.82%, ctx=218, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146803: Wed Sep 16 04:48:41 2026 read: IOPS=0, BW=506KiB/s (518kB/s)(2048KiB/4046msec) clat (usec): min=1188, max=192370, avg=96779.27, stdev=135185.87 lat (usec): min=1190, max=192371, avg=96780.96, stdev=135186.01 clat percentiles (usec): | 1.00th=[ 1188], 5.00th=[ 1188], 10.00th=[ 1188], 20.00th=[ 1188], | 30.00th=[ 1188], 40.00th=[ 1188], 50.00th=[ 1188], 60.00th=[191890], | 70.00th=[191890], 80.00th=[191890], 90.00th=[191890], 95.00th=[191890], | 99.00th=[191890], 99.50th=[191890], 99.90th=[191890], 99.95th=[191890], | 99.99th=[191890] bw ( KiB/s): min= 1452, max= 2039, per=34.49%, avg=1745.50, stdev=415.07, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2531KiB/s (2592kB/s)(10.0MiB/4046msec); 0 zone resets clat (msec): min=118, max=1068, avg=384.95, stdev=281.71 lat (msec): min=118, max=1068, avg=385.12, stdev=281.70 clat percentiles (msec): | 1.00th=[ 118], 5.00th=[ 118], 10.00th=[ 118], 20.00th=[ 176], | 30.00th=[ 197], 40.00th=[ 222], 50.00th=[ 251], 60.00th=[ 321], | 70.00th=[ 435], 80.00th=[ 498], 90.00th=[ 567], 95.00th=[ 1070], | 99.00th=[ 1070], 99.50th=[ 1070], 99.90th=[ 1070], 99.95th=[ 1070], | 99.99th=[ 1070] bw ( KiB/s): min= 1988, max= 4039, per=45.15%, avg=3199.40, stdev=1083.90, samples=5 iops : min= 1, max= 3, avg= 2.20, stdev= 1.10, samples=5 lat (msec) : 2=8.33%, 250=41.67%, 500=33.33%, 750=8.33%, 2000=8.33% cpu : usr=0.07%, sys=12.61%, ctx=137, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146804: Wed Sep 16 04:48:41 2026 read: IOPS=1, BW=1741KiB/s (1783kB/s)(6144KiB/3528msec) clat (usec): min=498, max=362993, avg=83403.08, stdev=146954.36 lat (usec): min=500, max=362994, avg=83404.38, stdev=146954.35 clat percentiles (usec): | 1.00th=[ 498], 5.00th=[ 498], 10.00th=[ 498], 20.00th=[ 562], | 30.00th=[ 562], 40.00th=[ 611], 50.00th=[ 611], 60.00th=[ 1795], | 70.00th=[133694], 80.00th=[133694], 90.00th=[362808], 95.00th=[362808], | 99.00th=[362808], 99.50th=[362808], 99.90th=[362808], 99.95th=[362808], | 99.99th=[362808] bw ( KiB/s): min= 4079, max= 5862, per=98.21%, avg=4970.50, stdev=1260.77, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=1, BW=1741KiB/s (1783kB/s)(6144KiB/3528msec); 0 zone resets clat (msec): min=211, max=1265, avg=504.27, stdev=393.07 lat (msec): min=211, max=1265, avg=504.33, stdev=393.07 clat percentiles (msec): | 1.00th=[ 211], 5.00th=[ 211], 10.00th=[ 211], 20.00th=[ 313], | 30.00th=[ 313], 40.00th=[ 317], 50.00th=[ 317], 60.00th=[ 334], | 70.00th=[ 584], 80.00th=[ 584], 90.00th=[ 1267], 95.00th=[ 1267], | 99.00th=[ 1267], 99.50th=[ 1267], 99.90th=[ 1267], 99.95th=[ 1267], | 99.99th=[ 1267] bw ( KiB/s): min= 1954, max= 3079, per=32.14%, avg=2277.75, stdev=535.67, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (usec) : 500=8.33%, 750=16.67% lat (msec) : 2=8.33%, 250=16.67%, 500=33.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=10.26%, ctx=100, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146805: Wed Sep 16 04:48:41 2026 read: IOPS=1, BW=1522KiB/s (1559kB/s)(5120KiB/3364msec) clat (usec): min=537, max=174537, avg=90769.09, stdev=86418.80 lat (usec): min=538, max=174539, avg=90770.63, stdev=86418.93 clat percentiles (usec): | 1.00th=[ 537], 5.00th=[ 537], 10.00th=[ 537], 20.00th=[ 537], | 30.00th=[ 652], 40.00th=[ 652], 50.00th=[108528], 60.00th=[108528], | 70.00th=[170918], 80.00th=[170918], 90.00th=[175113], 95.00th=[175113], | 99.00th=[175113], 99.50th=[175113], 99.90th=[175113], 99.95th=[175113], | 99.99th=[175113] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=2, BW=2131KiB/s (2182kB/s)(7168KiB/3364msec); 0 zone resets clat (msec): min=87, max=1202, avg=415.18, stdev=381.08 lat (msec): min=87, max=1202, avg=415.28, stdev=381.10 clat percentiles (msec): | 1.00th=[ 88], 5.00th=[ 88], 10.00th=[ 88], 20.00th=[ 211], | 30.00th=[ 213], 40.00th=[ 213], 50.00th=[ 239], 60.00th=[ 376], | 70.00th=[ 376], 80.00th=[ 584], 90.00th=[ 1200], 95.00th=[ 1200], | 99.00th=[ 1200], 99.50th=[ 1200], 99.90th=[ 1200], 99.95th=[ 1200], | 99.99th=[ 1200] bw ( KiB/s): min= 1505, max= 6119, per=54.25%, avg=3844.00, stdev=2307.67, samples=3 iops : min= 1, max= 5, avg= 3.00, stdev= 2.00, samples=3 lat (usec) : 750=16.67% lat (msec) : 100=8.33%, 250=50.00%, 500=8.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=11.95%, ctx=131, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5062KiB/s (5183kB/s), 506KiB/s-2050KiB/s (518kB/s-2099kB/s), io=20.0MiB (20.0MB), run=3364-4046msec WRITE: bw=7087KiB/s (7257kB/s), 1464KiB/s-2531KiB/s (1499kB/s-2592kB/s), io=28.0MiB (29.4MB), run=3364-4046msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146797: Wed Sep 16 04:48:41 2026 read: IOPS=3, BW=3144KiB/s (3219kB/s)(7168KiB/2280msec) clat (msec): min=92, max=206, avg=148.94, stdev=42.68 lat (msec): min=92, max=206, avg=148.94, stdev=42.68 clat percentiles (msec): | 1.00th=[ 93], 5.00th=[ 93], 10.00th=[ 93], 20.00th=[ 120], | 30.00th=[ 127], 40.00th=[ 127], 50.00th=[ 136], 60.00th=[ 161], | 70.00th=[ 161], 80.00th=[ 203], 90.00th=[ 207], 95.00th=[ 207], | 99.00th=[ 207], 99.50th=[ 207], 99.90th=[ 207], 99.95th=[ 207], | 99.99th=[ 207] bw ( KiB/s): min= 1988, max= 5520, per=75.11%, avg=3181.00, stdev=2025.77, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 write: IOPS=2, BW=2246KiB/s (2300kB/s)(5120KiB/2280msec); 0 zone resets clat (msec): min=142, max=444, avg=238.98, stdev=122.17 lat (msec): min=143, max=444, avg=239.11, stdev=122.14 clat percentiles (msec): | 1.00th=[ 144], 5.00th=[ 144], 10.00th=[ 144], 20.00th=[ 144], | 30.00th=[ 153], 40.00th=[ 153], 50.00th=[ 213], 60.00th=[ 213], | 70.00th=[ 243], 80.00th=[ 243], 90.00th=[ 443], 95.00th=[ 443], | 99.00th=[ 443], 99.50th=[ 443], 99.90th=[ 443], 99.95th=[ 443], | 99.99th=[ 443] bw ( KiB/s): min= 1380, max= 4071, per=53.34%, avg=3163.33, stdev=1544.49, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 lat (msec) : 100=8.33%, 250=83.33%, 500=8.33% cpu : usr=0.04%, sys=5.62%, ctx=50, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146798: Wed Sep 16 04:48:41 2026 read: IOPS=0, BW=424KiB/s (434kB/s)(2048KiB/4835msec) clat (msec): min=151, max=211, avg=181.28, stdev=42.46 lat (msec): min=151, max=211, avg=181.42, stdev=42.25 clat percentiles (msec): | 1.00th=[ 153], 5.00th=[ 153], 10.00th=[ 153], 20.00th=[ 153], | 30.00th=[ 153], 40.00th=[ 153], 50.00th=[ 153], 60.00th=[ 211], | 70.00th=[ 211], 80.00th=[ 211], 90.00th=[ 211], 95.00th=[ 211], | 99.00th=[ 211], 99.50th=[ 211], 99.90th=[ 211], 99.95th=[ 211], | 99.99th=[ 211] bw ( KiB/s): min= 1424, max= 2003, per=40.46%, avg=1713.50, stdev=409.41, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2118KiB/s (2169kB/s)(10.0MiB/4835msec); 0 zone resets clat (msec): min=77, max=1251, avg=432.82, stdev=380.56 lat (msec): min=77, max=1251, avg=432.93, stdev=380.54 clat percentiles (msec): | 1.00th=[ 78], 5.00th=[ 78], 10.00th=[ 78], 20.00th=[ 111], | 30.00th=[ 165], 40.00th=[ 182], 50.00th=[ 194], 60.00th=[ 426], | 70.00th=[ 523], 80.00th=[ 527], 90.00th=[ 877], 95.00th=[ 1250], | 99.00th=[ 1250], 99.50th=[ 1250], 99.90th=[ 1250], 99.95th=[ 1250], | 99.99th=[ 1250] bw ( KiB/s): min= 1424, max= 6071, per=49.00%, avg=2905.83, stdev=1797.07, samples=6 iops : min= 1, max= 5, avg= 2.00, stdev= 1.67, samples=6 lat (msec) : 100=8.33%, 250=50.00%, 500=8.33%, 750=16.67%, 1000=8.33% lat (msec) : 2000=8.33% cpu : usr=0.04%, sys=3.39%, ctx=79, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146799: Wed Sep 16 04:48:41 2026 read: IOPS=1, BW=1299KiB/s (1330kB/s)(6144KiB/4729msec) clat (msec): min=62, max=366, avg=190.45, stdev=116.43 lat (msec): min=62, max=366, avg=190.45, stdev=116.43 clat percentiles (msec): | 1.00th=[ 63], 5.00th=[ 63], 10.00th=[ 63], 20.00th=[ 108], | 30.00th=[ 108], 40.00th=[ 142], 50.00th=[ 142], 60.00th=[ 174], | 70.00th=[ 292], 80.00th=[ 292], 90.00th=[ 368], 95.00th=[ 368], | 99.00th=[ 368], 99.50th=[ 368], 99.90th=[ 368], 99.95th=[ 368], | 99.99th=[ 368] bw ( KiB/s): min= 4047, max= 4087, per=96.03%, avg=4067.00, stdev=28.28, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1299KiB/s (1330kB/s)(6144KiB/4729msec); 0 zone resets clat (msec): min=240, max=1148, avg=569.41, stdev=318.82 lat (msec): min=240, max=1148, avg=569.54, stdev=318.82 clat percentiles (msec): | 1.00th=[ 241], 5.00th=[ 241], 10.00th=[ 241], 20.00th=[ 355], | 30.00th=[ 355], 40.00th=[ 451], 50.00th=[ 451], 60.00th=[ 600], | 70.00th=[ 625], 80.00th=[ 625], 90.00th=[ 1150], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min= 1391, max= 2023, per=31.42%, avg=1863.00, stdev=267.31, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (msec) : 100=8.33%, 250=33.33%, 500=33.33%, 750=16.67%, 2000=8.33% cpu : usr=0.00%, sys=2.88%, ctx=56, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146800: Wed Sep 16 04:48:41 2026 read: IOPS=1, BW=1617KiB/s (1656kB/s)(5120KiB/3166msec) clat (msec): min=53, max=160, avg=113.20, stdev=46.89 lat (msec): min=53, max=160, avg=113.20, stdev=46.89 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 54], 10.00th=[ 54], 20.00th=[ 54], | 30.00th=[ 74], 40.00th=[ 74], 50.00th=[ 133], 60.00th=[ 133], | 70.00th=[ 146], 80.00th=[ 146], 90.00th=[ 161], 95.00th=[ 161], | 99.00th=[ 161], 99.50th=[ 161], 99.90th=[ 161], 99.95th=[ 161], | 99.99th=[ 161] bw ( KiB/s): min= 4039, max= 4071, per=95.75%, avg=4055.00, stdev=22.63, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2264KiB/s (2318kB/s)(7168KiB/3166msec); 0 zone resets clat (msec): min=155, max=631, avg=360.42, stdev=185.63 lat (msec): min=155, max=631, avg=360.61, stdev=185.67 clat percentiles (msec): | 1.00th=[ 157], 5.00th=[ 157], 10.00th=[ 157], 20.00th=[ 171], | 30.00th=[ 245], 40.00th=[ 245], 50.00th=[ 351], 60.00th=[ 409], | 70.00th=[ 409], 80.00th=[ 558], 90.00th=[ 634], 95.00th=[ 634], | 99.00th=[ 634], 99.50th=[ 634], 99.90th=[ 634], 99.95th=[ 634], | 99.99th=[ 634] bw ( KiB/s): min= 1988, max= 4047, per=46.94%, avg=2783.25, stdev=983.37, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 lat (msec) : 100=16.67%, 250=50.00%, 500=16.67%, 750=16.67% cpu : usr=0.00%, sys=3.89%, ctx=53, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4236KiB/s (4337kB/s), 424KiB/s-3144KiB/s (434kB/s-3219kB/s), io=20.0MiB (20.0MB), run=2280-4835msec WRITE: bw=5930KiB/s (6072kB/s), 1299KiB/s-2264KiB/s (1330kB/s-2318kB/s), io=28.0MiB (29.4MB), run=2280-4835msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146817: Wed Sep 16 04:48:56 2026 read: IOPS=1, BW=4430KiB/s (4536kB/s)(12.0MiB/2774msec) clat (msec): min=616, max=1142, avg=812.98, stdev=287.01 lat (msec): min=616, max=1142, avg=812.98, stdev=287.01 clat percentiles (msec): | 1.00th=[ 617], 5.00th=[ 617], 10.00th=[ 617], 20.00th=[ 617], | 30.00th=[ 617], 40.00th=[ 684], 50.00th=[ 684], 60.00th=[ 684], | 70.00th=[ 1150], 80.00th=[ 1150], 90.00th=[ 1150], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min= 8062, max= 8192, per=100.00%, avg=8127.00, stdev=91.92, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=12.04%, ctx=259, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146818: Wed Sep 16 04:48:56 2026 read: IOPS=0, BW=1298KiB/s (1329kB/s)(4096KiB/3156msec) clat (nsec): min=944420k, max=944420k, avg=944420451.00, stdev= 0.00 lat (nsec): min=944423k, max=944423k, avg=944423111.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 944], 5.00th=[ 944], 10.00th=[ 944], 20.00th=[ 944], | 30.00th=[ 944], 40.00th=[ 944], 50.00th=[ 944], 60.00th=[ 944], | 70.00th=[ 944], 80.00th=[ 944], 90.00th=[ 944], 95.00th=[ 944], | 99.00th=[ 944], 99.50th=[ 944], 99.90th=[ 944], 99.95th=[ 944], | 99.99th=[ 944] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2596KiB/s (2658kB/s)(8192KiB/3156msec); 0 zone resets clat (msec): min=609, max=1516, avg=1063.12, stdev=641.66 lat (msec): min=609, max=1517, avg=1063.54, stdev=641.74 clat percentiles (msec): | 1.00th=[ 609], 5.00th=[ 609], 10.00th=[ 609], 20.00th=[ 609], | 30.00th=[ 609], 40.00th=[ 609], 50.00th=[ 609], 60.00th=[ 1519], | 70.00th=[ 1519], 80.00th=[ 1519], 90.00th=[ 1519], 95.00th=[ 1519], | 99.00th=[ 1519], 99.50th=[ 1519], 99.90th=[ 1519], 99.95th=[ 1519], | 99.99th=[ 1519] bw ( KiB/s): min= 8000, max= 8000, per=89.49%, avg=8000.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=4.75%, ctx=117, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146819: Wed Sep 16 04:48:56 2026 write: IOPS=0, BW=3353KiB/s (3433kB/s)(12.0MiB/3665msec); 0 zone resets clat (msec): min=264, max=1654, avg=1128.93, stdev=754.11 lat (msec): min=265, max=1654, avg=1129.56, stdev=754.25 clat percentiles (msec): | 1.00th=[ 266], 5.00th=[ 266], 10.00th=[ 266], 20.00th=[ 266], | 30.00th=[ 266], 40.00th=[ 1469], 50.00th=[ 1469], 60.00th=[ 1469], | 70.00th=[ 1653], 80.00th=[ 1653], 90.00th=[ 1653], 95.00th=[ 1653], | 99.00th=[ 1653], 99.50th=[ 1653], 99.90th=[ 1653], 99.95th=[ 1653], | 99.99th=[ 1653] bw ( KiB/s): min= 8000, max= 8062, per=89.83%, avg=8031.00, stdev=43.84, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.00%, sys=4.31%, ctx=58, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146820: Wed Sep 16 04:48:56 2026 write: IOPS=1, BW=4543KiB/s (4652kB/s)(12.0MiB/2705msec); 0 zone resets clat (msec): min=353, max=1437, avg=788.47, stdev=573.19 lat (msec): min=353, max=1438, avg=788.83, stdev=573.07 clat percentiles (msec): | 1.00th=[ 355], 5.00th=[ 355], 10.00th=[ 355], 20.00th=[ 355], | 30.00th=[ 355], 40.00th=[ 575], 50.00th=[ 575], 60.00th=[ 575], | 70.00th=[ 1435], 80.00th=[ 1435], 90.00th=[ 1435], 95.00th=[ 1435], | 99.00th=[ 1435], 99.50th=[ 1435], 99.90th=[ 1435], 99.95th=[ 1435], | 99.99th=[ 1435] bw ( KiB/s): min= 8062, max= 8192, per=90.91%, avg=8127.00, stdev=91.92, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=5.84%, ctx=70, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5191KiB/s (5316kB/s), 1298KiB/s-4430KiB/s (1329kB/s-4536kB/s), io=16.0MiB (16.8MB), run=2774-3156msec WRITE: bw=8941KiB/s (9155kB/s), 2596KiB/s-4543KiB/s (2658kB/s-4652kB/s), io=32.0MiB (33.6MB), run=2705-3665msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146813: Wed Sep 16 04:48:56 2026 read: IOPS=0, BW=3963KiB/s (4058kB/s)(12.0MiB/3101msec) clat (msec): min=252, max=1362, avg=991.53, stdev=639.96 lat (msec): min=252, max=1362, avg=991.53, stdev=639.96 clat percentiles (msec): | 1.00th=[ 253], 5.00th=[ 253], 10.00th=[ 253], 20.00th=[ 253], | 30.00th=[ 253], 40.00th=[ 1368], 50.00th=[ 1368], 60.00th=[ 1368], | 70.00th=[ 1368], 80.00th=[ 1368], 90.00th=[ 1368], 95.00th=[ 1368], | 99.00th=[ 1368], 99.50th=[ 1368], 99.90th=[ 1368], 99.95th=[ 1368], | 99.99th=[ 1368] bw ( KiB/s): min= 7892, max= 8126, per=100.00%, avg=8009.00, stdev=165.46, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.00%, sys=3.32%, ctx=50, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146814: Wed Sep 16 04:48:56 2026 read: IOPS=0, BW=1035KiB/s (1060kB/s)(4096KiB/3957msec) clat (nsec): min=1150.2M, max=1150.2M, avg=1150209800.00, stdev= 0.00 lat (nsec): min=1150.2M, max=1150.2M, avg=1150211225.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1150], 5.00th=[ 1150], 10.00th=[ 1150], 20.00th=[ 1150], | 30.00th=[ 1150], 40.00th=[ 1150], 50.00th=[ 1150], 60.00th=[ 1150], | 70.00th=[ 1150], 80.00th=[ 1150], 90.00th=[ 1150], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min= 8126, max= 8126, per=100.00%, avg=8126.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2070KiB/s (2120kB/s)(8192KiB/3957msec); 0 zone resets clat (msec): min=815, max=1733, avg=1274.59, stdev=648.96 lat (msec): min=816, max=1734, avg=1275.26, stdev=649.36 clat percentiles (msec): | 1.00th=[ 818], 5.00th=[ 818], 10.00th=[ 818], 20.00th=[ 818], | 30.00th=[ 818], 40.00th=[ 818], 50.00th=[ 818], 60.00th=[ 1737], | 70.00th=[ 1737], 80.00th=[ 1737], 90.00th=[ 1737], 95.00th=[ 1737], | 99.00th=[ 1737], 99.50th=[ 1737], 99.90th=[ 1737], 99.95th=[ 1737], | 99.99th=[ 1737] bw ( KiB/s): min= 8159, max= 8159, per=98.53%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=3.06%, ctx=64, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146815: Wed Sep 16 04:48:56 2026 write: IOPS=2, BW=9423KiB/s (9649kB/s)(12.0MiB/1304msec); 0 zone resets clat (msec): min=316, max=500, avg=416.58, stdev=93.43 lat (msec): min=316, max=501, avg=417.03, stdev=93.30 clat percentiles (msec): | 1.00th=[ 317], 5.00th=[ 317], 10.00th=[ 317], 20.00th=[ 317], | 30.00th=[ 317], 40.00th=[ 435], 50.00th=[ 435], 60.00th=[ 435], | 70.00th=[ 502], 80.00th=[ 502], 90.00th=[ 502], 95.00th=[ 502], | 99.00th=[ 502], 99.50th=[ 502], 99.90th=[ 502], 99.95th=[ 502], | 99.99th=[ 502] bw ( KiB/s): min= 5843, max= 8175, per=84.64%, avg=7009.00, stdev=1648.97, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.00%, sys=6.52%, ctx=47, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146816: Wed Sep 16 04:48:56 2026 write: IOPS=1, BW=5525KiB/s (5658kB/s)(12.0MiB/2224msec); 0 zone resets clat (msec): min=303, max=903, avg=683.56, stdev=330.64 lat (msec): min=303, max=903, avg=683.99, stdev=330.78 clat percentiles (msec): | 1.00th=[ 305], 5.00th=[ 305], 10.00th=[ 305], 20.00th=[ 305], | 30.00th=[ 305], 40.00th=[ 844], 50.00th=[ 844], 60.00th=[ 844], | 70.00th=[ 902], 80.00th=[ 902], 90.00th=[ 902], 95.00th=[ 902], | 99.00th=[ 902], 99.50th=[ 902], 99.90th=[ 902], 99.95th=[ 902], | 99.99th=[ 902] bw ( KiB/s): min= 6291, max= 8126, per=87.05%, avg=7208.50, stdev=1297.54, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 1000=66.67% cpu : usr=0.00%, sys=4.41%, ctx=34, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4141KiB/s (4240kB/s), 1035KiB/s-3963KiB/s (1060kB/s-4058kB/s), io=16.0MiB (16.8MB), run=3101-3957msec WRITE: bw=8281KiB/s (8480kB/s), 2070KiB/s-9423KiB/s (2120kB/s-9649kB/s), io=32.0MiB (33.6MB), run=1304-3957msec