== sanity test 398b: DIO and buffer IO race ============== 18:35:13 (1763336113) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 4.3274 s, 11.6 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=149358: Sun Nov 16 18:36:48 2025 read: IOPS=18, BW=73.3KiB/s (75.0kB/s)(5992KiB/81779msec) clat (usec): min=106, max=653097, avg=26593.83, stdev=26235.19 lat (usec): min=106, max=653098, avg=26595.00, stdev=26235.21 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 180], 10.00th=[ 783], 20.00th=[ 13173], | 30.00th=[ 16188], 40.00th=[ 19006], 50.00th=[ 21627], 60.00th=[ 26608], | 70.00th=[ 31327], 80.00th=[ 38536], 90.00th=[ 49021], 95.00th=[ 60031], | 99.00th=[ 91751], 99.50th=[123208], 99.90th=[267387], 99.95th=[650118], | 99.99th=[650118] bw ( KiB/s): min= 23, max= 136, per=24.80%, avg=72.67, stdev=25.93, samples=162 iops : min= 5, max= 34, avg=17.59, stdev= 6.54, samples=162 write: IOPS=19, BW=76.0KiB/s (78.8kB/s)(6296KiB/81779msec); 0 zone resets clat (usec): min=827, max=287454, avg=26435.92, stdev=27329.06 lat (usec): min=828, max=287455, avg=26439.37, stdev=27327.37 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 1172], 10.00th=[ 1467], 20.00th=[ 3064], | 30.00th=[ 6652], 40.00th=[ 13304], 50.00th=[ 20055], 60.00th=[ 27132], | 70.00th=[ 34866], 80.00th=[ 44827], 90.00th=[ 60031], 95.00th=[ 77071], | 99.00th=[116917], 99.50th=[145753], 99.90th=[223347], 99.95th=[287310], | 99.99th=[287310] bw ( KiB/s): min= 31, max= 191, per=24.88%, avg=76.39, stdev=27.58, samples=162 iops : min= 7, max= 47, avg=18.53, stdev= 6.92, samples=162 lat (usec) : 250=3.42%, 500=1.27%, 750=0.16%, 1000=0.36% lat (msec) : 2=7.78%, 4=4.49%, 10=7.55%, 20=21.94%, 50=40.62% lat (msec) : 100=11.10%, 250=1.20%, 500=0.07%, 750=0.03% cpu : usr=0.09%, sys=8.68%, ctx=6733, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149359: Sun Nov 16 18:36:48 2025 read: IOPS=18, BW=73.6KiB/s (75.4kB/s)(5940KiB/80711msec) clat (usec): min=110, max=500608, avg=26618.38, stdev=24617.35 lat (usec): min=110, max=500609, avg=26620.29, stdev=24618.39 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 182], 10.00th=[ 6259], 20.00th=[ 13435], | 30.00th=[ 16581], 40.00th=[ 19268], 50.00th=[ 22414], 60.00th=[ 27132], | 70.00th=[ 32375], 80.00th=[ 38011], 90.00th=[ 47449], 95.00th=[ 56361], | 99.00th=[ 90702], 99.50th=[111674], 99.90th=[392168], 99.95th=[501220], | 99.99th=[501220] bw ( KiB/s): min= 15, max= 152, per=24.99%, avg=73.22, stdev=25.66, samples=160 iops : min= 3, max= 38, avg=17.73, stdev= 6.46, samples=160 write: IOPS=19, BW=78.7KiB/s (80.5kB/s)(6348KiB/80711msec); 0 zone resets clat (usec): min=854, max=191681, avg=25738.84, stdev=25489.90 lat (usec): min=855, max=191683, avg=25743.11, stdev=25489.09 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1139], 10.00th=[ 1319], 20.00th=[ 2540], | 30.00th=[ 5669], 40.00th=[ 12518], 50.00th=[ 19792], 60.00th=[ 27395], | 70.00th=[ 35390], 80.00th=[ 44303], 90.00th=[ 58459], 95.00th=[ 70779], | 99.00th=[111674], 99.50th=[143655], 99.90th=[170918], 99.95th=[191890], | 99.99th=[191890] bw ( KiB/s): min= 15, max= 184, per=25.46%, avg=78.16, stdev=26.86, samples=160 iops : min= 3, max= 46, avg=18.97, stdev= 6.76, samples=160 lat (usec) : 250=3.26%, 500=0.94%, 750=0.20%, 1000=0.33% lat (msec) : 2=8.24%, 4=5.31%, 10=6.84%, 20=21.16%, 50=41.89% lat (msec) : 100=10.77%, 250=0.98%, 500=0.07%, 750=0.03% cpu : usr=0.13%, sys=8.98%, ctx=6827, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149360: Sun Nov 16 18:36:48 2025 read: IOPS=18, BW=74.1KiB/s (75.8kB/s)(6028KiB/81384msec) clat (usec): min=100, max=478342, avg=25989.66, stdev=22184.27 lat (usec): min=100, max=478344, avg=25990.90, stdev=22184.29 clat percentiles (usec): | 1.00th=[ 130], 5.00th=[ 172], 10.00th=[ 478], 20.00th=[ 12518], | 30.00th=[ 16057], 40.00th=[ 19006], 50.00th=[ 22414], 60.00th=[ 26608], | 70.00th=[ 31327], 80.00th=[ 39060], 90.00th=[ 47973], 95.00th=[ 55313], | 99.00th=[ 83362], 99.50th=[111674], 99.90th=[242222], 99.95th=[480248], | 99.99th=[480248] bw ( KiB/s): min= 8, max= 165, per=25.04%, avg=73.37, stdev=27.39, samples=161 iops : min= 2, max= 41, avg=17.77, stdev= 6.88, samples=161 write: IOPS=19, BW=76.9KiB/s (78.8kB/s)(6260KiB/81384msec); 0 zone resets clat (usec): min=894, max=557306, avg=26746.14, stdev=29602.86 lat (usec): min=895, max=557307, avg=26747.95, stdev=29602.66 clat percentiles (usec): | 1.00th=[ 1012], 5.00th=[ 1172], 10.00th=[ 1418], 20.00th=[ 2999], | 30.00th=[ 6390], 40.00th=[ 14091], 50.00th=[ 20579], 60.00th=[ 26608], | 70.00th=[ 34341], 80.00th=[ 45351], 90.00th=[ 60556], 95.00th=[ 72877], | 99.00th=[122160], 99.50th=[149947], 99.90th=[198181], 99.95th=[557843], | 99.99th=[557843] bw ( KiB/s): min= 23, max= 181, per=24.78%, avg=76.07, stdev=24.72, samples=161 iops : min= 5, max= 45, avg=18.45, stdev= 6.18, samples=161 lat (usec) : 250=3.78%, 500=1.20%, 750=0.16%, 1000=0.52% lat (msec) : 2=7.32%, 4=4.75%, 10=7.10%, 20=21.48%, 50=41.44% lat (msec) : 100=10.90%, 250=1.27%, 500=0.03%, 750=0.03% cpu : usr=0.12%, sys=8.93%, ctx=6735, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149361: Sun Nov 16 18:36:48 2025 read: IOPS=18, BW=74.6KiB/s (76.3kB/s)(6052KiB/81171msec) clat (usec): min=103, max=510939, avg=26940.62, stdev=25537.77 lat (usec): min=103, max=510940, avg=26942.54, stdev=25540.32 clat percentiles (usec): | 1.00th=[ 130], 5.00th=[ 176], 10.00th=[ 1762], 20.00th=[ 13960], | 30.00th=[ 16712], 40.00th=[ 20055], 50.00th=[ 22938], 60.00th=[ 26870], | 70.00th=[ 32375], 80.00th=[ 38536], 90.00th=[ 48497], 95.00th=[ 56886], | 99.00th=[ 87557], 99.50th=[128451], 99.90th=[484443], 99.95th=[509608], | 99.99th=[509608] bw ( KiB/s): min= 7, max= 143, per=25.25%, avg=73.98, stdev=25.35, samples=161 iops : min= 1, max= 35, avg=17.93, stdev= 6.34, samples=161 write: IOPS=19, BW=76.8KiB/s (78.7kB/s)(6236KiB/81171msec); 0 zone resets clat (usec): min=841, max=495133, avg=25702.50, stdev=28957.58 lat (usec): min=842, max=495134, avg=25703.94, stdev=28957.59 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 1123], 10.00th=[ 1221], 20.00th=[ 2442], | 30.00th=[ 5538], 40.00th=[ 12125], 50.00th=[ 19530], 60.00th=[ 26084], | 70.00th=[ 34341], 80.00th=[ 43779], 90.00th=[ 58459], 95.00th=[ 74974], | 99.00th=[110625], 99.50th=[132645], 99.90th=[295699], 99.95th=[497026], | 99.99th=[497026] bw ( KiB/s): min= 8, max= 167, per=24.81%, avg=76.16, stdev=27.47, samples=161 iops : min= 2, max= 41, avg=18.48, stdev= 6.87, samples=161 lat (usec) : 250=3.45%, 500=1.20%, 750=0.13%, 1000=0.52% lat (msec) : 2=8.53%, 4=4.56%, 10=6.51%, 20=20.54%, 50=43.07% lat (msec) : 100=10.38%, 250=0.98%, 500=0.10%, 750=0.03% cpu : usr=0.09%, sys=8.95%, ctx=6738, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=294KiB/s (301kB/s), 73.3KiB/s-74.6KiB/s (75.0kB/s-76.3kB/s), io=23.4MiB (24.6MB), run=80711-81779msec WRITE: bw=307KiB/s (315kB/s), 76.8KiB/s-78.7KiB/s (78.7kB/s-80.5kB/s), io=24.6MiB (25.7MB), run=80711-81779msec rand-rw: (groupid=0, jobs=1): err= 0: pid=149363: Sun Nov 16 18:38:46 2025 read: IOPS=7, BW=30.2KiB/s (30.9kB/s)(5992KiB/198267msec) clat (msec): min=4, max=742, avg=27.97, stdev=29.37 lat (msec): min=4, max=742, avg=27.97, stdev=29.37 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 21], 60.00th=[ 28], | 70.00th=[ 34], 80.00th=[ 42], 90.00th=[ 54], 95.00th=[ 63], | 99.00th=[ 104], 99.50th=[ 188], 99.90th=[ 245], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 7, max= 119, per=26.17%, avg=31.40, stdev=19.99, samples=371 iops : min= 1, max= 29, avg= 7.25, stdev= 5.04, samples=371 write: IOPS=7, BW=31.8KiB/s (32.5kB/s)(6296KiB/198267msec); 0 zone resets clat (msec): min=26, max=495, avg=99.28, stdev=43.70 lat (msec): min=26, max=495, avg=99.28, stdev=43.70 clat percentiles (msec): | 1.00th=[ 40], 5.00th=[ 48], 10.00th=[ 56], 20.00th=[ 64], | 30.00th=[ 73], 40.00th=[ 82], 50.00th=[ 91], 60.00th=[ 101], | 70.00th=[ 113], 80.00th=[ 130], 90.00th=[ 155], 95.00th=[ 178], | 99.00th=[ 234], 99.50th=[ 268], 99.90th=[ 468], 99.95th=[ 498], | 99.99th=[ 498] bw ( KiB/s): min= 7, max= 72, per=24.80%, avg=31.25, stdev=13.20, samples=391 iops : min= 1, max= 18, avg= 7.21, stdev= 3.37, samples=391 lat (msec) : 10=7.81%, 20=15.69%, 50=22.66%, 100=33.04%, 250=20.35% lat (msec) : 500=0.42%, 750=0.03% cpu : usr=0.06%, sys=1.87%, ctx=5892, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149364: Sun Nov 16 18:38:46 2025 read: IOPS=7, BW=29.9KiB/s (30.6kB/s)(5940KiB/198552msec) clat (msec): min=4, max=593, avg=27.45, stdev=27.23 lat (msec): min=4, max=593, avg=27.45, stdev=27.23 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 21], 60.00th=[ 27], | 70.00th=[ 34], 80.00th=[ 42], 90.00th=[ 53], 95.00th=[ 63], | 99.00th=[ 99], 99.50th=[ 161], 99.90th=[ 321], 99.95th=[ 592], | 99.99th=[ 592] bw ( KiB/s): min= 7, max= 120, per=26.25%, avg=31.50, stdev=20.18, samples=367 iops : min= 1, max= 30, avg= 7.28, stdev= 5.08, samples=367 write: IOPS=7, BW=31.0KiB/s (32.7kB/s)(6348KiB/198552msec); 0 zone resets clat (msec): min=28, max=341, avg=99.35, stdev=42.75 lat (msec): min=28, max=341, avg=99.35, stdev=42.75 clat percentiles (msec): | 1.00th=[ 39], 5.00th=[ 48], 10.00th=[ 55], 20.00th=[ 65], | 30.00th=[ 73], 40.00th=[ 81], 50.00th=[ 90], 60.00th=[ 101], | 70.00th=[ 114], 80.00th=[ 132], 90.00th=[ 157], 95.00th=[ 180], | 99.00th=[ 234], 99.50th=[ 268], 99.90th=[ 330], 99.95th=[ 342], | 99.99th=[ 342] bw ( KiB/s): min= 7, max= 87, per=24.93%, avg=31.41, stdev=13.00, samples=393 iops : min= 1, max= 21, avg= 7.25, stdev= 3.31, samples=393 lat (msec) : 10=8.07%, 20=16.11%, 50=21.94%, 100=32.29%, 250=21.16% lat (msec) : 500=0.39%, 750=0.03% cpu : usr=0.05%, sys=1.87%, ctx=5946, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149365: Sun Nov 16 18:38:46 2025 read: IOPS=7, BW=30.5KiB/s (31.3kB/s)(6028KiB/197503msec) clat (msec): min=4, max=325, avg=27.96, stdev=23.27 lat (msec): min=4, max=325, avg=27.96, stdev=23.27 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 21], 60.00th=[ 28], | 70.00th=[ 34], 80.00th=[ 42], 90.00th=[ 55], 95.00th=[ 68], | 99.00th=[ 107], 99.50th=[ 122], 99.90th=[ 239], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 7, max= 119, per=26.40%, avg=31.68, stdev=19.61, samples=371 iops : min= 1, max= 29, avg= 7.31, stdev= 4.93, samples=371 write: IOPS=7, BW=31.7KiB/s (32.5kB/s)(6260KiB/197503msec); 0 zone resets clat (msec): min=33, max=625, avg=99.20, stdev=45.15 lat (msec): min=33, max=625, avg=99.20, stdev=45.15 clat percentiles (msec): | 1.00th=[ 41], 5.00th=[ 48], 10.00th=[ 55], 20.00th=[ 64], | 30.00th=[ 73], 40.00th=[ 82], 50.00th=[ 90], 60.00th=[ 101], | 70.00th=[ 113], 80.00th=[ 128], 90.00th=[ 153], 95.00th=[ 178], | 99.00th=[ 239], 99.50th=[ 255], 99.90th=[ 485], 99.95th=[ 625], | 99.99th=[ 625] bw ( KiB/s): min= 7, max= 79, per=24.73%, avg=31.16, stdev=13.11, samples=391 iops : min= 1, max= 19, avg= 7.18, stdev= 3.34, samples=391 lat (msec) : 10=7.45%, 20=16.18%, 50=22.07%, 100=33.14%, 250=20.74% lat (msec) : 500=0.39%, 750=0.03% cpu : usr=0.05%, sys=1.88%, ctx=5920, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149366: Sun Nov 16 18:38:46 2025 read: IOPS=7, BW=30.9KiB/s (31.6kB/s)(6052KiB/195946msec) clat (msec): min=4, max=247, avg=26.79, stdev=21.78 lat (msec): min=4, max=247, avg=26.79, stdev=21.78 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 26], | 70.00th=[ 33], 80.00th=[ 42], 90.00th=[ 53], 95.00th=[ 64], | 99.00th=[ 101], 99.50th=[ 124], 99.90th=[ 203], 99.95th=[ 249], | 99.99th=[ 249] bw ( KiB/s): min= 7, max= 143, per=26.64%, avg=31.96, stdev=20.81, samples=369 iops : min= 1, max= 35, avg= 7.39, stdev= 5.25, samples=369 write: IOPS=7, BW=31.8KiB/s (32.6kB/s)(6236KiB/195946msec); 0 zone resets clat (msec): min=31, max=675, avg=99.62, stdev=46.44 lat (msec): min=31, max=675, avg=99.62, stdev=46.44 clat percentiles (msec): | 1.00th=[ 40], 5.00th=[ 49], 10.00th=[ 56], 20.00th=[ 65], | 30.00th=[ 73], 40.00th=[ 82], 50.00th=[ 92], 60.00th=[ 101], | 70.00th=[ 113], 80.00th=[ 129], 90.00th=[ 155], 95.00th=[ 176], | 99.00th=[ 230], 99.50th=[ 330], 99.90th=[ 498], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 7, max= 71, per=24.83%, avg=31.29, stdev=12.93, samples=388 iops : min= 1, max= 17, avg= 7.21, stdev= 3.29, samples=388 lat (msec) : 10=7.91%, 20=17.35%, 50=21.09%, 100=32.45%, 250=20.70% lat (msec) : 500=0.46%, 750=0.03% cpu : usr=0.06%, sys=1.87%, ctx=5853, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=121KiB/s (124kB/s), 29.9KiB/s-30.9KiB/s (30.6kB/s-31.6kB/s), io=23.4MiB (24.6MB), run=195946-198552msec WRITE: bw=127KiB/s (130kB/s), 31.7KiB/s-31.0KiB/s (32.5kB/s-32.7kB/s), io=24.6MiB (25.7MB), run=195946-198552msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=149412: Sun Nov 16 18:39:16 2025 read: IOPS=17, BW=273KiB/s (279kB/s)(5920KiB/21698msec) clat (usec): min=120, max=321023, avg=31832.13, stdev=29775.55 lat (usec): min=120, max=321025, avg=31833.36, stdev=29775.60 clat percentiles (usec): | 1.00th=[ 131], 5.00th=[ 157], 10.00th=[ 174], 20.00th=[ 2573], | 30.00th=[ 19006], 40.00th=[ 22938], 50.00th=[ 27395], 60.00th=[ 32637], | 70.00th=[ 40633], 80.00th=[ 47973], 90.00th=[ 61604], 95.00th=[ 76022], | 99.00th=[135267], 99.50th=[191890], 99.90th=[320865], 99.95th=[320865], | 99.99th=[320865] bw ( KiB/s): min= 63, max= 571, per=25.71%, avg=272.79, stdev=109.08, samples=42 iops : min= 3, max= 35, avg=16.36, stdev= 6.78, samples=42 write: IOPS=18, BW=293KiB/s (301kB/s)(6368KiB/21698msec); 0 zone resets clat (usec): min=1259, max=163894, avg=24716.52, stdev=25693.57 lat (usec): min=1262, max=163899, avg=24723.42, stdev=25693.33 clat percentiles (usec): | 1.00th=[ 1352], 5.00th=[ 1549], 10.00th=[ 1745], 20.00th=[ 3097], | 30.00th=[ 6456], 40.00th=[ 10683], 50.00th=[ 16581], 60.00th=[ 23200], | 70.00th=[ 31065], 80.00th=[ 43254], 90.00th=[ 58983], 95.00th=[ 72877], | 99.00th=[120062], 99.50th=[135267], 99.90th=[164627], 99.95th=[164627], | 99.99th=[164627] bw ( KiB/s): min= 31, max= 535, per=26.39%, avg=294.79, stdev=115.92, samples=42 iops : min= 1, max= 33, avg=17.79, stdev= 7.24, samples=42 lat (usec) : 250=7.03%, 500=1.69%, 750=0.39% lat (msec) : 2=7.81%, 4=5.21%, 10=8.07%, 20=13.67%, 50=39.32% lat (msec) : 100=14.84%, 250=1.82%, 500=0.13% cpu : usr=0.08%, sys=8.32%, ctx=1507, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149413: Sun Nov 16 18:39:16 2025 read: IOPS=15, BW=255KiB/s (261kB/s)(5744KiB/22554msec) clat (usec): min=113, max=150089, avg=32197.10, stdev=26415.86 lat (usec): min=113, max=150091, avg=32198.34, stdev=26415.91 clat percentiles (usec): | 1.00th=[ 118], 5.00th=[ 149], 10.00th=[ 169], 20.00th=[ 13304], | 30.00th=[ 20055], 40.00th=[ 23725], 50.00th=[ 28443], 60.00th=[ 33817], | 70.00th=[ 40109], 80.00th=[ 45351], 90.00th=[ 64750], 95.00th=[ 81265], | 99.00th=[131597], 99.50th=[147850], 99.90th=[149947], 99.95th=[149947], | 99.99th=[149947] bw ( KiB/s): min= 31, max= 476, per=23.92%, avg=253.75, stdev=111.07, samples=44 iops : min= 1, max= 29, avg=15.16, stdev= 6.98, samples=44 write: IOPS=18, BW=290KiB/s (297kB/s)(6544KiB/22554msec); 0 zone resets clat (usec): min=1273, max=421590, avg=26318.49, stdev=36137.23 lat (usec): min=1274, max=421592, avg=26322.71, stdev=36137.43 clat percentiles (usec): | 1.00th=[ 1401], 5.00th=[ 1582], 10.00th=[ 1680], 20.00th=[ 2409], | 30.00th=[ 5800], 40.00th=[ 10159], 50.00th=[ 15401], 60.00th=[ 22938], | 70.00th=[ 30278], 80.00th=[ 43779], 90.00th=[ 63177], 95.00th=[ 83362], | 99.00th=[147850], 99.50th=[235930], 99.90th=[421528], 99.95th=[421528], | 99.99th=[421528] bw ( KiB/s): min= 63, max= 669, per=25.93%, avg=289.59, stdev=119.91, samples=44 iops : min= 3, max= 41, avg=17.45, stdev= 7.52, samples=44 lat (usec) : 250=6.90%, 500=1.04%, 750=0.13% lat (msec) : 2=9.51%, 4=4.43%, 10=7.68%, 20=14.71%, 50=40.23% lat (msec) : 100=12.89%, 250=2.21%, 500=0.26% cpu : usr=0.13%, sys=8.73%, ctx=1451, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149414: Sun Nov 16 18:39:16 2025 read: IOPS=17, BW=277KiB/s (284kB/s)(6256KiB/22555msec) clat (usec): min=105, max=158788, avg=30926.03, stdev=25975.03 lat (usec): min=105, max=158790, avg=30927.42, stdev=25975.35 clat percentiles (usec): | 1.00th=[ 114], 5.00th=[ 155], 10.00th=[ 172], 20.00th=[ 1188], | 30.00th=[ 16188], 40.00th=[ 21890], 50.00th=[ 27657], 60.00th=[ 32900], | 70.00th=[ 40633], 80.00th=[ 49546], 90.00th=[ 64226], 95.00th=[ 81265], | 99.00th=[114820], 99.50th=[145753], 99.90th=[158335], 99.95th=[158335], | 99.99th=[158335] bw ( KiB/s): min= 90, max= 541, per=26.18%, avg=277.75, stdev=119.27, samples=44 iops : min= 5, max= 33, avg=16.66, stdev= 7.44, samples=44 write: IOPS=16, BW=267KiB/s (274kB/s)(6032KiB/22555msec); 0 zone resets clat (usec): min=1350, max=238540, avg=27297.41, stdev=33468.51 lat (usec): min=1352, max=238543, avg=27305.00, stdev=33477.69 clat percentiles (usec): | 1.00th=[ 1401], 5.00th=[ 1614], 10.00th=[ 1795], 20.00th=[ 3261], | 30.00th=[ 5276], 40.00th=[ 9896], 50.00th=[ 17171], 60.00th=[ 23462], | 70.00th=[ 31589], 80.00th=[ 44827], 90.00th=[ 68682], 95.00th=[ 90702], | 99.00th=[152044], 99.50th=[238027], 99.90th=[238027], 99.95th=[238027], | 99.99th=[238027] bw ( KiB/s): min= 31, max= 507, per=23.55%, avg=263.02, stdev=118.65, samples=44 iops : min= 1, max= 31, avg=15.77, stdev= 7.50, samples=44 lat (usec) : 250=8.07%, 500=1.43%, 750=0.39%, 1000=0.26% lat (msec) : 2=6.90%, 4=6.25%, 10=7.94%, 20=13.41%, 50=37.50% lat (msec) : 100=15.10%, 250=2.73% cpu : usr=0.09%, sys=8.40%, ctx=1494, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149415: Sun Nov 16 18:39:16 2025 read: IOPS=17, BW=275KiB/s (281kB/s)(6032KiB/21969msec) clat (usec): min=102, max=346494, avg=32401.32, stdev=32211.51 lat (usec): min=103, max=346495, avg=32402.56, stdev=32211.55 clat percentiles (usec): | 1.00th=[ 118], 5.00th=[ 143], 10.00th=[ 163], 20.00th=[ 429], | 30.00th=[ 17433], 40.00th=[ 22414], 50.00th=[ 26870], 60.00th=[ 33162], | 70.00th=[ 41681], 80.00th=[ 50070], 90.00th=[ 66847], 95.00th=[ 85459], | 99.00th=[145753], 99.50th=[175113], 99.90th=[346031], 99.95th=[346031], | 99.99th=[346031] bw ( KiB/s): min= 92, max= 477, per=25.73%, avg=273.00, stdev=98.22, samples=43 iops : min= 5, max= 29, avg=16.40, stdev= 6.12, samples=43 write: IOPS=17, BW=285KiB/s (292kB/s)(6256KiB/21969msec); 0 zone resets clat (usec): min=1293, max=217720, avg=24410.97, stdev=28095.37 lat (usec): min=1297, max=217721, avg=24429.26, stdev=28084.05 clat percentiles (usec): | 1.00th=[ 1336], 5.00th=[ 1532], 10.00th=[ 1680], 20.00th=[ 2442], | 30.00th=[ 4228], 40.00th=[ 8717], 50.00th=[ 16057], 60.00th=[ 23200], | 70.00th=[ 30278], 80.00th=[ 40633], 90.00th=[ 59507], 95.00th=[ 79168], | 99.00th=[123208], 99.50th=[164627], 99.90th=[217056], 99.95th=[217056], | 99.99th=[217056] bw ( KiB/s): min= 94, max= 565, per=25.50%, avg=284.79, stdev=123.12, samples=43 iops : min= 5, max= 35, avg=17.14, stdev= 7.69, samples=43 lat (usec) : 250=8.07%, 500=1.82%, 750=0.26%, 1000=0.26% lat (msec) : 2=9.11%, 4=5.99%, 10=8.07%, 20=11.46%, 50=37.76% lat (msec) : 100=14.32%, 250=2.73%, 500=0.13% cpu : usr=0.10%, sys=8.46%, ctx=1429, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1062KiB/s (1087kB/s), 255KiB/s-277KiB/s (261kB/s-284kB/s), io=23.4MiB (24.5MB), run=21698-22555msec WRITE: bw=1117KiB/s (1144kB/s), 267KiB/s-293KiB/s (274kB/s-301kB/s), io=24.6MiB (25.8MB), run=21698-22555msec rand-rw: (groupid=0, jobs=1): err= 0: pid=149417: Sun Nov 16 18:39:50 2025 read: IOPS=6, BW=107KiB/s (109kB/s)(5920KiB/55567msec) clat (msec): min=6, max=243, avg=26.36, stdev=28.32 lat (msec): min=6, max=243, avg=26.36, stdev=28.32 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 33], 90.00th=[ 57], 95.00th=[ 83], | 99.00th=[ 140], 99.50th=[ 218], 99.90th=[ 245], 99.95th=[ 245], | 99.99th=[ 245] bw ( KiB/s): min= 30, max= 383, per=26.55%, avg=111.51, stdev=68.45, samples=105 iops : min= 1, max= 23, avg= 6.22, stdev= 4.39, samples=105 write: IOPS=7, BW=115KiB/s (117kB/s)(6368KiB/55567msec); 0 zone resets clat (msec): min=39, max=498, avg=114.40, stdev=55.00 lat (msec): min=39, max=498, avg=114.40, stdev=55.00 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 57], 10.00th=[ 67], 20.00th=[ 78], | 30.00th=[ 86], 40.00th=[ 94], 50.00th=[ 103], 60.00th=[ 111], | 70.00th=[ 125], 80.00th=[ 140], 90.00th=[ 176], 95.00th=[ 218], | 99.00th=[ 351], 99.50th=[ 414], 99.90th=[ 498], 99.95th=[ 498], | 99.99th=[ 498] bw ( KiB/s): min= 31, max= 192, per=26.33%, avg=116.36, stdev=43.36, samples=108 iops : min= 1, max= 12, avg= 6.53, stdev= 2.85, samples=108 lat (msec) : 10=5.47%, 20=25.65%, 50=12.24%, 100=27.86%, 250=27.21% lat (msec) : 500=1.56% cpu : usr=0.05%, sys=1.64%, ctx=1430, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149418: Sun Nov 16 18:39:50 2025 read: IOPS=6, BW=101KiB/s (103kB/s)(5744KiB/56970msec) clat (msec): min=5, max=467, avg=27.61, stdev=36.87 lat (msec): min=5, max=467, avg=27.61, stdev=36.87 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 32], 90.00th=[ 61], 95.00th=[ 92], | 99.00th=[ 165], 99.50th=[ 188], 99.90th=[ 468], 99.95th=[ 468], | 99.99th=[ 468] bw ( KiB/s): min= 30, max= 352, per=27.16%, avg=114.06, stdev=75.42, samples=98 iops : min= 1, max= 22, avg= 6.42, stdev= 4.84, samples=98 write: IOPS=7, BW=115KiB/s (118kB/s)(6544KiB/56970msec); 0 zone resets clat (msec): min=35, max=519, avg=114.51, stdev=59.72 lat (msec): min=35, max=519, avg=114.52, stdev=59.72 clat percentiles (msec): | 1.00th=[ 43], 5.00th=[ 55], 10.00th=[ 66], 20.00th=[ 75], | 30.00th=[ 84], 40.00th=[ 93], 50.00th=[ 102], 60.00th=[ 110], | 70.00th=[ 123], 80.00th=[ 140], 90.00th=[ 176], 95.00th=[ 220], | 99.00th=[ 359], 99.50th=[ 376], 99.90th=[ 518], 99.95th=[ 518], | 99.99th=[ 518] bw ( KiB/s): min= 31, max= 256, per=26.18%, avg=115.74, stdev=46.21, samples=110 iops : min= 1, max= 16, avg= 6.50, stdev= 3.04, samples=110 lat (msec) : 10=5.08%, 20=26.04%, 50=11.72%, 100=27.99%, 250=27.34% lat (msec) : 500=1.69%, 750=0.13% cpu : usr=0.05%, sys=1.61%, ctx=1456, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149419: Sun Nov 16 18:39:50 2025 read: IOPS=7, BW=114KiB/s (117kB/s)(6256KiB/54675msec) clat (msec): min=5, max=239, avg=25.41, stdev=29.12 lat (msec): min=5, max=239, avg=25.41, stdev=29.12 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 19], | 70.00th=[ 23], 80.00th=[ 29], 90.00th=[ 51], 95.00th=[ 78], | 99.00th=[ 178], 99.50th=[ 230], 99.90th=[ 241], 99.95th=[ 241], | 99.99th=[ 241] bw ( KiB/s): min= 31, max= 476, per=29.79%, avg=125.11, stdev=89.19, samples=99 iops : min= 1, max= 29, avg= 7.08, stdev= 5.63, samples=99 write: IOPS=6, BW=110KiB/s (113kB/s)(6032KiB/54675msec); 0 zone resets clat (msec): min=43, max=440, avg=118.34, stdev=60.57 lat (msec): min=43, max=440, avg=118.35, stdev=60.57 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 57], 10.00th=[ 64], 20.00th=[ 78], | 30.00th=[ 86], 40.00th=[ 93], 50.00th=[ 103], 60.00th=[ 114], | 70.00th=[ 126], 80.00th=[ 144], 90.00th=[ 194], 95.00th=[ 262], | 99.00th=[ 368], 99.50th=[ 435], 99.90th=[ 443], 99.95th=[ 443], | 99.99th=[ 443] bw ( KiB/s): min= 30, max= 221, per=24.87%, avg=109.94, stdev=46.70, samples=108 iops : min= 1, max= 13, avg= 6.12, stdev= 3.10, samples=108 lat (msec) : 10=5.73%, 20=26.56%, 50=14.32%, 100=25.78%, 250=25.00% lat (msec) : 500=2.60% cpu : usr=0.05%, sys=1.68%, ctx=1484, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149420: Sun Nov 16 18:39:50 2025 read: IOPS=6, BW=108KiB/s (110kB/s)(6032KiB/56032msec) clat (msec): min=6, max=174, avg=25.52, stdev=25.87 lat (msec): min=6, max=174, avg=25.52, stdev=25.87 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 32], 90.00th=[ 52], 95.00th=[ 80], | 99.00th=[ 157], 99.50th=[ 169], 99.90th=[ 174], 99.95th=[ 174], | 99.99th=[ 174] bw ( KiB/s): min= 31, max= 319, per=27.84%, avg=116.94, stdev=68.55, samples=101 iops : min= 1, max= 19, avg= 6.57, stdev= 4.36, samples=101 write: IOPS=6, BW=112KiB/s (114kB/s)(6256KiB/56032msec); 0 zone resets clat (msec): min=39, max=481, avg=118.20, stdev=57.88 lat (msec): min=39, max=481, avg=118.21, stdev=57.88 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 64], 10.00th=[ 69], 20.00th=[ 78], | 30.00th=[ 87], 40.00th=[ 93], 50.00th=[ 104], 60.00th=[ 115], | 70.00th=[ 127], 80.00th=[ 148], 90.00th=[ 178], 95.00th=[ 226], | 99.00th=[ 334], 99.50th=[ 477], 99.90th=[ 481], 99.95th=[ 481], | 99.99th=[ 481] bw ( KiB/s): min= 31, max= 224, per=25.23%, avg=111.52, stdev=46.36, samples=111 iops : min= 1, max= 14, avg= 6.23, stdev= 3.05, samples=111 lat (msec) : 10=7.29%, 20=23.31%, 50=13.41%, 100=27.86%, 250=26.17% lat (msec) : 500=1.95% cpu : usr=0.03%, sys=1.58%, ctx=1395, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=420KiB/s (431kB/s), 101KiB/s-114KiB/s (103kB/s-117kB/s), io=23.4MiB (24.5MB), run=54675-56970msec WRITE: bw=442KiB/s (453kB/s), 110KiB/s-115KiB/s (113kB/s-118kB/s), io=24.6MiB (25.8MB), run=54675-56970msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=149433: Sun Nov 16 18:40:05 2025 read: IOPS=1, BW=2023KiB/s (2072kB/s)(7168KiB/3543msec) clat (msec): min=103, max=294, avg=165.31, stdev=79.29 lat (msec): min=103, max=294, avg=165.32, stdev=79.29 clat percentiles (msec): | 1.00th=[ 104], 5.00th=[ 104], 10.00th=[ 104], 20.00th=[ 106], | 30.00th=[ 114], 40.00th=[ 114], 50.00th=[ 115], 60.00th=[ 167], | 70.00th=[ 167], 80.00th=[ 257], 90.00th=[ 296], 95.00th=[ 296], | 99.00th=[ 296], 99.50th=[ 296], 99.90th=[ 296], 99.95th=[ 296], | 99.99th=[ 296] bw ( KiB/s): min= 2048, max= 4123, per=83.00%, avg=3408.67, stdev=1178.86, samples=3 iops : min= 2, max= 4, avg= 3.00, stdev= 1.00, samples=3 write: IOPS=1, BW=1445KiB/s (1480kB/s)(5120KiB/3543msec); 0 zone resets clat (msec): min=192, max=750, avg=451.68, stdev=278.56 lat (msec): min=194, max=750, avg=452.05, stdev=278.25 clat percentiles (msec): | 1.00th=[ 192], 5.00th=[ 192], 10.00th=[ 192], 20.00th=[ 192], | 30.00th=[ 213], 40.00th=[ 213], 50.00th=[ 355], 60.00th=[ 355], | 70.00th=[ 751], 80.00th=[ 751], 90.00th=[ 751], 95.00th=[ 751], | 99.00th=[ 751], 99.50th=[ 751], 99.90th=[ 751], 99.95th=[ 751], | 99.99th=[ 751] bw ( KiB/s): min= 1374, max= 4096, per=41.10%, avg=2363.25, stdev=1192.17, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 lat (msec) : 250=58.33%, 500=25.00%, 750=8.33%, 1000=8.33% cpu : usr=0.06%, sys=4.86%, ctx=63, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149434: Sun Nov 16 18:40:05 2025 read: IOPS=0, BW=411KiB/s (421kB/s)(2048KiB/4986msec) clat (msec): min=137, max=585, avg=361.11, stdev=316.66 lat (msec): min=137, max=585, avg=361.11, stdev=316.66 clat percentiles (msec): | 1.00th=[ 138], 5.00th=[ 138], 10.00th=[ 138], 20.00th=[ 138], | 30.00th=[ 138], 40.00th=[ 138], 50.00th=[ 138], 60.00th=[ 584], | 70.00th=[ 584], 80.00th=[ 584], 90.00th=[ 584], 95.00th=[ 584], | 99.00th=[ 584], 99.50th=[ 584], 99.90th=[ 584], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 1368, max= 1935, per=40.21%, avg=1651.50, stdev=400.93, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2054KiB/s (2103kB/s)(10.0MiB/4986msec); 0 zone resets clat (msec): min=218, max=711, avg=420.91, stdev=174.45 lat (msec): min=219, max=711, avg=421.05, stdev=174.49 clat percentiles (msec): | 1.00th=[ 220], 5.00th=[ 220], 10.00th=[ 220], 20.00th=[ 224], | 30.00th=[ 279], 40.00th=[ 279], 50.00th=[ 368], 60.00th=[ 468], | 70.00th=[ 514], 80.00th=[ 514], 90.00th=[ 634], 95.00th=[ 709], | 99.00th=[ 709], 99.50th=[ 709], 99.90th=[ 709], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 1984, max= 4071, per=42.01%, avg=2415.71, stdev=777.56, samples=7 iops : min= 1, max= 3, avg= 1.71, stdev= 0.76, samples=7 lat (msec) : 250=25.00%, 500=33.33%, 750=41.67% cpu : usr=0.00%, sys=4.77%, ctx=69, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149435: Sun Nov 16 18:40:05 2025 read: IOPS=1, BW=1461KiB/s (1497kB/s)(6144KiB/4204msec) clat (msec): min=92, max=681, avg=342.67, stdev=266.89 lat (msec): min=92, max=681, avg=342.67, stdev=266.89 clat percentiles (msec): | 1.00th=[ 92], 5.00th=[ 92], 10.00th=[ 92], 20.00th=[ 124], | 30.00th=[ 124], 40.00th=[ 126], 50.00th=[ 126], 60.00th=[ 409], | 70.00th=[ 625], 80.00th=[ 625], 90.00th=[ 684], 95.00th=[ 684], | 99.00th=[ 684], 99.50th=[ 684], 99.90th=[ 684], 99.95th=[ 684], | 99.99th=[ 684] bw ( KiB/s): min= 1935, max= 2027, per=48.41%, avg=1988.25, stdev=39.61, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 write: IOPS=1, BW=1461KiB/s (1497kB/s)(6144KiB/4204msec); 0 zone resets clat (msec): min=146, max=714, avg=339.83, stdev=197.68 lat (msec): min=146, max=714, avg=339.92, stdev=197.68 clat percentiles (msec): | 1.00th=[ 146], 5.00th=[ 146], 10.00th=[ 146], 20.00th=[ 220], | 30.00th=[ 220], 40.00th=[ 284], 50.00th=[ 284], 60.00th=[ 326], | 70.00th=[ 351], 80.00th=[ 351], 90.00th=[ 718], 95.00th=[ 718], | 99.00th=[ 718], 99.50th=[ 718], 99.90th=[ 718], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 1984, max= 2801, per=38.52%, avg=2215.00, stdev=391.57, samples=4 iops : min= 1, max= 2, avg= 1.50, stdev= 0.58, samples=4 lat (msec) : 100=8.33%, 250=33.33%, 500=33.33%, 750=25.00% cpu : usr=0.00%, sys=2.55%, ctx=51, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149436: Sun Nov 16 18:40:05 2025 read: IOPS=1, BW=1197KiB/s (1226kB/s)(5120KiB/4278msec) clat (msec): min=70, max=294, avg=180.99, stdev=85.02 lat (msec): min=70, max=294, avg=180.99, stdev=85.02 clat percentiles (msec): | 1.00th=[ 70], 5.00th=[ 70], 10.00th=[ 70], 20.00th=[ 70], | 30.00th=[ 131], 40.00th=[ 131], 50.00th=[ 201], 60.00th=[ 201], | 70.00th=[ 209], 80.00th=[ 209], 90.00th=[ 296], 95.00th=[ 296], | 99.00th=[ 296], 99.50th=[ 296], 99.90th=[ 296], 99.95th=[ 296], | 99.99th=[ 296] bw ( KiB/s): min= 1935, max= 4096, per=65.57%, avg=2693.00, stdev=1216.35, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 write: IOPS=1, BW=1676KiB/s (1716kB/s)(7168KiB/4278msec); 0 zone resets clat (msec): min=192, max=911, avg=466.59, stdev=219.97 lat (msec): min=192, max=911, avg=467.14, stdev=219.75 clat percentiles (msec): | 1.00th=[ 192], 5.00th=[ 192], 10.00th=[ 192], 20.00th=[ 372], | 30.00th=[ 393], 40.00th=[ 393], 50.00th=[ 435], 60.00th=[ 456], | 70.00th=[ 456], 80.00th=[ 510], 90.00th=[ 911], 95.00th=[ 911], | 99.00th=[ 911], 99.50th=[ 911], 99.90th=[ 911], 99.95th=[ 911], | 99.99th=[ 911] bw ( KiB/s): min= 1935, max= 2797, per=37.53%, avg=2158.20, stdev=359.72, samples=5 iops : min= 1, max= 2, avg= 1.40, stdev= 0.55, samples=5 lat (msec) : 100=8.33%, 250=33.33%, 500=41.67%, 750=8.33%, 1000=8.33% cpu : usr=0.00%, sys=4.44%, ctx=91, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4108KiB/s (4206kB/s), 411KiB/s-2023KiB/s (421kB/s-2072kB/s), io=20.0MiB (20.0MB), run=3543-4986msec WRITE: bw=5751KiB/s (5889kB/s), 1445KiB/s-2054KiB/s (1480kB/s-2103kB/s), io=28.0MiB (29.4MB), run=3543-4986msec rand-rw: (groupid=0, jobs=1): err= 0: pid=149438: Sun Nov 16 18:40:05 2025 read: IOPS=2, BW=2145KiB/s (2197kB/s)(7168KiB/3341msec) clat (msec): min=2, max=462, avg=190.55, stdev=155.82 lat (msec): min=2, max=462, avg=190.55, stdev=155.82 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 3], 20.00th=[ 25], | 30.00th=[ 157], 40.00th=[ 157], 50.00th=[ 194], 60.00th=[ 220], | 70.00th=[ 220], 80.00th=[ 275], 90.00th=[ 464], 95.00th=[ 464], | 99.00th=[ 464], 99.50th=[ 464], 99.90th=[ 464], 99.95th=[ 464], | 99.99th=[ 464] bw ( KiB/s): min= 2000, max= 4357, per=75.44%, avg=3478.67, stdev=1288.08, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=1, BW=1532KiB/s (1569kB/s)(5120KiB/3341msec); 0 zone resets clat (msec): min=237, max=815, avg=401.17, stdev=239.77 lat (msec): min=237, max=815, avg=401.25, stdev=239.78 clat percentiles (msec): | 1.00th=[ 239], 5.00th=[ 239], 10.00th=[ 239], 20.00th=[ 239], | 30.00th=[ 247], 40.00th=[ 247], 50.00th=[ 313], 60.00th=[ 313], | 70.00th=[ 393], 80.00th=[ 393], 90.00th=[ 818], 95.00th=[ 818], | 99.00th=[ 818], 99.50th=[ 818], 99.90th=[ 818], 99.95th=[ 818], | 99.99th=[ 818] bw ( KiB/s): min= 1939, max= 4000, per=41.03%, avg=2648.67, stdev=1170.78, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 4=8.33%, 50=8.33%, 250=41.67%, 500=33.33%, 1000=8.33% cpu : usr=0.00%, sys=14.25%, ctx=188, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149439: Sun Nov 16 18:40:05 2025 read: IOPS=0, BW=461KiB/s (472kB/s)(2048KiB/4441msec) clat (msec): min=224, max=543, avg=383.98, stdev=224.91 lat (msec): min=224, max=543, avg=383.98, stdev=224.90 clat percentiles (msec): | 1.00th=[ 226], 5.00th=[ 226], 10.00th=[ 226], 20.00th=[ 226], | 30.00th=[ 226], 40.00th=[ 226], 50.00th=[ 226], 60.00th=[ 542], | 70.00th=[ 542], 80.00th=[ 542], 90.00th=[ 542], 95.00th=[ 542], | 99.00th=[ 542], 99.50th=[ 542], 99.90th=[ 542], 99.95th=[ 542], | 99.99th=[ 542] bw ( KiB/s): min= 1577, max= 2000, per=38.79%, avg=1788.50, stdev=299.11, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2306KiB/s (2361kB/s)(10.0MiB/4441msec); 0 zone resets clat (msec): min=115, max=883, avg=367.04, stdev=253.22 lat (msec): min=115, max=883, avg=367.13, stdev=253.23 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 184], | 30.00th=[ 205], 40.00th=[ 205], 50.00th=[ 259], 60.00th=[ 268], | 70.00th=[ 317], 80.00th=[ 550], 90.00th=[ 684], 95.00th=[ 885], | 99.00th=[ 885], 99.50th=[ 885], 99.90th=[ 885], 99.95th=[ 885], | 99.99th=[ 885] bw ( KiB/s): min= 1939, max= 6023, per=53.92%, avg=3481.20, stdev=2047.34, samples=5 iops : min= 1, max= 5, avg= 2.60, stdev= 2.19, samples=5 lat (msec) : 250=41.67%, 500=25.00%, 750=25.00%, 1000=8.33% cpu : usr=0.11%, sys=10.32%, ctx=140, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149440: Sun Nov 16 18:40:05 2025 read: IOPS=2, BW=2064KiB/s (2113kB/s)(6144KiB/2977msec) clat (usec): min=595, max=385239, avg=168418.87, stdev=166085.66 lat (usec): min=596, max=385241, avg=168419.94, stdev=166085.55 clat percentiles (usec): | 1.00th=[ 594], 5.00th=[ 594], 10.00th=[ 594], 20.00th=[ 611], | 30.00th=[ 611], 40.00th=[ 77071], 50.00th=[ 77071], 60.00th=[229639], | 70.00th=[316670], 80.00th=[316670], 90.00th=[383779], 95.00th=[383779], | 99.00th=[383779], 99.50th=[383779], 99.90th=[383779], 99.95th=[383779], | 99.99th=[383779] bw ( KiB/s): min= 1939, max= 6119, per=72.71%, avg=3352.67, stdev=2395.91, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 write: IOPS=2, BW=2064KiB/s (2113kB/s)(6144KiB/2977msec); 0 zone resets clat (msec): min=133, max=674, avg=327.01, stdev=209.49 lat (msec): min=133, max=674, avg=327.09, stdev=209.51 clat percentiles (msec): | 1.00th=[ 134], 5.00th=[ 134], 10.00th=[ 134], 20.00th=[ 163], | 30.00th=[ 163], 40.00th=[ 249], 50.00th=[ 249], 60.00th=[ 262], | 70.00th=[ 481], 80.00th=[ 481], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 1605, max= 4079, per=36.03%, avg=2326.00, stdev=993.81, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 lat (usec) : 750=16.67% lat (msec) : 100=8.33%, 250=33.33%, 500=33.33%, 750=8.33% cpu : usr=0.00%, sys=15.46%, ctx=128, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149441: Sun Nov 16 18:40:05 2025 read: IOPS=1, BW=1223KiB/s (1252kB/s)(5120KiB/4186msec) clat (usec): min=560, max=499651, avg=266137.09, stdev=180614.11 lat (usec): min=562, max=499652, avg=266138.40, stdev=180613.92 clat percentiles (usec): | 1.00th=[ 562], 5.00th=[ 562], 10.00th=[ 562], 20.00th=[ 562], | 30.00th=[223347], 40.00th=[223347], 50.00th=[283116], 60.00th=[283116], | 70.00th=[325059], 80.00th=[325059], 90.00th=[501220], 95.00th=[501220], | 99.00th=[501220], 99.50th=[501220], 99.90th=[501220], 99.95th=[501220], | 99.99th=[501220] bw ( KiB/s): min= 2000, max= 4015, per=57.99%, avg=2674.00, stdev=1161.35, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=1, BW=1712KiB/s (1753kB/s)(7168KiB/4186msec); 0 zone resets clat (msec): min=131, max=604, avg=406.74, stdev=199.08 lat (msec): min=131, max=604, avg=406.82, stdev=199.09 clat percentiles (msec): | 1.00th=[ 132], 5.00th=[ 132], 10.00th=[ 132], 20.00th=[ 222], | 30.00th=[ 253], 40.00th=[ 253], 50.00th=[ 481], 60.00th=[ 558], | 70.00th=[ 558], 80.00th=[ 600], 90.00th=[ 609], 95.00th=[ 609], | 99.00th=[ 609], 99.50th=[ 609], 99.90th=[ 609], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 1467, max= 4015, per=35.40%, avg=2285.60, stdev=992.50, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 lat (usec) : 750=8.33% lat (msec) : 250=25.00%, 500=41.67%, 750=25.00% cpu : usr=0.00%, sys=11.92%, ctx=137, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4612KiB/s (4722kB/s), 461KiB/s-2145KiB/s (472kB/s-2197kB/s), io=20.0MiB (20.0MB), run=2977-4441msec WRITE: bw=6456KiB/s (6611kB/s), 1532KiB/s-2306KiB/s (1569kB/s-2361kB/s), io=28.0MiB (29.4MB), run=2977-4441msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=149453: Sun Nov 16 18:40:16 2025 read: IOPS=2, BW=8393KiB/s (8595kB/s)(12.0MiB/1464msec) clat (msec): min=308, max=612, avg=487.72, stdev=159.21 lat (msec): min=308, max=612, avg=487.72, stdev=159.21 clat percentiles (msec): | 1.00th=[ 309], 5.00th=[ 309], 10.00th=[ 309], 20.00th=[ 309], | 30.00th=[ 309], 40.00th=[ 542], 50.00th=[ 542], 60.00th=[ 542], | 70.00th=[ 609], 80.00th=[ 609], 90.00th=[ 609], 95.00th=[ 609], | 99.00th=[ 609], 99.50th=[ 609], 99.90th=[ 609], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 6627, max= 6627, per=97.08%, avg=6627.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.00%, sys=10.53%, ctx=49, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149454: Sun Nov 16 18:40:16 2025 read: IOPS=0, BW=1707KiB/s (1748kB/s)(4096KiB/2400msec) clat (nsec): min=380571k, max=380571k, avg=380570711.00, stdev= 0.00 lat (nsec): min=380573k, max=380573k, avg=380573311.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 380], 5.00th=[ 380], 10.00th=[ 380], 20.00th=[ 380], | 30.00th=[ 380], 40.00th=[ 380], 50.00th=[ 380], 60.00th=[ 380], | 70.00th=[ 380], 80.00th=[ 380], 90.00th=[ 380], 95.00th=[ 380], | 99.00th=[ 380], 99.50th=[ 380], 99.90th=[ 380], 99.95th=[ 380], | 99.99th=[ 380] bw ( KiB/s): min= 6522, max= 6522, per=95.55%, avg=6522.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3413KiB/s (3495kB/s)(8192KiB/2400msec); 0 zone resets clat (msec): min=858, max=1149, avg=1003.91, stdev=205.82 lat (msec): min=858, max=1150, avg=1004.45, stdev=205.92 clat percentiles (msec): | 1.00th=[ 860], 5.00th=[ 860], 10.00th=[ 860], 20.00th=[ 860], | 30.00th=[ 860], 40.00th=[ 860], 50.00th=[ 860], 60.00th=[ 1150], | 70.00th=[ 1150], 80.00th=[ 1150], 90.00th=[ 1150], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min= 8126, max= 8126, per=59.52%, avg=8126.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=5.75%, ctx=36, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149455: Sun Nov 16 18:40:16 2025 write: IOPS=1, BW=6815KiB/s (6979kB/s)(12.0MiB/1803msec); 0 zone resets clat (msec): min=248, max=778, avg=594.05, stdev=299.24 lat (msec): min=249, max=796, avg=600.20, stdev=304.65 clat percentiles (msec): | 1.00th=[ 249], 5.00th=[ 249], 10.00th=[ 249], 20.00th=[ 249], | 30.00th=[ 249], 40.00th=[ 751], 50.00th=[ 751], 60.00th=[ 751], | 70.00th=[ 776], 80.00th=[ 776], 90.00th=[ 776], 95.00th=[ 776], | 99.00th=[ 776], 99.50th=[ 776], 99.90th=[ 776], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 8126, max= 8192, per=59.76%, avg=8159.00, stdev=46.67, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 1000=66.67% cpu : usr=0.00%, sys=7.99%, ctx=50, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149456: Sun Nov 16 18:40:16 2025 write: IOPS=1, BW=5606KiB/s (5740kB/s)(12.0MiB/2192msec); 0 zone resets clat (msec): min=234, max=1550, avg=729.54, stdev=715.58 lat (msec): min=235, max=1550, avg=730.18, stdev=715.34 clat percentiles (msec): | 1.00th=[ 234], 5.00th=[ 234], 10.00th=[ 234], 20.00th=[ 234], | 30.00th=[ 234], 40.00th=[ 405], 50.00th=[ 405], 60.00th=[ 405], | 70.00th=[ 1552], 80.00th=[ 1552], 90.00th=[ 1552], 95.00th=[ 1552], | 99.00th=[ 1552], 99.50th=[ 1552], 99.90th=[ 1552], 99.95th=[ 1552], | 99.99th=[ 1552] bw ( KiB/s): min= 6965, max= 8192, per=55.51%, avg=7578.50, stdev=867.62, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=8.85%, ctx=48, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6827KiB/s (6991kB/s), 1707KiB/s-8393KiB/s (1748kB/s-8595kB/s), io=16.0MiB (16.8MB), run=1464-2400msec WRITE: bw=13.3MiB/s (13.0MB/s), 3413KiB/s-6815KiB/s (3495kB/s-6979kB/s), io=32.0MiB (33.6MB), run=1803-2400msec rand-rw: (groupid=0, jobs=1): err= 0: pid=149448: Sun Nov 16 18:40:16 2025 read: IOPS=1, BW=6370KiB/s (6523kB/s)(12.0MiB/1929msec) clat (msec): min=569, max=613, avg=592.50, stdev=22.19 lat (msec): min=569, max=613, avg=592.51, stdev=22.19 clat percentiles (msec): | 1.00th=[ 567], 5.00th=[ 567], 10.00th=[ 567], 20.00th=[ 567], | 30.00th=[ 567], 40.00th=[ 592], 50.00th=[ 592], 60.00th=[ 592], | 70.00th=[ 617], 80.00th=[ 617], 90.00th=[ 617], 95.00th=[ 617], | 99.00th=[ 617], 99.50th=[ 617], 99.90th=[ 617], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 8094, max= 8126, per=100.00%, avg=8110.00, stdev=22.63, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=100.00% cpu : usr=0.00%, sys=26.40%, ctx=652, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149449: Sun Nov 16 18:40:16 2025 read: IOPS=0, BW=1720KiB/s (1761kB/s)(4096KiB/2382msec) clat (nsec): min=600683k, max=600683k, avg=600682757.00, stdev= 0.00 lat (nsec): min=600685k, max=600685k, avg=600685468.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 600], 5.00th=[ 600], 10.00th=[ 600], 20.00th=[ 600], | 30.00th=[ 600], 40.00th=[ 600], 50.00th=[ 600], 60.00th=[ 600], | 70.00th=[ 600], 80.00th=[ 600], 90.00th=[ 600], 95.00th=[ 600], | 99.00th=[ 600], 99.50th=[ 600], 99.90th=[ 600], 99.95th=[ 600], | 99.99th=[ 600] bw ( KiB/s): min= 8143, max= 8143, per=100.00%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3439KiB/s (3522kB/s)(8192KiB/2382msec); 0 zone resets clat (msec): min=592, max=1007, avg=799.72, stdev=293.57 lat (msec): min=592, max=1007, avg=800.16, stdev=293.57 clat percentiles (msec): | 1.00th=[ 592], 5.00th=[ 592], 10.00th=[ 592], 20.00th=[ 592], | 30.00th=[ 592], 40.00th=[ 592], 50.00th=[ 592], 60.00th=[ 1011], | 70.00th=[ 1011], 80.00th=[ 1011], 90.00th=[ 1011], 95.00th=[ 1011], | 99.00th=[ 1011], 99.50th=[ 1011], 99.90th=[ 1011], 99.95th=[ 1011], | 99.99th=[ 1011] bw ( KiB/s): min= 8159, max= 8159, per=74.68%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.04%, sys=9.58%, ctx=300, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149450: Sun Nov 16 18:40:16 2025 write: IOPS=1, BW=4097KiB/s (4196kB/s)(12.0MiB/2999msec); 0 zone resets clat (msec): min=215, max=1551, avg=933.53, stdev=673.48 lat (msec): min=216, max=1551, avg=933.86, stdev=673.44 clat percentiles (msec): | 1.00th=[ 215], 5.00th=[ 215], 10.00th=[ 215], 20.00th=[ 215], | 30.00th=[ 215], 40.00th=[ 1036], 50.00th=[ 1036], 60.00th=[ 1036], | 70.00th=[ 1552], 80.00th=[ 1552], 90.00th=[ 1552], 95.00th=[ 1552], | 99.00th=[ 1552], 99.50th=[ 1552], 99.90th=[ 1552], 99.95th=[ 1552], | 99.99th=[ 1552] bw ( KiB/s): min= 8078, max= 8078, per=73.93%, avg=8078.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=66.67% cpu : usr=0.00%, sys=10.47%, ctx=77, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149451: Sun Nov 16 18:40:16 2025 write: IOPS=1, BW=7979KiB/s (8171kB/s)(12.0MiB/1540msec); 0 zone resets clat (msec): min=168, max=950, avg=467.00, stdev=422.90 lat (msec): min=168, max=951, avg=467.51, stdev=423.09 clat percentiles (msec): | 1.00th=[ 169], 5.00th=[ 169], 10.00th=[ 169], 20.00th=[ 169], | 30.00th=[ 169], 40.00th=[ 284], 50.00th=[ 284], 60.00th=[ 284], | 70.00th=[ 953], 80.00th=[ 953], 90.00th=[ 953], 95.00th=[ 953], | 99.00th=[ 953], 99.50th=[ 953], 99.90th=[ 953], 99.95th=[ 953], | 99.99th=[ 953] bw ( KiB/s): min=11636, max=11636, per=100.00%, avg=11636.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 1000=33.33% cpu : usr=0.00%, sys=12.74%, ctx=61, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6878KiB/s (7043kB/s), 1720KiB/s-6370KiB/s (1761kB/s-6523kB/s), io=16.0MiB (16.8MB), run=1929-2382msec WRITE: bw=10.7MiB/s (11.2MB/s), 3439KiB/s-7979KiB/s (3522kB/s-8171kB/s), io=32.0MiB (33.6MB), run=1540-2999msec