== sanity test 398b: DIO and buffer IO race ============== 20:31:55 (1763083915) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.96342 s, 17.0 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148247: Thu Nov 13 20:33:00 2025 read: IOPS=27, BW=110KiB/s (113kB/s)(5992KiB/54412msec) clat (usec): min=106, max=685786, avg=18260.30, stdev=22035.06 lat (usec): min=106, max=685787, avg=18261.47, stdev=22035.04 clat percentiles (usec): | 1.00th=[ 141], 5.00th=[ 310], 10.00th=[ 5538], 20.00th=[ 9372], | 30.00th=[ 11076], 40.00th=[ 12780], 50.00th=[ 14746], 60.00th=[ 17433], | 70.00th=[ 21103], 80.00th=[ 26084], 90.00th=[ 31851], 95.00th=[ 38536], | 99.00th=[ 76022], 99.50th=[ 90702], 99.90th=[173016], 99.95th=[683672], | 99.99th=[683672] bw ( KiB/s): min= 6, max= 192, per=25.51%, avg=109.96, stdev=31.95, samples=108 iops : min= 1, max= 48, avg=27.00, stdev= 8.06, samples=108 write: IOPS=28, BW=116KiB/s (118kB/s)(6296KiB/54412msec); 0 zone resets clat (usec): min=860, max=193574, avg=17051.56, stdev=15916.56 lat (usec): min=861, max=193575, avg=17052.91, stdev=15916.61 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1254], 10.00th=[ 1713], 20.00th=[ 2704], | 30.00th=[ 4424], 40.00th=[ 9110], 50.00th=[ 14091], 60.00th=[ 17957], | 70.00th=[ 23725], 80.00th=[ 28443], 90.00th=[ 38011], 95.00th=[ 46924], | 99.00th=[ 63177], 99.50th=[ 73925], 99.90th=[125305], 99.95th=[193987], | 99.99th=[193987] bw ( KiB/s): min= 23, max= 208, per=25.80%, avg=116.64, stdev=34.46, samples=107 iops : min= 5, max= 52, avg=28.67, stdev= 8.65, samples=107 lat (usec) : 250=1.79%, 500=2.08%, 750=0.23%, 1000=0.46% lat (msec) : 2=6.45%, 4=8.14%, 10=13.48%, 20=32.94%, 50=31.67% lat (msec) : 100=2.54%, 250=0.20%, 750=0.03% cpu : usr=0.15%, sys=15.97%, ctx=7402, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148248: Thu Nov 13 20:33:00 2025 read: IOPS=26, BW=108KiB/s (110kB/s)(5940KiB/55092msec) clat (usec): min=76, max=281954, avg=17199.31, stdev=13565.34 lat (usec): min=76, max=281955, avg=17201.05, stdev=13565.17 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 255], 10.00th=[ 4948], 20.00th=[ 9503], | 30.00th=[ 11207], 40.00th=[ 13042], 50.00th=[ 15008], 60.00th=[ 17433], | 70.00th=[ 20841], 80.00th=[ 24249], 90.00th=[ 30802], 95.00th=[ 36439], | 99.00th=[ 49021], 99.50th=[ 62653], 99.90th=[173016], 99.95th=[283116], | 99.99th=[283116] bw ( KiB/s): min= 6, max= 192, per=24.88%, avg=107.23, stdev=34.49, samples=109 iops : min= 1, max= 48, avg=26.32, stdev= 8.73, samples=109 write: IOPS=28, BW=115KiB/s (118kB/s)(6348KiB/55092msec); 0 zone resets clat (usec): min=882, max=715905, avg=18470.27, stdev=26198.67 lat (usec): min=883, max=715907, avg=18473.18, stdev=26198.35 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 1303], 10.00th=[ 1958], 20.00th=[ 3064], | 30.00th=[ 5342], 40.00th=[ 10421], 50.00th=[ 14484], 60.00th=[ 19268], | 70.00th=[ 23200], 80.00th=[ 28967], 90.00th=[ 38011], 95.00th=[ 45876], | 99.00th=[ 79168], 99.50th=[ 89654], 99.90th=[354419], 99.95th=[717226], | 99.99th=[717226] bw ( KiB/s): min= 6, max= 216, per=25.26%, avg=114.18, stdev=37.43, samples=109 iops : min= 1, max= 54, avg=28.06, stdev= 9.40, samples=109 lat (usec) : 100=0.10%, 250=2.25%, 500=1.86%, 750=0.26%, 1000=0.23% lat (msec) : 2=5.53%, 4=7.78%, 10=13.35%, 20=33.89%, 50=32.45% lat (msec) : 100=2.02%, 250=0.16%, 500=0.10%, 750=0.03% cpu : usr=0.13%, sys=16.16%, ctx=7537, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148249: Thu Nov 13 20:33:00 2025 read: IOPS=27, BW=108KiB/s (111kB/s)(6028KiB/55584msec) clat (usec): min=102, max=208981, avg=17023.46, stdev=12822.64 lat (usec): min=103, max=208982, avg=17024.58, stdev=12822.62 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 277], 10.00th=[ 2024], 20.00th=[ 9372], | 30.00th=[ 11076], 40.00th=[ 12518], 50.00th=[ 14484], 60.00th=[ 17171], | 70.00th=[ 20317], 80.00th=[ 24511], 90.00th=[ 31589], 95.00th=[ 37487], | 99.00th=[ 49546], 99.50th=[ 70779], 99.90th=[143655], 99.95th=[208667], | 99.99th=[208667] bw ( KiB/s): min= 23, max= 216, per=25.45%, avg=109.69, stdev=37.25, samples=108 iops : min= 5, max= 54, avg=26.94, stdev= 9.38, samples=108 write: IOPS=28, BW=113KiB/s (115kB/s)(6260KiB/55584msec); 0 zone resets clat (usec): min=715, max=794035, avg=18988.25, stdev=25914.21 lat (usec): min=716, max=794037, avg=18989.95, stdev=25914.34 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 1254], 10.00th=[ 1876], 20.00th=[ 2999], | 30.00th=[ 6128], 40.00th=[ 11600], 50.00th=[ 15139], 60.00th=[ 19006], | 70.00th=[ 23987], 80.00th=[ 30278], 90.00th=[ 40633], 95.00th=[ 49546], | 99.00th=[ 72877], 99.50th=[ 99091], 99.90th=[156238], 99.95th=[792724], | 99.99th=[792724] bw ( KiB/s): min= 15, max= 215, per=24.92%, avg=112.65, stdev=30.49, samples=109 iops : min= 3, max= 53, avg=27.68, stdev= 7.64, samples=109 lat (usec) : 250=1.99%, 500=2.51%, 750=0.23%, 1000=0.49% lat (msec) : 2=5.37%, 4=7.19%, 10=12.66%, 20=35.16%, 50=31.41% lat (msec) : 100=2.70%, 250=0.26%, 1000=0.03% cpu : usr=0.13%, sys=15.78%, ctx=7483, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148250: Thu Nov 13 20:33:00 2025 read: IOPS=27, BW=111KiB/s (114kB/s)(6052KiB/54582msec) clat (usec): min=97, max=721080, avg=17780.34, stdev=22036.36 lat (usec): min=98, max=721081, avg=17781.45, stdev=22036.35 clat percentiles (usec): | 1.00th=[ 129], 5.00th=[ 247], 10.00th=[ 1172], 20.00th=[ 9241], | 30.00th=[ 10945], 40.00th=[ 12780], 50.00th=[ 14746], 60.00th=[ 17695], | 70.00th=[ 21103], 80.00th=[ 25035], 90.00th=[ 31589], 95.00th=[ 37487], | 99.00th=[ 58459], 99.50th=[ 71828], 99.90th=[164627], 99.95th=[717226], | 99.99th=[717226] bw ( KiB/s): min= 7, max= 184, per=25.83%, avg=111.35, stdev=32.61, samples=107 iops : min= 1, max= 46, avg=27.35, stdev= 8.19, samples=107 write: IOPS=28, BW=114KiB/s (117kB/s)(6236KiB/54582msec); 0 zone resets clat (usec): min=858, max=318798, avg=17607.57, stdev=19308.89 lat (usec): min=860, max=318799, avg=17610.90, stdev=19309.23 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1237], 10.00th=[ 1795], 20.00th=[ 2769], | 30.00th=[ 4686], 40.00th=[ 9634], 50.00th=[ 14091], 60.00th=[ 17957], | 70.00th=[ 22676], 80.00th=[ 29230], 90.00th=[ 38536], 95.00th=[ 47449], | 99.00th=[ 68682], 99.50th=[ 81265], 99.90th=[312476], 99.95th=[320865], | 99.99th=[320865] bw ( KiB/s): min= 26, max= 200, per=25.16%, avg=113.71, stdev=33.72, samples=108 iops : min= 6, max= 50, avg=27.94, stdev= 8.48, samples=108 lat (usec) : 100=0.07%, 250=2.41%, 500=2.15%, 750=0.16%, 1000=0.52% lat (msec) : 2=5.92%, 4=8.07%, 10=12.96%, 20=34.11%, 50=30.99% lat (msec) : 100=2.38%, 250=0.16%, 500=0.07%, 750=0.03% cpu : usr=0.13%, sys=16.00%, ctx=7479, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=432KiB/s (442kB/s), 108KiB/s-111KiB/s (110kB/s-114kB/s), io=23.4MiB (24.6MB), run=54412-55584msec WRITE: bw=452KiB/s (463kB/s), 113KiB/s-116KiB/s (115kB/s-118kB/s), io=24.6MiB (25.7MB), run=54412-55584msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148243: Thu Nov 13 20:34:05 2025 read: IOPS=12, BW=49.9KiB/s (51.1kB/s)(5992KiB/119996msec) clat (msec): min=3, max=691, avg=15.49, stdev=23.26 lat (msec): min=3, max=691, avg=15.49, stdev=23.26 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 10], 60.00th=[ 12], | 70.00th=[ 14], 80.00th=[ 20], 90.00th=[ 34], 95.00th=[ 43], | 99.00th=[ 67], 99.50th=[ 84], 99.90th=[ 284], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 7, max= 143, per=25.18%, avg=50.36, stdev=28.16, samples=233 iops : min= 1, max= 35, avg=12.08, stdev= 7.11, samples=233 write: IOPS=13, BW=52.5KiB/s (53.7kB/s)(6296KiB/119996msec); 0 zone resets clat (msec): min=20, max=319, avg=61.34, stdev=26.01 lat (msec): min=20, max=319, avg=61.34, stdev=26.01 clat percentiles (msec): | 1.00th=[ 27], 5.00th=[ 32], 10.00th=[ 35], 20.00th=[ 41], | 30.00th=[ 46], 40.00th=[ 52], 50.00th=[ 57], 60.00th=[ 63], | 70.00th=[ 70], 80.00th=[ 80], 90.00th=[ 95], 95.00th=[ 107], | 99.00th=[ 146], 99.50th=[ 171], 99.90th=[ 232], 99.95th=[ 321], | 99.99th=[ 321] bw ( KiB/s): min= 8, max= 104, per=24.93%, avg=52.10, stdev=19.93, samples=237 iops : min= 2, max= 26, avg=12.51, stdev= 5.01, samples=237 lat (msec) : 4=0.10%, 10=24.51%, 20=14.75%, 50=27.70%, 100=28.81% lat (msec) : 250=4.04%, 500=0.07%, 750=0.03% cpu : usr=0.07%, sys=2.96%, ctx=5458, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148244: Thu Nov 13 20:34:05 2025 read: IOPS=12, BW=49.5KiB/s (50.7kB/s)(5940KiB/119996msec) clat (msec): min=3, max=724, avg=15.55, stdev=23.35 lat (msec): min=3, max=724, avg=15.56, stdev=23.35 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 12], | 70.00th=[ 14], 80.00th=[ 20], 90.00th=[ 34], 95.00th=[ 45], | 99.00th=[ 70], 99.50th=[ 81], 99.90th=[ 209], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 7, max= 152, per=24.90%, avg=49.79, stdev=29.32, samples=234 iops : min= 1, max= 38, avg=11.94, stdev= 7.37, samples=234 write: IOPS=13, BW=52.9KiB/s (54.2kB/s)(6348KiB/119996msec); 0 zone resets clat (msec): min=21, max=381, avg=60.90, stdev=27.07 lat (msec): min=21, max=381, avg=60.91, stdev=27.07 clat percentiles (msec): | 1.00th=[ 27], 5.00th=[ 32], 10.00th=[ 35], 20.00th=[ 41], | 30.00th=[ 46], 40.00th=[ 51], 50.00th=[ 56], 60.00th=[ 62], | 70.00th=[ 68], 80.00th=[ 79], 90.00th=[ 94], 95.00th=[ 108], | 99.00th=[ 144], 99.50th=[ 157], 99.90th=[ 347], 99.95th=[ 380], | 99.99th=[ 380] bw ( KiB/s): min= 7, max= 96, per=25.11%, avg=52.47, stdev=19.90, samples=237 iops : min= 1, max= 24, avg=12.61, stdev= 5.03, samples=237 lat (msec) : 4=0.03%, 10=24.12%, 20=14.84%, 50=27.73%, 100=29.30% lat (msec) : 250=3.84%, 500=0.10%, 750=0.03% cpu : usr=0.07%, sys=3.01%, ctx=5524, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148245: Thu Nov 13 20:34:05 2025 read: IOPS=12, BW=50.5KiB/s (51.7kB/s)(6028KiB/119422msec) clat (msec): min=3, max=212, avg=15.19, stdev=14.18 lat (msec): min=3, max=212, avg=15.19, stdev=14.18 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 12], | 70.00th=[ 15], 80.00th=[ 21], 90.00th=[ 33], 95.00th=[ 44], | 99.00th=[ 66], 99.50th=[ 77], 99.90th=[ 165], 99.95th=[ 213], | 99.99th=[ 213] bw ( KiB/s): min= 7, max= 167, per=25.16%, avg=50.33, stdev=29.94, samples=235 iops : min= 1, max= 41, avg=12.07, stdev= 7.49, samples=235 write: IOPS=13, BW=52.4KiB/s (53.7kB/s)(6260KiB/119422msec); 0 zone resets clat (msec): min=19, max=784, avg=61.52, stdev=31.79 lat (msec): min=19, max=784, avg=61.53, stdev=31.79 clat percentiles (msec): | 1.00th=[ 28], 5.00th=[ 32], 10.00th=[ 36], 20.00th=[ 41], | 30.00th=[ 46], 40.00th=[ 51], 50.00th=[ 56], 60.00th=[ 62], | 70.00th=[ 69], 80.00th=[ 79], 90.00th=[ 95], 95.00th=[ 110], | 99.00th=[ 148], 99.50th=[ 169], 99.90th=[ 342], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 15, max= 104, per=24.87%, avg=51.99, stdev=19.90, samples=236 iops : min= 3, max= 26, avg=12.49, stdev= 5.03, samples=236 lat (msec) : 4=0.03%, 10=24.41%, 20=14.68%, 50=28.97%, 100=27.80% lat (msec) : 250=4.04%, 500=0.03%, 1000=0.03% cpu : usr=0.07%, sys=3.14%, ctx=5481, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148246: Thu Nov 13 20:34:05 2025 read: IOPS=12, BW=50.8KiB/s (51.0kB/s)(6052KiB/119236msec) clat (msec): min=3, max=205, avg=14.68, stdev=13.85 lat (msec): min=3, max=205, avg=14.68, stdev=13.85 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 10], 60.00th=[ 12], | 70.00th=[ 14], 80.00th=[ 20], 90.00th=[ 31], 95.00th=[ 42], | 99.00th=[ 61], 99.50th=[ 74], 99.90th=[ 165], 99.95th=[ 207], | 99.99th=[ 207] bw ( KiB/s): min= 7, max= 159, per=25.89%, avg=51.78, stdev=29.10, samples=230 iops : min= 1, max= 39, avg=12.43, stdev= 7.33, samples=230 write: IOPS=13, BW=52.3KiB/s (53.6kB/s)(6236KiB/119236msec); 0 zone resets clat (msec): min=22, max=789, avg=62.07, stdev=33.03 lat (msec): min=22, max=789, avg=62.07, stdev=33.03 clat percentiles (msec): | 1.00th=[ 27], 5.00th=[ 32], 10.00th=[ 35], 20.00th=[ 41], | 30.00th=[ 47], 40.00th=[ 51], 50.00th=[ 56], 60.00th=[ 61], | 70.00th=[ 69], 80.00th=[ 80], 90.00th=[ 96], 95.00th=[ 112], | 99.00th=[ 140], 99.50th=[ 159], 99.90th=[ 384], 99.95th=[ 793], | 99.99th=[ 793] bw ( KiB/s): min= 7, max= 104, per=24.86%, avg=51.96, stdev=19.40, samples=236 iops : min= 1, max= 26, avg=12.48, stdev= 4.91, samples=236 lat (msec) : 4=0.03%, 10=25.03%, 20=15.01%, 50=28.06%, 100=27.18% lat (msec) : 250=4.56%, 500=0.10%, 1000=0.03% cpu : usr=0.07%, sys=3.05%, ctx=5486, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=200KiB/s (205kB/s), 49.5KiB/s-50.8KiB/s (50.7kB/s-51.0kB/s), io=23.4MiB (24.6MB), run=119236-119996msec WRITE: bw=210KiB/s (215kB/s), 52.3KiB/s-52.9KiB/s (53.6kB/s-54.2kB/s), io=24.6MiB (25.7MB), run=119236-119996msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148283: Thu Nov 13 20:34:22 2025 read: IOPS=31, BW=504KiB/s (516kB/s)(5920KiB/11757msec) clat (usec): min=124, max=120087, avg=15466.42, stdev=13050.89 lat (usec): min=125, max=120089, avg=15467.37, stdev=13050.90 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 174], 10.00th=[ 314], 20.00th=[ 529], | 30.00th=[ 10421], 40.00th=[ 12780], 50.00th=[ 14746], 60.00th=[ 16712], | 70.00th=[ 19006], 80.00th=[ 22938], 90.00th=[ 29230], 95.00th=[ 33817], | 99.00th=[ 60031], 99.50th=[ 87557], 99.90th=[120062], 99.95th=[120062], | 99.99th=[120062] bw ( KiB/s): min= 222, max= 734, per=26.17%, avg=500.65, stdev=131.74, samples=23 iops : min= 13, max= 45, avg=30.61, stdev= 8.26, samples=23 write: IOPS=33, BW=542KiB/s (555kB/s)(6368KiB/11757msec); 0 zone resets clat (usec): min=1298, max=83583, avg=14938.93, stdev=14577.26 lat (usec): min=1299, max=83662, avg=14940.97, stdev=14578.22 clat percentiles (usec): | 1.00th=[ 1532], 5.00th=[ 2180], 10.00th=[ 2868], 20.00th=[ 3752], | 30.00th=[ 4948], 40.00th=[ 6521], 50.00th=[10028], 60.00th=[13829], | 70.00th=[18482], 80.00th=[24249], 90.00th=[34866], 95.00th=[43254], | 99.00th=[71828], 99.50th=[79168], 99.90th=[83362], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 315, max= 800, per=26.94%, avg=542.39, stdev=129.23, samples=23 iops : min= 19, max= 50, avg=33.22, stdev= 8.17, samples=23 lat (usec) : 250=2.99%, 500=6.51%, 750=0.65%, 1000=0.13% lat (msec) : 2=2.47%, 4=9.51%, 10=16.93%, 20=34.51%, 50=23.83% lat (msec) : 100=2.34%, 250=0.13% cpu : usr=0.14%, sys=21.93%, ctx=1801, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148284: Thu Nov 13 20:34:22 2025 read: IOPS=28, BW=459KiB/s (470kB/s)(5744KiB/12517msec) clat (usec): min=117, max=105255, avg=17909.19, stdev=13121.33 lat (usec): min=118, max=105256, avg=17910.21, stdev=13121.30 clat percentiles (usec): | 1.00th=[ 157], 5.00th=[ 262], 10.00th=[ 359], 20.00th=[ 9634], | 30.00th=[ 11994], 40.00th=[ 14091], 50.00th=[ 16450], 60.00th=[ 18744], | 70.00th=[ 21627], 80.00th=[ 24773], 90.00th=[ 32637], 95.00th=[ 37487], | 99.00th=[ 63177], 99.50th=[ 91751], 99.90th=[105382], 99.95th=[105382], | 99.99th=[105382] bw ( KiB/s): min= 52, max= 736, per=23.91%, avg=457.38, stdev=132.49, samples=24 iops : min= 3, max= 46, avg=27.92, stdev= 8.36, samples=24 write: IOPS=32, BW=523KiB/s (535kB/s)(6544KiB/12517msec); 0 zone resets clat (usec): min=1279, max=98784, avg=14781.18, stdev=14254.56 lat (usec): min=1280, max=98786, avg=14783.07, stdev=14254.68 clat percentiles (usec): | 1.00th=[ 1401], 5.00th=[ 1827], 10.00th=[ 2540], 20.00th=[ 3490], | 30.00th=[ 4621], 40.00th=[ 6456], 50.00th=[11600], 60.00th=[15139], | 70.00th=[18482], 80.00th=[23462], 90.00th=[32900], 95.00th=[39060], | 99.00th=[69731], 99.50th=[78119], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 190, max= 698, per=25.26%, avg=508.46, stdev=127.58, samples=24 iops : min= 11, max= 43, avg=31.12, stdev= 8.10, samples=24 lat (usec) : 250=2.21%, 500=4.04%, 750=0.13% lat (msec) : 2=3.26%, 4=10.29%, 10=14.45%, 20=34.90%, 50=28.26% lat (msec) : 100=2.34%, 250=0.13% cpu : usr=0.13%, sys=21.69%, ctx=1951, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148285: Thu Nov 13 20:34:22 2025 read: IOPS=32, BW=514KiB/s (526kB/s)(6256KiB/12175msec) clat (usec): min=114, max=147757, avg=17007.30, stdev=16074.57 lat (usec): min=115, max=147758, avg=17008.40, stdev=16074.55 clat percentiles (usec): | 1.00th=[ 143], 5.00th=[ 245], 10.00th=[ 310], 20.00th=[ 717], | 30.00th=[ 11207], 40.00th=[ 13042], 50.00th=[ 15139], 60.00th=[ 17695], | 70.00th=[ 21103], 80.00th=[ 24249], 90.00th=[ 29754], 95.00th=[ 38011], | 99.00th=[ 88605], 99.50th=[121111], 99.90th=[147850], 99.95th=[147850], | 99.99th=[147850] bw ( KiB/s): min= 319, max= 862, per=26.93%, avg=515.08, stdev=121.82, samples=24 iops : min= 19, max= 53, avg=31.54, stdev= 7.55, samples=24 write: IOPS=30, BW=495KiB/s (507kB/s)(6032KiB/12175msec); 0 zone resets clat (usec): min=1090, max=68717, avg=14378.30, stdev=11985.29 lat (usec): min=1092, max=68719, avg=14380.85, stdev=11986.44 clat percentiles (usec): | 1.00th=[ 1532], 5.00th=[ 2278], 10.00th=[ 2966], 20.00th=[ 4146], | 30.00th=[ 5211], 40.00th=[ 7046], 50.00th=[10945], 60.00th=[15533], | 70.00th=[18482], 80.00th=[22938], 90.00th=[31851], 95.00th=[38536], | 99.00th=[53216], 99.50th=[60031], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 286, max= 702, per=24.53%, avg=493.83, stdev=123.64, samples=24 iops : min= 17, max= 43, avg=30.21, stdev= 7.87, samples=24 lat (usec) : 250=2.86%, 500=6.38%, 750=1.04%, 1000=0.13% lat (msec) : 2=1.95%, 4=7.03%, 10=16.28%, 20=34.90%, 50=27.34% lat (msec) : 100=1.82%, 250=0.26% cpu : usr=0.08%, sys=21.71%, ctx=1908, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148286: Thu Nov 13 20:34:22 2025 read: IOPS=30, BW=492KiB/s (504kB/s)(6032KiB/12253msec) clat (usec): min=117, max=98900, avg=16294.43, stdev=13300.65 lat (usec): min=118, max=98901, avg=16295.82, stdev=13300.76 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 180], 10.00th=[ 318], 20.00th=[ 693], | 30.00th=[10421], 40.00th=[12518], 50.00th=[15008], 60.00th=[17171], | 70.00th=[20841], 80.00th=[23987], 90.00th=[30802], 95.00th=[40109], | 99.00th=[62653], 99.50th=[82314], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 211, max= 731, per=25.63%, avg=490.29, stdev=125.60, samples=24 iops : min= 13, max= 45, avg=30.00, stdev= 7.87, samples=24 write: IOPS=31, BW=511KiB/s (523kB/s)(6256KiB/12253msec); 0 zone resets clat (usec): min=1300, max=113646, avg=15449.09, stdev=14159.23 lat (usec): min=1301, max=113648, avg=15450.82, stdev=14159.19 clat percentiles (usec): | 1.00th=[ 1516], 5.00th=[ 2040], 10.00th=[ 2704], 20.00th=[ 4047], | 30.00th=[ 4948], 40.00th=[ 7701], 50.00th=[ 11076], 60.00th=[ 15139], | 70.00th=[ 20317], 80.00th=[ 25822], 90.00th=[ 34341], 95.00th=[ 41681], | 99.00th=[ 62129], 99.50th=[ 69731], 99.90th=[113771], 99.95th=[113771], | 99.99th=[113771] bw ( KiB/s): min= 290, max= 798, per=25.38%, avg=511.00, stdev=134.10, samples=24 iops : min= 18, max= 49, avg=31.29, stdev= 8.43, samples=24 lat (usec) : 250=3.12%, 500=5.86%, 750=0.91% lat (msec) : 2=2.60%, 4=7.68%, 10=17.45%, 20=30.86%, 50=29.30% lat (msec) : 100=2.08%, 250=0.13% cpu : usr=0.11%, sys=21.60%, ctx=1891, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1914KiB/s (1959kB/s), 459KiB/s-514KiB/s (470kB/s-526kB/s), io=23.4MiB (24.5MB), run=11757-12517msec WRITE: bw=2013KiB/s (2062kB/s), 495KiB/s-542KiB/s (507kB/s-555kB/s), io=24.6MiB (25.8MB), run=11757-12517msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148278: Thu Nov 13 20:34:38 2025 read: IOPS=13, BW=216KiB/s (221kB/s)(5920KiB/27459msec) clat (usec): min=4254, max=92204, avg=15186.06, stdev=13540.81 lat (usec): min=4255, max=92205, avg=15187.71, stdev=13540.45 clat percentiles (usec): | 1.00th=[ 4883], 5.00th=[ 5800], 10.00th=[ 6259], 20.00th=[ 6915], | 30.00th=[ 7767], 40.00th=[ 8586], 50.00th=[ 9765], 60.00th=[11469], | 70.00th=[14091], 80.00th=[21365], 90.00th=[30802], 95.00th=[42206], | 99.00th=[65274], 99.50th=[90702], 99.90th=[91751], 99.95th=[91751], | 99.99th=[91751] bw ( KiB/s): min= 63, max= 544, per=24.82%, avg=213.98, stdev=96.60, samples=54 iops : min= 3, max= 34, avg=12.81, stdev= 6.12, samples=54 write: IOPS=14, BW=232KiB/s (237kB/s)(6368KiB/27459msec); 0 zone resets clat (msec): min=22, max=166, avg=54.74, stdev=19.80 lat (msec): min=22, max=166, avg=54.74, stdev=19.80 clat percentiles (msec): | 1.00th=[ 26], 5.00th=[ 32], 10.00th=[ 35], 20.00th=[ 40], | 30.00th=[ 43], 40.00th=[ 47], 50.00th=[ 52], 60.00th=[ 55], | 70.00th=[ 61], 80.00th=[ 66], 90.00th=[ 83], 95.00th=[ 94], | 99.00th=[ 129], 99.50th=[ 144], 99.90th=[ 167], 99.95th=[ 167], | 99.99th=[ 167] bw ( KiB/s): min= 95, max= 384, per=25.53%, avg=231.56, stdev=76.22, samples=54 iops : min= 5, max= 24, avg=13.91, stdev= 4.81, samples=54 lat (msec) : 10=25.26%, 20=12.89%, 50=32.68%, 100=27.21%, 250=1.95% cpu : usr=0.07%, sys=4.30%, ctx=1397, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148279: Thu Nov 13 20:34:38 2025 read: IOPS=12, BW=207KiB/s (212kB/s)(5744KiB/27775msec) clat (msec): min=4, max=142, avg=14.90, stdev=14.93 lat (msec): min=4, max=142, avg=14.90, stdev=14.93 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 10], 60.00th=[ 11], | 70.00th=[ 14], 80.00th=[ 20], 90.00th=[ 33], 95.00th=[ 45], | 99.00th=[ 75], 99.50th=[ 88], 99.90th=[ 142], 99.95th=[ 142], | 99.99th=[ 142] bw ( KiB/s): min= 31, max= 416, per=23.99%, avg=206.80, stdev=97.68, samples=55 iops : min= 1, max= 26, avg=12.36, stdev= 6.16, samples=55 write: IOPS=14, BW=236KiB/s (241kB/s)(6544KiB/27775msec); 0 zone resets clat (msec): min=25, max=248, avg=54.68, stdev=22.63 lat (msec): min=25, max=248, avg=54.68, stdev=22.63 clat percentiles (msec): | 1.00th=[ 27], 5.00th=[ 32], 10.00th=[ 35], 20.00th=[ 39], | 30.00th=[ 41], 40.00th=[ 45], 50.00th=[ 50], 60.00th=[ 54], | 70.00th=[ 62], 80.00th=[ 69], 90.00th=[ 83], 95.00th=[ 101], | 99.00th=[ 125], 99.50th=[ 129], 99.90th=[ 249], 99.95th=[ 249], | 99.99th=[ 249] bw ( KiB/s): min= 95, max= 352, per=25.93%, avg=235.16, stdev=79.66, samples=55 iops : min= 5, max= 22, avg=14.13, stdev= 5.00, samples=55 lat (msec) : 10=26.17%, 20=11.98%, 50=35.29%, 100=23.70%, 250=2.86% cpu : usr=0.08%, sys=4.27%, ctx=1495, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148280: Thu Nov 13 20:34:38 2025 read: IOPS=14, BW=231KiB/s (237kB/s)(6256KiB/27072msec) clat (msec): min=3, max=142, avg=14.57, stdev=13.73 lat (msec): min=3, max=142, avg=14.58, stdev=13.73 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 10], 60.00th=[ 11], | 70.00th=[ 13], 80.00th=[ 18], 90.00th=[ 31], 95.00th=[ 41], | 99.00th=[ 68], 99.50th=[ 75], 99.90th=[ 144], 99.95th=[ 144], | 99.99th=[ 144] bw ( KiB/s): min= 31, max= 576, per=27.16%, avg=234.15, stdev=139.48, samples=53 iops : min= 1, max= 36, avg=14.06, stdev= 8.81, samples=53 write: IOPS=13, BW=223KiB/s (228kB/s)(6032KiB/27072msec); 0 zone resets clat (msec): min=26, max=249, avg=56.49, stdev=24.14 lat (msec): min=26, max=249, avg=56.49, stdev=24.14 clat percentiles (msec): | 1.00th=[ 29], 5.00th=[ 32], 10.00th=[ 34], 20.00th=[ 39], | 30.00th=[ 42], 40.00th=[ 47], 50.00th=[ 51], 60.00th=[ 57], | 70.00th=[ 63], 80.00th=[ 72], 90.00th=[ 83], 95.00th=[ 104], | 99.00th=[ 134], 99.50th=[ 163], 99.90th=[ 251], 99.95th=[ 251], | 99.99th=[ 251] bw ( KiB/s): min= 63, max= 351, per=24.47%, avg=221.98, stdev=74.83, samples=53 iops : min= 3, max= 21, avg=13.30, stdev= 4.70, samples=53 lat (msec) : 4=0.13%, 10=26.69%, 20=14.84%, 50=30.99%, 100=24.35% lat (msec) : 250=2.99% cpu : usr=0.07%, sys=4.33%, ctx=1475, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148281: Thu Nov 13 20:34:38 2025 read: IOPS=13, BW=221KiB/s (226kB/s)(6032KiB/27284msec) clat (msec): min=4, max=110, avg=15.28, stdev=14.57 lat (msec): min=4, max=110, avg=15.28, stdev=14.57 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 10], 60.00th=[ 11], | 70.00th=[ 14], 80.00th=[ 21], 90.00th=[ 34], 95.00th=[ 54], | 99.00th=[ 69], 99.50th=[ 73], 99.90th=[ 110], 99.95th=[ 110], | 99.99th=[ 110] bw ( KiB/s): min= 32, max= 478, per=25.48%, avg=219.61, stdev=113.03, samples=54 iops : min= 2, max= 29, avg=13.17, stdev= 6.98, samples=54 write: IOPS=14, BW=229KiB/s (235kB/s)(6256KiB/27284msec); 0 zone resets clat (msec): min=24, max=152, avg=54.96, stdev=19.34 lat (msec): min=24, max=152, avg=54.96, stdev=19.34 clat percentiles (msec): | 1.00th=[ 28], 5.00th=[ 32], 10.00th=[ 35], 20.00th=[ 40], | 30.00th=[ 43], 40.00th=[ 48], 50.00th=[ 51], 60.00th=[ 55], | 70.00th=[ 63], 80.00th=[ 69], 90.00th=[ 80], 95.00th=[ 92], | 99.00th=[ 121], 99.50th=[ 128], 99.90th=[ 153], 99.95th=[ 153], | 99.99th=[ 153] bw ( KiB/s): min= 63, max= 351, per=25.22%, avg=228.78, stdev=72.26, samples=54 iops : min= 3, max= 21, avg=13.74, stdev= 4.56, samples=54 lat (msec) : 10=25.52%, 20=13.80%, 50=31.90%, 100=26.82%, 250=1.95% cpu : usr=0.06%, sys=4.39%, ctx=1500, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=862KiB/s (883kB/s), 207KiB/s-231KiB/s (212kB/s-237kB/s), io=23.4MiB (24.5MB), run=27072-27775msec WRITE: bw=907KiB/s (929kB/s), 223KiB/s-236KiB/s (228kB/s-241kB/s), io=24.6MiB (25.8MB), run=27072-27775msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148304: Thu Nov 13 20:34:46 2025 read: IOPS=3, BW=3323KiB/s (3403kB/s)(7168KiB/2157msec) clat (usec): min=882, max=700230, avg=210098.27, stdev=236309.88 lat (usec): min=883, max=700232, avg=210099.72, stdev=236309.99 clat percentiles (usec): | 1.00th=[ 881], 5.00th=[ 881], 10.00th=[ 881], 20.00th=[ 95945], | 30.00th=[101188], 40.00th=[101188], 50.00th=[117965], 60.00th=[135267], | 70.00th=[135267], 80.00th=[320865], 90.00th=[700449], 95.00th=[700449], | 99.00th=[700449], 99.50th=[700449], 99.90th=[700449], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 2043, max= 4096, per=46.09%, avg=2901.50, stdev=1026.34, samples=4 iops : min= 1, max= 4, avg= 2.50, stdev= 1.29, samples=4 write: IOPS=2, BW=2374KiB/s (2431kB/s)(5120KiB/2157msec); 0 zone resets clat (msec): min=74, max=191, avg=132.69, stdev=53.76 lat (msec): min=74, max=191, avg=132.75, stdev=53.74 clat percentiles (msec): | 1.00th=[ 75], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 75], | 30.00th=[ 104], 40.00th=[ 104], 50.00th=[ 105], 60.00th=[ 105], | 70.00th=[ 190], 80.00th=[ 190], 90.00th=[ 192], 95.00th=[ 192], | 99.00th=[ 192], 99.50th=[ 192], 99.90th=[ 192], 99.95th=[ 192], | 99.99th=[ 192] bw ( KiB/s): min= 2048, max= 4096, per=38.69%, avg=3410.33, stdev=1179.82, samples=3 iops : min= 2, max= 4, avg= 3.00, stdev= 1.00, samples=3 lat (usec) : 1000=8.33% lat (msec) : 100=16.67%, 250=58.33%, 500=8.33%, 750=8.33% cpu : usr=0.00%, sys=44.25%, ctx=788, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148305: Thu Nov 13 20:34:46 2025 read: IOPS=0, BW=630KiB/s (645kB/s)(2048KiB/3253msec) clat (msec): min=121, max=200, avg=160.81, stdev=55.46 lat (msec): min=121, max=200, avg=160.81, stdev=55.46 clat percentiles (msec): | 1.00th=[ 122], 5.00th=[ 122], 10.00th=[ 122], 20.00th=[ 122], | 30.00th=[ 122], 40.00th=[ 122], 50.00th=[ 122], 60.00th=[ 201], | 70.00th=[ 201], 80.00th=[ 201], 90.00th=[ 201], 95.00th=[ 201], | 99.00th=[ 201], 99.50th=[ 201], 99.90th=[ 201], 99.95th=[ 201], | 99.99th=[ 201] bw ( KiB/s): min= 1747, max= 2048, per=30.14%, avg=1897.50, stdev=212.84, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3148KiB/s (3223kB/s)(10.0MiB/3253msec); 0 zone resets clat (msec): min=74, max=1196, avg=279.09, stdev=351.49 lat (msec): min=74, max=1197, avg=279.18, stdev=351.48 clat percentiles (msec): | 1.00th=[ 74], 5.00th=[ 74], 10.00th=[ 74], 20.00th=[ 92], | 30.00th=[ 96], 40.00th=[ 116], 50.00th=[ 117], 60.00th=[ 163], | 70.00th=[ 182], 80.00th=[ 201], 90.00th=[ 558], 95.00th=[ 1200], | 99.00th=[ 1200], 99.50th=[ 1200], 99.90th=[ 1200], 99.95th=[ 1200], | 99.99th=[ 1200] bw ( KiB/s): min= 1747, max= 6144, per=41.12%, avg=3624.20, stdev=2303.42, samples=5 iops : min= 1, max= 6, avg= 3.00, stdev= 2.74, samples=5 lat (msec) : 100=25.00%, 250=58.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=31.37%, ctx=253, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148306: Thu Nov 13 20:34:46 2025 read: IOPS=1, BW=2036KiB/s (2085kB/s)(6144KiB/3018msec) clat (usec): min=898, max=206986, avg=136015.94, stdev=71383.40 lat (usec): min=899, max=206987, avg=136016.92, stdev=71383.16 clat percentiles (usec): | 1.00th=[ 898], 5.00th=[ 898], 10.00th=[ 898], 20.00th=[135267], | 30.00th=[135267], 40.00th=[135267], 50.00th=[135267], 60.00th=[162530], | 70.00th=[173016], 80.00th=[173016], 90.00th=[206570], 95.00th=[206570], | 99.00th=[206570], 99.50th=[206570], 99.90th=[206570], 99.95th=[206570], | 99.99th=[206570] bw ( KiB/s): min= 2048, max= 6131, per=64.96%, avg=4089.50, stdev=2887.12, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 write: IOPS=1, BW=2036KiB/s (2085kB/s)(6144KiB/3018msec); 0 zone resets clat (msec): min=132, max=1024, avg=353.20, stdev=335.39 lat (msec): min=132, max=1024, avg=353.27, stdev=335.39 clat percentiles (msec): | 1.00th=[ 133], 5.00th=[ 133], 10.00th=[ 133], 20.00th=[ 153], | 30.00th=[ 153], 40.00th=[ 234], 50.00th=[ 234], 60.00th=[ 275], | 70.00th=[ 300], 80.00th=[ 300], 90.00th=[ 1028], 95.00th=[ 1028], | 99.00th=[ 1028], 99.50th=[ 1028], 99.90th=[ 1028], 99.95th=[ 1028], | 99.99th=[ 1028] bw ( KiB/s): min= 2048, max= 4087, per=36.26%, avg=3196.00, stdev=1043.51, samples=3 iops : min= 2, max= 3, avg= 2.67, stdev= 0.58, samples=3 lat (usec) : 1000=8.33% lat (msec) : 250=66.67%, 500=16.67%, 2000=8.33% cpu : usr=0.00%, sys=35.17%, ctx=523, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148307: Thu Nov 13 20:34:46 2025 read: IOPS=1, BW=1779KiB/s (1822kB/s)(5120KiB/2878msec) clat (usec): min=528, max=140299, avg=41137.63, stdev=61727.92 lat (usec): min=528, max=140301, avg=41138.90, stdev=61728.22 clat percentiles (usec): | 1.00th=[ 529], 5.00th=[ 529], 10.00th=[ 529], 20.00th=[ 529], | 30.00th=[ 685], 40.00th=[ 685], 50.00th=[ 799], 60.00th=[ 799], | 70.00th=[ 63177], 80.00th=[ 63177], 90.00th=[139461], 95.00th=[139461], | 99.00th=[139461], 99.50th=[139461], 99.90th=[139461], 99.95th=[139461], | 99.99th=[139461] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=2, BW=2491KiB/s (2550kB/s)(7168KiB/2878msec); 0 zone resets clat (msec): min=139, max=841, avg=361.74, stdev=278.96 lat (msec): min=139, max=841, avg=361.81, stdev=278.96 clat percentiles (msec): | 1.00th=[ 140], 5.00th=[ 140], 10.00th=[ 140], 20.00th=[ 188], | 30.00th=[ 197], 40.00th=[ 197], 50.00th=[ 228], 60.00th=[ 262], | 70.00th=[ 262], 80.00th=[ 684], 90.00th=[ 844], 95.00th=[ 844], | 99.00th=[ 844], 99.50th=[ 844], 99.90th=[ 844], 99.95th=[ 844], | 99.99th=[ 844] bw ( KiB/s): min= 1735, max= 4096, per=28.14%, avg=2480.50, stdev=1086.90, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 lat (usec) : 750=16.67%, 1000=8.33% lat (msec) : 100=8.33%, 250=41.67%, 500=8.33%, 750=8.33%, 1000=8.33% cpu : usr=0.03%, sys=29.02%, ctx=119, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6296KiB/s (6447kB/s), 630KiB/s-3323KiB/s (645kB/s-3403kB/s), io=20.0MiB (20.0MB), run=2157-3253msec WRITE: bw=8814KiB/s (9026kB/s), 2036KiB/s-3148KiB/s (2085kB/s-3223kB/s), io=28.0MiB (29.4MB), run=2157-3253msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148300: Thu Nov 13 20:34:46 2025 read: IOPS=2, BW=2645KiB/s (2708kB/s)(7168KiB/2710msec) clat (msec): min=59, max=667, avg=191.97, stdev=213.53 lat (msec): min=59, max=667, avg=191.97, stdev=213.53 clat percentiles (msec): | 1.00th=[ 60], 5.00th=[ 60], 10.00th=[ 60], 20.00th=[ 64], | 30.00th=[ 103], 40.00th=[ 103], 50.00th=[ 144], 60.00th=[ 144], | 70.00th=[ 144], 80.00th=[ 163], 90.00th=[ 667], 95.00th=[ 667], | 99.00th=[ 667], 99.50th=[ 667], 99.90th=[ 667], 99.95th=[ 667], | 99.99th=[ 667] bw ( KiB/s): min= 2035, max= 3976, per=47.82%, avg=2878.00, stdev=989.24, samples=4 iops : min= 1, max= 3, avg= 2.25, stdev= 0.96, samples=4 write: IOPS=1, BW=1889KiB/s (1935kB/s)(5120KiB/2710msec); 0 zone resets clat (msec): min=151, max=498, avg=262.29, stdev=136.90 lat (msec): min=151, max=498, avg=262.37, stdev=136.90 clat percentiles (msec): | 1.00th=[ 153], 5.00th=[ 153], 10.00th=[ 153], 20.00th=[ 153], | 30.00th=[ 186], 40.00th=[ 186], 50.00th=[ 236], 60.00th=[ 236], | 70.00th=[ 243], 80.00th=[ 243], 90.00th=[ 498], 95.00th=[ 498], | 99.00th=[ 498], 99.50th=[ 498], 99.90th=[ 498], 99.95th=[ 498], | 99.99th=[ 498] bw ( KiB/s): min= 1988, max= 4096, per=32.15%, avg=2709.00, stdev=1201.49, samples=3 iops : min= 1, max= 4, avg= 2.00, stdev= 1.73, samples=3 lat (msec) : 100=16.67%, 250=66.67%, 500=8.33%, 750=8.33% cpu : usr=0.07%, sys=15.10%, ctx=110, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148301: Thu Nov 13 20:34:46 2025 read: IOPS=0, BW=602KiB/s (616kB/s)(2048KiB/3402msec) clat (msec): min=52, max=367, avg=210.24, stdev=222.99 lat (msec): min=52, max=367, avg=210.24, stdev=222.99 clat percentiles (msec): | 1.00th=[ 53], 5.00th=[ 53], 10.00th=[ 53], 20.00th=[ 53], | 30.00th=[ 53], 40.00th=[ 53], 50.00th=[ 53], 60.00th=[ 368], | 70.00th=[ 368], 80.00th=[ 368], 90.00th=[ 368], 95.00th=[ 368], | 99.00th=[ 368], 99.50th=[ 368], 99.90th=[ 368], 99.95th=[ 368], | 99.99th=[ 368] bw ( KiB/s): min= 1735, max= 2048, per=31.43%, avg=1891.50, stdev=221.32, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=3010KiB/s (3082kB/s)(10.0MiB/3402msec); 0 zone resets clat (msec): min=77, max=1086, avg=296.41, stdev=302.65 lat (msec): min=77, max=1086, avg=296.48, stdev=302.64 clat percentiles (msec): | 1.00th=[ 79], 5.00th=[ 79], 10.00th=[ 79], 20.00th=[ 107], | 30.00th=[ 110], 40.00th=[ 120], 50.00th=[ 174], 60.00th=[ 174], | 70.00th=[ 342], 80.00th=[ 384], 90.00th=[ 397], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 2043, max= 4087, per=37.31%, avg=3144.00, stdev=1033.07, samples=5 iops : min= 1, max= 3, avg= 2.40, stdev= 0.89, samples=5 lat (msec) : 100=16.67%, 250=41.67%, 500=33.33%, 2000=8.33% cpu : usr=0.06%, sys=11.56%, ctx=267, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148302: Thu Nov 13 20:34:46 2025 read: IOPS=1, BW=1931KiB/s (1978kB/s)(6144KiB/3181msec) clat (msec): min=67, max=331, avg=167.39, stdev=109.00 lat (msec): min=67, max=331, avg=167.39, stdev=109.00 clat percentiles (msec): | 1.00th=[ 68], 5.00th=[ 68], 10.00th=[ 68], 20.00th=[ 73], | 30.00th=[ 73], 40.00th=[ 95], 50.00th=[ 95], 60.00th=[ 184], | 70.00th=[ 255], 80.00th=[ 255], 90.00th=[ 334], 95.00th=[ 334], | 99.00th=[ 334], 99.50th=[ 334], 99.90th=[ 334], 99.95th=[ 334], | 99.99th=[ 334] bw ( KiB/s): min= 1988, max= 4087, per=42.22%, avg=2541.50, stdev=1030.69, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 write: IOPS=1, BW=1931KiB/s (1978kB/s)(6144KiB/3181msec); 0 zone resets clat (msec): min=93, max=935, avg=348.64, stdev=329.81 lat (msec): min=93, max=935, avg=348.72, stdev=329.83 clat percentiles (msec): | 1.00th=[ 94], 5.00th=[ 94], 10.00th=[ 94], 20.00th=[ 101], | 30.00th=[ 101], 40.00th=[ 132], 50.00th=[ 132], 60.00th=[ 326], | 70.00th=[ 506], 80.00th=[ 506], 90.00th=[ 936], 95.00th=[ 936], | 99.00th=[ 936], 99.50th=[ 936], 99.90th=[ 936], 99.95th=[ 936], | 99.99th=[ 936] bw ( KiB/s): min= 1988, max= 4087, per=34.29%, avg=2890.00, stdev=1043.80, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 lat (msec) : 100=33.33%, 250=25.00%, 500=25.00%, 750=8.33%, 1000=8.33% cpu : usr=0.00%, sys=11.45%, ctx=130, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148303: Thu Nov 13 20:34:46 2025 read: IOPS=1, BW=1524KiB/s (1561kB/s)(5120KiB/3359msec) clat (msec): min=40, max=246, avg=161.47, stdev=97.33 lat (msec): min=41, max=246, avg=161.48, stdev=97.33 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 42], 10.00th=[ 42], 20.00th=[ 42], | 30.00th=[ 73], 40.00th=[ 73], 50.00th=[ 205], 60.00th=[ 205], | 70.00th=[ 243], 80.00th=[ 243], 90.00th=[ 247], 95.00th=[ 247], | 99.00th=[ 247], 99.50th=[ 247], 99.90th=[ 247], 99.95th=[ 247], | 99.99th=[ 247] bw ( KiB/s): min= 2043, max= 6144, per=68.01%, avg=4093.50, stdev=2899.84, samples=2 iops : min= 1, max= 6, avg= 3.50, stdev= 3.54, samples=2 write: IOPS=2, BW=2134KiB/s (2185kB/s)(7168KiB/3359msec); 0 zone resets clat (msec): min=123, max=894, avg=353.20, stdev=279.43 lat (msec): min=123, max=894, avg=353.28, stdev=279.43 clat percentiles (msec): | 1.00th=[ 124], 5.00th=[ 124], 10.00th=[ 124], 20.00th=[ 129], | 30.00th=[ 194], 40.00th=[ 194], 50.00th=[ 203], 60.00th=[ 460], | 70.00th=[ 460], 80.00th=[ 468], 90.00th=[ 894], 95.00th=[ 894], | 99.00th=[ 894], 99.50th=[ 894], 99.90th=[ 894], 99.95th=[ 894], | 99.99th=[ 894] bw ( KiB/s): min= 1759, max= 3976, per=29.14%, avg=2455.25, stdev=1022.63, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 50=8.33%, 100=8.33%, 250=58.33%, 500=16.67%, 1000=8.33% cpu : usr=0.03%, sys=9.83%, ctx=74, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6020KiB/s (6164kB/s), 602KiB/s-2645KiB/s (616kB/s-2708kB/s), io=20.0MiB (20.0MB), run=2710-3402msec WRITE: bw=8428KiB/s (8630kB/s), 1889KiB/s-3010KiB/s (1935kB/s-3082kB/s), io=28.0MiB (29.4MB), run=2710-3402msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148312: Thu Nov 13 20:34:54 2025 read: IOPS=3, BW=13.3MiB/s (13.0MB/s)(12.0MiB/900msec) clat (msec): min=163, max=385, avg=287.27, stdev=113.01 lat (msec): min=163, max=385, avg=287.27, stdev=113.01 clat percentiles (msec): | 1.00th=[ 165], 5.00th=[ 165], 10.00th=[ 165], 20.00th=[ 165], | 30.00th=[ 165], 40.00th=[ 313], 50.00th=[ 313], 60.00th=[ 313], | 70.00th=[ 384], 80.00th=[ 384], 90.00th=[ 384], 95.00th=[ 384], | 99.00th=[ 384], 99.50th=[ 384], 99.90th=[ 384], 99.95th=[ 384], | 99.99th=[ 384] bw ( KiB/s): min=15312, max=15312, per=100.00%, avg=15312.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=66.67% cpu : usr=0.00%, sys=23.80%, ctx=72, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148313: Thu Nov 13 20:34:54 2025 read: IOPS=0, BW=2022KiB/s (2070kB/s)(4096KiB/2026msec) clat (nsec): min=481662k, max=481662k, avg=481661567.00, stdev= 0.00 lat (nsec): min=481663k, max=481663k, avg=481662669.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 481], 5.00th=[ 481], 10.00th=[ 481], 20.00th=[ 481], | 30.00th=[ 481], 40.00th=[ 481], 50.00th=[ 481], 60.00th=[ 481], | 70.00th=[ 481], 80.00th=[ 481], 90.00th=[ 481], 95.00th=[ 481], | 99.00th=[ 481], 99.50th=[ 481], 99.90th=[ 481], 99.95th=[ 481], | 99.99th=[ 481] bw ( KiB/s): min= 7787, max= 7787, per=96.30%, avg=7787.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=4043KiB/s (4140kB/s)(8192KiB/2026msec); 0 zone resets clat (msec): min=721, max=792, avg=757.12, stdev=50.14 lat (msec): min=721, max=792, avg=757.40, stdev=50.12 clat percentiles (msec): | 1.00th=[ 726], 5.00th=[ 726], 10.00th=[ 726], 20.00th=[ 726], | 30.00th=[ 726], 40.00th=[ 726], 50.00th=[ 726], 60.00th=[ 793], | 70.00th=[ 793], 80.00th=[ 793], 90.00th=[ 793], 95.00th=[ 793], | 99.00th=[ 793], 99.50th=[ 793], 99.90th=[ 793], 99.95th=[ 793], | 99.99th=[ 793] bw ( KiB/s): min= 8192, max= 8192, per=54.05%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=10.12%, ctx=54, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148314: Thu Nov 13 20:34:54 2025 write: IOPS=1, BW=5684KiB/s (5820kB/s)(12.0MiB/2162msec); 0 zone resets clat (msec): min=547, max=916, avg=705.70, stdev=190.13 lat (msec): min=548, max=917, avg=706.04, stdev=190.11 clat percentiles (msec): | 1.00th=[ 550], 5.00th=[ 550], 10.00th=[ 550], 20.00th=[ 550], | 30.00th=[ 550], 40.00th=[ 651], 50.00th=[ 651], 60.00th=[ 651], | 70.00th=[ 919], 80.00th=[ 919], 90.00th=[ 919], 95.00th=[ 919], | 99.00th=[ 919], 99.50th=[ 919], 99.90th=[ 919], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 8126, max= 8159, per=53.72%, avg=8142.50, stdev=23.33, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=66.67%, 1000=33.33% cpu : usr=0.00%, sys=19.90%, ctx=95, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148315: Thu Nov 13 20:34:54 2025 write: IOPS=2, BW=8771KiB/s (8981kB/s)(12.0MiB/1401msec); 0 zone resets clat (msec): min=167, max=719, avg=459.58, stdev=277.09 lat (msec): min=168, max=719, avg=459.90, stdev=277.01 clat percentiles (msec): | 1.00th=[ 169], 5.00th=[ 169], 10.00th=[ 169], 20.00th=[ 169], | 30.00th=[ 169], 40.00th=[ 493], 50.00th=[ 493], 60.00th=[ 493], | 70.00th=[ 718], 80.00th=[ 718], 90.00th=[ 718], 95.00th=[ 718], | 99.00th=[ 718], 99.50th=[ 718], 99.90th=[ 718], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 8015, max= 8126, per=53.25%, avg=8070.50, stdev=78.49, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 500=33.33%, 750=33.33% cpu : usr=0.00%, sys=14.64%, ctx=65, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8087KiB/s (8281kB/s), 2022KiB/s-13.3MiB/s (2070kB/s-13.0MB/s), io=16.0MiB (16.8MB), run=900-2026msec WRITE: bw=14.8MiB/s (15.5MB/s), 4043KiB/s-8771KiB/s (4140kB/s-8981kB/s), io=32.0MiB (33.6MB), run=1401-2162msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148317: Thu Nov 13 20:34:55 2025 read: IOPS=1, BW=7506KiB/s (7687kB/s)(12.0MiB/1637msec) clat (msec): min=510, max=535, avg=521.44, stdev=12.43 lat (msec): min=510, max=535, avg=521.44, stdev=12.43 clat percentiles (msec): | 1.00th=[ 510], 5.00th=[ 510], 10.00th=[ 510], 20.00th=[ 510], | 30.00th=[ 510], 40.00th=[ 518], 50.00th=[ 518], 60.00th=[ 518], | 70.00th=[ 535], 80.00th=[ 535], 90.00th=[ 535], 95.00th=[ 535], | 99.00th=[ 535], 99.50th=[ 535], 99.90th=[ 535], 99.95th=[ 535], | 99.99th=[ 535] bw ( KiB/s): min= 8094, max= 8192, per=100.00%, avg=8143.00, stdev=69.30, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=100.00% cpu : usr=0.00%, sys=35.64%, ctx=1178, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148318: Thu Nov 13 20:34:55 2025 read: IOPS=0, BW=2000KiB/s (2048kB/s)(4096KiB/2048msec) clat (nsec): min=567292k, max=567292k, avg=567292453.00, stdev= 0.00 lat (nsec): min=567294k, max=567294k, avg=567294313.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 567], 5.00th=[ 567], 10.00th=[ 567], 20.00th=[ 567], | 30.00th=[ 567], 40.00th=[ 567], 50.00th=[ 567], 60.00th=[ 567], | 70.00th=[ 567], 80.00th=[ 567], 90.00th=[ 567], 95.00th=[ 567], | 99.00th=[ 567], 99.50th=[ 567], 99.90th=[ 567], 99.95th=[ 567], | 99.99th=[ 567] bw ( KiB/s): min= 5587, max= 5587, per=69.84%, avg=5587.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=4000KiB/s (4096kB/s)(8192KiB/2048msec); 0 zone resets clat (msec): min=639, max=816, avg=728.00, stdev=125.55 lat (msec): min=639, max=817, avg=728.36, stdev=125.65 clat percentiles (msec): | 1.00th=[ 642], 5.00th=[ 642], 10.00th=[ 642], 20.00th=[ 642], | 30.00th=[ 642], 40.00th=[ 642], 50.00th=[ 642], 60.00th=[ 818], | 70.00th=[ 818], 80.00th=[ 818], 90.00th=[ 818], 95.00th=[ 818], | 99.00th=[ 818], 99.50th=[ 818], 99.90th=[ 818], 99.95th=[ 818], | 99.99th=[ 818] bw ( KiB/s): min= 8126, max= 8126, per=50.79%, avg=8126.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=66.67%, 1000=33.33% cpu : usr=0.00%, sys=16.41%, ctx=752, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148319: Thu Nov 13 20:34:55 2025 write: IOPS=1, BW=6481KiB/s (6637kB/s)(12.0MiB/1896msec); 0 zone resets clat (msec): min=171, max=972, avg=604.34, stdev=404.13 lat (msec): min=172, max=972, avg=604.74, stdev=404.07 clat percentiles (msec): | 1.00th=[ 171], 5.00th=[ 171], 10.00th=[ 171], 20.00th=[ 171], | 30.00th=[ 171], 40.00th=[ 667], 50.00th=[ 667], 60.00th=[ 667], | 70.00th=[ 969], 80.00th=[ 969], 90.00th=[ 969], 95.00th=[ 969], | 99.00th=[ 969], 99.50th=[ 969], 99.90th=[ 969], 99.95th=[ 969], | 99.99th=[ 969] bw ( KiB/s): min= 8126, max= 8143, per=50.84%, avg=8134.50, stdev=12.02, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.05%, sys=10.34%, ctx=56, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148320: Thu Nov 13 20:34:55 2025 write: IOPS=2, BW=8416KiB/s (8618kB/s)(12.0MiB/1460msec); 0 zone resets clat (msec): min=276, max=724, avg=479.23, stdev=227.17 lat (msec): min=276, max=725, avg=479.54, stdev=227.14 clat percentiles (msec): | 1.00th=[ 275], 5.00th=[ 275], 10.00th=[ 275], 20.00th=[ 275], | 30.00th=[ 275], 40.00th=[ 439], 50.00th=[ 439], 60.00th=[ 439], | 70.00th=[ 726], 80.00th=[ 726], 90.00th=[ 726], 95.00th=[ 726], | 99.00th=[ 726], 99.50th=[ 726], 99.90th=[ 726], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 8094, max= 8192, per=50.89%, avg=8143.00, stdev=69.30, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.07%, sys=16.59%, ctx=74, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8000KiB/s (8192kB/s), 2000KiB/s-7506KiB/s (2048kB/s-7687kB/s), io=16.0MiB (16.8MB), run=1637-2048msec WRITE: bw=15.6MiB/s (16.4MB/s), 4000KiB/s-8416KiB/s (4096kB/s-8618kB/s), io=32.0MiB (33.6MB), run=1460-2048msec