== sanity test 398b: DIO and buffer IO race ============== 10:10:50 (1758809450) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.26287 s, 15.4 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=158816: Thu Sep 25 10:11:51 2025 read: IOPS=29, BW=117KiB/s (120kB/s)(5992KiB/51108msec) clat (usec): min=78, max=695572, avg=17523.27, stdev=21240.35 lat (usec): min=79, max=695573, avg=17523.92, stdev=21240.39 clat percentiles (usec): | 1.00th=[ 249], 5.00th=[ 510], 10.00th=[ 6652], 20.00th=[ 8979], | 30.00th=[ 10290], 40.00th=[ 12256], 50.00th=[ 14615], 60.00th=[ 17171], | 70.00th=[ 20055], 80.00th=[ 23987], 90.00th=[ 31327], 95.00th=[ 38011], | 99.00th=[ 61080], 99.50th=[ 81265], 99.90th=[112722], 99.95th=[692061], | 99.99th=[692061] bw ( KiB/s): min= 13, max= 232, per=25.03%, avg=116.91, stdev=39.59, samples=101 iops : min= 3, max= 58, avg=28.89, stdev= 9.98, samples=101 write: IOPS=30, BW=123KiB/s (126kB/s)(6296KiB/51108msec); 0 zone resets clat (usec): min=636, max=159646, avg=15656.19, stdev=13996.25 lat (usec): min=637, max=159647, avg=15657.15, stdev=13996.12 clat percentiles (usec): | 1.00th=[ 734], 5.00th=[ 1827], 10.00th=[ 2638], 20.00th=[ 3556], | 30.00th=[ 5866], 40.00th=[ 9372], 50.00th=[ 12518], 60.00th=[ 15664], | 70.00th=[ 19268], 80.00th=[ 25297], 90.00th=[ 33424], 95.00th=[ 41157], | 99.00th=[ 58983], 99.50th=[ 74974], 99.90th=[133694], 99.95th=[160433], | 99.99th=[160433] bw ( KiB/s): min= 31, max= 216, per=25.33%, avg=123.85, stdev=40.03, samples=100 iops : min= 7, max= 54, avg=30.63, stdev=10.09, samples=100 lat (usec) : 100=0.16%, 250=0.33%, 500=1.89%, 750=1.20%, 1000=0.81% lat (msec) : 2=1.66%, 4=8.98%, 10=19.99%, 20=35.74%, 50=27.02% lat (msec) : 100=2.05%, 250=0.13%, 750=0.03% cpu : usr=0.09%, sys=27.40%, ctx=8581, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158817: Thu Sep 25 10:11:51 2025 read: IOPS=29, BW=116KiB/s (119kB/s)(5940KiB/51178msec) clat (usec): min=81, max=145983, avg=16970.91, stdev=12201.44 lat (usec): min=82, max=145984, avg=16971.58, stdev=12201.46 clat percentiles (usec): | 1.00th=[ 258], 5.00th=[ 4113], 10.00th=[ 7242], 20.00th=[ 8979], | 30.00th=[ 10290], 40.00th=[ 12125], 50.00th=[ 14353], 60.00th=[ 16581], | 70.00th=[ 19268], 80.00th=[ 23462], 90.00th=[ 30016], 95.00th=[ 37487], | 99.00th=[ 60031], 99.50th=[ 82314], 99.90th=[117965], 99.95th=[145753], | 99.99th=[145753] bw ( KiB/s): min= 6, max= 192, per=24.80%, avg=115.81, stdev=38.17, samples=101 iops : min= 1, max= 48, avg=28.61, stdev= 9.67, samples=101 write: IOPS=31, BW=124KiB/s (127kB/s)(6348KiB/51178msec); 0 zone resets clat (usec): min=637, max=706073, avg=16229.41, stdev=22114.52 lat (usec): min=638, max=706074, avg=16230.24, stdev=22114.62 clat percentiles (usec): | 1.00th=[ 750], 5.00th=[ 2008], 10.00th=[ 2704], 20.00th=[ 3654], | 30.00th=[ 6194], 40.00th=[ 9503], 50.00th=[ 12649], 60.00th=[ 15926], | 70.00th=[ 19792], 80.00th=[ 25297], 90.00th=[ 33162], 95.00th=[ 41157], | 99.00th=[ 62653], 99.50th=[ 69731], 99.90th=[139461], 99.95th=[708838], | 99.99th=[708838] bw ( KiB/s): min= 6, max= 216, per=25.17%, avg=123.07, stdev=42.76, samples=101 iops : min= 1, max= 54, avg=30.44, stdev=10.78, samples=101 lat (usec) : 100=0.16%, 250=0.29%, 500=1.30%, 750=0.94%, 1000=0.62% lat (msec) : 2=1.60%, 4=8.85%, 10=21.13%, 20=36.17%, 50=26.86% lat (msec) : 100=1.82%, 250=0.23%, 750=0.03% cpu : usr=0.07%, sys=27.44%, ctx=8764, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158818: Thu Sep 25 10:11:51 2025 read: IOPS=29, BW=117KiB/s (120kB/s)(6028KiB/51406msec) clat (usec): min=82, max=723146, avg=17187.68, stdev=21622.24 lat (usec): min=82, max=723146, avg=17188.59, stdev=21622.30 clat percentiles (usec): | 1.00th=[ 219], 5.00th=[ 668], 10.00th=[ 6980], 20.00th=[ 8848], | 30.00th=[ 10159], 40.00th=[ 11994], 50.00th=[ 14222], 60.00th=[ 16712], | 70.00th=[ 19530], 80.00th=[ 23725], 90.00th=[ 29230], 95.00th=[ 35914], | 99.00th=[ 61604], 99.50th=[ 82314], 99.90th=[130548], 99.95th=[725615], | 99.99th=[725615] bw ( KiB/s): min= 23, max= 232, per=25.24%, avg=117.87, stdev=42.34, samples=101 iops : min= 5, max= 58, avg=29.14, stdev=10.71, samples=101 write: IOPS=30, BW=122KiB/s (125kB/s)(6260KiB/51406msec); 0 zone resets clat (usec): min=608, max=143916, avg=16169.11, stdev=14399.44 lat (usec): min=609, max=143918, avg=16169.99, stdev=14399.49 clat percentiles (usec): | 1.00th=[ 758], 5.00th=[ 1713], 10.00th=[ 2573], 20.00th=[ 3621], | 30.00th=[ 6390], 40.00th=[ 10159], 50.00th=[ 13042], 60.00th=[ 16450], | 70.00th=[ 20579], 80.00th=[ 25560], 90.00th=[ 34341], 95.00th=[ 40633], | 99.00th=[ 62129], 99.50th=[ 77071], 99.90th=[141558], 99.95th=[143655], | 99.99th=[143655] bw ( KiB/s): min= 19, max= 208, per=24.76%, avg=121.08, stdev=36.22, samples=102 iops : min= 4, max= 52, avg=29.93, stdev= 9.18, samples=102 lat (usec) : 100=0.16%, 250=0.36%, 500=1.53%, 750=0.98%, 1000=1.04% lat (msec) : 2=2.12%, 4=7.94%, 10=20.25%, 20=35.87%, 50=27.70% lat (msec) : 100=1.79%, 250=0.23%, 750=0.03% cpu : usr=0.09%, sys=27.21%, ctx=8714, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158819: Thu Sep 25 10:11:51 2025 read: IOPS=29, BW=118KiB/s (121kB/s)(6052KiB/51264msec) clat (usec): min=74, max=106724, avg=16809.98, stdev=11571.00 lat (usec): min=74, max=106725, avg=16811.26, stdev=11571.32 clat percentiles (usec): | 1.00th=[ 184], 5.00th=[ 482], 10.00th=[ 6521], 20.00th=[ 8717], | 30.00th=[ 10028], 40.00th=[ 11994], 50.00th=[ 14353], 60.00th=[ 17171], | 70.00th=[ 20317], 80.00th=[ 23987], 90.00th=[ 30802], 95.00th=[ 37487], | 99.00th=[ 56886], 99.50th=[ 70779], 99.90th=[ 94897], 99.95th=[106431], | 99.99th=[106431] bw ( KiB/s): min= 8, max= 232, per=25.53%, avg=119.22, stdev=39.78, samples=101 iops : min= 2, max= 58, avg=29.48, stdev=10.00, samples=101 write: IOPS=30, BW=122KiB/s (125kB/s)(6236KiB/51264msec); 0 zone resets clat (usec): min=643, max=767079, avg=16436.84, stdev=24679.93 lat (usec): min=643, max=767080, avg=16437.69, stdev=24679.98 clat percentiles (usec): | 1.00th=[ 725], 5.00th=[ 2073], 10.00th=[ 2638], 20.00th=[ 3687], | 30.00th=[ 6194], 40.00th=[ 9110], 50.00th=[ 12256], 60.00th=[ 15795], | 70.00th=[ 20055], 80.00th=[ 25035], 90.00th=[ 33424], 95.00th=[ 43779], | 99.00th=[ 65799], 99.50th=[ 91751], 99.90th=[214959], 99.95th=[767558], | 99.99th=[767558] bw ( KiB/s): min= 32, max= 224, per=25.11%, avg=122.79, stdev=41.85, samples=101 iops : min= 8, max= 56, avg=30.37, stdev=10.55, samples=101 lat (usec) : 100=0.13%, 250=0.49%, 500=1.99%, 750=1.17%, 1000=0.94% lat (msec) : 2=1.11%, 4=8.98%, 10=21.42%, 20=33.24%, 50=28.45% lat (msec) : 100=1.86%, 250=0.20%, 1000=0.03% cpu : usr=0.11%, sys=27.05%, ctx=8493, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=467KiB/s (478kB/s), 116KiB/s-118KiB/s (119kB/s-121kB/s), io=23.4MiB (24.6MB), run=51108-51406msec WRITE: bw=489KiB/s (501kB/s), 122KiB/s-124KiB/s (125kB/s-127kB/s), io=24.6MiB (25.7MB), run=51108-51406msec rand-rw: (groupid=0, jobs=1): err= 0: pid=158821: Thu Sep 25 10:12:23 2025 read: IOPS=17, BW=71.9KiB/s (73.6kB/s)(5992KiB/83323msec) clat (msec): min=2, max=707, avg=14.94, stdev=22.73 lat (msec): min=2, max=707, avg=14.94, stdev=22.73 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 10], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 31], 95.00th=[ 41], | 99.00th=[ 74], 99.50th=[ 95], 99.90th=[ 138], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 8, max= 240, per=25.19%, avg=72.31, stdev=41.02, samples=165 iops : min= 2, max= 60, avg=17.84, stdev=10.40, samples=165 write: IOPS=18, BW=75.6KiB/s (77.4kB/s)(6296KiB/83323msec); 0 zone resets clat (msec): min=15, max=188, avg=38.61, stdev=18.75 lat (msec): min=15, max=188, avg=38.61, stdev=18.75 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 31], 50.00th=[ 34], 60.00th=[ 37], | 70.00th=[ 42], 80.00th=[ 50], 90.00th=[ 62], 95.00th=[ 74], | 99.00th=[ 106], 99.50th=[ 131], 99.90th=[ 169], 99.95th=[ 188], | 99.99th=[ 188] bw ( KiB/s): min= 15, max= 152, per=25.23%, avg=75.95, stdev=31.95, samples=165 iops : min= 3, max= 38, avg=18.75, stdev= 8.16, samples=165 lat (msec) : 4=5.21%, 10=20.48%, 20=13.57%, 50=49.22%, 100=10.68% lat (msec) : 250=0.81%, 750=0.03% cpu : usr=0.07%, sys=7.87%, ctx=5805, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158822: Thu Sep 25 10:12:23 2025 read: IOPS=17, BW=71.2KiB/s (72.9kB/s)(5940KiB/83400msec) clat (msec): min=2, max=715, avg=15.01, stdev=23.43 lat (msec): min=2, max=715, avg=15.01, stdev=23.43 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 10], 60.00th=[ 13], | 70.00th=[ 16], 80.00th=[ 22], 90.00th=[ 31], 95.00th=[ 44], | 99.00th=[ 77], 99.50th=[ 94], 99.90th=[ 155], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 7, max= 192, per=24.92%, avg=71.53, stdev=41.11, samples=165 iops : min= 1, max= 48, avg=17.64, stdev=10.42, samples=165 write: IOPS=19, BW=76.1KiB/s (77.9kB/s)(6348KiB/83400msec); 0 zone resets clat (msec): min=14, max=240, avg=38.41, stdev=19.62 lat (msec): min=14, max=240, avg=38.41, stdev=19.62 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 26], | 30.00th=[ 28], 40.00th=[ 31], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 41], 80.00th=[ 50], 90.00th=[ 62], 95.00th=[ 72], | 99.00th=[ 110], 99.50th=[ 138], 99.90th=[ 228], 99.95th=[ 241], | 99.99th=[ 241] bw ( KiB/s): min= 23, max= 152, per=25.39%, avg=76.43, stdev=32.64, samples=165 iops : min= 5, max= 38, avg=18.87, stdev= 8.33, samples=165 lat (msec) : 4=5.27%, 10=19.95%, 20=14.68%, 50=48.76%, 100=10.48% lat (msec) : 250=0.81%, 750=0.03% cpu : usr=0.05%, sys=7.78%, ctx=5808, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158823: Thu Sep 25 10:12:23 2025 read: IOPS=18, BW=72.3KiB/s (74.0kB/s)(6028KiB/83382msec) clat (msec): min=2, max=162, avg=14.93, stdev=15.21 lat (msec): min=2, max=162, avg=14.93, stdev=15.21 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 10], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 32], 95.00th=[ 43], | 99.00th=[ 73], 99.50th=[ 94], 99.90th=[ 155], 99.95th=[ 163], | 99.99th=[ 163] bw ( KiB/s): min= 15, max= 216, per=25.31%, avg=72.65, stdev=37.63, samples=165 iops : min= 3, max= 54, avg=17.92, stdev= 9.55, samples=165 write: IOPS=18, BW=75.1KiB/s (76.9kB/s)(6260KiB/83382msec); 0 zone resets clat (msec): min=14, max=738, avg=38.80, stdev=26.30 lat (msec): min=14, max=738, avg=38.80, stdev=26.30 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 31], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 41], 80.00th=[ 48], 90.00th=[ 63], 95.00th=[ 74], | 99.00th=[ 102], 99.50th=[ 150], 99.90th=[ 228], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 15, max= 152, per=25.05%, avg=75.41, stdev=33.43, samples=165 iops : min= 3, max= 38, avg=18.61, stdev= 8.52, samples=165 lat (msec) : 4=4.92%, 10=20.12%, 20=15.04%, 50=49.19%, 100=9.93% lat (msec) : 250=0.78%, 750=0.03% cpu : usr=0.07%, sys=7.86%, ctx=5788, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158824: Thu Sep 25 10:12:23 2025 read: IOPS=18, BW=73.1KiB/s (74.8kB/s)(6052KiB/82806msec) clat (msec): min=2, max=186, avg=14.60, stdev=15.50 lat (msec): min=2, max=186, avg=14.61, stdev=15.50 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 10], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 31], 95.00th=[ 43], | 99.00th=[ 73], 99.50th=[ 97], 99.90th=[ 165], 99.95th=[ 186], | 99.99th=[ 186] bw ( KiB/s): min= 8, max= 192, per=25.57%, avg=73.39, stdev=39.45, samples=163 iops : min= 2, max= 48, avg=18.10, stdev= 9.99, samples=163 write: IOPS=18, BW=75.3KiB/s (77.1kB/s)(6236KiB/82806msec); 0 zone resets clat (msec): min=15, max=762, avg=38.84, stdev=25.79 lat (msec): min=15, max=762, avg=38.84, stdev=25.79 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 22], 10.00th=[ 23], 20.00th=[ 26], | 30.00th=[ 28], 40.00th=[ 31], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 42], 80.00th=[ 50], 90.00th=[ 63], 95.00th=[ 74], | 99.00th=[ 100], 99.50th=[ 120], 99.90th=[ 226], 99.95th=[ 760], | 99.99th=[ 760] bw ( KiB/s): min= 7, max= 152, per=25.12%, avg=75.60, stdev=32.17, samples=163 iops : min= 1, max= 38, avg=18.66, stdev= 8.22, samples=163 lat (msec) : 4=5.66%, 10=20.05%, 20=14.68%, 50=48.14%, 100=10.77% lat (msec) : 250=0.65%, 1000=0.03% cpu : usr=0.08%, sys=7.91%, ctx=5849, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=288KiB/s (295kB/s), 71.2KiB/s-73.1KiB/s (72.9kB/s-74.8kB/s), io=23.4MiB (24.6MB), run=82806-83400msec WRITE: bw=301KiB/s (309kB/s), 75.1KiB/s-76.1KiB/s (76.9kB/s-77.9kB/s), io=24.6MiB (25.7MB), run=82806-83400msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=158854: Thu Sep 25 10:12:46 2025 read: IOPS=22, BW=363KiB/s (372kB/s)(5920KiB/16302msec) clat (usec): min=260, max=93200, avg=22071.06, stdev=14015.10 lat (usec): min=261, max=93201, avg=22071.70, stdev=14015.14 clat percentiles (usec): | 1.00th=[ 318], 5.00th=[ 594], 10.00th=[ 848], 20.00th=[11076], | 30.00th=[15795], 40.00th=[18220], 50.00th=[21103], 60.00th=[23987], | 70.00th=[27395], 80.00th=[32113], 90.00th=[37487], 95.00th=[45351], | 99.00th=[66847], 99.50th=[78119], 99.90th=[92799], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 223, max= 736, per=25.56%, avg=362.22, stdev=117.79, samples=32 iops : min= 13, max= 46, avg=22.00, stdev= 7.51, samples=32 write: IOPS=24, BW=391KiB/s (400kB/s)(6368KiB/16302msec); 0 zone resets clat (usec): min=968, max=103295, avg=20347.42, stdev=14001.05 lat (usec): min=969, max=103296, avg=20348.61, stdev=14001.09 clat percentiles (usec): | 1.00th=[ 1188], 5.00th=[ 4113], 10.00th=[ 5538], 20.00th=[ 8356], | 30.00th=[ 11731], 40.00th=[ 14484], 50.00th=[ 17433], 60.00th=[ 20579], | 70.00th=[ 24773], 80.00th=[ 30278], 90.00th=[ 40109], 95.00th=[ 46400], | 99.00th=[ 66323], 99.50th=[ 83362], 99.90th=[103285], 99.95th=[103285], | 99.99th=[103285] bw ( KiB/s): min= 191, max= 672, per=26.21%, avg=390.75, stdev=121.74, samples=32 iops : min= 11, max= 42, avg=23.78, stdev= 7.75, samples=32 lat (usec) : 500=1.95%, 750=1.69%, 1000=1.95% lat (msec) : 2=1.17%, 4=1.56%, 10=12.89%, 20=30.86%, 50=44.66% lat (msec) : 100=3.12%, 250=0.13% cpu : usr=0.06%, sys=32.64%, ctx=2764, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158855: Thu Sep 25 10:12:46 2025 read: IOPS=21, BW=340KiB/s (348kB/s)(5744KiB/16897msec) clat (usec): min=93, max=94851, avg=22444.46, stdev=13305.48 lat (usec): min=94, max=94852, avg=22445.44, stdev=13305.97 clat percentiles (usec): | 1.00th=[ 343], 5.00th=[ 619], 10.00th=[ 7832], 20.00th=[12649], | 30.00th=[16188], 40.00th=[18482], 50.00th=[21103], 60.00th=[23725], | 70.00th=[27132], 80.00th=[30802], 90.00th=[38536], 95.00th=[43254], | 99.00th=[64750], 99.50th=[78119], 99.90th=[94897], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 53, max= 698, per=23.91%, avg=338.79, stdev=131.22, samples=33 iops : min= 3, max= 43, avg=20.52, stdev= 8.24, samples=33 write: IOPS=24, BW=387KiB/s (397kB/s)(6544KiB/16897msec); 0 zone resets clat (usec): min=844, max=119276, avg=21277.22, stdev=15814.12 lat (usec): min=845, max=119278, avg=21278.91, stdev=15814.14 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 5], 10.00th=[ 7], 20.00th=[ 10], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 18], 60.00th=[ 21], | 70.00th=[ 25], 80.00th=[ 30], 90.00th=[ 41], 95.00th=[ 51], | 99.00th=[ 81], 99.50th=[ 96], 99.90th=[ 120], 99.95th=[ 120], | 99.99th=[ 120] bw ( KiB/s): min= 192, max= 544, per=25.63%, avg=382.21, stdev=96.28, samples=33 iops : min= 12, max= 34, avg=23.24, stdev= 6.11, samples=33 lat (usec) : 100=0.13%, 250=0.13%, 500=1.04%, 750=1.56%, 1000=1.04% lat (msec) : 2=0.65%, 4=1.30%, 10=12.24%, 20=33.98%, 50=43.49% lat (msec) : 100=4.30%, 250=0.13% cpu : usr=0.09%, sys=32.77%, ctx=2838, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158856: Thu Sep 25 10:12:46 2025 read: IOPS=24, BW=386KiB/s (395kB/s)(6256KiB/16218msec) clat (usec): min=165, max=74039, avg=22143.81, stdev=12897.61 lat (usec): min=166, max=74040, avg=22144.52, stdev=12897.63 clat percentiles (usec): | 1.00th=[ 326], 5.00th=[ 611], 10.00th=[ 881], 20.00th=[12125], | 30.00th=[16057], 40.00th=[19006], 50.00th=[22152], 60.00th=[24511], | 70.00th=[28181], 80.00th=[32637], 90.00th=[37487], 95.00th=[43779], | 99.00th=[56886], 99.50th=[72877], 99.90th=[73925], 99.95th=[73925], | 99.99th=[73925] bw ( KiB/s): min= 192, max= 672, per=27.38%, avg=388.03, stdev=129.88, samples=32 iops : min= 12, max= 42, avg=23.59, stdev= 8.23, samples=32 write: IOPS=23, BW=372KiB/s (381kB/s)(6032KiB/16218msec); 0 zone resets clat (usec): min=1030, max=77594, avg=19747.53, stdev=13676.50 lat (usec): min=1031, max=77596, avg=19748.85, stdev=13676.46 clat percentiles (usec): | 1.00th=[ 2606], 5.00th=[ 4490], 10.00th=[ 5473], 20.00th=[ 7504], | 30.00th=[10159], 40.00th=[13173], 50.00th=[16450], 60.00th=[19792], | 70.00th=[24249], 80.00th=[30016], 90.00th=[40633], 95.00th=[47449], | 99.00th=[61080], 99.50th=[65274], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 191, max= 768, per=25.00%, avg=372.78, stdev=127.50, samples=32 iops : min= 11, max= 48, avg=22.66, stdev= 8.12, samples=32 lat (usec) : 250=0.13%, 500=1.56%, 750=2.47%, 1000=1.30% lat (msec) : 2=0.52%, 4=1.17%, 10=14.97%, 20=30.08%, 50=44.79% lat (msec) : 100=2.99% cpu : usr=0.06%, sys=32.98%, ctx=2902, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158857: Thu Sep 25 10:12:46 2025 read: IOPS=23, BW=372KiB/s (381kB/s)(6032KiB/16211msec) clat (usec): min=100, max=83925, avg=22003.50, stdev=13318.15 lat (usec): min=100, max=83925, avg=22004.18, stdev=13318.15 clat percentiles (usec): | 1.00th=[ 243], 5.00th=[ 545], 10.00th=[ 914], 20.00th=[11469], | 30.00th=[15533], 40.00th=[18220], 50.00th=[21103], 60.00th=[24511], | 70.00th=[27657], 80.00th=[32375], 90.00th=[38011], 95.00th=[43254], | 99.00th=[66323], 99.50th=[83362], 99.90th=[84411], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 223, max= 640, per=26.21%, avg=371.41, stdev=101.81, samples=32 iops : min= 13, max= 40, avg=22.56, stdev= 6.56, samples=32 write: IOPS=24, BW=386KiB/s (395kB/s)(6256KiB/16211msec); 0 zone resets clat (usec): min=859, max=119109, avg=20119.13, stdev=15230.88 lat (usec): min=860, max=119111, avg=20120.54, stdev=15230.79 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 4015], 10.00th=[ 5735], 20.00th=[ 7832], | 30.00th=[ 9765], 40.00th=[ 12911], 50.00th=[ 16581], 60.00th=[ 20055], | 70.00th=[ 23987], 80.00th=[ 31851], 90.00th=[ 41681], 95.00th=[ 46924], | 99.00th=[ 63701], 99.50th=[101188], 99.90th=[119014], 99.95th=[119014], | 99.99th=[119014] bw ( KiB/s): min= 191, max= 864, per=25.92%, avg=386.47, stdev=139.57, samples=32 iops : min= 11, max= 54, avg=23.50, stdev= 8.87, samples=32 lat (usec) : 250=0.52%, 500=1.56%, 750=2.08%, 1000=1.43% lat (msec) : 2=1.17%, 4=1.43%, 10=15.63%, 20=28.78%, 50=44.53% lat (msec) : 100=2.60%, 250=0.26% cpu : usr=0.11%, sys=32.54%, ctx=2807, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1418KiB/s (1452kB/s), 340KiB/s-386KiB/s (348kB/s-395kB/s), io=23.4MiB (24.5MB), run=16211-16897msec WRITE: bw=1491KiB/s (1527kB/s), 372KiB/s-391KiB/s (381kB/s-400kB/s), io=24.6MiB (25.8MB), run=16211-16897msec rand-rw: (groupid=0, jobs=1): err= 0: pid=158850: Thu Sep 25 10:12:53 2025 read: IOPS=15, BW=248KiB/s (254kB/s)(5920KiB/23911msec) clat (msec): min=2, max=117, avg=20.25, stdev=20.18 lat (msec): min=2, max=117, avg=20.25, stdev=20.18 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 17], | 70.00th=[ 22], 80.00th=[ 28], 90.00th=[ 46], 95.00th=[ 68], | 99.00th=[ 105], 99.50th=[ 112], 99.90th=[ 118], 99.95th=[ 118], | 99.99th=[ 118] bw ( KiB/s): min= 95, max= 736, per=24.40%, avg=243.02, stdev=137.37, samples=47 iops : min= 5, max= 46, avg=14.83, stdev= 8.75, samples=47 write: IOPS=16, BW=266KiB/s (273kB/s)(6368KiB/23911msec); 0 zone resets clat (msec): min=15, max=176, avg=40.90, stdev=20.53 lat (msec): min=15, max=176, avg=40.91, stdev=20.53 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 26], | 30.00th=[ 29], 40.00th=[ 32], 50.00th=[ 36], 60.00th=[ 39], | 70.00th=[ 45], 80.00th=[ 54], 90.00th=[ 67], 95.00th=[ 82], | 99.00th=[ 121], 99.50th=[ 144], 99.90th=[ 178], 99.95th=[ 178], | 99.99th=[ 178] bw ( KiB/s): min= 95, max= 512, per=25.29%, avg=265.04, stdev=128.60, samples=47 iops : min= 5, max= 32, avg=16.21, stdev= 8.27, samples=47 lat (msec) : 4=3.78%, 10=13.80%, 20=17.19%, 50=49.61%, 100=14.06% lat (msec) : 250=1.56% cpu : usr=0.06%, sys=10.89%, ctx=1769, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158851: Thu Sep 25 10:12:53 2025 read: IOPS=15, BW=241KiB/s (246kB/s)(5744KiB/23866msec) clat (msec): min=3, max=121, avg=18.81, stdev=18.13 lat (msec): min=3, max=121, avg=18.81, stdev=18.13 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 11], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 20], 80.00th=[ 28], 90.00th=[ 42], 95.00th=[ 57], | 99.00th=[ 93], 99.50th=[ 112], 99.90th=[ 123], 99.95th=[ 123], | 99.99th=[ 123] bw ( KiB/s): min= 53, max= 640, per=24.59%, avg=244.89, stdev=137.55, samples=46 iops : min= 3, max= 40, avg=14.93, stdev= 8.84, samples=46 write: IOPS=17, BW=274KiB/s (281kB/s)(6544KiB/23866msec); 0 zone resets clat (msec): min=15, max=150, avg=41.78, stdev=20.81 lat (msec): min=15, max=150, avg=41.79, stdev=20.81 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 22], 10.00th=[ 24], 20.00th=[ 27], | 30.00th=[ 29], 40.00th=[ 33], 50.00th=[ 36], 60.00th=[ 40], | 70.00th=[ 45], 80.00th=[ 54], 90.00th=[ 71], 95.00th=[ 86], | 99.00th=[ 110], 99.50th=[ 128], 99.90th=[ 150], 99.95th=[ 150], | 99.99th=[ 150] bw ( KiB/s): min= 127, max= 512, per=25.96%, avg=272.09, stdev=117.55, samples=47 iops : min= 7, max= 32, avg=16.64, stdev= 7.60, samples=47 lat (msec) : 4=2.21%, 10=16.02%, 20=15.49%, 50=50.65%, 100=14.06% lat (msec) : 250=1.56% cpu : usr=0.08%, sys=10.49%, ctx=1681, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158852: Thu Sep 25 10:12:53 2025 read: IOPS=16, BW=270KiB/s (277kB/s)(6256KiB/23151msec) clat (msec): min=2, max=127, avg=18.59, stdev=17.81 lat (msec): min=2, max=127, avg=18.59, stdev=17.81 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 8], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 17], | 70.00th=[ 21], 80.00th=[ 27], 90.00th=[ 41], 95.00th=[ 58], | 99.00th=[ 87], 99.50th=[ 115], 99.90th=[ 128], 99.95th=[ 128], | 99.99th=[ 128] bw ( KiB/s): min= 95, max= 832, per=27.15%, avg=270.44, stdev=177.40, samples=45 iops : min= 5, max= 52, avg=16.53, stdev=11.25, samples=45 write: IOPS=16, BW=261KiB/s (267kB/s)(6032KiB/23151msec); 0 zone resets clat (msec): min=15, max=207, avg=42.00, stdev=21.24 lat (msec): min=15, max=207, avg=42.00, stdev=21.24 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 22], 10.00th=[ 24], 20.00th=[ 27], | 30.00th=[ 30], 40.00th=[ 32], 50.00th=[ 35], 60.00th=[ 40], | 70.00th=[ 47], 80.00th=[ 56], 90.00th=[ 70], 95.00th=[ 84], | 99.00th=[ 104], 99.50th=[ 122], 99.90th=[ 207], 99.95th=[ 207], | 99.99th=[ 207] bw ( KiB/s): min= 63, max= 512, per=24.51%, avg=256.89, stdev=119.17, samples=45 iops : min= 3, max= 32, avg=15.69, stdev= 7.65, samples=45 lat (msec) : 4=2.21%, 10=17.71%, 20=16.28%, 50=47.53%, 100=15.10% lat (msec) : 250=1.17% cpu : usr=0.07%, sys=10.99%, ctx=1743, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158853: Thu Sep 25 10:12:53 2025 read: IOPS=15, BW=251KiB/s (257kB/s)(6032KiB/24034msec) clat (msec): min=2, max=124, avg=19.83, stdev=19.52 lat (msec): min=2, max=124, avg=19.84, stdev=19.52 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 8], 40.00th=[ 11], 50.00th=[ 14], 60.00th=[ 16], | 70.00th=[ 20], 80.00th=[ 30], 90.00th=[ 48], 95.00th=[ 63], | 99.00th=[ 99], 99.50th=[ 114], 99.90th=[ 125], 99.95th=[ 125], | 99.99th=[ 125] bw ( KiB/s): min= 95, max= 576, per=24.58%, avg=244.83, stdev=124.17, samples=47 iops : min= 5, max= 36, avg=14.94, stdev= 8.01, samples=47 write: IOPS=16, BW=260KiB/s (267kB/s)(6256KiB/24034msec); 0 zone resets clat (msec): min=14, max=208, avg=42.14, stdev=22.58 lat (msec): min=14, max=208, avg=42.14, stdev=22.58 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 24], 20.00th=[ 27], | 30.00th=[ 30], 40.00th=[ 33], 50.00th=[ 36], 60.00th=[ 40], | 70.00th=[ 44], 80.00th=[ 55], 90.00th=[ 72], 95.00th=[ 87], | 99.00th=[ 122], 99.50th=[ 144], 99.90th=[ 209], 99.95th=[ 209], | 99.99th=[ 209] bw ( KiB/s): min= 95, max= 608, per=24.43%, avg=256.06, stdev=130.15, samples=47 iops : min= 5, max= 38, avg=15.64, stdev= 8.38, samples=47 lat (msec) : 4=3.39%, 10=14.97%, 20=18.36%, 50=46.61%, 100=14.97% lat (msec) : 250=1.69% cpu : usr=0.07%, sys=10.69%, ctx=1760, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=997KiB/s (1021kB/s), 241KiB/s-270KiB/s (246kB/s-277kB/s), io=23.4MiB (24.5MB), run=23151-24034msec WRITE: bw=1049KiB/s (1074kB/s), 260KiB/s-274KiB/s (267kB/s-281kB/s), io=24.6MiB (25.8MB), run=23151-24034msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=158870: Thu Sep 25 10:13:02 2025 read: IOPS=2, BW=2700KiB/s (2765kB/s)(7168KiB/2655msec) clat (usec): min=755, max=960544, avg=209199.63, stdev=337920.41 lat (usec): min=756, max=960545, avg=209200.74, stdev=337920.46 clat percentiles (usec): | 1.00th=[ 758], 5.00th=[ 758], 10.00th=[ 758], 20.00th=[ 807], | 30.00th=[ 66847], 40.00th=[ 66847], 50.00th=[120062], 60.00th=[145753], | 70.00th=[145753], 80.00th=[168821], 90.00th=[960496], 95.00th=[960496], | 99.00th=[960496], 99.50th=[960496], 99.90th=[960496], 99.95th=[960496], | 99.99th=[960496] bw ( KiB/s): min= 1928, max= 4096, per=53.68%, avg=2883.50, stdev=1092.47, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.50, samples=4 write: IOPS=1, BW=1928KiB/s (1975kB/s)(5120KiB/2655msec); 0 zone resets clat (msec): min=95, max=459, avg=232.63, stdev=135.80 lat (msec): min=95, max=459, avg=232.69, stdev=135.80 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 96], 10.00th=[ 96], 20.00th=[ 96], | 30.00th=[ 188], 40.00th=[ 188], 50.00th=[ 192], 60.00th=[ 192], | 70.00th=[ 230], 80.00th=[ 230], 90.00th=[ 460], 95.00th=[ 460], | 99.00th=[ 460], 99.50th=[ 460], 99.90th=[ 460], 99.95th=[ 460], | 99.99th=[ 460] bw ( KiB/s): min= 1928, max= 3984, per=35.28%, avg=2653.33, stdev=1153.95, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 lat (usec) : 1000=16.67% lat (msec) : 100=16.67%, 250=50.00%, 500=8.33%, 1000=8.33% cpu : usr=0.04%, sys=42.73%, ctx=1339, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158871: Thu Sep 25 10:13:02 2025 read: IOPS=0, BW=677KiB/s (693kB/s)(2048KiB/3027msec) clat (msec): min=93, max=136, avg=114.97, stdev=29.97 lat (msec): min=93, max=136, avg=114.97, stdev=29.97 clat percentiles (msec): | 1.00th=[ 94], 5.00th=[ 94], 10.00th=[ 94], 20.00th=[ 94], | 30.00th=[ 94], 40.00th=[ 94], 50.00th=[ 94], 60.00th=[ 136], | 70.00th=[ 136], 80.00th=[ 136], 90.00th=[ 136], 95.00th=[ 136], | 99.00th=[ 136], 99.50th=[ 136], 99.90th=[ 136], 99.95th=[ 136], | 99.99th=[ 136] bw ( KiB/s): min= 1762, max= 1928, per=34.34%, avg=1845.00, stdev=117.38, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3383KiB/s (3464kB/s)(10.0MiB/3027msec); 0 zone resets clat (msec): min=86, max=1222, avg=268.54, stdev=346.77 lat (msec): min=87, max=1222, avg=268.60, stdev=346.76 clat percentiles (msec): | 1.00th=[ 87], 5.00th=[ 87], 10.00th=[ 87], 20.00th=[ 107], | 30.00th=[ 113], 40.00th=[ 126], 50.00th=[ 136], 60.00th=[ 148], | 70.00th=[ 171], 80.00th=[ 174], 90.00th=[ 405], 95.00th=[ 1217], | 99.00th=[ 1217], 99.50th=[ 1217], 99.90th=[ 1217], 99.95th=[ 1217], | 99.99th=[ 1217] bw ( KiB/s): min= 3856, max= 8192, per=80.10%, avg=6024.00, stdev=3066.02, samples=2 iops : min= 3, max= 8, avg= 5.50, stdev= 3.54, samples=2 lat (msec) : 100=16.67%, 250=66.67%, 500=8.33%, 2000=8.33% cpu : usr=0.00%, sys=37.97%, ctx=116, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158872: Thu Sep 25 10:13:02 2025 read: IOPS=1, BW=1619KiB/s (1657kB/s)(6144KiB/3796msec) clat (usec): min=863, max=220415, avg=111493.27, stdev=90155.59 lat (usec): min=865, max=220416, avg=111494.39, stdev=90155.59 clat percentiles (usec): | 1.00th=[ 865], 5.00th=[ 865], 10.00th=[ 865], 20.00th=[ 947], | 30.00th=[ 947], 40.00th=[141558], 50.00th=[141558], 60.00th=[149947], | 70.00th=[156238], 80.00th=[156238], 90.00th=[221250], 95.00th=[221250], | 99.00th=[221250], 99.50th=[221250], 99.90th=[221250], 99.95th=[221250], | 99.99th=[221250] bw ( KiB/s): min= 2048, max= 3984, per=61.36%, avg=3296.00, stdev=1082.69, samples=3 iops : min= 2, max= 3, avg= 2.67, stdev= 0.58, samples=3 write: IOPS=1, BW=1619KiB/s (1657kB/s)(6144KiB/3796msec); 0 zone resets clat (msec): min=141, max=1284, avg=507.63, stdev=470.77 lat (msec): min=141, max=1284, avg=507.67, stdev=470.78 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 205], | 30.00th=[ 205], 40.00th=[ 207], 50.00th=[ 207], 60.00th=[ 313], | 70.00th=[ 894], 80.00th=[ 894], 90.00th=[ 1284], 95.00th=[ 1284], | 99.00th=[ 1284], 99.50th=[ 1284], 99.90th=[ 1284], 99.95th=[ 1284], | 99.99th=[ 1284] bw ( KiB/s): min= 1735, max= 2048, per=25.93%, avg=1950.20, stdev=130.08, samples=5 iops : min= 1, max= 2, avg= 1.40, stdev= 0.55, samples=5 lat (usec) : 1000=16.67% lat (msec) : 250=58.33%, 500=8.33%, 1000=8.33%, 2000=8.33% cpu : usr=0.00%, sys=28.14%, ctx=660, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158873: Thu Sep 25 10:13:02 2025 read: IOPS=1, BW=1343KiB/s (1375kB/s)(5120KiB/3812msec) clat (usec): min=606, max=902599, avg=287578.59, stdev=370178.47 lat (usec): min=607, max=902600, avg=287579.50, stdev=370178.58 clat percentiles (usec): | 1.00th=[ 611], 5.00th=[ 611], 10.00th=[ 611], 20.00th=[ 611], | 30.00th=[ 742], 40.00th=[ 742], 50.00th=[219153], 60.00th=[219153], | 70.00th=[312476], 80.00th=[312476], 90.00th=[901776], 95.00th=[901776], | 99.00th=[901776], 99.50th=[901776], 99.90th=[901776], 99.95th=[901776], | 99.99th=[901776] bw ( KiB/s): min= 3856, max= 4096, per=74.01%, avg=3976.00, stdev=169.71, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1880KiB/s (1926kB/s)(7168KiB/3812msec); 0 zone resets clat (msec): min=32, max=1258, avg=329.65, stdev=418.81 lat (msec): min=32, max=1258, avg=329.72, stdev=418.82 clat percentiles (msec): | 1.00th=[ 33], 5.00th=[ 33], 10.00th=[ 33], 20.00th=[ 142], | 30.00th=[ 169], 40.00th=[ 169], 50.00th=[ 186], 60.00th=[ 190], | 70.00th=[ 190], 80.00th=[ 330], 90.00th=[ 1267], 95.00th=[ 1267], | 99.00th=[ 1267], 99.50th=[ 1267], 99.90th=[ 1267], 99.95th=[ 1267], | 99.99th=[ 1267] bw ( KiB/s): min= 1759, max= 4096, per=39.51%, avg=2971.75, stdev=1239.98, samples=4 iops : min= 1, max= 4, avg= 2.50, stdev= 1.29, samples=4 lat (usec) : 750=16.67% lat (msec) : 50=8.33%, 250=41.67%, 500=16.67%, 1000=8.33%, 2000=8.33% cpu : usr=0.00%, sys=30.70%, ctx=585, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5373KiB/s (5501kB/s), 677KiB/s-2700KiB/s (693kB/s-2765kB/s), io=20.0MiB (20.0MB), run=2655-3812msec WRITE: bw=7522KiB/s (7702kB/s), 1619KiB/s-3383KiB/s (1657kB/s-3464kB/s), io=28.0MiB (29.4MB), run=2655-3812msec rand-rw: (groupid=0, jobs=1): err= 0: pid=158866: Thu Sep 25 10:13:03 2025 read: IOPS=1, BW=1706KiB/s (1747kB/s)(7168KiB/4201msec) clat (msec): min=50, max=1261, avg=332.42, stdev=485.81 lat (msec): min=50, max=1261, avg=332.42, stdev=485.81 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 51], 10.00th=[ 51], 20.00th=[ 58], | 30.00th=[ 61], 40.00th=[ 61], 50.00th=[ 68], 60.00th=[ 69], | 70.00th=[ 69], 80.00th=[ 760], 90.00th=[ 1267], 95.00th=[ 1267], | 99.00th=[ 1267], 99.50th=[ 1267], 99.90th=[ 1267], 99.95th=[ 1267], | 99.99th=[ 1267] bw ( KiB/s): min= 1715, max= 4096, per=49.68%, avg=2375.60, stdev=971.10, samples=5 iops : min= 1, max= 4, avg= 1.80, stdev= 1.30, samples=5 write: IOPS=1, BW=1219KiB/s (1248kB/s)(5120KiB/4201msec); 0 zone resets clat (msec): min=57, max=1016, avg=364.55, stdev=392.10 lat (msec): min=57, max=1016, avg=364.61, stdev=392.10 clat percentiles (msec): | 1.00th=[ 58], 5.00th=[ 58], 10.00th=[ 58], 20.00th=[ 58], | 30.00th=[ 69], 40.00th=[ 69], 50.00th=[ 279], 60.00th=[ 279], | 70.00th=[ 401], 80.00th=[ 401], 90.00th=[ 1020], 95.00th=[ 1020], | 99.00th=[ 1020], 99.50th=[ 1020], 99.90th=[ 1020], 99.95th=[ 1020], | 99.99th=[ 1020] bw ( KiB/s): min= 2048, max= 3984, per=40.23%, avg=2693.33, stdev=1117.75, samples=3 iops : min= 2, max= 3, avg= 2.33, stdev= 0.58, samples=3 lat (msec) : 100=58.33%, 500=16.67%, 1000=8.33%, 2000=16.67% cpu : usr=0.00%, sys=7.64%, ctx=95, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158867: Thu Sep 25 10:13:03 2025 read: IOPS=0, BW=490KiB/s (501kB/s)(2048KiB/4182msec) clat (msec): min=78, max=364, avg=221.48, stdev=201.56 lat (msec): min=78, max=364, avg=221.48, stdev=201.56 clat percentiles (msec): | 1.00th=[ 80], 5.00th=[ 80], 10.00th=[ 80], 20.00th=[ 80], | 30.00th=[ 80], 40.00th=[ 80], 50.00th=[ 80], 60.00th=[ 363], | 70.00th=[ 363], 80.00th=[ 363], 90.00th=[ 363], 95.00th=[ 363], | 99.00th=[ 363], 99.50th=[ 363], 99.90th=[ 363], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 1712, max= 2027, per=39.09%, avg=1869.50, stdev=222.74, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2449KiB/s (2507kB/s)(10.0MiB/4182msec); 0 zone resets clat (msec): min=57, max=1138, avg=369.95, stdev=418.11 lat (msec): min=57, max=1138, avg=370.02, stdev=418.10 clat percentiles (msec): | 1.00th=[ 58], 5.00th=[ 58], 10.00th=[ 58], 20.00th=[ 81], | 30.00th=[ 103], 40.00th=[ 157], 50.00th=[ 174], 60.00th=[ 184], | 70.00th=[ 236], 80.00th=[ 439], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 1712, max= 7968, per=66.66%, avg=4462.75, stdev=3088.30, samples=4 iops : min= 1, max= 7, avg= 3.75, stdev= 3.20, samples=4 lat (msec) : 100=25.00%, 250=41.67%, 500=16.67%, 2000=16.67% cpu : usr=0.00%, sys=9.42%, ctx=115, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158868: Thu Sep 25 10:13:03 2025 read: IOPS=1, BW=1435KiB/s (1469kB/s)(6144KiB/4282msec) clat (msec): min=18, max=496, avg=208.90, stdev=194.07 lat (msec): min=18, max=496, avg=208.90, stdev=194.07 clat percentiles (msec): | 1.00th=[ 19], 5.00th=[ 19], 10.00th=[ 19], 20.00th=[ 46], | 30.00th=[ 46], 40.00th=[ 95], 50.00th=[ 95], 60.00th=[ 218], | 70.00th=[ 380], 80.00th=[ 380], 90.00th=[ 498], 95.00th=[ 498], | 99.00th=[ 498], 99.50th=[ 498], 99.90th=[ 498], 99.95th=[ 498], | 99.99th=[ 498] bw ( KiB/s): min= 3984, max= 4055, per=84.05%, avg=4019.50, stdev=50.20, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1435KiB/s (1469kB/s)(6144KiB/4282msec); 0 zone resets clat (msec): min=47, max=1217, avg=497.52, stdev=524.10 lat (msec): min=47, max=1217, avg=497.58, stdev=524.12 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 48], 10.00th=[ 48], 20.00th=[ 103], | 30.00th=[ 103], 40.00th=[ 157], 50.00th=[ 157], 60.00th=[ 363], | 70.00th=[ 1099], 80.00th=[ 1099], 90.00th=[ 1217], 95.00th=[ 1217], | 99.00th=[ 1217], 99.50th=[ 1217], 99.90th=[ 1217], 99.95th=[ 1217], | 99.99th=[ 1217] bw ( KiB/s): min= 2048, max= 3430, per=37.47%, avg=2508.67, stdev=797.90, samples=3 iops : min= 2, max= 3, avg= 2.33, stdev= 0.58, samples=3 lat (msec) : 20=8.33%, 50=16.67%, 100=8.33%, 250=25.00%, 500=25.00% lat (msec) : 2000=16.67% cpu : usr=0.02%, sys=7.57%, ctx=88, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158869: Thu Sep 25 10:13:03 2025 read: IOPS=1, BW=1203KiB/s (1232kB/s)(5120KiB/4257msec) clat (msec): min=23, max=653, avg=302.84, stdev=263.07 lat (msec): min=23, max=653, avg=302.84, stdev=263.07 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 24], 10.00th=[ 24], 20.00th=[ 24], | 30.00th=[ 99], 40.00th=[ 99], 50.00th=[ 259], 60.00th=[ 259], | 70.00th=[ 481], 80.00th=[ 481], 90.00th=[ 651], 95.00th=[ 651], | 99.00th=[ 651], 99.50th=[ 651], 99.90th=[ 651], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 2027, max= 3984, per=56.18%, avg=2686.33, stdev=1123.86, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 write: IOPS=1, BW=1684KiB/s (1724kB/s)(7168KiB/4257msec); 0 zone resets clat (msec): min=69, max=1574, avg=389.07, stdev=554.18 lat (msec): min=69, max=1574, avg=389.13, stdev=554.18 clat percentiles (msec): | 1.00th=[ 70], 5.00th=[ 70], 10.00th=[ 70], 20.00th=[ 74], | 30.00th=[ 111], 40.00th=[ 111], 50.00th=[ 123], 60.00th=[ 178], | 70.00th=[ 178], 80.00th=[ 592], 90.00th=[ 1569], 95.00th=[ 1569], | 99.00th=[ 1569], 99.50th=[ 1569], 99.90th=[ 1569], 99.95th=[ 1569], | 99.99th=[ 1569] bw ( KiB/s): min= 1712, max= 4055, per=36.62%, avg=2451.75, stdev=1078.89, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 50=8.33%, 100=25.00%, 250=25.00%, 500=16.67%, 750=16.67% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=10.43%, ctx=159, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4783KiB/s (4898kB/s), 490KiB/s-1706KiB/s (501kB/s-1747kB/s), io=20.0MiB (20.0MB), run=4182-4282msec WRITE: bw=6696KiB/s (6857kB/s), 1219KiB/s-2449KiB/s (1248kB/s-2507kB/s), io=28.0MiB (29.4MB), run=4182-4282msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=158880: Thu Sep 25 10:13:13 2025 read: IOPS=1, BW=5322KiB/s (5450kB/s)(12.0MiB/2309msec) clat (msec): min=615, max=988, avg=761.96, stdev=198.76 lat (msec): min=615, max=988, avg=761.96, stdev=198.76 clat percentiles (msec): | 1.00th=[ 617], 5.00th=[ 617], 10.00th=[ 617], 20.00th=[ 617], | 30.00th=[ 617], 40.00th=[ 684], 50.00th=[ 684], 60.00th=[ 684], | 70.00th=[ 986], 80.00th=[ 986], 90.00th=[ 986], 95.00th=[ 986], | 99.00th=[ 986], 99.50th=[ 986], 99.90th=[ 986], 99.95th=[ 986], | 99.99th=[ 986] bw ( KiB/s): min= 8015, max= 8192, per=100.00%, avg=8103.50, stdev=125.16, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 1000=33.33% cpu : usr=0.00%, sys=38.78%, ctx=1980, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158882: Thu Sep 25 10:13:13 2025 read: IOPS=0, BW=1040KiB/s (1065kB/s)(4096KiB/3937msec) clat (nsec): min=999029k, max=999029k, avg=999029191.00, stdev= 0.00 lat (nsec): min=999032k, max=999032k, avg=999031681.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1003], 5.00th=[ 1003], 10.00th=[ 1003], 20.00th=[ 1003], | 30.00th=[ 1003], 40.00th=[ 1003], 50.00th=[ 1003], 60.00th=[ 1003], | 70.00th=[ 1003], 80.00th=[ 1003], 90.00th=[ 1003], 95.00th=[ 1003], | 99.00th=[ 1003], 99.50th=[ 1003], 99.90th=[ 1003], 99.95th=[ 1003], | 99.99th=[ 1003] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2081KiB/s (2131kB/s)(8192KiB/3937msec); 0 zone resets clat (msec): min=1402, max=1522, avg=1462.66, stdev=84.83 lat (msec): min=1402, max=1522, avg=1462.92, stdev=84.84 clat percentiles (msec): | 1.00th=[ 1401], 5.00th=[ 1401], 10.00th=[ 1401], 20.00th=[ 1401], | 30.00th=[ 1401], 40.00th=[ 1401], 50.00th=[ 1401], 60.00th=[ 1519], | 70.00th=[ 1519], 80.00th=[ 1519], 90.00th=[ 1519], 95.00th=[ 1519], | 99.00th=[ 1519], 99.50th=[ 1519], 99.90th=[ 1519], 99.95th=[ 1519], | 99.99th=[ 1519] bw ( KiB/s): min= 8175, max= 8175, per=98.22%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=32.47%, ctx=1163, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158884: Thu Sep 25 10:13:13 2025 write: IOPS=0, BW=3277KiB/s (3355kB/s)(12.0MiB/3750msec); 0 zone resets clat (msec): min=961, max=1547, avg=1246.37, stdev=293.49 lat (msec): min=961, max=1548, avg=1246.68, stdev=293.49 clat percentiles (msec): | 1.00th=[ 961], 5.00th=[ 961], 10.00th=[ 961], 20.00th=[ 961], | 30.00th=[ 961], 40.00th=[ 1234], 50.00th=[ 1234], 60.00th=[ 1234], | 70.00th=[ 1552], 80.00th=[ 1552], 90.00th=[ 1552], 95.00th=[ 1552], | 99.00th=[ 1552], 99.50th=[ 1552], 99.90th=[ 1552], 99.95th=[ 1552], | 99.99th=[ 1552] bw ( KiB/s): min= 7922, max= 8015, per=95.74%, avg=7968.50, stdev=65.76, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=44.15%, ctx=252, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158886: Thu Sep 25 10:13:13 2025 write: IOPS=0, BW=3747KiB/s (3837kB/s)(12.0MiB/3279msec); 0 zone resets clat (msec): min=592, max=1440, avg=1089.05, stdev=442.45 lat (msec): min=592, max=1440, avg=1089.33, stdev=442.46 clat percentiles (msec): | 1.00th=[ 592], 5.00th=[ 592], 10.00th=[ 592], 20.00th=[ 592], | 30.00th=[ 592], 40.00th=[ 1234], 50.00th=[ 1234], 60.00th=[ 1234], | 70.00th=[ 1435], 80.00th=[ 1435], 90.00th=[ 1435], 95.00th=[ 1435], | 99.00th=[ 1435], 99.50th=[ 1435], 99.90th=[ 1435], 99.95th=[ 1435], | 99.99th=[ 1435] bw ( KiB/s): min= 8015, max= 8175, per=97.26%, avg=8095.00, stdev=113.14, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=44.69%, ctx=171, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4162KiB/s (4261kB/s), 1040KiB/s-5322KiB/s (1065kB/s-5450kB/s), io=16.0MiB (16.8MB), run=2309-3937msec WRITE: bw=8323KiB/s (8523kB/s), 2081KiB/s-3747KiB/s (2131kB/s-3837kB/s), io=32.0MiB (33.6MB), run=3279-3937msec rand-rw: (groupid=0, jobs=1): err= 0: pid=158881: Thu Sep 25 10:13:14 2025 read: IOPS=0, BW=3001KiB/s (3073kB/s)(12.0MiB/4095msec) clat (msec): min=1149, max=1463, avg=1354.97, stdev=177.94 lat (msec): min=1149, max=1463, avg=1354.97, stdev=177.94 clat percentiles (msec): | 1.00th=[ 1150], 5.00th=[ 1150], 10.00th=[ 1150], 20.00th=[ 1150], | 30.00th=[ 1150], 40.00th=[ 1452], 50.00th=[ 1452], 60.00th=[ 1452], | 70.00th=[ 1469], 80.00th=[ 1469], 90.00th=[ 1469], 95.00th=[ 1469], | 99.00th=[ 1469], 99.50th=[ 1469], 99.90th=[ 1469], 99.95th=[ 1469], | 99.99th=[ 1469] bw ( KiB/s): min= 8126, max= 8192, per=100.00%, avg=8159.00, stdev=46.67, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 2000=100.00% cpu : usr=0.00%, sys=7.03%, ctx=70, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158883: Thu Sep 25 10:13:14 2025 read: IOPS=0, BW=961KiB/s (984kB/s)(4096KiB/4261msec) clat (nsec): min=1642.9M, max=1642.9M, avg=1642865182.00, stdev= 0.00 lat (nsec): min=1642.9M, max=1642.9M, avg=1642867789.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1636], 5.00th=[ 1636], 10.00th=[ 1636], 20.00th=[ 1636], | 30.00th=[ 1636], 40.00th=[ 1636], 50.00th=[ 1636], 60.00th=[ 1636], | 70.00th=[ 1636], 80.00th=[ 1636], 90.00th=[ 1636], 95.00th=[ 1636], | 99.00th=[ 1636], 99.50th=[ 1636], 99.90th=[ 1636], 99.95th=[ 1636], | 99.99th=[ 1636] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1923KiB/s (1969kB/s)(8192KiB/4261msec); 0 zone resets clat (msec): min=372, max=2218, avg=1295.26, stdev=1305.52 lat (msec): min=372, max=2218, avg=1295.46, stdev=1305.63 clat percentiles (msec): | 1.00th=[ 372], 5.00th=[ 372], 10.00th=[ 372], 20.00th=[ 372], | 30.00th=[ 372], 40.00th=[ 372], 50.00th=[ 372], 60.00th=[ 2232], | 70.00th=[ 2232], 80.00th=[ 2232], 90.00th=[ 2232], 95.00th=[ 2232], | 99.00th=[ 2232], 99.50th=[ 2232], 99.90th=[ 2232], 99.95th=[ 2232], | 99.99th=[ 2232] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.02%, sys=9.46%, ctx=99, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158885: Thu Sep 25 10:13:14 2025 write: IOPS=0, BW=2904KiB/s (2974kB/s)(12.0MiB/4231msec); 0 zone resets clat (msec): min=1094, max=1787, avg=1401.79, stdev=353.36 lat (msec): min=1094, max=1787, avg=1402.01, stdev=353.29 clat percentiles (msec): | 1.00th=[ 1099], 5.00th=[ 1099], 10.00th=[ 1099], 20.00th=[ 1099], | 30.00th=[ 1099], 40.00th=[ 1318], 50.00th=[ 1318], 60.00th=[ 1318], | 70.00th=[ 1787], 80.00th=[ 1787], 90.00th=[ 1787], 95.00th=[ 1787], | 99.00th=[ 1787], 99.50th=[ 1787], 99.90th=[ 1787], 99.95th=[ 1787], | 99.99th=[ 1787] bw ( KiB/s): min= 8126, max= 8192, per=100.00%, avg=8159.00, stdev=46.67, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 2000=100.00% cpu : usr=0.00%, sys=6.05%, ctx=75, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158887: Thu Sep 25 10:13:14 2025 write: IOPS=0, BW=2783KiB/s (2849kB/s)(12.0MiB/4416msec); 0 zone resets clat (msec): min=109, max=2667, avg=1458.44, stdev=1284.48 lat (msec): min=109, max=2667, avg=1458.72, stdev=1284.47 clat percentiles (msec): | 1.00th=[ 110], 5.00th=[ 110], 10.00th=[ 110], 20.00th=[ 110], | 30.00th=[ 110], 40.00th=[ 1603], 50.00th=[ 1603], 60.00th=[ 1603], | 70.00th=[ 2668], 80.00th=[ 2668], 90.00th=[ 2668], 95.00th=[ 2668], | 99.00th=[ 2668], 99.50th=[ 2668], 99.90th=[ 2668], 99.95th=[ 2668], | 99.99th=[ 2668] bw ( KiB/s): min= 8126, max= 8126, per=100.00%, avg=8126.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.05%, sys=9.83%, ctx=213, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3845KiB/s (3937kB/s), 961KiB/s-3001KiB/s (984kB/s-3073kB/s), io=16.0MiB (16.8MB), run=4095-4261msec WRITE: bw=7420KiB/s (7598kB/s), 1923KiB/s-2904KiB/s (1969kB/s-2974kB/s), io=32.0MiB (33.6MB), run=4231-4416msec