== sanity test 398b: DIO and buffer IO race ============== 23:28:05 (1764736085) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.95748 s, 17.0 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146707: Tue Dec 2 23:29:05 2025 read: IOPS=29, BW=119KiB/s (122kB/s)(5992KiB/50221msec) clat (usec): min=86, max=718569, avg=16726.23, stdev=21847.07 lat (usec): min=87, max=718570, avg=16728.11, stdev=21847.52 clat percentiles (usec): | 1.00th=[ 188], 5.00th=[ 359], 10.00th=[ 5932], 20.00th=[ 8455], | 30.00th=[ 10028], 40.00th=[ 11731], 50.00th=[ 13960], 60.00th=[ 16319], | 70.00th=[ 19530], 80.00th=[ 23462], 90.00th=[ 28705], 95.00th=[ 34341], | 99.00th=[ 62653], 99.50th=[ 83362], 99.90th=[154141], 99.95th=[717226], | 99.99th=[717226] bw ( KiB/s): min= 7, max= 264, per=24.91%, avg=119.06, stdev=35.39, samples=100 iops : min= 1, max= 66, avg=29.35, stdev= 8.91, samples=100 write: IOPS=31, BW=125KiB/s (128kB/s)(6296KiB/50221msec); 0 zone resets clat (usec): min=578, max=125808, avg=15857.22, stdev=14565.13 lat (usec): min=579, max=125809, avg=15859.67, stdev=14564.20 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 1516], 10.00th=[ 2180], 20.00th=[ 2933], | 30.00th=[ 4817], 40.00th=[ 9503], 50.00th=[ 13042], 60.00th=[ 16450], | 70.00th=[ 20317], 80.00th=[ 25560], 90.00th=[ 34341], 95.00th=[ 41157], | 99.00th=[ 64750], 99.50th=[ 84411], 99.90th=[115868], 99.95th=[125305], | 99.99th=[125305] bw ( KiB/s): min= 40, max= 280, per=25.25%, avg=126.27, stdev=35.70, samples=99 iops : min= 10, max= 70, avg=31.15, stdev= 8.95, samples=99 lat (usec) : 100=0.03%, 250=0.81%, 500=2.77%, 750=0.39%, 1000=0.39% lat (msec) : 2=3.87%, 4=9.34%, 10=18.00%, 20=34.64%, 50=27.64% lat (msec) : 100=1.86%, 250=0.23%, 750=0.03% cpu : usr=0.14%, sys=20.63%, ctx=7570, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146708: Tue Dec 2 23:29:05 2025 read: IOPS=30, BW=121KiB/s (123kB/s)(5940KiB/49265msec) clat (usec): min=96, max=170525, avg=15811.91, stdev=12152.91 lat (usec): min=96, max=170526, avg=15812.83, stdev=12152.90 clat percentiles (usec): | 1.00th=[ 151], 5.00th=[ 334], 10.00th=[ 6128], 20.00th=[ 8586], | 30.00th=[ 10159], 40.00th=[ 11469], 50.00th=[ 13304], 60.00th=[ 15926], | 70.00th=[ 18744], 80.00th=[ 22414], 90.00th=[ 28443], 95.00th=[ 33817], | 99.00th=[ 55313], 99.50th=[ 71828], 99.90th=[166724], 99.95th=[170918], | 99.99th=[170918] bw ( KiB/s): min= 7, max= 207, per=25.19%, avg=120.43, stdev=31.02, samples=98 iops : min= 1, max= 51, avg=29.69, stdev= 7.73, samples=98 write: IOPS=32, BW=129KiB/s (132kB/s)(6348KiB/49265msec); 0 zone resets clat (usec): min=718, max=712399, avg=16116.40, stdev=22907.08 lat (usec): min=719, max=712400, avg=16117.59, stdev=22907.19 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1598], 10.00th=[ 2180], 20.00th=[ 3064], | 30.00th=[ 4883], 40.00th=[ 9372], 50.00th=[ 12911], 60.00th=[ 16188], | 70.00th=[ 20055], 80.00th=[ 25822], 90.00th=[ 33424], 95.00th=[ 41681], | 99.00th=[ 61604], 99.50th=[ 77071], 99.90th=[204473], 99.95th=[708838], | 99.99th=[708838] bw ( KiB/s): min= 63, max= 224, per=26.02%, avg=130.10, stdev=32.86, samples=97 iops : min= 15, max= 56, avg=32.11, stdev= 8.33, samples=97 lat (usec) : 100=0.03%, 250=1.20%, 500=2.60%, 750=0.46%, 1000=0.39% lat (msec) : 2=3.74%, 4=9.51%, 10=17.71%, 20=36.20%, 50=26.37% lat (msec) : 100=1.53%, 250=0.23%, 750=0.03% cpu : usr=0.15%, sys=20.75%, ctx=7520, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146710: Tue Dec 2 23:29:05 2025 read: IOPS=30, BW=123KiB/s (126kB/s)(6028KiB/48862msec) clat (usec): min=117, max=715467, avg=16175.94, stdev=21098.49 lat (usec): min=118, max=715468, avg=16177.15, stdev=21098.52 clat percentiles (usec): | 1.00th=[ 163], 5.00th=[ 326], 10.00th=[ 5800], 20.00th=[ 8586], | 30.00th=[ 10028], 40.00th=[ 11338], 50.00th=[ 13173], 60.00th=[ 16188], | 70.00th=[ 19006], 80.00th=[ 22676], 90.00th=[ 27919], 95.00th=[ 33162], | 99.00th=[ 55837], 99.50th=[ 66323], 99.90th=[ 96994], 99.95th=[717226], | 99.99th=[717226] bw ( KiB/s): min= 7, max= 191, per=25.73%, avg=123.00, stdev=32.05, samples=97 iops : min= 1, max= 47, avg=30.33, stdev= 8.03, samples=97 write: IOPS=32, BW=128KiB/s (131kB/s)(6260KiB/48862msec); 0 zone resets clat (usec): min=673, max=141105, avg=15514.76, stdev=14224.09 lat (usec): min=674, max=141107, avg=15516.32, stdev=14224.42 clat percentiles (usec): | 1.00th=[ 979], 5.00th=[ 1418], 10.00th=[ 2024], 20.00th=[ 2769], | 30.00th=[ 4146], 40.00th=[ 8586], 50.00th=[ 12387], 60.00th=[ 15926], | 70.00th=[ 20579], 80.00th=[ 25822], 90.00th=[ 33817], 95.00th=[ 42206], | 99.00th=[ 59507], 99.50th=[ 66847], 99.90th=[123208], 99.95th=[141558], | 99.99th=[141558] bw ( KiB/s): min= 21, max= 224, per=25.48%, avg=127.42, stdev=35.45, samples=97 iops : min= 5, max= 56, avg=31.44, stdev= 8.89, samples=97 lat (usec) : 250=1.14%, 500=2.70%, 750=0.29%, 1000=0.55% lat (msec) : 2=4.65%, 4=9.80%, 10=17.09%, 20=34.51%, 50=27.12% lat (msec) : 100=2.05%, 250=0.07%, 750=0.03% cpu : usr=0.17%, sys=20.87%, ctx=7459, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146712: Tue Dec 2 23:29:05 2025 read: IOPS=30, BW=123KiB/s (126kB/s)(6052KiB/49371msec) clat (usec): min=113, max=650051, avg=15898.01, stdev=19642.80 lat (usec): min=114, max=650052, avg=15899.03, stdev=19642.79 clat percentiles (usec): | 1.00th=[ 151], 5.00th=[ 334], 10.00th=[ 6456], 20.00th=[ 8586], | 30.00th=[ 10028], 40.00th=[ 11469], 50.00th=[ 13042], 60.00th=[ 15401], | 70.00th=[ 18220], 80.00th=[ 21890], 90.00th=[ 28181], 95.00th=[ 33424], | 99.00th=[ 55313], 99.50th=[ 63177], 99.90th=[162530], 99.95th=[650118], | 99.99th=[650118] bw ( KiB/s): min= 7, max= 200, per=25.62%, avg=122.48, stdev=33.58, samples=98 iops : min= 1, max= 50, avg=30.20, stdev= 8.47, samples=98 write: IOPS=31, BW=126KiB/s (129kB/s)(6236KiB/49371msec); 0 zone resets clat (usec): min=676, max=106741, avg=16107.26, stdev=14370.71 lat (usec): min=677, max=106743, avg=16108.48, stdev=14370.61 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1614], 10.00th=[ 2245], 20.00th=[ 3064], | 30.00th=[ 4948], 40.00th=[ 9372], 50.00th=[ 13042], 60.00th=[ 16909], | 70.00th=[ 20841], 80.00th=[ 26608], 90.00th=[ 34866], 95.00th=[ 42730], | 99.00th=[ 62129], 99.50th=[ 82314], 99.90th=[ 99091], 99.95th=[106431], | 99.99th=[106431] bw ( KiB/s): min= 29, max= 240, per=25.23%, avg=126.13, stdev=31.23, samples=98 iops : min= 7, max= 60, avg=31.12, stdev= 7.83, samples=98 lat (usec) : 250=1.37%, 500=2.67%, 750=0.23%, 1000=0.46% lat (msec) : 2=3.22%, 4=9.57%, 10=18.10%, 20=35.77%, 50=26.63% lat (msec) : 100=1.86%, 250=0.10%, 750=0.03% cpu : usr=0.15%, sys=20.63%, ctx=7587, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=478KiB/s (490kB/s), 119KiB/s-123KiB/s (122kB/s-126kB/s), io=23.4MiB (24.6MB), run=48862-50221msec WRITE: bw=501KiB/s (513kB/s), 125KiB/s-129KiB/s (128kB/s-132kB/s), io=24.6MiB (25.7MB), run=48862-50221msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146709: Tue Dec 2 23:30:53 2025 read: IOPS=9, BW=37.9KiB/s (38.8kB/s)(5992KiB/158288msec) clat (msec): min=3, max=720, avg=17.65, stdev=21.79 lat (msec): min=3, max=720, avg=17.65, stdev=21.79 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 19], 80.00th=[ 23], 90.00th=[ 31], 95.00th=[ 41], | 99.00th=[ 63], 99.50th=[ 91], 99.90th=[ 138], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 7, max= 120, per=26.05%, avg=39.33, stdev=22.33, samples=299 iops : min= 1, max= 30, avg= 9.38, stdev= 5.65, samples=299 write: IOPS=9, BW=39.8KiB/s (40.7kB/s)(6296KiB/158288msec); 0 zone resets clat (msec): min=19, max=869, avg=83.63, stdev=40.59 lat (msec): min=19, max=869, avg=83.63, stdev=40.59 clat percentiles (msec): | 1.00th=[ 27], 5.00th=[ 35], 10.00th=[ 44], 20.00th=[ 56], | 30.00th=[ 64], 40.00th=[ 71], 50.00th=[ 78], 60.00th=[ 86], | 70.00th=[ 95], 80.00th=[ 108], 90.00th=[ 131], 95.00th=[ 150], | 99.00th=[ 188], 99.50th=[ 205], 99.90th=[ 409], 99.95th=[ 869], | 99.99th=[ 869] bw ( KiB/s): min= 6, max= 111, per=24.96%, avg=39.44, stdev=14.45, samples=313 iops : min= 1, max= 27, avg= 9.41, stdev= 3.69, samples=313 lat (msec) : 4=0.13%, 10=12.86%, 20=22.79%, 50=19.95%, 100=30.79% lat (msec) : 250=13.35%, 500=0.07%, 750=0.03%, 1000=0.03% cpu : usr=0.06%, sys=2.51%, ctx=5408, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146711: Tue Dec 2 23:30:53 2025 read: IOPS=9, BW=37.5KiB/s (38.4kB/s)(5940KiB/158300msec) clat (msec): min=4, max=665, avg=17.43, stdev=21.53 lat (msec): min=4, max=665, avg=17.43, stdev=21.53 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 16], | 70.00th=[ 19], 80.00th=[ 23], 90.00th=[ 31], 95.00th=[ 37], | 99.00th=[ 65], 99.50th=[ 82], 99.90th=[ 230], 99.95th=[ 667], | 99.99th=[ 667] bw ( KiB/s): min= 7, max= 143, per=25.33%, avg=38.24, stdev=24.59, samples=305 iops : min= 1, max= 35, avg= 9.11, stdev= 6.21, samples=305 write: IOPS=10, BW=40.1KiB/s (41.1kB/s)(6348KiB/158300msec); 0 zone resets clat (msec): min=19, max=778, avg=83.31, stdev=38.33 lat (msec): min=19, max=778, avg=83.31, stdev=38.33 clat percentiles (msec): | 1.00th=[ 27], 5.00th=[ 35], 10.00th=[ 44], 20.00th=[ 55], | 30.00th=[ 63], 40.00th=[ 70], 50.00th=[ 79], 60.00th=[ 88], | 70.00th=[ 96], 80.00th=[ 109], 90.00th=[ 128], 95.00th=[ 144], | 99.00th=[ 190], 99.50th=[ 218], 99.90th=[ 249], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 6, max= 119, per=25.15%, avg=39.74, stdev=14.51, samples=313 iops : min= 1, max= 29, avg= 9.48, stdev= 3.69, samples=313 lat (msec) : 10=13.15%, 20=23.14%, 50=18.91%, 100=30.86%, 250=13.87% lat (msec) : 750=0.03%, 1000=0.03% cpu : usr=0.06%, sys=2.52%, ctx=5427, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146713: Tue Dec 2 23:30:53 2025 read: IOPS=9, BW=38.4KiB/s (39.3kB/s)(6028KiB/157157msec) clat (msec): min=3, max=283, avg=17.45, stdev=13.72 lat (msec): min=3, max=283, avg=17.45, stdev=13.72 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 11], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 20], 80.00th=[ 24], 90.00th=[ 32], 95.00th=[ 40], | 99.00th=[ 63], 99.50th=[ 75], 99.90th=[ 127], 99.95th=[ 284], | 99.99th=[ 284] bw ( KiB/s): min= 7, max= 168, per=26.15%, avg=39.49, stdev=24.37, samples=300 iops : min= 1, max= 42, avg= 9.42, stdev= 6.15, samples=300 write: IOPS=9, BW=39.8KiB/s (40.8kB/s)(6260KiB/157157msec); 0 zone resets clat (msec): min=21, max=778, avg=83.48, stdev=41.34 lat (msec): min=21, max=778, avg=83.48, stdev=41.34 clat percentiles (msec): | 1.00th=[ 28], 5.00th=[ 36], 10.00th=[ 44], 20.00th=[ 55], | 30.00th=[ 64], 40.00th=[ 72], 50.00th=[ 80], 60.00th=[ 87], | 70.00th=[ 96], 80.00th=[ 108], 90.00th=[ 125], 95.00th=[ 144], | 99.00th=[ 184], 99.50th=[ 199], 99.90th=[ 701], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 6, max= 103, per=25.03%, avg=39.54, stdev=14.15, samples=311 iops : min= 1, max= 25, avg= 9.43, stdev= 3.62, samples=311 lat (msec) : 4=0.03%, 10=13.18%, 20=22.53%, 50=20.31%, 100=30.76% lat (msec) : 250=13.02%, 500=0.10%, 750=0.03%, 1000=0.03% cpu : usr=0.05%, sys=2.54%, ctx=5401, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146714: Tue Dec 2 23:30:53 2025 read: IOPS=9, BW=38.7KiB/s (39.7kB/s)(6052KiB/156243msec) clat (msec): min=4, max=707, avg=17.77, stdev=23.37 lat (msec): min=4, max=707, avg=17.77, stdev=23.37 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 16], | 70.00th=[ 19], 80.00th=[ 23], 90.00th=[ 31], 95.00th=[ 38], | 99.00th=[ 67], 99.50th=[ 118], 99.90th=[ 284], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 6, max= 128, per=26.52%, avg=40.04, stdev=22.45, samples=297 iops : min= 1, max= 32, avg= 9.56, stdev= 5.66, samples=297 write: IOPS=9, BW=39.9KiB/s (40.9kB/s)(6236KiB/156243msec); 0 zone resets clat (msec): min=20, max=698, avg=82.83, stdev=36.85 lat (msec): min=20, max=698, avg=82.83, stdev=36.86 clat percentiles (msec): | 1.00th=[ 28], 5.00th=[ 37], 10.00th=[ 44], 20.00th=[ 56], | 30.00th=[ 64], 40.00th=[ 71], 50.00th=[ 78], 60.00th=[ 86], | 70.00th=[ 94], 80.00th=[ 107], 90.00th=[ 128], 95.00th=[ 144], | 99.00th=[ 190], 99.50th=[ 218], 99.90th=[ 266], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 15, max= 111, per=25.17%, avg=39.76, stdev=13.90, samples=308 iops : min= 3, max= 27, avg= 9.48, stdev= 3.56, samples=308 lat (msec) : 10=13.35%, 20=22.98%, 50=19.34%, 100=31.09%, 250=13.12% lat (msec) : 500=0.07%, 750=0.07% cpu : usr=0.04%, sys=2.57%, ctx=5300, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=152KiB/s (155kB/s), 37.5KiB/s-38.7KiB/s (38.4kB/s-39.7kB/s), io=23.4MiB (24.6MB), run=156243-158300msec WRITE: bw=159KiB/s (163kB/s), 39.8KiB/s-40.1KiB/s (40.7kB/s-41.1kB/s), io=24.6MiB (25.7MB), run=156243-158300msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146752: Tue Dec 2 23:31:16 2025 read: IOPS=21, BW=349KiB/s (358kB/s)(5920KiB/16953msec) clat (usec): min=97, max=221621, avg=22698.06, stdev=17698.98 lat (usec): min=97, max=221621, avg=22699.18, stdev=17699.07 clat percentiles (usec): | 1.00th=[ 130], 5.00th=[ 149], 10.00th=[ 247], 20.00th=[ 11731], | 30.00th=[ 14746], 40.00th=[ 18482], 50.00th=[ 21890], 60.00th=[ 25297], | 70.00th=[ 28967], 80.00th=[ 32637], 90.00th=[ 41157], 95.00th=[ 47973], | 99.00th=[ 65799], 99.50th=[ 67634], 99.90th=[221250], 99.95th=[221250], | 99.99th=[221250] bw ( KiB/s): min= 127, max= 604, per=25.21%, avg=349.36, stdev=114.56, samples=33 iops : min= 7, max= 37, avg=21.18, stdev= 7.19, samples=33 write: IOPS=23, BW=376KiB/s (385kB/s)(6368KiB/16953msec); 0 zone resets clat (usec): min=1166, max=193671, avg=21233.92, stdev=22682.12 lat (usec): min=1167, max=193672, avg=21236.53, stdev=22681.58 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 1549], 10.00th=[ 1876], 20.00th=[ 2966], | 30.00th=[ 4817], 40.00th=[ 8717], 50.00th=[ 13566], 60.00th=[ 20579], | 70.00th=[ 26870], 80.00th=[ 38011], 90.00th=[ 50070], 95.00th=[ 61604], | 99.00th=[103285], 99.50th=[115868], 99.90th=[193987], 99.95th=[193987], | 99.99th=[193987] bw ( KiB/s): min= 159, max= 544, per=25.89%, avg=377.52, stdev=103.52, samples=33 iops : min= 9, max= 34, avg=22.97, stdev= 6.59, samples=33 lat (usec) : 100=0.13%, 250=4.82%, 500=1.56%, 750=0.39% lat (msec) : 2=6.12%, 4=8.72%, 10=9.38%, 20=21.22%, 50=40.36% lat (msec) : 100=6.64%, 250=0.65% cpu : usr=0.08%, sys=11.41%, ctx=1605, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146753: Tue Dec 2 23:31:16 2025 read: IOPS=21, BW=338KiB/s (347kB/s)(5744KiB/16971msec) clat (usec): min=121, max=213701, avg=25370.54, stdev=20482.99 lat (usec): min=122, max=213702, avg=25371.72, stdev=20483.13 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 153], 10.00th=[ 258], 20.00th=[ 11207], | 30.00th=[ 15008], 40.00th=[ 19006], 50.00th=[ 22676], 60.00th=[ 26608], | 70.00th=[ 32637], 80.00th=[ 38536], 90.00th=[ 47973], 95.00th=[ 54789], | 99.00th=[ 79168], 99.50th=[133694], 99.90th=[212861], 99.95th=[212861], | 99.99th=[212861] bw ( KiB/s): min= 186, max= 535, per=24.44%, avg=338.73, stdev=83.88, samples=33 iops : min= 11, max= 33, avg=20.52, stdev= 5.23, samples=33 write: IOPS=24, BW=386KiB/s (395kB/s)(6544KiB/16971msec); 0 zone resets clat (usec): min=1282, max=168270, avg=19104.32, stdev=19248.49 lat (usec): min=1284, max=168272, avg=19112.46, stdev=19243.11 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 1598], 10.00th=[ 1909], 20.00th=[ 3195], | 30.00th=[ 5604], 40.00th=[ 9110], 50.00th=[ 12911], 60.00th=[ 17171], | 70.00th=[ 25035], 80.00th=[ 32900], 90.00th=[ 46400], 95.00th=[ 55837], | 99.00th=[ 69731], 99.50th=[ 81265], 99.90th=[168821], 99.95th=[168821], | 99.99th=[168821] bw ( KiB/s): min= 125, max= 795, per=26.40%, avg=384.85, stdev=136.52, samples=33 iops : min= 7, max= 49, avg=23.42, stdev= 8.61, samples=33 lat (usec) : 250=4.56%, 500=2.08%, 750=0.39% lat (msec) : 2=5.86%, 4=6.38%, 10=11.59%, 20=23.05%, 50=37.89% lat (msec) : 100=7.81%, 250=0.39% cpu : usr=0.08%, sys=11.89%, ctx=1580, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146754: Tue Dec 2 23:31:16 2025 read: IOPS=22, BW=362KiB/s (371kB/s)(6256KiB/17275msec) clat (usec): min=112, max=124760, avg=23917.61, stdev=18687.51 lat (usec): min=112, max=124761, avg=23918.62, stdev=18687.50 clat percentiles (usec): | 1.00th=[ 121], 5.00th=[ 147], 10.00th=[ 194], 20.00th=[ 8979], | 30.00th=[ 15270], 40.00th=[ 19530], 50.00th=[ 22414], 60.00th=[ 25297], | 70.00th=[ 29754], 80.00th=[ 34866], 90.00th=[ 44303], 95.00th=[ 55313], | 99.00th=[102237], 99.50th=[117965], 99.90th=[124257], 99.95th=[124257], | 99.99th=[124257] bw ( KiB/s): min= 160, max= 598, per=26.18%, avg=362.85, stdev=102.03, samples=34 iops : min= 10, max= 37, avg=22.09, stdev= 6.41, samples=34 write: IOPS=21, BW=349KiB/s (358kB/s)(6032KiB/17275msec); 0 zone resets clat (usec): min=1401, max=99566, avg=20768.42, stdev=20175.85 lat (usec): min=1403, max=99567, avg=20770.28, stdev=20175.91 clat percentiles (usec): | 1.00th=[ 1450], 5.00th=[ 1582], 10.00th=[ 1795], 20.00th=[ 2933], | 30.00th=[ 5604], 40.00th=[ 9634], 50.00th=[14746], 60.00th=[20579], | 70.00th=[26346], 80.00th=[34866], 90.00th=[49546], 95.00th=[60031], | 99.00th=[92799], 99.50th=[96994], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 93, max= 608, per=23.76%, avg=346.44, stdev=102.38, samples=34 iops : min= 5, max= 38, avg=21.00, stdev= 6.53, samples=34 lat (usec) : 250=5.86%, 500=2.34% lat (msec) : 2=6.90%, 4=6.12%, 10=9.38%, 20=19.79%, 50=40.89% lat (msec) : 100=8.20%, 250=0.52% cpu : usr=0.09%, sys=11.33%, ctx=1620, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146755: Tue Dec 2 23:31:16 2025 read: IOPS=22, BW=359KiB/s (368kB/s)(6032KiB/16792msec) clat (usec): min=88, max=192942, avg=24534.10, stdev=22233.99 lat (usec): min=88, max=192942, avg=24535.07, stdev=22233.95 clat percentiles (usec): | 1.00th=[ 130], 5.00th=[ 147], 10.00th=[ 186], 20.00th=[ 8586], | 30.00th=[ 14615], 40.00th=[ 17695], 50.00th=[ 21627], 60.00th=[ 25822], | 70.00th=[ 30278], 80.00th=[ 35390], 90.00th=[ 46924], 95.00th=[ 59507], | 99.00th=[133694], 99.50th=[164627], 99.90th=[193987], 99.95th=[193987], | 99.99th=[193987] bw ( KiB/s): min= 188, max= 512, per=25.76%, avg=357.06, stdev=97.64, samples=33 iops : min= 11, max= 32, avg=21.70, stdev= 6.20, samples=33 write: IOPS=23, BW=373KiB/s (381kB/s)(6256KiB/16792msec); 0 zone resets clat (usec): min=1179, max=106475, avg=19043.68, stdev=19356.71 lat (usec): min=1180, max=106477, avg=19046.92, stdev=19357.79 clat percentiles (usec): | 1.00th=[ 1369], 5.00th=[ 1516], 10.00th=[ 1696], 20.00th=[ 2966], | 30.00th=[ 5604], 40.00th=[ 8160], 50.00th=[ 12649], 60.00th=[ 18482], | 70.00th=[ 23200], 80.00th=[ 32375], 90.00th=[ 45351], 95.00th=[ 60031], | 99.00th=[ 87557], 99.50th=[102237], 99.90th=[106431], 99.95th=[106431], | 99.99th=[106431] bw ( KiB/s): min= 127, max= 692, per=25.65%, avg=373.94, stdev=125.86, samples=33 iops : min= 7, max= 43, avg=22.76, stdev= 7.96, samples=33 lat (usec) : 100=0.13%, 250=5.86%, 500=2.47%, 750=0.13%, 1000=0.39% lat (msec) : 2=7.03%, 4=5.86%, 10=11.33%, 20=20.70%, 50=38.54% lat (msec) : 100=6.77%, 250=0.78% cpu : usr=0.07%, sys=11.62%, ctx=1562, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1387KiB/s (1420kB/s), 338KiB/s-362KiB/s (347kB/s-371kB/s), io=23.4MiB (24.5MB), run=16792-17275msec WRITE: bw=1459KiB/s (1494kB/s), 349KiB/s-386KiB/s (358kB/s-395kB/s), io=24.6MiB (25.8MB), run=16792-17275msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146757: Tue Dec 2 23:31:47 2025 read: IOPS=7, BW=124KiB/s (127kB/s)(5920KiB/47738msec) clat (msec): min=5, max=194, avg=24.62, stdev=24.32 lat (msec): min=5, max=194, avg=24.63, stdev=24.32 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 31], 90.00th=[ 46], 95.00th=[ 69], | 99.00th=[ 131], 99.50th=[ 161], 99.90th=[ 194], 99.95th=[ 194], | 99.99th=[ 194] bw ( KiB/s): min= 31, max= 510, per=25.06%, avg=125.53, stdev=78.72, samples=92 iops : min= 1, max= 31, avg= 7.16, stdev= 4.96, samples=92 write: IOPS=8, BW=133KiB/s (137kB/s)(6368KiB/47738msec); 0 zone resets clat (msec): min=40, max=270, avg=96.95, stdev=34.15 lat (msec): min=40, max=270, avg=96.95, stdev=34.15 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 54], 10.00th=[ 59], 20.00th=[ 70], | 30.00th=[ 78], 40.00th=[ 84], 50.00th=[ 92], 60.00th=[ 99], | 70.00th=[ 108], 80.00th=[ 122], 90.00th=[ 142], 95.00th=[ 159], | 99.00th=[ 207], 99.50th=[ 253], 99.90th=[ 271], 99.95th=[ 271], | 99.99th=[ 271] bw ( KiB/s): min= 31, max= 224, per=25.24%, avg=132.99, stdev=43.91, samples=94 iops : min= 1, max= 14, avg= 7.62, stdev= 2.86, samples=94 lat (msec) : 10=6.12%, 20=23.70%, 50=15.63%, 100=32.81%, 250=21.48% lat (msec) : 500=0.26% cpu : usr=0.06%, sys=1.95%, ctx=1381, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146758: Tue Dec 2 23:31:47 2025 read: IOPS=7, BW=121KiB/s (124kB/s)(5744KiB/47584msec) clat (msec): min=5, max=166, avg=24.61, stdev=21.99 lat (msec): min=5, max=166, avg=24.61, stdev=21.99 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 35], 90.00th=[ 52], 95.00th=[ 73], | 99.00th=[ 127], 99.50th=[ 148], 99.90th=[ 167], 99.95th=[ 167], | 99.99th=[ 167] bw ( KiB/s): min= 31, max= 352, per=25.52%, avg=127.84, stdev=76.34, samples=89 iops : min= 1, max= 22, avg= 7.28, stdev= 4.80, samples=89 write: IOPS=8, BW=138KiB/s (141kB/s)(6544KiB/47584msec); 0 zone resets clat (msec): min=45, max=208, avg=94.62, stdev=29.26 lat (msec): min=45, max=208, avg=94.62, stdev=29.26 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 56], 10.00th=[ 63], 20.00th=[ 69], | 30.00th=[ 77], 40.00th=[ 84], 50.00th=[ 90], 60.00th=[ 99], | 70.00th=[ 107], 80.00th=[ 116], 90.00th=[ 136], 95.00th=[ 150], | 99.00th=[ 180], 99.50th=[ 199], 99.90th=[ 209], 99.95th=[ 209], | 99.99th=[ 209] bw ( KiB/s): min= 63, max= 224, per=25.98%, avg=136.93, stdev=39.05, samples=94 iops : min= 3, max= 14, avg= 7.87, stdev= 2.58, samples=94 lat (msec) : 10=6.51%, 20=22.92%, 50=13.02%, 100=37.89%, 250=19.66% cpu : usr=0.05%, sys=1.92%, ctx=1363, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146759: Tue Dec 2 23:31:47 2025 read: IOPS=8, BW=135KiB/s (138kB/s)(6256KiB/46291msec) clat (msec): min=6, max=113, avg=22.02, stdev=16.98 lat (msec): min=6, max=113, avg=22.03, stdev=16.97 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 19], | 70.00th=[ 23], 80.00th=[ 31], 90.00th=[ 43], 95.00th=[ 55], | 99.00th=[ 102], 99.50th=[ 108], 99.90th=[ 114], 99.95th=[ 114], | 99.99th=[ 114] bw ( KiB/s): min= 31, max= 384, per=27.41%, avg=137.33, stdev=84.45, samples=90 iops : min= 1, max= 24, avg= 7.88, stdev= 5.40, samples=90 write: IOPS=8, BW=130KiB/s (133kB/s)(6032KiB/46291msec); 0 zone resets clat (msec): min=43, max=282, avg=99.82, stdev=35.42 lat (msec): min=43, max=282, avg=99.82, stdev=35.42 clat percentiles (msec): | 1.00th=[ 49], 5.00th=[ 56], 10.00th=[ 62], 20.00th=[ 70], | 30.00th=[ 79], 40.00th=[ 86], 50.00th=[ 94], 60.00th=[ 104], | 70.00th=[ 111], 80.00th=[ 127], 90.00th=[ 148], 95.00th=[ 163], | 99.00th=[ 220], 99.50th=[ 245], 99.90th=[ 284], 99.95th=[ 284], | 99.99th=[ 284] bw ( KiB/s): min= 63, max= 223, per=24.58%, avg=129.52, stdev=37.76, samples=91 iops : min= 3, max= 13, avg= 7.40, stdev= 2.50, samples=91 lat (msec) : 10=7.03%, 20=25.39%, 50=15.63%, 100=30.60%, 250=21.22% lat (msec) : 500=0.13% cpu : usr=0.05%, sys=2.00%, ctx=1436, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146760: Tue Dec 2 23:31:47 2025 read: IOPS=7, BW=127KiB/s (130kB/s)(6032KiB/47399msec) clat (msec): min=4, max=230, avg=24.65, stdev=23.15 lat (msec): min=4, max=230, avg=24.65, stdev=23.15 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 21], | 70.00th=[ 25], 80.00th=[ 36], 90.00th=[ 50], 95.00th=[ 64], | 99.00th=[ 130], 99.50th=[ 165], 99.90th=[ 230], 99.95th=[ 230], | 99.99th=[ 230] bw ( KiB/s): min= 31, max= 320, per=25.39%, avg=127.20, stdev=74.55, samples=92 iops : min= 1, max= 20, avg= 7.26, stdev= 4.80, samples=92 write: IOPS=8, BW=132KiB/s (135kB/s)(6256KiB/47399msec); 0 zone resets clat (msec): min=39, max=270, avg=97.40, stdev=34.04 lat (msec): min=39, max=270, avg=97.40, stdev=34.05 clat percentiles (msec): | 1.00th=[ 44], 5.00th=[ 55], 10.00th=[ 64], 20.00th=[ 71], | 30.00th=[ 79], 40.00th=[ 84], 50.00th=[ 90], 60.00th=[ 97], | 70.00th=[ 109], 80.00th=[ 121], 90.00th=[ 140], 95.00th=[ 169], | 99.00th=[ 220], 99.50th=[ 262], 99.90th=[ 271], 99.95th=[ 271], | 99.99th=[ 271] bw ( KiB/s): min= 31, max= 192, per=24.92%, avg=131.32, stdev=39.10, samples=93 iops : min= 1, max= 12, avg= 7.52, stdev= 2.57, samples=93 lat (msec) : 10=7.81%, 20=21.61%, 50=16.02%, 100=35.29%, 250=19.01% lat (msec) : 500=0.26% cpu : usr=0.06%, sys=2.06%, ctx=1447, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=502KiB/s (514kB/s), 121KiB/s-135KiB/s (124kB/s-138kB/s), io=23.4MiB (24.5MB), run=46291-47738msec WRITE: bw=528KiB/s (541kB/s), 130KiB/s-138KiB/s (133kB/s-141kB/s), io=24.6MiB (25.8MB), run=46291-47738msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146775: Tue Dec 2 23:31:58 2025 read: IOPS=1, BW=1579KiB/s (1617kB/s)(7168KiB/4539msec) clat (msec): min=49, max=362, avg=144.64, stdev=120.91 lat (msec): min=49, max=362, avg=144.64, stdev=120.91 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 51], 10.00th=[ 51], 20.00th=[ 65], | 30.00th=[ 70], 40.00th=[ 70], 50.00th=[ 87], 60.00th=[ 113], | 70.00th=[ 113], 80.00th=[ 268], 90.00th=[ 363], 95.00th=[ 363], | 99.00th=[ 363], 99.50th=[ 363], 99.90th=[ 363], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 1965, max= 5572, per=82.61%, avg=3185.33, stdev=2067.09, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 write: IOPS=1, BW=1128KiB/s (1155kB/s)(5120KiB/4539msec); 0 zone resets clat (msec): min=117, max=2484, avg=700.89, stdev=1002.88 lat (msec): min=117, max=2484, avg=700.95, stdev=1002.88 clat percentiles (msec): | 1.00th=[ 118], 5.00th=[ 118], 10.00th=[ 118], 20.00th=[ 118], | 30.00th=[ 205], 40.00th=[ 205], 50.00th=[ 292], 60.00th=[ 292], | 70.00th=[ 405], 80.00th=[ 405], 90.00th=[ 2500], 95.00th=[ 2500], | 99.00th=[ 2500], 99.50th=[ 2500], 99.90th=[ 2500], 99.95th=[ 2500], | 99.99th=[ 2500] bw ( KiB/s): min= 1393, max= 4031, per=57.75%, avg=3118.00, stdev=1494.75, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 lat (msec) : 50=8.33%, 100=25.00%, 250=25.00%, 500=33.33%, >=2000=8.33% cpu : usr=0.07%, sys=4.47%, ctx=89, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146776: Tue Dec 2 23:31:58 2025 read: IOPS=0, BW=386KiB/s (395kB/s)(2048KiB/5310msec) clat (msec): min=87, max=469, avg=278.46, stdev=270.27 lat (msec): min=87, max=469, avg=278.46, stdev=270.27 clat percentiles (msec): | 1.00th=[ 88], 5.00th=[ 88], 10.00th=[ 88], 20.00th=[ 88], | 30.00th=[ 88], 40.00th=[ 88], 50.00th=[ 88], 60.00th=[ 468], | 70.00th=[ 468], 80.00th=[ 468], 90.00th=[ 468], 95.00th=[ 468], | 99.00th=[ 468], 99.50th=[ 468], 99.90th=[ 468], 99.95th=[ 468], | 99.99th=[ 468] bw ( KiB/s): min= 1374, max= 1965, per=43.30%, avg=1669.50, stdev=417.90, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1928KiB/s (1975kB/s)(10.0MiB/5310msec); 0 zone resets clat (msec): min=119, max=2643, avg=466.34, stdev=773.54 lat (msec): min=119, max=2643, avg=466.41, stdev=773.55 clat percentiles (msec): | 1.00th=[ 120], 5.00th=[ 120], 10.00th=[ 120], 20.00th=[ 130], | 30.00th=[ 142], 40.00th=[ 159], 50.00th=[ 169], 60.00th=[ 222], | 70.00th=[ 228], 80.00th=[ 384], 90.00th=[ 472], 95.00th=[ 2635], | 99.00th=[ 2635], 99.50th=[ 2635], 99.90th=[ 2635], 99.95th=[ 2635], | 99.99th=[ 2635] bw ( KiB/s): min= 2023, max= 4071, per=62.71%, avg=3385.60, stdev=948.37, samples=5 iops : min= 1, max= 3, avg= 2.40, stdev= 0.89, samples=5 lat (msec) : 100=8.33%, 250=58.33%, 500=25.00%, >=2000=8.33% cpu : usr=0.00%, sys=5.09%, ctx=126, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146777: Tue Dec 2 23:31:58 2025 read: IOPS=1, BW=1227KiB/s (1256kB/s)(6144KiB/5008msec) clat (msec): min=62, max=614, avg=221.00, stdev=200.26 lat (msec): min=62, max=614, avg=221.00, stdev=200.26 clat percentiles (msec): | 1.00th=[ 63], 5.00th=[ 63], 10.00th=[ 63], 20.00th=[ 113], | 30.00th=[ 113], 40.00th=[ 146], 50.00th=[ 146], 60.00th=[ 165], | 70.00th=[ 224], 80.00th=[ 224], 90.00th=[ 617], 95.00th=[ 617], | 99.00th=[ 617], 99.50th=[ 617], 99.90th=[ 617], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 2015, max= 6107, per=87.70%, avg=3381.67, stdev=2360.21, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 write: IOPS=1, BW=1227KiB/s (1256kB/s)(6144KiB/5008msec); 0 zone resets clat (msec): min=129, max=2693, avg=601.72, stdev=1027.14 lat (msec): min=129, max=2693, avg=601.78, stdev=1027.15 clat percentiles (msec): | 1.00th=[ 130], 5.00th=[ 130], 10.00th=[ 130], 20.00th=[ 138], | 30.00th=[ 138], 40.00th=[ 157], 50.00th=[ 157], 60.00th=[ 180], | 70.00th=[ 313], 80.00th=[ 313], 90.00th=[ 2702], 95.00th=[ 2702], | 99.00th=[ 2702], 99.50th=[ 2702], 99.90th=[ 2702], 99.95th=[ 2702], | 99.99th=[ 2702] bw ( KiB/s): min= 2015, max= 2805, per=41.11%, avg=2219.50, stdev=390.42, samples=4 iops : min= 1, max= 2, avg= 1.25, stdev= 0.50, samples=4 lat (msec) : 100=8.33%, 250=66.67%, 500=8.33%, 750=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=3.87%, ctx=62, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146778: Tue Dec 2 23:31:58 2025 read: IOPS=1, BW=1044KiB/s (1069kB/s)(5120KiB/4903msec) clat (msec): min=107, max=310, avg=191.57, stdev=100.10 lat (msec): min=107, max=310, avg=191.57, stdev=100.10 clat percentiles (msec): | 1.00th=[ 108], 5.00th=[ 108], 10.00th=[ 108], 20.00th=[ 108], | 30.00th=[ 111], 40.00th=[ 111], 50.00th=[ 140], 60.00th=[ 140], | 70.00th=[ 292], 80.00th=[ 292], 90.00th=[ 313], 95.00th=[ 313], | 99.00th=[ 313], 99.50th=[ 313], 99.90th=[ 313], 99.95th=[ 313], | 99.99th=[ 313] bw ( KiB/s): min= 3930, max= 4031, per=100.00%, avg=3980.50, stdev=71.42, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1462KiB/s (1497kB/s)(7168KiB/4903msec); 0 zone resets clat (msec): min=114, max=2658, avg=554.87, stdev=931.92 lat (msec): min=115, max=2658, avg=554.95, stdev=931.93 clat percentiles (msec): | 1.00th=[ 115], 5.00th=[ 115], 10.00th=[ 115], 20.00th=[ 138], | 30.00th=[ 144], 40.00th=[ 144], 50.00th=[ 190], 60.00th=[ 271], | 70.00th=[ 271], 80.00th=[ 368], 90.00th=[ 2668], 95.00th=[ 2668], | 99.00th=[ 2668], 99.50th=[ 2668], 99.90th=[ 2668], 99.95th=[ 2668], | 99.99th=[ 2668] bw ( KiB/s): min= 1965, max= 4168, per=47.19%, avg=2547.75, stdev=1080.60, samples=4 iops : min= 1, max= 4, avg= 1.75, stdev= 1.50, samples=4 lat (msec) : 250=58.33%, 500=33.33%, >=2000=8.33% cpu : usr=0.00%, sys=4.94%, ctx=66, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3857KiB/s (3949kB/s), 386KiB/s-1579KiB/s (395kB/s-1617kB/s), io=20.0MiB (20.0MB), run=4539-5310msec WRITE: bw=5400KiB/s (5529kB/s), 1128KiB/s-1928KiB/s (1155kB/s-1975kB/s), io=28.0MiB (29.4MB), run=4539-5310msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146780: Tue Dec 2 23:31:58 2025 read: IOPS=1, BW=1595KiB/s (1634kB/s)(7168KiB/4493msec) clat (msec): min=2, max=2280, avg=478.02, stdev=800.86 lat (msec): min=2, max=2280, avg=478.02, stdev=800.86 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 3], 20.00th=[ 97], | 30.00th=[ 205], 40.00th=[ 205], 50.00th=[ 215], 60.00th=[ 266], | 70.00th=[ 266], 80.00th=[ 284], 90.00th=[ 2265], 95.00th=[ 2265], | 99.00th=[ 2265], 99.50th=[ 2265], 99.90th=[ 2265], 99.95th=[ 2265], | 99.99th=[ 2265] bw ( KiB/s): min= 1460, max= 4031, per=55.72%, avg=2270.00, stdev=1008.54, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 write: IOPS=1, BW=1140KiB/s (1167kB/s)(5120KiB/4493msec); 0 zone resets clat (msec): min=136, max=344, avg=228.97, stdev=100.29 lat (msec): min=136, max=344, avg=229.04, stdev=100.30 clat percentiles (msec): | 1.00th=[ 138], 5.00th=[ 138], 10.00th=[ 138], 20.00th=[ 138], | 30.00th=[ 157], 40.00th=[ 157], 50.00th=[ 176], 60.00th=[ 176], | 70.00th=[ 330], 80.00th=[ 330], 90.00th=[ 347], 95.00th=[ 347], | 99.00th=[ 347], 99.50th=[ 347], 99.90th=[ 347], 99.95th=[ 347], | 99.99th=[ 347] bw ( KiB/s): min= 1954, max= 3764, per=45.25%, avg=2580.33, stdev=1025.67, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 4=8.33%, 100=8.33%, 250=41.67%, 500=33.33%, >=2000=8.33% cpu : usr=0.00%, sys=18.30%, ctx=741, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146781: Tue Dec 2 23:31:58 2025 read: IOPS=0, BW=407KiB/s (417kB/s)(2048KiB/5027msec) clat (msec): min=89, max=248, avg=169.28, stdev=112.69 lat (msec): min=89, max=248, avg=169.28, stdev=112.69 clat percentiles (msec): | 1.00th=[ 90], 5.00th=[ 90], 10.00th=[ 90], 20.00th=[ 90], | 30.00th=[ 90], 40.00th=[ 90], 50.00th=[ 90], 60.00th=[ 249], | 70.00th=[ 249], 80.00th=[ 249], 90.00th=[ 249], 95.00th=[ 249], | 99.00th=[ 249], 99.50th=[ 249], 99.90th=[ 249], 99.95th=[ 249], | 99.99th=[ 249] bw ( KiB/s): min= 1530, max= 2023, per=43.61%, avg=1776.50, stdev=348.60, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=2037KiB/s (2086kB/s)(10.0MiB/5027msec); 0 zone resets clat (msec): min=79, max=2479, avg=468.63, stdev=717.40 lat (msec): min=79, max=2479, avg=468.70, stdev=717.39 clat percentiles (msec): | 1.00th=[ 80], 5.00th=[ 80], 10.00th=[ 80], 20.00th=[ 108], | 30.00th=[ 150], 40.00th=[ 176], 50.00th=[ 239], 60.00th=[ 279], | 70.00th=[ 309], 80.00th=[ 401], 90.00th=[ 472], 95.00th=[ 2467], | 99.00th=[ 2467], 99.50th=[ 2467], 99.90th=[ 2467], 99.95th=[ 2467], | 99.99th=[ 2467] bw ( KiB/s): min= 1882, max= 5862, per=60.49%, avg=3449.50, stdev=1884.17, samples=4 iops : min= 1, max= 5, avg= 2.50, stdev= 1.91, samples=4 lat (msec) : 100=16.67%, 250=41.67%, 500=33.33%, >=2000=8.33% cpu : usr=0.00%, sys=14.96%, ctx=167, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146782: Tue Dec 2 23:31:58 2025 read: IOPS=1, BW=1226KiB/s (1255kB/s)(6144KiB/5012msec) clat (usec): min=524, max=282959, avg=128773.81, stdev=117200.66 lat (usec): min=525, max=282960, avg=128774.73, stdev=117200.75 clat percentiles (usec): | 1.00th=[ 523], 5.00th=[ 523], 10.00th=[ 523], 20.00th=[ 668], | 30.00th=[ 668], 40.00th=[107480], 50.00th=[107480], 60.00th=[145753], | 70.00th=[235930], 80.00th=[235930], 90.00th=[283116], 95.00th=[283116], | 99.00th=[283116], 99.50th=[283116], 99.90th=[283116], 99.95th=[283116], | 99.99th=[283116] bw ( KiB/s): min= 2015, max= 4047, per=80.40%, avg=3275.33, stdev=1100.61, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=1, BW=1226KiB/s (1255kB/s)(6144KiB/5012msec); 0 zone resets clat (msec): min=186, max=2539, avg=704.09, stdev=910.10 lat (msec): min=186, max=2539, avg=704.15, stdev=910.11 clat percentiles (msec): | 1.00th=[ 188], 5.00th=[ 188], 10.00th=[ 188], 20.00th=[ 243], | 30.00th=[ 243], 40.00th=[ 268], 50.00th=[ 268], 60.00th=[ 418], | 70.00th=[ 575], 80.00th=[ 575], 90.00th=[ 2534], 95.00th=[ 2534], | 99.00th=[ 2534], 99.50th=[ 2534], 99.90th=[ 2534], 99.95th=[ 2534], | 99.99th=[ 2534] bw ( KiB/s): min= 1954, max= 3025, per=39.53%, avg=2254.25, stdev=514.76, samples=4 iops : min= 1, max= 2, avg= 1.25, stdev= 0.50, samples=4 lat (usec) : 750=16.67% lat (msec) : 250=41.67%, 500=25.00%, 750=8.33%, >=2000=8.33% cpu : usr=0.02%, sys=10.30%, ctx=251, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146783: Tue Dec 2 23:31:58 2025 read: IOPS=1, BW=1049KiB/s (1074kB/s)(5120KiB/4880msec) clat (usec): min=741, max=286099, avg=137571.92, stdev=130080.93 lat (usec): min=742, max=286100, avg=137573.77, stdev=130081.36 clat percentiles (usec): | 1.00th=[ 742], 5.00th=[ 742], 10.00th=[ 742], 20.00th=[ 742], | 30.00th=[ 742], 40.00th=[ 742], 50.00th=[187696], 60.00th=[187696], | 70.00th=[212861], 80.00th=[212861], 90.00th=[287310], 95.00th=[287310], | 99.00th=[287310], 99.50th=[287310], 99.90th=[287310], 99.95th=[287310], | 99.99th=[287310] bw ( KiB/s): min= 4031, max= 4047, per=99.14%, avg=4039.00, stdev=11.31, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1469KiB/s (1504kB/s)(7168KiB/4880msec); 0 zone resets clat (msec): min=131, max=2674, avg=597.81, stdev=923.33 lat (msec): min=131, max=2674, avg=597.88, stdev=923.33 clat percentiles (msec): | 1.00th=[ 132], 5.00th=[ 132], 10.00th=[ 132], 20.00th=[ 161], | 30.00th=[ 176], 40.00th=[ 176], 50.00th=[ 245], 60.00th=[ 330], | 70.00th=[ 330], 80.00th=[ 472], 90.00th=[ 2668], 95.00th=[ 2668], | 99.00th=[ 2668], 99.50th=[ 2668], 99.90th=[ 2668], 99.95th=[ 2668], | 99.99th=[ 2668] bw ( KiB/s): min= 1471, max= 4031, per=49.18%, avg=2805.00, stdev=1281.47, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 lat (usec) : 750=16.67% lat (msec) : 250=50.00%, 500=25.00%, >=2000=8.33% cpu : usr=0.00%, sys=13.28%, ctx=348, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4074KiB/s (4172kB/s), 407KiB/s-1595KiB/s (417kB/s-1634kB/s), io=20.0MiB (20.0MB), run=4493-5027msec WRITE: bw=5704KiB/s (5840kB/s), 1140KiB/s-2037KiB/s (1167kB/s-2086kB/s), io=28.0MiB (29.4MB), run=4493-5027msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146789: Tue Dec 2 23:32:09 2025 read: IOPS=0, BW=3281KiB/s (3360kB/s)(12.0MiB/3745msec) clat (msec): min=478, max=2352, avg=1184.08, stdev=1019.30 lat (msec): min=478, max=2352, avg=1184.08, stdev=1019.30 clat percentiles (msec): | 1.00th=[ 481], 5.00th=[ 481], 10.00th=[ 481], 20.00th=[ 481], | 30.00th=[ 481], 40.00th=[ 718], 50.00th=[ 718], 60.00th=[ 718], | 70.00th=[ 2366], 80.00th=[ 2366], 90.00th=[ 2366], 95.00th=[ 2366], | 99.00th=[ 2366], 99.50th=[ 2366], 99.90th=[ 2366], 99.95th=[ 2366], | 99.99th=[ 2366] bw ( KiB/s): min= 8143, max= 8159, per=100.00%, avg=8151.00, stdev=11.31, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=19.74%, ctx=888, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146790: Tue Dec 2 23:32:09 2025 read: IOPS=0, BW=1012KiB/s (1036kB/s)(4096KiB/4049msec) clat (nsec): min=2350.3M, max=2350.3M, avg=2350336930.00, stdev= 0.00 lat (nsec): min=2350.3M, max=2350.3M, avg=2350337937.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 2366], 5.00th=[ 2366], 10.00th=[ 2366], 20.00th=[ 2366], | 30.00th=[ 2366], 40.00th=[ 2366], 50.00th=[ 2366], 60.00th=[ 2366], | 70.00th=[ 2366], 80.00th=[ 2366], 90.00th=[ 2366], 95.00th=[ 2366], | 99.00th=[ 2366], 99.50th=[ 2366], 99.90th=[ 2366], 99.95th=[ 2366], | 99.99th=[ 2366] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2023KiB/s (2072kB/s)(8192KiB/4049msec); 0 zone resets clat (msec): min=334, max=1204, avg=769.23, stdev=614.88 lat (msec): min=334, max=1204, avg=769.52, stdev=614.87 clat percentiles (msec): | 1.00th=[ 334], 5.00th=[ 334], 10.00th=[ 334], 20.00th=[ 334], | 30.00th=[ 334], 40.00th=[ 334], 50.00th=[ 334], 60.00th=[ 1200], | 70.00th=[ 1200], 80.00th=[ 1200], 90.00th=[ 1200], 95.00th=[ 1200], | 99.00th=[ 1200], 99.50th=[ 1200], 99.90th=[ 1200], 99.95th=[ 1200], | 99.99th=[ 1200] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=6.50%, ctx=175, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146791: Tue Dec 2 23:32:09 2025 write: IOPS=0, BW=3946KiB/s (4041kB/s)(12.0MiB/3114msec); 0 zone resets clat (msec): min=351, max=2117, avg=978.95, stdev=987.42 lat (msec): min=351, max=2117, avg=979.33, stdev=987.60 clat percentiles (msec): | 1.00th=[ 351], 5.00th=[ 351], 10.00th=[ 351], 20.00th=[ 351], | 30.00th=[ 351], 40.00th=[ 468], 50.00th=[ 468], 60.00th=[ 468], | 70.00th=[ 2123], 80.00th=[ 2123], 90.00th=[ 2123], 95.00th=[ 2123], | 99.00th=[ 2123], 99.50th=[ 2123], 99.90th=[ 2123], 99.95th=[ 2123], | 99.99th=[ 2123] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=7.42%, ctx=79, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146792: Tue Dec 2 23:32:09 2025 write: IOPS=0, BW=3714KiB/s (3803kB/s)(12.0MiB/3309msec); 0 zone resets clat (msec): min=352, max=1939, avg=1057.21, stdev=807.99 lat (msec): min=352, max=1939, avg=1057.63, stdev=808.06 clat percentiles (msec): | 1.00th=[ 355], 5.00th=[ 355], 10.00th=[ 355], 20.00th=[ 355], | 30.00th=[ 355], 40.00th=[ 877], 50.00th=[ 877], 60.00th=[ 877], | 70.00th=[ 1938], 80.00th=[ 1938], 90.00th=[ 1938], 95.00th=[ 1938], | 99.00th=[ 1938], 99.50th=[ 1938], 99.90th=[ 1938], 99.95th=[ 1938], | 99.99th=[ 1938] bw ( KiB/s): min= 8159, max= 8192, per=100.00%, avg=8175.50, stdev=23.33, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=6.65%, ctx=55, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4046KiB/s (4144kB/s), 1012KiB/s-3281KiB/s (1036kB/s-3360kB/s), io=16.0MiB (16.8MB), run=3745-4049msec WRITE: bw=8093KiB/s (8287kB/s), 2023KiB/s-3946KiB/s (2072kB/s-4041kB/s), io=32.0MiB (33.6MB), run=3114-4049msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146794: Tue Dec 2 23:32:09 2025 read: IOPS=0, BW=3631KiB/s (3718kB/s)(12.0MiB/3384msec) clat (msec): min=455, max=2259, avg=1127.51, stdev=986.26 lat (msec): min=455, max=2259, avg=1127.52, stdev=986.26 clat percentiles (msec): | 1.00th=[ 456], 5.00th=[ 456], 10.00th=[ 456], 20.00th=[ 456], | 30.00th=[ 456], 40.00th=[ 667], 50.00th=[ 667], 60.00th=[ 667], | 70.00th=[ 2265], 80.00th=[ 2265], 90.00th=[ 2265], 95.00th=[ 2265], | 99.00th=[ 2265], 99.50th=[ 2265], 99.90th=[ 2265], 99.95th=[ 2265], | 99.99th=[ 2265] bw ( KiB/s): min= 7892, max= 8192, per=100.00%, avg=8042.00, stdev=212.13, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 750=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=3.64%, ctx=47, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146795: Tue Dec 2 23:32:09 2025 read: IOPS=0, BW=1031KiB/s (1056kB/s)(4096KiB/3971msec) clat (nsec): min=2736.8M, max=2736.8M, avg=2736796374.00, stdev= 0.00 lat (nsec): min=2736.8M, max=2736.8M, avg=2736797788.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 2735], 5.00th=[ 2735], 10.00th=[ 2735], 20.00th=[ 2735], | 30.00th=[ 2735], 40.00th=[ 2735], 50.00th=[ 2735], 60.00th=[ 2735], | 70.00th=[ 2735], 80.00th=[ 2735], 90.00th=[ 2735], 95.00th=[ 2735], | 99.00th=[ 2735], 99.50th=[ 2735], 99.90th=[ 2735], 99.95th=[ 2735], | 99.99th=[ 2735] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2063KiB/s (2112kB/s)(8192KiB/3971msec); 0 zone resets clat (msec): min=238, max=993, avg=616.23, stdev=533.95 lat (msec): min=238, max=994, avg=616.45, stdev=534.06 clat percentiles (msec): | 1.00th=[ 239], 5.00th=[ 239], 10.00th=[ 239], 20.00th=[ 239], | 30.00th=[ 239], 40.00th=[ 239], 50.00th=[ 239], 60.00th=[ 995], | 70.00th=[ 995], 80.00th=[ 995], 90.00th=[ 995], 95.00th=[ 995], | 99.00th=[ 995], 99.50th=[ 995], 99.90th=[ 995], 99.95th=[ 995], | 99.99th=[ 995] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 1000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=3.80%, ctx=52, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146796: Tue Dec 2 23:32:09 2025 write: IOPS=0, BW=3063KiB/s (3136kB/s)(12.0MiB/4012msec); 0 zone resets clat (msec): min=209, max=2580, avg=1336.76, stdev=1189.77 lat (msec): min=209, max=2580, avg=1337.07, stdev=1189.83 clat percentiles (msec): | 1.00th=[ 209], 5.00th=[ 209], 10.00th=[ 209], 20.00th=[ 209], | 30.00th=[ 209], 40.00th=[ 1217], 50.00th=[ 1217], 60.00th=[ 1217], | 70.00th=[ 2567], 80.00th=[ 2567], 90.00th=[ 2567], 95.00th=[ 2567], | 99.00th=[ 2567], 99.50th=[ 2567], 99.90th=[ 2567], 99.95th=[ 2567], | 99.99th=[ 2567] bw ( KiB/s): min= 7892, max= 7892, per=96.63%, avg=7892.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=5.06%, ctx=47, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146797: Tue Dec 2 23:32:09 2025 write: IOPS=0, BW=3353KiB/s (3433kB/s)(12.0MiB/3665msec); 0 zone resets clat (msec): min=231, max=2012, avg=1220.77, stdev=906.77 lat (msec): min=231, max=2012, avg=1221.19, stdev=906.85 clat percentiles (msec): | 1.00th=[ 232], 5.00th=[ 232], 10.00th=[ 232], 20.00th=[ 232], | 30.00th=[ 232], 40.00th=[ 1418], 50.00th=[ 1418], 60.00th=[ 1418], | 70.00th=[ 2005], 80.00th=[ 2005], 90.00th=[ 2005], 95.00th=[ 2005], | 99.00th=[ 2005], 99.50th=[ 2005], 99.90th=[ 2005], 99.95th=[ 2005], | 99.99th=[ 2005] bw ( KiB/s): min= 8031, max= 8031, per=98.33%, avg=8031.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=4.67%, ctx=51, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4126KiB/s (4225kB/s), 1031KiB/s-3631KiB/s (1056kB/s-3718kB/s), io=16.0MiB (16.8MB), run=3384-3971msec WRITE: bw=8167KiB/s (8364kB/s), 2063KiB/s-3353KiB/s (2112kB/s-3433kB/s), io=32.0MiB (33.6MB), run=3665-4012msec