== sanity test 398b: DIO and buffer IO race ============== 06:09:24 (1762168164) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.88018 s, 17.5 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155094: Mon Nov 3 06:10:23 2025 read: IOPS=30, BW=123KiB/s (126kB/s)(5992KiB/48615msec) clat (usec): min=78, max=745015, avg=16283.03, stdev=21889.19 lat (usec): min=79, max=745016, avg=16283.99, stdev=21889.22 clat percentiles (usec): | 1.00th=[ 159], 5.00th=[ 383], 10.00th=[ 6587], 20.00th=[ 8848], | 30.00th=[ 10159], 40.00th=[ 11600], 50.00th=[ 13829], 60.00th=[ 16057], | 70.00th=[ 18744], 80.00th=[ 22414], 90.00th=[ 28181], 95.00th=[ 32900], | 99.00th=[ 56361], 99.50th=[ 63177], 99.90th=[149947], 99.95th=[742392], | 99.99th=[742392] bw ( KiB/s): min= 13, max= 199, per=25.69%, avg=123.30, stdev=33.80, samples=96 iops : min= 3, max= 49, avg=30.49, stdev= 8.45, samples=96 write: IOPS=32, BW=130KiB/s (133kB/s)(6296KiB/48615msec); 0 zone resets clat (usec): min=754, max=174961, avg=15253.11, stdev=14154.04 lat (usec): min=755, max=174963, avg=15254.33, stdev=14154.03 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1532], 10.00th=[ 2245], 20.00th=[ 3032], | 30.00th=[ 4883], 40.00th=[ 9110], 50.00th=[ 12387], 60.00th=[ 15401], | 70.00th=[ 19268], 80.00th=[ 24773], 90.00th=[ 32113], 95.00th=[ 39584], | 99.00th=[ 63701], 99.50th=[ 80217], 99.90th=[109577], 99.95th=[175113], | 99.99th=[175113] bw ( KiB/s): min= 39, max= 216, per=26.04%, avg=130.96, stdev=39.58, samples=95 iops : min= 9, max= 54, avg=32.40, stdev= 9.91, samples=95 lat (usec) : 100=0.07%, 250=1.11%, 500=2.38%, 750=0.33%, 1000=0.36% lat (msec) : 2=3.48%, 4=9.99%, 10=18.33%, 20=36.33%, 50=25.94% lat (msec) : 100=1.50%, 250=0.16%, 750=0.03% cpu : usr=0.14%, sys=22.39%, ctx=7690, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155095: Mon Nov 3 06:10:23 2025 read: IOPS=30, BW=121KiB/s (124kB/s)(5940KiB/48966msec) clat (usec): min=85, max=741838, avg=16517.60, stdev=21633.29 lat (usec): min=86, max=741838, avg=16518.54, stdev=21633.29 clat percentiles (usec): | 1.00th=[ 169], 5.00th=[ 429], 10.00th=[ 7111], 20.00th=[ 9110], | 30.00th=[ 10421], 40.00th=[ 11863], 50.00th=[ 13829], 60.00th=[ 16188], | 70.00th=[ 19006], 80.00th=[ 22676], 90.00th=[ 27919], 95.00th=[ 33162], | 99.00th=[ 55313], 99.50th=[ 72877], 99.90th=[125305], 99.95th=[742392], | 99.99th=[742392] bw ( KiB/s): min= 6, max= 182, per=25.18%, avg=120.85, stdev=29.81, samples=97 iops : min= 1, max= 45, avg=29.88, stdev= 7.45, samples=97 write: IOPS=32, BW=130KiB/s (133kB/s)(6348KiB/48966msec); 0 zone resets clat (usec): min=699, max=163723, avg=15262.46, stdev=14417.63 lat (usec): min=701, max=163724, avg=15263.63, stdev=14417.61 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 1745], 10.00th=[ 2311], 20.00th=[ 3064], | 30.00th=[ 4686], 40.00th=[ 8848], 50.00th=[ 11994], 60.00th=[ 15401], | 70.00th=[ 19268], 80.00th=[ 24773], 90.00th=[ 32637], 95.00th=[ 40633], | 99.00th=[ 63701], 99.50th=[ 77071], 99.90th=[141558], 99.95th=[164627], | 99.99th=[164627] bw ( KiB/s): min= 34, max= 208, per=25.61%, avg=128.82, stdev=34.27, samples=97 iops : min= 8, max= 52, avg=31.87, stdev= 8.62, samples=97 lat (usec) : 100=0.03%, 250=0.98%, 500=1.89%, 750=0.42%, 1000=0.49% lat (msec) : 2=3.42%, 4=9.99%, 10=18.00%, 20=36.59%, 50=26.33% lat (msec) : 100=1.69%, 250=0.13%, 750=0.03% cpu : usr=0.15%, sys=22.45%, ctx=7715, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155096: Mon Nov 3 06:10:23 2025 read: IOPS=30, BW=123KiB/s (126kB/s)(6028KiB/49056msec) clat (usec): min=88, max=193956, avg=15863.75, stdev=11922.20 lat (usec): min=89, max=193957, avg=15864.92, stdev=11922.20 clat percentiles (usec): | 1.00th=[ 180], 5.00th=[ 363], 10.00th=[ 6718], 20.00th=[ 8717], | 30.00th=[ 10159], 40.00th=[ 11600], 50.00th=[ 13304], 60.00th=[ 15795], | 70.00th=[ 18744], 80.00th=[ 22414], 90.00th=[ 28181], 95.00th=[ 33162], | 99.00th=[ 58459], 99.50th=[ 77071], 99.90th=[115868], 99.95th=[193987], | 99.99th=[193987] bw ( KiB/s): min= 13, max= 200, per=25.50%, avg=122.41, stdev=35.84, samples=97 iops : min= 3, max= 50, avg=30.27, stdev= 9.04, samples=97 write: IOPS=31, BW=128KiB/s (131kB/s)(6260KiB/49056msec); 0 zone resets clat (usec): min=903, max=724288, avg=15939.19, stdev=22559.68 lat (usec): min=904, max=724290, avg=15940.50, stdev=22559.70 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1614], 10.00th=[ 2180], 20.00th=[ 3130], | 30.00th=[ 5014], 40.00th=[ 8979], 50.00th=[ 12518], 60.00th=[ 16188], | 70.00th=[ 20317], 80.00th=[ 25560], 90.00th=[ 33817], 95.00th=[ 41157], | 99.00th=[ 60031], 99.50th=[ 67634], 99.90th=[125305], 99.95th=[725615], | 99.99th=[725615] bw ( KiB/s): min= 20, max= 216, per=25.22%, avg=126.87, stdev=34.89, samples=97 iops : min= 5, max= 54, avg=31.38, stdev= 8.74, samples=97 lat (usec) : 100=0.03%, 250=0.88%, 500=2.60%, 750=0.20%, 1000=0.52% lat (msec) : 2=3.81%, 4=9.41%, 10=18.68%, 20=35.16%, 50=26.76% lat (msec) : 100=1.79%, 250=0.13%, 750=0.03% cpu : usr=0.12%, sys=22.32%, ctx=7789, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155097: Mon Nov 3 06:10:23 2025 read: IOPS=30, BW=121KiB/s (124kB/s)(6052KiB/49929msec) clat (usec): min=83, max=241131, avg=16146.82, stdev=12524.08 lat (usec): min=83, max=241133, avg=16147.97, stdev=12524.09 clat percentiles (usec): | 1.00th=[ 204], 5.00th=[ 404], 10.00th=[ 6718], 20.00th=[ 8848], | 30.00th=[ 10290], 40.00th=[ 11994], 50.00th=[ 13698], 60.00th=[ 16188], | 70.00th=[ 19268], 80.00th=[ 22676], 90.00th=[ 28705], 95.00th=[ 33817], | 99.00th=[ 49021], 99.50th=[ 65274], 99.90th=[189793], 99.95th=[240124], | 99.99th=[240124] bw ( KiB/s): min= 56, max= 263, per=25.61%, avg=122.95, stdev=33.20, samples=97 iops : min= 14, max= 65, avg=30.39, stdev= 8.29, samples=97 write: IOPS=31, BW=125KiB/s (128kB/s)(6236KiB/49929msec); 0 zone resets clat (usec): min=627, max=778638, avg=16215.15, stdev=24273.86 lat (usec): min=627, max=778639, avg=16216.61, stdev=24273.92 clat percentiles (usec): | 1.00th=[ 988], 5.00th=[ 1532], 10.00th=[ 2278], 20.00th=[ 3097], | 30.00th=[ 5145], 40.00th=[ 9372], 50.00th=[ 12518], 60.00th=[ 15926], | 70.00th=[ 19530], 80.00th=[ 25560], 90.00th=[ 33817], 95.00th=[ 41157], | 99.00th=[ 64750], 99.50th=[ 79168], 99.90th=[147850], 99.95th=[775947], | 99.99th=[775947] bw ( KiB/s): min= 24, max= 271, per=24.97%, avg=125.60, stdev=38.24, samples=98 iops : min= 6, max= 67, avg=31.06, stdev= 9.57, samples=98 lat (usec) : 100=0.03%, 250=0.75%, 500=2.28%, 750=0.42%, 1000=0.55% lat (msec) : 2=3.35%, 4=8.89%, 10=18.59%, 20=36.72%, 50=26.53% lat (msec) : 100=1.63%, 250=0.23%, 1000=0.03% cpu : usr=0.15%, sys=22.07%, ctx=7873, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=481KiB/s (492kB/s), 121KiB/s-123KiB/s (124kB/s-126kB/s), io=23.4MiB (24.6MB), run=48615-49929msec WRITE: bw=504KiB/s (516kB/s), 125KiB/s-130KiB/s (128kB/s-133kB/s), io=24.6MiB (25.7MB), run=48615-49929msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155099: Mon Nov 3 06:11:06 2025 read: IOPS=16, BW=64.8KiB/s (66.4kB/s)(5992KiB/92437msec) clat (msec): min=3, max=751, avg=14.87, stdev=23.19 lat (msec): min=3, max=751, avg=14.87, stdev=23.19 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 20], 90.00th=[ 28], 95.00th=[ 40], | 99.00th=[ 65], 99.50th=[ 78], 99.90th=[ 213], 99.95th=[ 751], | 99.99th=[ 751] bw ( KiB/s): min= 7, max= 160, per=25.39%, avg=65.75, stdev=36.10, samples=181 iops : min= 1, max= 40, avg=16.13, stdev= 9.12, samples=181 write: IOPS=17, BW=68.1KiB/s (69.7kB/s)(6296KiB/92437msec); 0 zone resets clat (msec): min=14, max=265, avg=44.51, stdev=21.88 lat (msec): min=14, max=265, avg=44.51, stdev=21.88 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 22], 10.00th=[ 24], 20.00th=[ 28], | 30.00th=[ 32], 40.00th=[ 34], 50.00th=[ 39], 60.00th=[ 44], | 70.00th=[ 51], 80.00th=[ 60], 90.00th=[ 74], 95.00th=[ 87], | 99.00th=[ 117], 99.50th=[ 126], 99.90th=[ 157], 99.95th=[ 266], | 99.99th=[ 266] bw ( KiB/s): min= 16, max= 136, per=25.19%, avg=68.26, stdev=28.51, samples=183 iops : min= 4, max= 34, avg=16.75, stdev= 7.27, samples=183 lat (msec) : 4=1.01%, 10=22.82%, 20=16.54%, 50=43.10%, 100=15.07% lat (msec) : 250=1.40%, 500=0.03%, 1000=0.03% cpu : usr=0.09%, sys=5.54%, ctx=5792, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155100: Mon Nov 3 06:11:06 2025 read: IOPS=16, BW=64.1KiB/s (65.6kB/s)(5940KiB/92710msec) clat (msec): min=3, max=706, avg=14.76, stdev=21.64 lat (msec): min=3, max=706, avg=14.76, stdev=21.64 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 9], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 16], 80.00th=[ 20], 90.00th=[ 27], 95.00th=[ 35], | 99.00th=[ 69], 99.50th=[ 79], 99.90th=[ 138], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 7, max= 160, per=24.74%, avg=64.07, stdev=35.76, samples=183 iops : min= 1, max= 40, avg=15.70, stdev= 9.02, samples=183 write: IOPS=17, BW=68.5KiB/s (70.1kB/s)(6348KiB/92710msec); 0 zone resets clat (msec): min=13, max=268, avg=44.51, stdev=22.69 lat (msec): min=13, max=268, avg=44.52, stdev=22.69 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 22], 10.00th=[ 24], 20.00th=[ 28], | 30.00th=[ 31], 40.00th=[ 34], 50.00th=[ 39], 60.00th=[ 44], | 70.00th=[ 51], 80.00th=[ 60], 90.00th=[ 73], 95.00th=[ 86], | 99.00th=[ 130], 99.50th=[ 148], 99.90th=[ 224], 99.95th=[ 268], | 99.99th=[ 268] bw ( KiB/s): min= 23, max= 144, per=25.19%, avg=68.26, stdev=28.02, samples=183 iops : min= 5, max= 36, avg=16.75, stdev= 7.14, samples=183 lat (msec) : 4=0.91%, 10=21.65%, 20=17.87%, 50=43.20%, 100=15.14% lat (msec) : 250=1.17%, 500=0.03%, 750=0.03% cpu : usr=0.08%, sys=5.57%, ctx=5696, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155101: Mon Nov 3 06:11:06 2025 read: IOPS=16, BW=65.5KiB/s (67.0kB/s)(6028KiB/92062msec) clat (msec): min=3, max=140, avg=15.06, stdev=12.98 lat (msec): min=3, max=140, avg=15.06, stdev=12.98 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 30], 95.00th=[ 40], | 99.00th=[ 66], 99.50th=[ 87], 99.90th=[ 110], 99.95th=[ 142], | 99.99th=[ 142] bw ( KiB/s): min= 15, max= 168, per=25.31%, avg=65.55, stdev=33.62, samples=182 iops : min= 3, max= 42, avg=16.08, stdev= 8.48, samples=182 write: IOPS=16, BW=67.0KiB/s (69.6kB/s)(6260KiB/92062msec); 0 zone resets clat (msec): min=16, max=746, avg=44.25, stdev=27.13 lat (msec): min=16, max=746, avg=44.25, stdev=27.13 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 22], 10.00th=[ 24], 20.00th=[ 28], | 30.00th=[ 31], 40.00th=[ 34], 50.00th=[ 38], 60.00th=[ 43], | 70.00th=[ 50], 80.00th=[ 61], 90.00th=[ 73], 95.00th=[ 85], | 99.00th=[ 112], 99.50th=[ 127], 99.90th=[ 148], 99.95th=[ 751], | 99.99th=[ 751] bw ( KiB/s): min= 7, max= 152, per=25.13%, avg=68.11, stdev=29.22, samples=182 iops : min= 1, max= 38, avg=16.71, stdev= 7.44, samples=182 lat (msec) : 4=0.59%, 10=21.13%, 20=18.46%, 50=43.88%, 100=14.84% lat (msec) : 250=1.07%, 750=0.03% cpu : usr=0.08%, sys=5.63%, ctx=5754, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155102: Mon Nov 3 06:11:06 2025 read: IOPS=16, BW=65.7KiB/s (67.3kB/s)(6052KiB/92104msec) clat (msec): min=3, max=139, avg=14.50, stdev=12.15 lat (msec): min=3, max=139, avg=14.50, stdev=12.15 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 14], | 70.00th=[ 16], 80.00th=[ 21], 90.00th=[ 29], 95.00th=[ 37], | 99.00th=[ 61], 99.50th=[ 78], 99.90th=[ 111], 99.95th=[ 140], | 99.99th=[ 140] bw ( KiB/s): min= 7, max= 192, per=25.57%, avg=66.23, stdev=36.83, samples=181 iops : min= 1, max= 48, avg=16.25, stdev= 9.33, samples=181 write: IOPS=16, BW=67.7KiB/s (69.3kB/s)(6236KiB/92104msec); 0 zone resets clat (msec): min=15, max=744, avg=44.94, stdev=27.96 lat (msec): min=15, max=744, avg=44.94, stdev=27.96 clat percentiles (msec): | 1.00th=[ 19], 5.00th=[ 22], 10.00th=[ 24], 20.00th=[ 28], | 30.00th=[ 31], 40.00th=[ 34], 50.00th=[ 39], 60.00th=[ 44], | 70.00th=[ 51], 80.00th=[ 60], 90.00th=[ 74], 95.00th=[ 89], | 99.00th=[ 113], 99.50th=[ 142], 99.90th=[ 224], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 23, max= 144, per=24.99%, avg=67.71, stdev=28.19, samples=182 iops : min= 5, max= 36, avg=16.62, stdev= 7.18, samples=182 lat (msec) : 4=0.65%, 10=22.62%, 20=17.35%, 50=43.10%, 100=15.04% lat (msec) : 250=1.20%, 750=0.03% cpu : usr=0.08%, sys=5.70%, ctx=5806, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=259KiB/s (265kB/s), 64.1KiB/s-65.7KiB/s (65.6kB/s-67.3kB/s), io=23.4MiB (24.6MB), run=92062-92710msec WRITE: bw=271KiB/s (278kB/s), 67.7KiB/s-68.5KiB/s (69.3kB/s-70.1kB/s), io=24.6MiB (25.7MB), run=92062-92710msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155130: Mon Nov 3 06:11:21 2025 read: IOPS=34, BW=546KiB/s (560kB/s)(5920KiB/10833msec) clat (usec): min=115, max=61016, avg=14878.90, stdev=9116.50 lat (usec): min=115, max=61017, avg=14881.19, stdev=9114.38 clat percentiles (usec): | 1.00th=[ 208], 5.00th=[ 355], 10.00th=[ 474], 20.00th=[ 9503], | 30.00th=[10814], 40.00th=[12387], 50.00th=[13566], 60.00th=[15270], | 70.00th=[17695], 80.00th=[20317], 90.00th=[24773], 95.00th=[31327], | 99.00th=[46924], 99.50th=[59507], 99.90th=[61080], 99.95th=[61080], | 99.99th=[61080] bw ( KiB/s): min= 317, max= 736, per=25.77%, avg=536.71, stdev=119.58, samples=21 iops : min= 19, max= 46, avg=33.05, stdev= 7.54, samples=21 write: IOPS=36, BW=588KiB/s (602kB/s)(6368KiB/10833msec); 0 zone resets clat (usec): min=1209, max=75847, avg=13247.39, stdev=10172.43 lat (usec): min=1211, max=75849, avg=13249.00, stdev=10172.47 clat percentiles (usec): | 1.00th=[ 1434], 5.00th=[ 3097], 10.00th=[ 3621], 20.00th=[ 4359], | 30.00th=[ 5145], 40.00th=[ 7832], 50.00th=[11076], 60.00th=[13173], | 70.00th=[16450], 80.00th=[21103], 90.00th=[27919], 95.00th=[31851], | 99.00th=[42206], 99.50th=[55313], 99.90th=[76022], 99.95th=[76022], | 99.99th=[76022] bw ( KiB/s): min= 319, max= 800, per=26.68%, avg=584.81, stdev=115.49, samples=21 iops : min= 19, max= 50, avg=36.10, stdev= 7.40, samples=21 lat (usec) : 250=0.78%, 500=4.17%, 750=0.13%, 1000=0.13% lat (msec) : 2=1.30%, 4=6.25%, 10=22.27%, 20=42.97%, 50=21.48% lat (msec) : 100=0.52% cpu : usr=0.24%, sys=30.31%, ctx=2129, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155132: Mon Nov 3 06:11:21 2025 read: IOPS=31, BW=500KiB/s (512kB/s)(5744KiB/11494msec) clat (usec): min=130, max=93663, avg=16365.17, stdev=11803.66 lat (usec): min=131, max=93664, avg=16366.13, stdev=11803.72 clat percentiles (usec): | 1.00th=[ 147], 5.00th=[ 338], 10.00th=[ 502], 20.00th=[ 8979], | 30.00th=[11207], 40.00th=[13173], 50.00th=[14746], 60.00th=[17171], | 70.00th=[19530], 80.00th=[22152], 90.00th=[27657], 95.00th=[33162], | 99.00th=[67634], 99.50th=[81265], 99.90th=[93848], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 57, max= 768, per=23.75%, avg=494.64, stdev=153.57, samples=22 iops : min= 3, max= 48, avg=30.45, stdev= 9.72, samples=22 write: IOPS=35, BW=569KiB/s (583kB/s)(6544KiB/11494msec); 0 zone resets clat (usec): min=1305, max=75607, avg=13520.30, stdev=12029.45 lat (usec): min=1306, max=75609, avg=13522.02, stdev=12029.45 clat percentiles (usec): | 1.00th=[ 1565], 5.00th=[ 2507], 10.00th=[ 3326], 20.00th=[ 4146], | 30.00th=[ 4883], 40.00th=[ 7046], 50.00th=[10290], 60.00th=[13173], | 70.00th=[15795], 80.00th=[20841], 90.00th=[29230], 95.00th=[36439], | 99.00th=[58459], 99.50th=[70779], 99.90th=[76022], 99.95th=[76022], | 99.99th=[76022] bw ( KiB/s): min= 285, max= 954, per=25.28%, avg=554.23, stdev=160.93, samples=22 iops : min= 17, max= 59, avg=34.18, stdev=10.14, samples=22 lat (usec) : 250=0.91%, 500=3.65%, 750=0.39%, 1000=0.13% lat (msec) : 2=2.34%, 4=7.29%, 10=22.79%, 20=38.41%, 50=22.53% lat (msec) : 100=1.56% cpu : usr=0.10%, sys=28.64%, ctx=2198, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155134: Mon Nov 3 06:11:21 2025 read: IOPS=34, BW=560KiB/s (573kB/s)(6256KiB/11179msec) clat (usec): min=185, max=83870, avg=15649.75, stdev=10185.71 lat (usec): min=187, max=83871, avg=15650.77, stdev=10185.73 clat percentiles (usec): | 1.00th=[ 208], 5.00th=[ 347], 10.00th=[ 529], 20.00th=[ 9241], | 30.00th=[11076], 40.00th=[12911], 50.00th=[14615], 60.00th=[16450], | 70.00th=[18482], 80.00th=[21627], 90.00th=[26346], 95.00th=[32113], | 99.00th=[53216], 99.50th=[72877], 99.90th=[83362], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 318, max= 768, per=26.88%, avg=559.82, stdev=124.60, samples=22 iops : min= 19, max= 48, avg=34.55, stdev= 7.93, samples=22 write: IOPS=33, BW=540KiB/s (553kB/s)(6032KiB/11179msec); 0 zone resets clat (usec): min=1637, max=65471, avg=13191.43, stdev=11226.55 lat (usec): min=1638, max=65473, avg=13216.42, stdev=11221.26 clat percentiles (usec): | 1.00th=[ 1778], 5.00th=[ 2933], 10.00th=[ 3621], 20.00th=[ 4146], | 30.00th=[ 4948], 40.00th=[ 6980], 50.00th=[10290], 60.00th=[12780], | 70.00th=[15664], 80.00th=[20841], 90.00th=[28443], 95.00th=[34341], | 99.00th=[56361], 99.50th=[60031], 99.90th=[65274], 99.95th=[65274], | 99.99th=[65274] bw ( KiB/s): min= 288, max= 800, per=24.20%, avg=530.50, stdev=139.73, samples=22 iops : min= 18, max= 50, avg=32.73, stdev= 8.88, samples=22 lat (usec) : 250=1.04%, 500=3.91%, 750=0.65%, 1000=0.13% lat (msec) : 2=0.91%, 4=7.16%, 10=22.40%, 20=40.49%, 50=21.74% lat (msec) : 100=1.56% cpu : usr=0.17%, sys=29.44%, ctx=2239, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155135: Mon Nov 3 06:11:21 2025 read: IOPS=34, BW=548KiB/s (561kB/s)(6032KiB/11007msec) clat (usec): min=111, max=68020, avg=15287.51, stdev=9993.56 lat (usec): min=111, max=68021, avg=15288.53, stdev=9993.61 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 330], 10.00th=[ 433], 20.00th=[ 9110], | 30.00th=[10683], 40.00th=[12256], 50.00th=[14222], 60.00th=[16057], | 70.00th=[18744], 80.00th=[21365], 90.00th=[26608], 95.00th=[31065], | 99.00th=[49546], 99.50th=[59507], 99.90th=[67634], 99.95th=[67634], | 99.99th=[67634] bw ( KiB/s): min= 319, max= 800, per=25.89%, avg=539.29, stdev=125.32, samples=21 iops : min= 19, max= 50, avg=33.24, stdev= 7.78, samples=21 write: IOPS=35, BW=568KiB/s (582kB/s)(6256KiB/11007msec); 0 zone resets clat (usec): min=1501, max=81852, avg=13176.70, stdev=10361.20 lat (usec): min=1504, max=81854, avg=13178.65, stdev=10361.36 clat percentiles (usec): | 1.00th=[ 1598], 5.00th=[ 2737], 10.00th=[ 3458], 20.00th=[ 4359], | 30.00th=[ 5407], 40.00th=[ 7439], 50.00th=[10159], 60.00th=[12649], | 70.00th=[16712], 80.00th=[21627], 90.00th=[27657], 95.00th=[33162], | 99.00th=[43254], 99.50th=[55313], 99.90th=[82314], 99.95th=[82314], | 99.99th=[82314] bw ( KiB/s): min= 288, max= 800, per=25.76%, avg=564.76, stdev=143.23, samples=21 iops : min= 18, max= 50, avg=34.86, stdev= 8.86, samples=21 lat (usec) : 250=1.17%, 500=4.69%, 750=0.13%, 1000=0.13% lat (msec) : 2=0.91%, 4=7.94%, 10=22.53%, 20=37.50%, 50=24.35% lat (msec) : 100=0.65% cpu : usr=0.15%, sys=29.37%, ctx=2210, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2084KiB/s (2134kB/s), 500KiB/s-560KiB/s (512kB/s-573kB/s), io=23.4MiB (24.5MB), run=10833-11494msec WRITE: bw=2192KiB/s (2245kB/s), 540KiB/s-588KiB/s (553kB/s-602kB/s), io=24.6MiB (25.8MB), run=10833-11494msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155128: Mon Nov 3 06:11:31 2025 read: IOPS=17, BW=285KiB/s (292kB/s)(5920KiB/20757msec) clat (usec): min=3530, max=63150, avg=13702.62, stdev=11539.56 lat (usec): min=3531, max=63151, avg=13703.64, stdev=11539.47 clat percentiles (usec): | 1.00th=[ 3654], 5.00th=[ 4490], 10.00th=[ 4948], 20.00th=[ 5866], | 30.00th=[ 6849], 40.00th=[ 8225], 50.00th=[ 9372], 60.00th=[11338], | 70.00th=[14353], 80.00th=[18482], 90.00th=[26870], 95.00th=[40633], | 99.00th=[60556], 99.50th=[62653], 99.90th=[63177], 99.95th=[63177], | 99.99th=[63177] bw ( KiB/s): min= 64, max= 608, per=24.65%, avg=281.98, stdev=116.42, samples=41 iops : min= 4, max= 38, avg=17.34, stdev= 7.35, samples=41 write: IOPS=19, BW=307KiB/s (314kB/s)(6368KiB/20757msec); 0 zone resets clat (msec): min=16, max=120, avg=39.19, stdev=16.73 lat (msec): min=16, max=120, avg=39.19, stdev=16.73 clat percentiles (msec): | 1.00th=[ 19], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 27], | 30.00th=[ 30], 40.00th=[ 33], 50.00th=[ 36], 60.00th=[ 39], | 70.00th=[ 44], 80.00th=[ 48], 90.00th=[ 62], 95.00th=[ 75], | 99.00th=[ 92], 99.50th=[ 111], 99.90th=[ 121], 99.95th=[ 121], | 99.99th=[ 121] bw ( KiB/s): min= 127, max= 512, per=25.32%, avg=304.80, stdev=115.73, samples=41 iops : min= 7, max= 32, avg=18.78, stdev= 7.36, samples=41 lat (msec) : 4=1.30%, 10=24.61%, 20=15.89%, 50=48.18%, 100=9.64% lat (msec) : 250=0.39% cpu : usr=0.09%, sys=7.48%, ctx=1523, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155129: Mon Nov 3 06:11:31 2025 read: IOPS=17, BW=275KiB/s (281kB/s)(5744KiB/20925msec) clat (msec): min=3, max=111, avg=13.22, stdev=13.55 lat (msec): min=3, max=111, avg=13.22, stdev=13.55 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 9], 60.00th=[ 11], | 70.00th=[ 13], 80.00th=[ 17], 90.00th=[ 28], 95.00th=[ 43], | 99.00th=[ 69], 99.50th=[ 102], 99.90th=[ 111], 99.95th=[ 111], | 99.99th=[ 111] bw ( KiB/s): min= 57, max= 672, per=23.83%, avg=272.63, stdev=154.45, samples=41 iops : min= 3, max= 42, avg=16.76, stdev= 9.76, samples=41 write: IOPS=19, BW=313KiB/s (320kB/s)(6544KiB/20925msec); 0 zone resets clat (msec): min=17, max=131, avg=39.42, stdev=17.29 lat (msec): min=17, max=131, avg=39.43, stdev=17.29 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 22], 10.00th=[ 24], 20.00th=[ 27], | 30.00th=[ 29], 40.00th=[ 32], 50.00th=[ 35], 60.00th=[ 39], | 70.00th=[ 43], 80.00th=[ 49], 90.00th=[ 64], 95.00th=[ 75], | 99.00th=[ 93], 99.50th=[ 109], 99.90th=[ 132], 99.95th=[ 132], | 99.99th=[ 132] bw ( KiB/s): min= 127, max= 480, per=25.57%, avg=307.88, stdev=108.14, samples=41 iops : min= 7, max= 30, avg=18.95, stdev= 6.89, samples=41 lat (msec) : 4=1.17%, 10=25.52%, 20=14.45%, 50=47.66%, 100=10.55% lat (msec) : 250=0.65% cpu : usr=0.10%, sys=7.03%, ctx=1444, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155131: Mon Nov 3 06:11:31 2025 read: IOPS=18, BW=304KiB/s (311kB/s)(6256KiB/20592msec) clat (msec): min=3, max=103, avg=14.27, stdev=13.34 lat (msec): min=3, max=103, avg=14.27, stdev=13.34 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 11], 60.00th=[ 12], | 70.00th=[ 14], 80.00th=[ 19], 90.00th=[ 30], 95.00th=[ 44], | 99.00th=[ 64], 99.50th=[ 83], 99.90th=[ 104], 99.95th=[ 104], | 99.99th=[ 104] bw ( KiB/s): min= 96, max= 830, per=26.63%, avg=304.70, stdev=157.13, samples=40 iops : min= 6, max= 51, avg=18.75, stdev= 9.89, samples=40 write: IOPS=18, BW=293KiB/s (300kB/s)(6032KiB/20592msec); 0 zone resets clat (msec): min=18, max=160, avg=39.61, stdev=18.23 lat (msec): min=18, max=160, avg=39.61, stdev=18.23 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 22], 10.00th=[ 23], 20.00th=[ 26], | 30.00th=[ 29], 40.00th=[ 31], 50.00th=[ 36], 60.00th=[ 39], | 70.00th=[ 44], 80.00th=[ 51], 90.00th=[ 64], 95.00th=[ 78], | 99.00th=[ 101], 99.50th=[ 122], 99.90th=[ 161], 99.95th=[ 161], | 99.99th=[ 161] bw ( KiB/s): min= 95, max= 512, per=24.03%, avg=289.33, stdev=118.02, samples=40 iops : min= 5, max= 32, avg=17.80, stdev= 7.50, samples=40 lat (msec) : 4=0.39%, 10=25.00%, 20=17.84%, 50=44.40%, 100=11.85% lat (msec) : 250=0.52% cpu : usr=0.11%, sys=7.30%, ctx=1533, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155133: Mon Nov 3 06:11:31 2025 read: IOPS=18, BW=296KiB/s (303kB/s)(6032KiB/20401msec) clat (usec): min=3445, max=85844, avg=13649.28, stdev=11734.40 lat (usec): min=3446, max=85845, avg=13650.22, stdev=11734.32 clat percentiles (usec): | 1.00th=[ 3884], 5.00th=[ 4490], 10.00th=[ 4948], 20.00th=[ 5735], | 30.00th=[ 6652], 40.00th=[ 7832], 50.00th=[ 9765], 60.00th=[11469], | 70.00th=[13829], 80.00th=[18482], 90.00th=[29230], 95.00th=[39060], | 99.00th=[61604], 99.50th=[69731], 99.90th=[85459], 99.95th=[85459], | 99.99th=[85459] bw ( KiB/s): min= 127, max= 544, per=25.72%, avg=294.25, stdev=117.95, samples=40 iops : min= 7, max= 34, avg=18.10, stdev= 7.50, samples=40 write: IOPS=19, BW=307KiB/s (314kB/s)(6256KiB/20401msec); 0 zone resets clat (msec): min=16, max=109, avg=38.94, stdev=16.63 lat (msec): min=16, max=109, avg=38.95, stdev=16.63 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 26], | 30.00th=[ 29], 40.00th=[ 32], 50.00th=[ 35], 60.00th=[ 39], | 70.00th=[ 43], 80.00th=[ 50], 90.00th=[ 63], 95.00th=[ 74], | 99.00th=[ 95], 99.50th=[ 97], 99.90th=[ 110], 99.95th=[ 110], | 99.99th=[ 110] bw ( KiB/s): min= 96, max= 576, per=25.42%, avg=306.05, stdev=120.82, samples=40 iops : min= 6, max= 36, avg=18.85, stdev= 7.65, samples=40 lat (msec) : 4=0.91%, 10=23.96%, 20=17.45%, 50=47.40%, 100=10.16% lat (msec) : 250=0.13% cpu : usr=0.11%, sys=7.28%, ctx=1468, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1145KiB/s (1172kB/s), 275KiB/s-304KiB/s (281kB/s-311kB/s), io=23.4MiB (24.5MB), run=20401-20925msec WRITE: bw=1204KiB/s (1233kB/s), 293KiB/s-313KiB/s (300kB/s-320kB/s), io=24.6MiB (25.8MB), run=20401-20925msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155144: Mon Nov 3 06:11:41 2025 read: IOPS=1, BW=1834KiB/s (1878kB/s)(7168KiB/3909msec) clat (msec): min=80, max=2068, avg=426.67, stdev=727.74 lat (msec): min=80, max=2068, avg=426.67, stdev=727.74 clat percentiles (msec): | 1.00th=[ 81], 5.00th=[ 81], 10.00th=[ 81], 20.00th=[ 96], | 30.00th=[ 129], 40.00th=[ 129], 50.00th=[ 142], 60.00th=[ 171], | 70.00th=[ 171], 80.00th=[ 300], 90.00th=[ 2072], 95.00th=[ 2072], | 99.00th=[ 2072], 99.50th=[ 2072], 99.90th=[ 2072], 99.95th=[ 2072], | 99.99th=[ 2072] bw ( KiB/s): min= 2048, max= 4087, per=61.30%, avg=3211.67, stdev=1049.63, samples=3 iops : min= 2, max= 3, avg= 2.67, stdev= 0.58, samples=3 write: IOPS=1, BW=1310KiB/s (1341kB/s)(5120KiB/3909msec); 0 zone resets clat (msec): min=68, max=509, avg=173.34, stdev=189.12 lat (msec): min=68, max=510, avg=173.41, stdev=189.12 clat percentiles (msec): | 1.00th=[ 69], 5.00th=[ 69], 10.00th=[ 69], 20.00th=[ 69], | 30.00th=[ 75], 40.00th=[ 75], 50.00th=[ 96], 60.00th=[ 96], | 70.00th=[ 117], 80.00th=[ 117], 90.00th=[ 510], 95.00th=[ 510], | 99.00th=[ 510], 99.50th=[ 510], 99.90th=[ 510], 99.95th=[ 510], | 99.99th=[ 510] bw ( KiB/s): min= 2048, max= 4096, per=41.89%, avg=3072.00, stdev=1448.15, samples=2 iops : min= 2, max= 4, avg= 3.00, stdev= 1.41, samples=2 lat (msec) : 100=41.67%, 250=33.33%, 500=8.33%, 750=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=27.28%, ctx=1318, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155145: Mon Nov 3 06:11:41 2025 read: IOPS=0, BW=637KiB/s (652kB/s)(2048KiB/3215msec) clat (msec): min=150, max=164, avg=157.48, stdev= 9.23 lat (msec): min=150, max=164, avg=157.49, stdev= 9.23 clat percentiles (msec): | 1.00th=[ 150], 5.00th=[ 150], 10.00th=[ 150], 20.00th=[ 150], | 30.00th=[ 150], 40.00th=[ 150], 50.00th=[ 150], 60.00th=[ 165], | 70.00th=[ 165], 80.00th=[ 165], 90.00th=[ 165], 95.00th=[ 165], | 99.00th=[ 165], 99.50th=[ 165], 99.90th=[ 165], 99.95th=[ 165], | 99.99th=[ 165] bw ( KiB/s): min= 1753, max= 2039, per=36.19%, avg=1896.00, stdev=202.23, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3185KiB/s (3262kB/s)(10.0MiB/3215msec); 0 zone resets clat (msec): min=98, max=983, avg=284.01, stdev=272.95 lat (msec): min=99, max=983, avg=284.09, stdev=272.96 clat percentiles (msec): | 1.00th=[ 100], 5.00th=[ 100], 10.00th=[ 100], 20.00th=[ 107], | 30.00th=[ 129], 40.00th=[ 132], 50.00th=[ 146], 60.00th=[ 146], | 70.00th=[ 288], 80.00th=[ 388], 90.00th=[ 422], 95.00th=[ 986], | 99.00th=[ 986], 99.50th=[ 986], 99.90th=[ 986], 99.95th=[ 986], | 99.99th=[ 986] bw ( KiB/s): min= 2043, max= 6144, per=62.73%, avg=4600.50, stdev=1956.49, samples=4 iops : min= 1, max= 6, avg= 4.00, stdev= 2.16, samples=4 lat (msec) : 100=8.33%, 250=58.33%, 500=25.00%, 1000=8.33% cpu : usr=0.09%, sys=38.74%, ctx=324, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155147: Mon Nov 3 06:11:41 2025 read: IOPS=1, BW=1677KiB/s (1718kB/s)(6144KiB/3663msec) clat (usec): min=673, max=614335, avg=137205.04, stdev=241605.93 lat (usec): min=674, max=614336, avg=137206.18, stdev=241605.92 clat percentiles (usec): | 1.00th=[ 676], 5.00th=[ 676], 10.00th=[ 676], 20.00th=[ 816], | 30.00th=[ 816], 40.00th=[ 865], 50.00th=[ 865], 60.00th=[ 47973], | 70.00th=[158335], 80.00th=[158335], 90.00th=[616563], 95.00th=[616563], | 99.00th=[616563], 99.50th=[616563], 99.90th=[616563], 99.95th=[616563], | 99.99th=[616563] bw ( KiB/s): min= 2048, max= 8192, per=97.73%, avg=5120.00, stdev=4344.46, samples=2 iops : min= 2, max= 8, avg= 5.00, stdev= 4.24, samples=2 write: IOPS=1, BW=1677KiB/s (1718kB/s)(6144KiB/3663msec); 0 zone resets clat (msec): min=150, max=1396, avg=470.51, stdev=491.41 lat (msec): min=150, max=1396, avg=470.57, stdev=491.39 clat percentiles (msec): | 1.00th=[ 150], 5.00th=[ 150], 10.00th=[ 150], 20.00th=[ 157], | 30.00th=[ 157], 40.00th=[ 190], 50.00th=[ 190], 60.00th=[ 279], | 70.00th=[ 651], 80.00th=[ 651], 90.00th=[ 1401], 95.00th=[ 1401], | 99.00th=[ 1401], 99.50th=[ 1401], 99.90th=[ 1401], 99.95th=[ 1401], | 99.99th=[ 1401] bw ( KiB/s): min= 2043, max= 3482, per=32.80%, avg=2405.25, stdev=717.84, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (usec) : 750=8.33%, 1000=16.67% lat (msec) : 50=8.33%, 250=33.33%, 500=8.33%, 750=16.67%, 2000=8.33% cpu : usr=0.00%, sys=23.68%, ctx=491, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155148: Mon Nov 3 06:11:41 2025 read: IOPS=1, BW=1514KiB/s (1550kB/s)(5120KiB/3382msec) clat (usec): min=682, max=270527, avg=130696.56, stdev=95758.97 lat (usec): min=683, max=270528, avg=130697.73, stdev=95758.90 clat percentiles (usec): | 1.00th=[ 685], 5.00th=[ 685], 10.00th=[ 685], 20.00th=[ 685], | 30.00th=[116917], 40.00th=[116917], 50.00th=[130548], 60.00th=[130548], | 70.00th=[135267], 80.00th=[135267], 90.00th=[270533], 95.00th=[270533], | 99.00th=[270533], 99.50th=[270533], 99.90th=[270533], 99.95th=[270533], | 99.99th=[270533] bw ( KiB/s): min= 2048, max= 6144, per=78.18%, avg=4096.00, stdev=2896.31, samples=2 iops : min= 2, max= 6, avg= 4.00, stdev= 2.83, samples=2 write: IOPS=2, BW=2119KiB/s (2170kB/s)(7168KiB/3382msec); 0 zone resets clat (msec): min=97, max=1289, avg=382.38, stdev=423.85 lat (msec): min=97, max=1289, avg=382.50, stdev=423.83 clat percentiles (msec): | 1.00th=[ 99], 5.00th=[ 99], 10.00th=[ 99], 20.00th=[ 122], | 30.00th=[ 142], 40.00th=[ 142], 50.00th=[ 174], 60.00th=[ 422], | 70.00th=[ 422], 80.00th=[ 430], 90.00th=[ 1284], 95.00th=[ 1284], | 99.00th=[ 1284], 99.50th=[ 1284], 99.90th=[ 1284], 99.95th=[ 1284], | 99.99th=[ 1284] bw ( KiB/s): min= 1744, max= 6144, per=40.83%, avg=2994.75, stdev=2104.31, samples=4 iops : min= 1, max= 6, avg= 2.50, stdev= 2.38, samples=4 lat (usec) : 750=8.33% lat (msec) : 100=8.33%, 250=50.00%, 500=25.00%, 2000=8.33% cpu : usr=0.09%, sys=31.85%, ctx=653, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5239KiB/s (5365kB/s), 637KiB/s-1834KiB/s (652kB/s-1878kB/s), io=20.0MiB (20.0MB), run=3215-3909msec WRITE: bw=7335KiB/s (7511kB/s), 1310KiB/s-3185KiB/s (1341kB/s-3262kB/s), io=28.0MiB (29.4MB), run=3215-3909msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155149: Mon Nov 3 06:11:41 2025 read: IOPS=1, BW=1759KiB/s (1801kB/s)(7168KiB/4075msec) clat (msec): min=18, max=1664, avg=390.52, stdev=582.41 lat (msec): min=18, max=1664, avg=390.52, stdev=582.41 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 20], 10.00th=[ 20], 20.00th=[ 24], | 30.00th=[ 115], 40.00th=[ 115], 50.00th=[ 220], 60.00th=[ 224], | 70.00th=[ 224], 80.00th=[ 472], 90.00th=[ 1670], 95.00th=[ 1670], | 99.00th=[ 1670], 99.50th=[ 1670], 99.90th=[ 1670], 99.95th=[ 1670], | 99.99th=[ 1670] bw ( KiB/s): min= 1759, max= 4055, per=49.28%, avg=2476.25, stdev=1061.13, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 write: IOPS=1, BW=1256KiB/s (1287kB/s)(5120KiB/4075msec); 0 zone resets clat (msec): min=30, max=890, avg=256.92, stdev=360.98 lat (msec): min=30, max=890, avg=256.99, stdev=360.99 clat percentiles (msec): | 1.00th=[ 31], 5.00th=[ 31], 10.00th=[ 31], 20.00th=[ 31], | 30.00th=[ 37], 40.00th=[ 37], 50.00th=[ 128], 60.00th=[ 128], | 70.00th=[ 201], 80.00th=[ 201], 90.00th=[ 894], 95.00th=[ 894], | 99.00th=[ 894], 99.50th=[ 894], 99.90th=[ 894], 99.95th=[ 894], | 99.99th=[ 894] bw ( KiB/s): min= 2027, max= 2027, per=28.81%, avg=2027.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 20=8.33%, 50=25.00%, 250=41.67%, 500=8.33%, 1000=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=6.09%, ctx=64, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155150: Mon Nov 3 06:11:41 2025 read: IOPS=0, BW=618KiB/s (633kB/s)(2048KiB/3314msec) clat (msec): min=47, max=372, avg=210.40, stdev=229.92 lat (msec): min=47, max=372, avg=210.40, stdev=229.92 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 48], 10.00th=[ 48], 20.00th=[ 48], | 30.00th=[ 48], 40.00th=[ 48], 50.00th=[ 48], 60.00th=[ 372], | 70.00th=[ 372], 80.00th=[ 372], 90.00th=[ 372], 95.00th=[ 372], | 99.00th=[ 372], 99.50th=[ 372], 99.90th=[ 372], 99.95th=[ 372], | 99.99th=[ 372] bw ( KiB/s): min= 1765, max= 2048, per=37.94%, avg=1906.50, stdev=200.11, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3090KiB/s (3164kB/s)(10.0MiB/3314msec); 0 zone resets clat (msec): min=41, max=1097, avg=282.83, stdev=353.36 lat (msec): min=41, max=1097, avg=282.91, stdev=353.35 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 42], 10.00th=[ 42], 20.00th=[ 59], | 30.00th=[ 69], 40.00th=[ 71], 50.00th=[ 74], 60.00th=[ 159], | 70.00th=[ 186], 80.00th=[ 359], 90.00th=[ 718], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 2043, max= 8192, per=58.20%, avg=4094.75, stdev=2897.49, samples=4 iops : min= 1, max= 8, avg= 3.75, stdev= 3.10, samples=4 lat (msec) : 50=16.67%, 100=33.33%, 250=16.67%, 500=16.67%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.03%, sys=11.44%, ctx=63, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155151: Mon Nov 3 06:11:41 2025 read: IOPS=1, BW=1563KiB/s (1600kB/s)(6144KiB/3931msec) clat (msec): min=47, max=201, avg=150.69, stdev=53.65 lat (msec): min=47, max=201, avg=150.69, stdev=53.65 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 48], 10.00th=[ 48], 20.00th=[ 144], | 30.00th=[ 144], 40.00th=[ 165], 50.00th=[ 165], 60.00th=[ 174], | 70.00th=[ 174], 80.00th=[ 174], 90.00th=[ 201], 95.00th=[ 201], | 99.00th=[ 201], 99.50th=[ 201], 99.90th=[ 201], 99.95th=[ 201], | 99.99th=[ 201] bw ( KiB/s): min= 2039, max= 6144, per=67.87%, avg=3410.33, stdev=2367.43, samples=3 iops : min= 1, max= 6, avg= 3.00, stdev= 2.65, samples=3 write: IOPS=1, BW=1563KiB/s (1600kB/s)(6144KiB/3931msec); 0 zone resets clat (msec): min=78, max=1105, avg=483.01, stdev=372.22 lat (msec): min=78, max=1105, avg=483.08, stdev=372.22 clat percentiles (msec): | 1.00th=[ 79], 5.00th=[ 79], 10.00th=[ 79], 20.00th=[ 236], | 30.00th=[ 236], 40.00th=[ 368], 50.00th=[ 368], 60.00th=[ 388], | 70.00th=[ 726], 80.00th=[ 726], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 2039, max= 3506, per=34.26%, avg=2410.25, stdev=730.51, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (msec) : 50=8.33%, 100=8.33%, 250=50.00%, 500=16.67%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=9.82%, ctx=114, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155152: Mon Nov 3 06:11:41 2025 read: IOPS=1, BW=1449KiB/s (1484kB/s)(5120KiB/3534msec) clat (msec): min=46, max=240, avg=115.63, stdev=78.22 lat (msec): min=46, max=240, avg=115.63, stdev=78.22 clat percentiles (msec): | 1.00th=[ 47], 5.00th=[ 47], 10.00th=[ 47], 20.00th=[ 47], | 30.00th=[ 66], 40.00th=[ 66], 50.00th=[ 85], 60.00th=[ 85], | 70.00th=[ 142], 80.00th=[ 142], 90.00th=[ 241], 95.00th=[ 241], | 99.00th=[ 241], 99.50th=[ 241], 99.90th=[ 241], 99.95th=[ 241], | 99.99th=[ 241] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=1, BW=2028KiB/s (2077kB/s)(7168KiB/3534msec); 0 zone resets clat (msec): min=90, max=1115, avg=416.73, stdev=383.59 lat (msec): min=90, max=1115, avg=416.81, stdev=383.60 clat percentiles (msec): | 1.00th=[ 90], 5.00th=[ 90], 10.00th=[ 90], 20.00th=[ 118], | 30.00th=[ 136], 40.00th=[ 136], 50.00th=[ 279], 60.00th=[ 447], | 70.00th=[ 447], 80.00th=[ 735], 90.00th=[ 1116], 95.00th=[ 1116], | 99.00th=[ 1116], 99.50th=[ 1116], 99.90th=[ 1116], 99.95th=[ 1116], | 99.99th=[ 1116] bw ( KiB/s): min= 1741, max= 4096, per=35.22%, avg=2478.00, stdev=1087.72, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 lat (msec) : 50=8.33%, 100=25.00%, 250=33.33%, 500=16.67%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=14.66%, ctx=122, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5026KiB/s (5146kB/s), 618KiB/s-1759KiB/s (633kB/s-1801kB/s), io=20.0MiB (20.0MB), run=3314-4075msec WRITE: bw=7036KiB/s (7205kB/s), 1256KiB/s-3090KiB/s (1287kB/s-3164kB/s), io=28.0MiB (29.4MB), run=3314-4075msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155158: Mon Nov 3 06:11:49 2025 read: IOPS=1, BW=5956KiB/s (6099kB/s)(12.0MiB/2063msec) clat (msec): min=479, max=1080, avg=680.36, stdev=346.84 lat (msec): min=479, max=1080, avg=680.37, stdev=346.84 clat percentiles (msec): | 1.00th=[ 481], 5.00th=[ 481], 10.00th=[ 481], 20.00th=[ 481], | 30.00th=[ 481], 40.00th=[ 481], 50.00th=[ 481], 60.00th=[ 481], | 70.00th=[ 1083], 80.00th=[ 1083], 90.00th=[ 1083], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 7907, max= 7937, per=100.00%, avg=7922.00, stdev=21.21, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.05%, sys=33.75%, ctx=2629, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155159: Mon Nov 3 06:11:49 2025 read: IOPS=0, BW=1690KiB/s (1730kB/s)(4096KiB/2424msec) clat (nsec): min=1086.9M, max=1086.9M, avg=1086900078.00, stdev= 0.00 lat (nsec): min=1086.9M, max=1086.9M, avg=1086901904.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1083], 5.00th=[ 1083], 10.00th=[ 1083], 20.00th=[ 1083], | 30.00th=[ 1083], 40.00th=[ 1083], 50.00th=[ 1083], 60.00th=[ 1083], | 70.00th=[ 1083], 80.00th=[ 1083], 90.00th=[ 1083], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 7937, max= 7937, per=100.00%, avg=7937.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3380KiB/s (3461kB/s)(8192KiB/2424msec); 0 zone resets clat (msec): min=631, max=691, avg=661.70, stdev=42.18 lat (msec): min=632, max=691, avg=661.95, stdev=42.04 clat percentiles (msec): | 1.00th=[ 634], 5.00th=[ 634], 10.00th=[ 634], 20.00th=[ 634], | 30.00th=[ 634], 40.00th=[ 634], 50.00th=[ 634], 60.00th=[ 693], | 70.00th=[ 693], 80.00th=[ 693], 90.00th=[ 693], 95.00th=[ 693], | 99.00th=[ 693], 99.50th=[ 693], 99.90th=[ 693], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 7907, max= 7907, per=58.49%, avg=7907.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=19.27%, ctx=1150, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155160: Mon Nov 3 06:11:49 2025 write: IOPS=1, BW=5240KiB/s (5366kB/s)(12.0MiB/2345msec); 0 zone resets clat (msec): min=346, max=1016, avg=773.85, stdev=371.33 lat (msec): min=346, max=1016, avg=774.16, stdev=371.33 clat percentiles (msec): | 1.00th=[ 347], 5.00th=[ 347], 10.00th=[ 347], 20.00th=[ 347], | 30.00th=[ 347], 40.00th=[ 961], 50.00th=[ 961], 60.00th=[ 961], | 70.00th=[ 1020], 80.00th=[ 1020], 90.00th=[ 1020], 95.00th=[ 1020], | 99.00th=[ 1020], 99.50th=[ 1020], 99.90th=[ 1020], 99.95th=[ 1020], | 99.99th=[ 1020] bw ( KiB/s): min= 7907, max= 7937, per=58.60%, avg=7922.00, stdev=21.21, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=11.86%, ctx=73, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155161: Mon Nov 3 06:11:49 2025 write: IOPS=1, BW=5624KiB/s (5759kB/s)(12.0MiB/2185msec); 0 zone resets clat (msec): min=430, max=929, avg=723.85, stdev=260.69 lat (msec): min=430, max=929, avg=724.15, stdev=260.68 clat percentiles (msec): | 1.00th=[ 430], 5.00th=[ 430], 10.00th=[ 430], 20.00th=[ 430], | 30.00th=[ 430], 40.00th=[ 810], 50.00th=[ 810], 60.00th=[ 810], | 70.00th=[ 927], 80.00th=[ 927], 90.00th=[ 927], 95.00th=[ 927], | 99.00th=[ 927], 99.50th=[ 927], 99.90th=[ 927], 99.95th=[ 927], | 99.99th=[ 927] bw ( KiB/s): min= 7937, max= 8192, per=59.66%, avg=8064.50, stdev=180.31, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 1000=66.67% cpu : usr=0.00%, sys=11.22%, ctx=78, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6759KiB/s (6921kB/s), 1690KiB/s-5956KiB/s (1730kB/s-6099kB/s), io=16.0MiB (16.8MB), run=2063-2424msec WRITE: bw=13.2MiB/s (13.8MB/s), 3380KiB/s-5624KiB/s (3461kB/s-5759kB/s), io=32.0MiB (33.6MB), run=2185-2424msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155163: Mon Nov 3 06:11:50 2025 read: IOPS=1, BW=5263KiB/s (5389kB/s)(12.0MiB/2335msec) clat (msec): min=315, max=1454, avg=765.80, stdev=605.53 lat (msec): min=315, max=1454, avg=765.80, stdev=605.53 clat percentiles (msec): | 1.00th=[ 317], 5.00th=[ 317], 10.00th=[ 317], 20.00th=[ 317], | 30.00th=[ 317], 40.00th=[ 527], 50.00th=[ 527], 60.00th=[ 527], | 70.00th=[ 1452], 80.00th=[ 1452], 90.00th=[ 1452], 95.00th=[ 1452], | 99.00th=[ 1452], 99.50th=[ 1452], 99.90th=[ 1452], 99.95th=[ 1452], | 99.99th=[ 1452] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=10.71%, ctx=72, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155164: Mon Nov 3 06:11:50 2025 read: IOPS=0, BW=1631KiB/s (1670kB/s)(4096KiB/2511msec) clat (nsec): min=1584.2M, max=1584.2M, avg=1584197678.00, stdev= 0.00 lat (nsec): min=1584.2M, max=1584.2M, avg=1584199978.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1586], 5.00th=[ 1586], 10.00th=[ 1586], 20.00th=[ 1586], | 30.00th=[ 1586], 40.00th=[ 1586], 50.00th=[ 1586], 60.00th=[ 1586], | 70.00th=[ 1586], 80.00th=[ 1586], 90.00th=[ 1586], 95.00th=[ 1586], | 99.00th=[ 1586], 99.50th=[ 1586], 99.90th=[ 1586], 99.95th=[ 1586], | 99.99th=[ 1586] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3262KiB/s (3341kB/s)(8192KiB/2511msec); 0 zone resets clat (msec): min=405, max=470, avg=438.33, stdev=45.79 lat (msec): min=406, max=470, avg=438.61, stdev=45.79 clat percentiles (msec): | 1.00th=[ 405], 5.00th=[ 405], 10.00th=[ 405], 20.00th=[ 405], | 30.00th=[ 405], 40.00th=[ 405], 50.00th=[ 405], 60.00th=[ 472], | 70.00th=[ 472], 80.00th=[ 472], 90.00th=[ 472], 95.00th=[ 472], | 99.00th=[ 472], 99.50th=[ 472], 99.90th=[ 472], 99.95th=[ 472], | 99.99th=[ 472] bw ( KiB/s): min= 8192, max= 8192, per=63.48%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=11.59%, ctx=269, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155165: Mon Nov 3 06:11:50 2025 write: IOPS=1, BW=4840KiB/s (4956kB/s)(12.0MiB/2539msec); 0 zone resets clat (msec): min=147, max=1820, avg=838.12, stdev=873.79 lat (msec): min=147, max=1820, avg=838.45, stdev=873.82 clat percentiles (msec): | 1.00th=[ 148], 5.00th=[ 148], 10.00th=[ 148], 20.00th=[ 148], | 30.00th=[ 148], 40.00th=[ 550], 50.00th=[ 550], 60.00th=[ 550], | 70.00th=[ 1821], 80.00th=[ 1821], 90.00th=[ 1821], 95.00th=[ 1821], | 99.00th=[ 1821], 99.50th=[ 1821], 99.90th=[ 1821], 99.95th=[ 1821], | 99.99th=[ 1821] bw ( KiB/s): min= 8192, max= 8192, per=63.48%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.04%, sys=10.76%, ctx=71, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155166: Mon Nov 3 06:11:50 2025 write: IOPS=1, BW=5430KiB/s (5560kB/s)(12.0MiB/2263msec); 0 zone resets clat (msec): min=205, max=1092, avg=747.81, stdev=475.64 lat (msec): min=205, max=1092, avg=748.16, stdev=475.67 clat percentiles (msec): | 1.00th=[ 205], 5.00th=[ 205], 10.00th=[ 205], 20.00th=[ 205], | 30.00th=[ 205], 40.00th=[ 944], 50.00th=[ 944], 60.00th=[ 944], | 70.00th=[ 1099], 80.00th=[ 1099], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 8192, max= 8192, per=63.48%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.13%, sys=12.07%, ctx=76, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6525KiB/s (6681kB/s), 1631KiB/s-5263KiB/s (1670kB/s-5389kB/s), io=16.0MiB (16.8MB), run=2335-2511msec WRITE: bw=12.6MiB/s (13.2MB/s), 3262KiB/s-5430KiB/s (3341kB/s-5560kB/s), io=32.0MiB (33.6MB), run=2263-2539msec