== sanity test 398b: DIO and buffer IO race ============== 15:26:53 (1758742013) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.78843 s, 18.1 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=182529: Wed Sep 24 15:28:08 2025 read: IOPS=23, BW=92.1KiB/s (94.3kB/s)(5992KiB/65043msec) clat (usec): min=251, max=655794, avg=23542.38, stdev=19962.42 lat (usec): min=253, max=655795, avg=23543.44, stdev=19962.43 clat percentiles (usec): | 1.00th=[ 668], 5.00th=[ 8848], 10.00th=[ 10814], 20.00th=[ 13960], | 30.00th=[ 16450], 40.00th=[ 18744], 50.00th=[ 21103], 60.00th=[ 24249], | 70.00th=[ 27395], 80.00th=[ 31851], 90.00th=[ 37487], 95.00th=[ 43254], | 99.00th=[ 60031], 99.50th=[ 67634], 99.90th=[112722], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 39, max= 174, per=25.28%, avg=92.26, stdev=23.93, samples=128 iops : min= 9, max= 43, avg=22.63, stdev= 6.03, samples=128 write: IOPS=24, BW=96.8KiB/s (99.1kB/s)(6296KiB/65043msec); 0 zone resets clat (usec): min=734, max=86291, avg=18768.97, stdev=13134.07 lat (usec): min=734, max=86293, avg=18770.38, stdev=13133.99 clat percentiles (usec): | 1.00th=[ 1401], 5.00th=[ 3294], 10.00th=[ 4490], 20.00th=[ 6259], | 30.00th=[ 9896], 40.00th=[13304], 50.00th=[15926], 60.00th=[19530], | 70.00th=[23462], 80.00th=[29492], 90.00th=[36963], 95.00th=[44303], | 99.00th=[56886], 99.50th=[62653], 99.90th=[85459], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 32, max= 192, per=25.33%, avg=97.01, stdev=29.76, samples=128 iops : min= 8, max= 48, avg=23.82, stdev= 7.48, samples=128 lat (usec) : 500=0.20%, 750=0.55%, 1000=0.39% lat (msec) : 2=0.88%, 4=2.80%, 10=14.26%, 20=34.57%, 50=43.88% lat (msec) : 100=2.41%, 250=0.03%, 750=0.03% cpu : usr=0.15%, sys=27.22%, ctx=10383, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182531: Wed Sep 24 15:28:08 2025 read: IOPS=22, BW=90.5KiB/s (92.7kB/s)(5940KiB/65628msec) clat (usec): min=128, max=76877, avg=23705.15, stdev=11308.72 lat (usec): min=128, max=76878, avg=23706.31, stdev=11308.90 clat percentiles (usec): | 1.00th=[ 660], 5.00th=[ 9241], 10.00th=[11469], 20.00th=[14484], | 30.00th=[16712], 40.00th=[19006], 50.00th=[21890], 60.00th=[24773], | 70.00th=[28443], 80.00th=[32637], 90.00th=[38011], 95.00th=[44303], | 99.00th=[60556], 99.50th=[64226], 99.90th=[72877], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 7, max= 168, per=24.70%, avg=90.15, stdev=26.51, samples=130 iops : min= 1, max= 42, avg=22.10, stdev= 6.62, samples=130 write: IOPS=24, BW=96.7KiB/s (99.0kB/s)(6348KiB/65628msec); 0 zone resets clat (usec): min=632, max=645688, avg=19021.21, stdev=20413.72 lat (usec): min=632, max=645689, avg=19022.38, stdev=20413.72 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 3064], 10.00th=[ 4228], 20.00th=[ 6259], | 30.00th=[ 9765], 40.00th=[ 13566], 50.00th=[ 16450], 60.00th=[ 19792], | 70.00th=[ 23462], 80.00th=[ 28967], 90.00th=[ 36439], 95.00th=[ 44303], | 99.00th=[ 57934], 99.50th=[ 65274], 99.90th=[ 85459], 99.95th=[641729], | 99.99th=[641729] bw ( KiB/s): min= 47, max= 199, per=25.31%, avg=96.92, stdev=31.12, samples=129 iops : min= 11, max= 49, avg=23.80, stdev= 7.84, samples=129 lat (usec) : 250=0.07%, 500=0.16%, 750=0.39%, 1000=0.42% lat (msec) : 2=1.01%, 4=3.19%, 10=14.00%, 20=33.40%, 50=45.05% lat (msec) : 100=2.28%, 750=0.03% cpu : usr=0.15%, sys=26.96%, ctx=10245, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182533: Wed Sep 24 15:28:08 2025 read: IOPS=22, BW=91.0KiB/s (94.2kB/s)(6028KiB/65553msec) clat (usec): min=122, max=92958, avg=23107.30, stdev=11549.85 lat (usec): min=123, max=92960, avg=23108.34, stdev=11549.86 clat percentiles (usec): | 1.00th=[ 523], 5.00th=[ 8586], 10.00th=[10683], 20.00th=[13566], | 30.00th=[16057], 40.00th=[18482], 50.00th=[21103], 60.00th=[24511], | 70.00th=[28181], 80.00th=[32113], 90.00th=[37487], 95.00th=[42206], | 99.00th=[58983], 99.50th=[68682], 99.90th=[80217], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 31, max= 168, per=25.23%, avg=92.10, stdev=27.80, samples=129 iops : min= 7, max= 42, avg=22.59, stdev= 7.06, samples=129 write: IOPS=23, BW=95.5KiB/s (97.8kB/s)(6260KiB/65553msec); 0 zone resets clat (usec): min=809, max=646923, avg=19495.97, stdev=21218.73 lat (usec): min=810, max=646924, avg=19503.25, stdev=21217.46 clat percentiles (usec): | 1.00th=[ 1172], 5.00th=[ 2769], 10.00th=[ 4015], 20.00th=[ 5669], | 30.00th=[ 8848], 40.00th=[ 13042], 50.00th=[ 16450], 60.00th=[ 20055], | 70.00th=[ 24511], 80.00th=[ 30540], 90.00th=[ 39060], 95.00th=[ 45351], | 99.00th=[ 59507], 99.50th=[ 74974], 99.90th=[101188], 99.95th=[650118], | 99.99th=[650118] bw ( KiB/s): min= 15, max= 191, per=24.76%, avg=94.82, stdev=28.85, samples=130 iops : min= 3, max= 47, avg=23.27, stdev= 7.24, samples=130 lat (usec) : 250=0.03%, 500=0.42%, 750=0.42%, 1000=0.46% lat (msec) : 2=1.07%, 4=3.87%, 10=14.36%, 20=32.29%, 50=44.40% lat (msec) : 100=2.60%, 250=0.03%, 750=0.03% cpu : usr=0.12%, sys=26.72%, ctx=10142, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182535: Wed Sep 24 15:28:08 2025 read: IOPS=23, BW=92.3KiB/s (94.5kB/s)(6052KiB/65578msec) clat (usec): min=102, max=83290, avg=23445.54, stdev=11667.83 lat (usec): min=102, max=83291, avg=23452.15, stdev=11693.66 clat percentiles (usec): | 1.00th=[ 519], 5.00th=[ 9241], 10.00th=[11207], 20.00th=[13960], | 30.00th=[16188], 40.00th=[18744], 50.00th=[21627], 60.00th=[24511], | 70.00th=[27657], 80.00th=[31851], 90.00th=[38536], 95.00th=[45351], | 99.00th=[59507], 99.50th=[67634], 99.90th=[77071], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 24, max= 184, per=25.39%, avg=92.67, stdev=24.76, samples=129 iops : min= 6, max= 46, avg=22.73, stdev= 6.19, samples=129 write: IOPS=23, BW=95.1KiB/s (97.4kB/s)(6236KiB/65578msec); 0 zone resets clat (usec): min=594, max=671675, avg=19164.93, stdev=21218.61 lat (usec): min=594, max=671675, avg=19168.18, stdev=21217.54 clat percentiles (usec): | 1.00th=[ 1729], 5.00th=[ 2999], 10.00th=[ 4113], 20.00th=[ 5604], | 30.00th=[ 9503], 40.00th=[ 13304], 50.00th=[ 16450], 60.00th=[ 19792], | 70.00th=[ 24249], 80.00th=[ 29492], 90.00th=[ 37487], 95.00th=[ 44303], | 99.00th=[ 57410], 99.50th=[ 64226], 99.90th=[ 89654], 99.95th=[675283], | 99.99th=[675283] bw ( KiB/s): min= 23, max= 208, per=24.71%, avg=94.65, stdev=30.35, samples=130 iops : min= 5, max= 52, avg=23.22, stdev= 7.66, samples=130 lat (usec) : 250=0.03%, 500=0.39%, 750=0.49%, 1000=0.23% lat (msec) : 2=0.59%, 4=4.17%, 10=13.05%, 20=33.46%, 50=44.89% lat (msec) : 100=2.67%, 750=0.03% cpu : usr=0.13%, sys=26.66%, ctx=10176, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=366KiB/s (375kB/s), 90.5KiB/s-92.3KiB/s (92.7kB/s-94.5kB/s), io=23.4MiB (24.6MB), run=65043-65628msec WRITE: bw=383KiB/s (392kB/s), 95.1KiB/s-96.8KiB/s (97.4kB/s-99.1kB/s), io=24.6MiB (25.7MB), run=65043-65628msec rand-rw: (groupid=0, jobs=1): err= 0: pid=182530: Wed Sep 24 15:28:15 2025 read: IOPS=20, BW=82.5KiB/s (84.5kB/s)(5992KiB/72599msec) clat (msec): min=3, max=653, avg=21.28, stdev=22.82 lat (msec): min=3, max=653, avg=21.28, stdev=22.82 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 31], 90.00th=[ 45], 95.00th=[ 55], | 99.00th=[ 75], 99.50th=[ 82], 99.90th=[ 122], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 15, max= 280, per=24.99%, avg=82.48, stdev=54.15, samples=143 iops : min= 3, max= 70, avg=20.18, stdev=13.64, samples=143 write: IOPS=21, BW=86.7KiB/s (88.8kB/s)(6296KiB/72599msec); 0 zone resets clat (msec): min=4, max=131, avg=25.73, stdev=17.18 lat (msec): min=4, max=131, avg=25.73, stdev=17.18 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 27], | 70.00th=[ 32], 80.00th=[ 40], 90.00th=[ 50], 95.00th=[ 59], | 99.00th=[ 80], 99.50th=[ 94], 99.90th=[ 114], 99.95th=[ 132], | 99.99th=[ 132] bw ( KiB/s): min= 31, max= 328, per=25.09%, avg=86.55, stdev=57.14, samples=143 iops : min= 7, max= 82, avg=21.20, stdev=14.38, samples=143 lat (msec) : 4=0.36%, 10=23.08%, 20=31.28%, 50=37.24%, 100=7.81% lat (msec) : 250=0.20%, 750=0.03% cpu : usr=0.14%, sys=12.88%, ctx=6262, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182532: Wed Sep 24 15:28:15 2025 read: IOPS=20, BW=81.9KiB/s (83.9kB/s)(5940KiB/72528msec) clat (msec): min=2, max=681, avg=21.15, stdev=23.14 lat (msec): min=2, max=681, avg=21.15, stdev=23.14 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 32], 90.00th=[ 45], 95.00th=[ 54], | 99.00th=[ 72], 99.50th=[ 77], 99.90th=[ 123], 99.95th=[ 684], | 99.99th=[ 684] bw ( KiB/s): min= 16, max= 336, per=24.84%, avg=81.98, stdev=57.32, samples=143 iops : min= 4, max= 84, avg=20.06, stdev=14.42, samples=143 write: IOPS=21, BW=87.5KiB/s (89.6kB/s)(6348KiB/72528msec); 0 zone resets clat (msec): min=4, max=133, avg=25.78, stdev=17.37 lat (msec): min=4, max=133, avg=25.78, stdev=17.37 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 27], | 70.00th=[ 32], 80.00th=[ 39], 90.00th=[ 49], 95.00th=[ 59], | 99.00th=[ 83], 99.50th=[ 95], 99.90th=[ 128], 99.95th=[ 134], | 99.99th=[ 134] bw ( KiB/s): min= 31, max= 320, per=25.35%, avg=87.45, stdev=55.47, samples=143 iops : min= 7, max= 80, avg=21.43, stdev=13.98, samples=143 lat (msec) : 4=0.42%, 10=23.08%, 20=30.99%, 50=37.34%, 100=7.98% lat (msec) : 250=0.16%, 750=0.03% cpu : usr=0.14%, sys=12.98%, ctx=6253, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182534: Wed Sep 24 15:28:15 2025 read: IOPS=20, BW=82.0KiB/s (84.0kB/s)(6028KiB/72636msec) clat (msec): min=3, max=103, avg=20.92, stdev=15.68 lat (msec): min=3, max=103, avg=20.93, stdev=15.68 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 32], 90.00th=[ 44], 95.00th=[ 56], | 99.00th=[ 70], 99.50th=[ 77], 99.90th=[ 100], 99.95th=[ 104], | 99.99th=[ 104] bw ( KiB/s): min= 23, max= 312, per=25.13%, avg=82.92, stdev=58.46, samples=143 iops : min= 5, max= 78, avg=20.29, stdev=14.71, samples=143 write: IOPS=21, BW=86.2KiB/s (88.3kB/s)(6260KiB/72636msec); 0 zone resets clat (msec): min=4, max=676, avg=26.13, stdev=23.79 lat (msec): min=4, max=676, avg=26.13, stdev=23.79 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 27], | 70.00th=[ 32], 80.00th=[ 39], 90.00th=[ 50], 95.00th=[ 59], | 99.00th=[ 86], 99.50th=[ 96], 99.90th=[ 114], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 24, max= 311, per=24.88%, avg=85.83, stdev=54.27, samples=143 iops : min= 6, max= 77, avg=21.02, stdev=13.66, samples=143 lat (msec) : 4=0.36%, 10=23.80%, 20=30.01%, 50=37.89%, 100=7.68% lat (msec) : 250=0.23%, 750=0.03% cpu : usr=0.12%, sys=13.12%, ctx=6311, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182536: Wed Sep 24 15:28:15 2025 read: IOPS=20, BW=83.2KiB/s (85.2kB/s)(6052KiB/72760msec) clat (usec): min=2424, max=97806, avg=21031.88, stdev=15643.44 lat (usec): min=2425, max=97806, avg=21032.91, stdev=15643.54 clat percentiles (usec): | 1.00th=[ 3884], 5.00th=[ 5014], 10.00th=[ 5932], 20.00th=[ 7701], | 30.00th=[11076], 40.00th=[13960], 50.00th=[16909], 60.00th=[19530], | 70.00th=[23987], 80.00th=[31589], 90.00th=[44303], 95.00th=[53216], | 99.00th=[71828], 99.50th=[80217], 99.90th=[94897], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 7, max= 320, per=24.96%, avg=82.37, stdev=54.79, samples=143 iops : min= 1, max= 80, avg=20.15, stdev=13.79, samples=143 write: IOPS=21, BW=85.7KiB/s (87.8kB/s)(6236KiB/72760msec); 0 zone resets clat (msec): min=4, max=684, avg=26.12, stdev=24.40 lat (msec): min=4, max=684, avg=26.12, stdev=24.40 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 22], 60.00th=[ 27], | 70.00th=[ 32], 80.00th=[ 40], 90.00th=[ 50], 95.00th=[ 60], | 99.00th=[ 86], 99.50th=[ 100], 99.90th=[ 129], 99.95th=[ 684], | 99.99th=[ 684] bw ( KiB/s): min= 31, max= 328, per=24.60%, avg=84.87, stdev=56.52, samples=143 iops : min= 7, max= 82, avg=20.78, stdev=14.24, samples=143 lat (msec) : 4=0.72%, 10=22.62%, 20=30.89%, 50=37.43%, 100=8.14% lat (msec) : 250=0.16%, 750=0.03% cpu : usr=0.12%, sys=12.87%, ctx=6386, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=330KiB/s (338kB/s), 81.9KiB/s-83.2KiB/s (83.9kB/s-85.2kB/s), io=23.4MiB (24.6MB), run=72528-72760msec WRITE: bw=346KiB/s (354kB/s), 85.7KiB/s-87.5KiB/s (87.8kB/s-89.6kB/s), io=24.6MiB (25.7MB), run=72528-72760msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=182573: Wed Sep 24 15:28:41 2025 read: IOPS=18, BW=296KiB/s (303kB/s)(5920KiB/20020msec) clat (usec): min=316, max=98846, avg=26892.34, stdev=14196.95 lat (usec): min=317, max=98848, avg=26893.33, stdev=14197.00 clat percentiles (usec): | 1.00th=[ 515], 5.00th=[ 8979], 10.00th=[12911], 20.00th=[16188], | 30.00th=[19268], 40.00th=[21365], 50.00th=[24511], 60.00th=[27919], | 70.00th=[31851], 80.00th=[36439], 90.00th=[44303], 95.00th=[53216], | 99.00th=[76022], 99.50th=[82314], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 127, max= 507, per=24.33%, avg=289.77, stdev=69.83, samples=39 iops : min= 7, max= 31, avg=17.49, stdev= 4.37, samples=39 write: IOPS=19, BW=318KiB/s (326kB/s)(6368KiB/20020msec); 0 zone resets clat (usec): min=2410, max=98405, avg=25111.91, stdev=15752.48 lat (usec): min=2411, max=98407, avg=25113.82, stdev=15752.51 clat percentiles (usec): | 1.00th=[ 3523], 5.00th=[ 5669], 10.00th=[ 7373], 20.00th=[10290], | 30.00th=[14353], 40.00th=[17957], 50.00th=[21890], 60.00th=[26084], | 70.00th=[32900], 80.00th=[38011], 90.00th=[46924], 95.00th=[53740], | 99.00th=[70779], 99.50th=[79168], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 158, max= 576, per=25.12%, avg=314.72, stdev=85.93, samples=39 iops : min= 9, max= 36, avg=19.05, stdev= 5.40, samples=39 lat (usec) : 500=0.39%, 750=1.17%, 1000=0.26% lat (msec) : 2=0.13%, 4=1.04%, 10=10.03%, 20=25.78%, 50=54.04% lat (msec) : 100=7.16% cpu : usr=0.08%, sys=29.08%, ctx=3043, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182575: Wed Sep 24 15:28:41 2025 read: IOPS=18, BW=294KiB/s (301kB/s)(5744KiB/19527msec) clat (usec): min=454, max=90497, avg=27416.53, stdev=13994.91 lat (usec): min=454, max=90498, avg=27417.53, stdev=13995.06 clat percentiles (usec): | 1.00th=[ 570], 5.00th=[ 824], 10.00th=[11863], 20.00th=[16909], | 30.00th=[20841], 40.00th=[23462], 50.00th=[25822], 60.00th=[29492], | 70.00th=[33424], 80.00th=[36963], 90.00th=[45876], 95.00th=[52167], | 99.00th=[65799], 99.50th=[80217], 99.90th=[90702], 99.95th=[90702], | 99.99th=[90702] bw ( KiB/s): min= 159, max= 448, per=24.67%, avg=293.87, stdev=82.94, samples=38 iops : min= 9, max= 28, avg=17.74, stdev= 5.17, samples=38 write: IOPS=20, BW=335KiB/s (343kB/s)(6544KiB/19527msec); 0 zone resets clat (usec): min=1513, max=85801, avg=23429.66, stdev=14311.17 lat (usec): min=1514, max=85801, avg=23431.60, stdev=14311.17 clat percentiles (usec): | 1.00th=[ 4621], 5.00th=[ 6128], 10.00th=[ 7439], 20.00th=[10159], | 30.00th=[14353], 40.00th=[17171], 50.00th=[20579], 60.00th=[23725], | 70.00th=[28705], 80.00th=[35390], 90.00th=[43779], 95.00th=[52167], | 99.00th=[65274], 99.50th=[69731], 99.90th=[85459], 99.95th=[85459], | 99.99th=[85459] bw ( KiB/s): min= 159, max= 510, per=26.69%, avg=334.42, stdev=90.19, samples=38 iops : min= 9, max= 31, avg=20.26, stdev= 5.70, samples=38 lat (usec) : 500=0.26%, 750=1.30%, 1000=1.17% lat (msec) : 2=0.26%, 4=0.39%, 10=10.81%, 20=24.35%, 50=55.34% lat (msec) : 100=6.12% cpu : usr=0.10%, sys=29.86%, ctx=3077, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182577: Wed Sep 24 15:28:41 2025 read: IOPS=19, BW=311KiB/s (319kB/s)(6256KiB/20107msec) clat (usec): min=238, max=92953, avg=27555.11, stdev=13697.48 lat (usec): min=238, max=92954, avg=27556.28, stdev=13697.48 clat percentiles (usec): | 1.00th=[ 502], 5.00th=[ 5538], 10.00th=[12911], 20.00th=[16909], | 30.00th=[20317], 40.00th=[23462], 50.00th=[26084], 60.00th=[29754], | 70.00th=[33817], 80.00th=[38011], 90.00th=[44303], 95.00th=[50070], | 99.00th=[72877], 99.50th=[84411], 99.90th=[92799], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 190, max= 445, per=25.91%, avg=308.54, stdev=71.06, samples=39 iops : min= 11, max= 27, avg=18.67, stdev= 4.44, samples=39 write: IOPS=18, BW=300KiB/s (307kB/s)(6032KiB/20107msec); 0 zone resets clat (msec): min=2, max=101, avg=24.48, stdev=15.58 lat (msec): min=2, max=101, avg=24.49, stdev=15.58 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 6], 10.00th=[ 8], 20.00th=[ 11], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 27], | 70.00th=[ 31], 80.00th=[ 36], 90.00th=[ 44], 95.00th=[ 57], | 99.00th=[ 74], 99.50th=[ 79], 99.90th=[ 102], 99.95th=[ 102], | 99.99th=[ 102] bw ( KiB/s): min= 95, max= 480, per=23.18%, avg=290.44, stdev=91.89, samples=39 iops : min= 5, max= 30, avg=17.54, stdev= 5.83, samples=39 lat (usec) : 250=0.13%, 500=0.26%, 750=1.43%, 1000=0.52% lat (msec) : 4=1.04%, 10=9.51%, 20=24.74%, 50=56.12%, 100=6.12% lat (msec) : 250=0.13% cpu : usr=0.09%, sys=28.85%, ctx=3271, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182579: Wed Sep 24 15:28:41 2025 read: IOPS=19, BW=311KiB/s (319kB/s)(6032KiB/19384msec) clat (usec): min=98, max=82744, avg=26817.52, stdev=13211.44 lat (usec): min=100, max=82745, avg=26818.62, stdev=13211.51 clat percentiles (usec): | 1.00th=[ 424], 5.00th=[ 742], 10.00th=[12780], 20.00th=[17433], | 30.00th=[20317], 40.00th=[23462], 50.00th=[25822], 60.00th=[28705], | 70.00th=[31851], 80.00th=[36963], 90.00th=[43254], 95.00th=[50070], | 99.00th=[63177], 99.50th=[76022], 99.90th=[82314], 99.95th=[82314], | 99.99th=[82314] bw ( KiB/s): min= 127, max= 508, per=25.83%, avg=307.66, stdev=81.93, samples=38 iops : min= 7, max= 31, avg=18.58, stdev= 5.18, samples=38 write: IOPS=20, BW=323KiB/s (330kB/s)(6256KiB/19384msec); 0 zone resets clat (usec): min=1177, max=76703, avg=23574.92, stdev=14858.39 lat (usec): min=1178, max=76705, avg=23576.96, stdev=14858.39 clat percentiles (usec): | 1.00th=[ 3654], 5.00th=[ 6652], 10.00th=[ 7832], 20.00th=[ 9765], | 30.00th=[12911], 40.00th=[17695], 50.00th=[19792], 60.00th=[23987], | 70.00th=[29492], 80.00th=[34341], 90.00th=[46400], 95.00th=[53216], | 99.00th=[68682], 99.50th=[74974], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 158, max= 477, per=25.60%, avg=320.74, stdev=71.01, samples=38 iops : min= 9, max= 29, avg=19.42, stdev= 4.47, samples=38 lat (usec) : 100=0.13%, 500=0.39%, 750=1.95%, 1000=0.65% lat (msec) : 2=0.39%, 4=0.39%, 10=10.94%, 20=25.00%, 50=54.04% lat (msec) : 100=6.12% cpu : usr=0.11%, sys=30.21%, ctx=3094, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1191KiB/s (1220kB/s), 294KiB/s-311KiB/s (301kB/s-319kB/s), io=23.4MiB (24.5MB), run=19384-20107msec WRITE: bw=1253KiB/s (1283kB/s), 300KiB/s-335KiB/s (307kB/s-343kB/s), io=24.6MiB (25.8MB), run=19384-20107msec rand-rw: (groupid=0, jobs=1): err= 0: pid=182574: Wed Sep 24 15:28:43 2025 read: IOPS=17, BW=276KiB/s (282kB/s)(5920KiB/21476msec) clat (msec): min=5, max=100, avg=24.67, stdev=17.98 lat (msec): min=5, max=100, avg=24.67, stdev=17.98 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 19], 60.00th=[ 22], | 70.00th=[ 27], 80.00th=[ 36], 90.00th=[ 53], 95.00th=[ 63], | 99.00th=[ 89], 99.50th=[ 101], 99.90th=[ 102], 99.95th=[ 102], | 99.99th=[ 102] bw ( KiB/s): min= 127, max= 800, per=24.01%, avg=264.83, stdev=132.35, samples=42 iops : min= 7, max= 50, avg=15.93, stdev= 8.35, samples=42 write: IOPS=18, BW=297KiB/s (304kB/s)(6368KiB/21476msec); 0 zone resets clat (usec): min=6534, max=99325, avg=30942.93, stdev=18529.89 lat (usec): min=6536, max=99328, avg=30945.16, stdev=18529.92 clat percentiles (usec): | 1.00th=[ 7832], 5.00th=[ 9634], 10.00th=[11338], 20.00th=[14484], | 30.00th=[17957], 40.00th=[21365], 50.00th=[25822], 60.00th=[31589], | 70.00th=[37487], 80.00th=[46924], 90.00th=[58459], 95.00th=[67634], | 99.00th=[82314], 99.50th=[99091], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 63, max= 736, per=24.93%, avg=289.14, stdev=127.20, samples=42 iops : min= 3, max= 46, avg=17.48, stdev= 8.06, samples=42 lat (msec) : 10=10.55%, 20=35.55%, 50=39.06%, 100=14.58%, 250=0.26% cpu : usr=0.15%, sys=13.76%, ctx=1822, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182576: Wed Sep 24 15:28:43 2025 read: IOPS=16, BW=265KiB/s (271kB/s)(5744KiB/21680msec) clat (usec): min=3485, max=81679, avg=25365.14, stdev=17676.72 lat (usec): min=3486, max=81680, avg=25366.28, stdev=17676.77 clat percentiles (usec): | 1.00th=[ 4686], 5.00th=[ 6849], 10.00th=[ 8160], 20.00th=[11338], | 30.00th=[14091], 40.00th=[16188], 50.00th=[20055], 60.00th=[23725], | 70.00th=[28443], 80.00th=[38011], 90.00th=[55313], 95.00th=[63701], | 99.00th=[76022], 99.50th=[80217], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 54, max= 800, per=24.11%, avg=265.95, stdev=156.71, samples=43 iops : min= 3, max= 50, avg=16.02, stdev= 9.92, samples=43 write: IOPS=18, BW=302KiB/s (309kB/s)(6544KiB/21680msec); 0 zone resets clat (msec): min=5, max=114, avg=30.48, stdev=20.47 lat (msec): min=5, max=114, avg=30.48, stdev=20.47 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 13], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 31], | 70.00th=[ 38], 80.00th=[ 47], 90.00th=[ 62], 95.00th=[ 70], | 99.00th=[ 90], 99.50th=[ 94], 99.90th=[ 114], 99.95th=[ 114], | 99.99th=[ 114] bw ( KiB/s): min= 127, max= 1088, per=25.98%, avg=301.42, stdev=195.11, samples=43 iops : min= 7, max= 68, avg=18.23, stdev=12.37, samples=43 lat (msec) : 4=0.39%, 10=12.89%, 20=32.16%, 50=39.32%, 100=14.97% lat (msec) : 250=0.26% cpu : usr=0.16%, sys=13.38%, ctx=1827, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182578: Wed Sep 24 15:28:43 2025 read: IOPS=18, BW=288KiB/s (295kB/s)(6256KiB/21706msec) clat (usec): min=3867, max=96528, avg=25467.88, stdev=18195.50 lat (usec): min=3867, max=96530, avg=25468.98, stdev=18195.55 clat percentiles (usec): | 1.00th=[ 4621], 5.00th=[ 6652], 10.00th=[ 8029], 20.00th=[11076], | 30.00th=[13829], 40.00th=[16319], 50.00th=[18744], 60.00th=[23462], | 70.00th=[28443], 80.00th=[39060], 90.00th=[55313], 95.00th=[62653], | 99.00th=[81265], 99.50th=[87557], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 94, max= 1024, per=26.15%, avg=288.49, stdev=183.43, samples=43 iops : min= 5, max= 64, avg=17.44, stdev=11.62, samples=43 write: IOPS=17, BW=278KiB/s (285kB/s)(6032KiB/21706msec); 0 zone resets clat (msec): min=4, max=107, avg=30.86, stdev=18.73 lat (msec): min=4, max=107, avg=30.87, stdev=18.73 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 15], | 30.00th=[ 19], 40.00th=[ 23], 50.00th=[ 28], 60.00th=[ 32], | 70.00th=[ 37], 80.00th=[ 45], 90.00th=[ 59], 95.00th=[ 65], | 99.00th=[ 97], 99.50th=[ 101], 99.90th=[ 108], 99.95th=[ 108], | 99.99th=[ 108] bw ( KiB/s): min= 95, max= 1024, per=23.71%, avg=275.00, stdev=151.22, samples=43 iops : min= 5, max= 64, avg=16.58, stdev= 9.55, samples=43 lat (msec) : 4=0.13%, 10=12.63%, 20=30.99%, 50=40.76%, 100=15.23% lat (msec) : 250=0.26% cpu : usr=0.06%, sys=13.37%, ctx=1761, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182580: Wed Sep 24 15:28:43 2025 read: IOPS=17, BW=279KiB/s (286kB/s)(6032KiB/21614msec) clat (usec): min=3267, max=92499, avg=25454.09, stdev=18549.93 lat (usec): min=3268, max=92500, avg=25455.15, stdev=18549.96 clat percentiles (usec): | 1.00th=[ 4293], 5.00th=[ 6063], 10.00th=[ 8029], 20.00th=[10945], | 30.00th=[14222], 40.00th=[17171], 50.00th=[19530], 60.00th=[22938], | 70.00th=[28443], 80.00th=[38011], 90.00th=[52691], 95.00th=[68682], | 99.00th=[85459], 99.50th=[90702], 99.90th=[92799], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 95, max= 864, per=23.87%, avg=263.33, stdev=139.36, samples=42 iops : min= 5, max= 54, avg=15.86, stdev= 8.81, samples=42 write: IOPS=18, BW=289KiB/s (296kB/s)(6256KiB/21614msec); 0 zone resets clat (msec): min=5, max=104, avg=30.42, stdev=18.47 lat (msec): min=5, max=104, avg=30.43, stdev=18.47 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 15], | 30.00th=[ 18], 40.00th=[ 22], 50.00th=[ 26], 60.00th=[ 32], | 70.00th=[ 37], 80.00th=[ 45], 90.00th=[ 56], 95.00th=[ 66], | 99.00th=[ 91], 99.50th=[ 103], 99.90th=[ 105], 99.95th=[ 105], | 99.99th=[ 105] bw ( KiB/s): min= 127, max= 768, per=23.94%, avg=277.76, stdev=120.43, samples=42 iops : min= 7, max= 48, avg=16.74, stdev= 7.70, samples=42 lat (msec) : 4=0.39%, 10=12.89%, 20=30.21%, 50=42.58%, 100=13.67% lat (msec) : 250=0.26% cpu : usr=0.11%, sys=13.45%, ctx=1732, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1103KiB/s (1130kB/s), 265KiB/s-288KiB/s (271kB/s-295kB/s), io=23.4MiB (24.5MB), run=21476-21706msec WRITE: bw=1161KiB/s (1189kB/s), 278KiB/s-302KiB/s (285kB/s-309kB/s), io=24.6MiB (25.8MB), run=21476-21706msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=182589: Wed Sep 24 15:28:54 2025 read: IOPS=1, BW=1804KiB/s (1847kB/s)(7168KiB/3974msec) clat (usec): min=1257, max=1487.2k, avg=332618.02, stdev=519361.17 lat (usec): min=1258, max=1487.2k, avg=332619.71, stdev=519361.26 clat percentiles (usec): | 1.00th=[ 1254], 5.00th=[ 1254], 10.00th=[ 1254], | 20.00th=[ 73925], 30.00th=[ 101188], 40.00th=[ 101188], | 50.00th=[ 154141], 60.00th=[ 179307], 70.00th=[ 179307], | 80.00th=[ 329253], 90.00th=[1484784], 95.00th=[1484784], | 99.00th=[1484784], 99.50th=[1484784], 99.90th=[1484784], | 99.95th=[1484784], 99.99th=[1484784] bw ( KiB/s): min= 1768, max= 6119, per=75.26%, avg=2946.25, stdev=2118.28, samples=4 iops : min= 1, max= 5, avg= 2.00, stdev= 2.00, samples=4 write: IOPS=1, BW=1288KiB/s (1319kB/s)(5120KiB/3974msec); 0 zone resets clat (msec): min=133, max=552, avg=317.65, stdev=150.43 lat (msec): min=133, max=552, avg=317.73, stdev=150.44 clat percentiles (msec): | 1.00th=[ 133], 5.00th=[ 133], 10.00th=[ 133], 20.00th=[ 133], | 30.00th=[ 288], 40.00th=[ 288], 50.00th=[ 292], 60.00th=[ 292], | 70.00th=[ 321], 80.00th=[ 321], 90.00th=[ 550], 95.00th=[ 550], | 99.00th=[ 550], 99.50th=[ 550], 99.90th=[ 550], 99.95th=[ 550], | 99.99th=[ 550] bw ( KiB/s): min= 1762, max= 2043, per=35.12%, avg=1924.75, stdev=139.51, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 lat (msec) : 2=8.33%, 100=8.33%, 250=33.33%, 500=33.33%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.03%, sys=50.57%, ctx=1488, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182590: Wed Sep 24 15:28:54 2025 read: IOPS=0, BW=392KiB/s (401kB/s)(2048KiB/5231msec) clat (msec): min=155, max=408, avg=281.79, stdev=178.60 lat (msec): min=155, max=408, avg=281.79, stdev=178.60 clat percentiles (msec): | 1.00th=[ 157], 5.00th=[ 157], 10.00th=[ 157], 20.00th=[ 157], | 30.00th=[ 157], 40.00th=[ 157], 50.00th=[ 157], 60.00th=[ 409], | 70.00th=[ 409], 80.00th=[ 409], 90.00th=[ 409], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 1768, max= 1855, per=46.27%, avg=1811.50, stdev=61.52, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1958KiB/s (2005kB/s)(10.0MiB/5231msec); 0 zone resets clat (msec): min=173, max=1648, avg=465.12, stdev=436.43 lat (msec): min=173, max=1648, avg=465.22, stdev=436.43 clat percentiles (msec): | 1.00th=[ 174], 5.00th=[ 174], 10.00th=[ 174], 20.00th=[ 201], | 30.00th=[ 205], 40.00th=[ 284], 50.00th=[ 300], 60.00th=[ 363], | 70.00th=[ 388], 80.00th=[ 531], 90.00th=[ 558], 95.00th=[ 1653], | 99.00th=[ 1653], 99.50th=[ 1653], 99.90th=[ 1653], 99.95th=[ 1653], | 99.99th=[ 1653] bw ( KiB/s): min= 1762, max= 4096, per=54.54%, avg=2989.17, stdev=1212.48, samples=6 iops : min= 1, max= 4, avg= 2.33, stdev= 1.51, samples=6 lat (msec) : 250=33.33%, 500=41.67%, 750=16.67%, 2000=8.33% cpu : usr=0.00%, sys=40.36%, ctx=574, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182591: Wed Sep 24 15:28:54 2025 read: IOPS=1, BW=1334KiB/s (1366kB/s)(6144KiB/4606msec) clat (usec): min=1240, max=262813, avg=200116.54, stdev=99026.99 lat (usec): min=1241, max=262815, avg=200118.09, stdev=99027.21 clat percentiles (usec): | 1.00th=[ 1237], 5.00th=[ 1237], 10.00th=[ 1237], 20.00th=[214959], | 30.00th=[214959], 40.00th=[227541], 50.00th=[227541], 60.00th=[238027], | 70.00th=[256902], 80.00th=[256902], 90.00th=[263193], 95.00th=[263193], | 99.00th=[263193], 99.50th=[263193], 99.90th=[263193], 99.95th=[263193], | 99.99th=[263193] bw ( KiB/s): min= 1762, max= 3710, per=61.01%, avg=2388.50, stdev=890.76, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=1, BW=1334KiB/s (1366kB/s)(6144KiB/4606msec); 0 zone resets clat (msec): min=316, max=1476, avg=559.25, stdev=451.83 lat (msec): min=316, max=1476, avg=559.33, stdev=451.84 clat percentiles (msec): | 1.00th=[ 317], 5.00th=[ 317], 10.00th=[ 317], 20.00th=[ 342], | 30.00th=[ 342], 40.00th=[ 380], 50.00th=[ 380], 60.00th=[ 388], | 70.00th=[ 456], 80.00th=[ 456], 90.00th=[ 1485], 95.00th=[ 1485], | 99.00th=[ 1485], 99.50th=[ 1485], 99.90th=[ 1485], 99.95th=[ 1485], | 99.99th=[ 1485] bw ( KiB/s): min= 1762, max= 2043, per=35.23%, avg=1931.00, stdev=151.56, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (msec) : 2=8.33%, 250=25.00%, 500=58.33%, 2000=8.33% cpu : usr=0.09%, sys=42.04%, ctx=282, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182592: Wed Sep 24 15:28:54 2025 read: IOPS=0, BW=1019KiB/s (1044kB/s)(5120KiB/5023msec) clat (usec): min=789, max=588067, avg=256243.99, stdev=254091.86 lat (usec): min=791, max=588069, avg=256245.18, stdev=254091.67 clat percentiles (usec): | 1.00th=[ 791], 5.00th=[ 791], 10.00th=[ 791], 20.00th=[ 791], | 30.00th=[ 1270], 40.00th=[ 1270], 50.00th=[312476], 60.00th=[312476], | 70.00th=[375391], 80.00th=[375391], 90.00th=[591397], 95.00th=[591397], | 99.00th=[591397], 99.50th=[591397], 99.90th=[591397], 99.95th=[591397], | 99.99th=[591397] bw ( KiB/s): min= 2039, max= 5565, per=97.11%, avg=3802.00, stdev=2493.26, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=1, BW=1427KiB/s (1461kB/s)(7168KiB/5023msec); 0 zone resets clat (msec): min=219, max=1575, avg=531.66, stdev=467.27 lat (msec): min=219, max=1576, avg=531.75, stdev=467.28 clat percentiles (msec): | 1.00th=[ 220], 5.00th=[ 220], 10.00th=[ 220], 20.00th=[ 288], | 30.00th=[ 380], 40.00th=[ 380], 50.00th=[ 384], 60.00th=[ 426], | 70.00th=[ 426], 80.00th=[ 447], 90.00th=[ 1569], 95.00th=[ 1569], | 99.00th=[ 1569], 99.50th=[ 1569], 99.90th=[ 1569], 99.95th=[ 1569], | 99.99th=[ 1569] bw ( KiB/s): min= 1762, max= 2048, per=35.48%, avg=1944.60, stdev=137.42, samples=5 iops : min= 1, max= 2, avg= 1.20, stdev= 0.45, samples=5 lat (usec) : 1000=8.33% lat (msec) : 2=8.33%, 250=8.33%, 500=58.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=38.37%, ctx=288, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3915KiB/s (4009kB/s), 392KiB/s-1804KiB/s (401kB/s-1847kB/s), io=20.0MiB (20.0MB), run=3974-5231msec WRITE: bw=5481KiB/s (5613kB/s), 1288KiB/s-1958KiB/s (1319kB/s-2005kB/s), io=28.0MiB (29.4MB), run=3974-5231msec rand-rw: (groupid=0, jobs=1): err= 0: pid=182593: Wed Sep 24 15:28:54 2025 read: IOPS=1, BW=1351KiB/s (1384kB/s)(7168KiB/5305msec) clat (msec): min=64, max=1457, avg=402.66, stdev=521.27 lat (msec): min=64, max=1457, avg=402.66, stdev=521.27 clat percentiles (msec): | 1.00th=[ 65], 5.00th=[ 65], 10.00th=[ 65], 20.00th=[ 79], | 30.00th=[ 109], 40.00th=[ 109], 50.00th=[ 150], 60.00th=[ 220], | 70.00th=[ 220], 80.00th=[ 743], 90.00th=[ 1452], 95.00th=[ 1452], | 99.00th=[ 1452], 99.50th=[ 1452], 99.90th=[ 1452], 99.95th=[ 1452], | 99.99th=[ 1452] bw ( KiB/s): min= 1802, max= 4096, per=65.16%, avg=2437.50, stdev=1111.32, samples=4 iops : min= 1, max= 4, avg= 1.75, stdev= 1.50, samples=4 write: IOPS=0, BW=965KiB/s (988kB/s)(5120KiB/5305msec); 0 zone resets clat (msec): min=91, max=726, avg=483.74, stdev=254.44 lat (msec): min=91, max=726, avg=483.83, stdev=254.43 clat percentiles (msec): | 1.00th=[ 92], 5.00th=[ 92], 10.00th=[ 92], 20.00th=[ 92], | 30.00th=[ 409], 40.00th=[ 409], 50.00th=[ 510], 60.00th=[ 510], | 70.00th=[ 684], 80.00th=[ 684], 90.00th=[ 726], 95.00th=[ 726], | 99.00th=[ 726], 99.50th=[ 726], 99.90th=[ 726], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 2035, max= 2048, per=39.00%, avg=2042.25, stdev= 5.38, samples=4 iops : min= 1, max= 2, avg= 1.25, stdev= 0.50, samples=4 lat (msec) : 100=25.00%, 250=25.00%, 500=8.33%, 750=33.33%, 2000=8.33% cpu : usr=0.04%, sys=12.07%, ctx=122, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182594: Wed Sep 24 15:28:54 2025 read: IOPS=0, BW=380KiB/s (389kB/s)(2048KiB/5388msec) clat (msec): min=222, max=572, avg=397.57, stdev=246.92 lat (msec): min=222, max=572, avg=397.57, stdev=246.92 clat percentiles (msec): | 1.00th=[ 224], 5.00th=[ 224], 10.00th=[ 224], 20.00th=[ 224], | 30.00th=[ 224], 40.00th=[ 224], 50.00th=[ 224], 60.00th=[ 575], | 70.00th=[ 575], 80.00th=[ 575], 90.00th=[ 575], 95.00th=[ 575], | 99.00th=[ 575], 99.50th=[ 575], 99.90th=[ 575], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 2035, max= 2048, per=54.57%, avg=2041.50, stdev= 9.19, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1901KiB/s (1946kB/s)(10.0MiB/5388msec); 0 zone resets clat (msec): min=78, max=1947, avg=449.99, stdev=561.36 lat (msec): min=78, max=1947, avg=450.08, stdev=561.37 clat percentiles (msec): | 1.00th=[ 80], 5.00th=[ 80], 10.00th=[ 80], 20.00th=[ 95], | 30.00th=[ 100], 40.00th=[ 161], 50.00th=[ 213], 60.00th=[ 275], | 70.00th=[ 502], 80.00th=[ 510], 90.00th=[ 625], 95.00th=[ 1955], | 99.00th=[ 1955], 99.50th=[ 1955], 99.90th=[ 1955], 99.95th=[ 1955], | 99.99th=[ 1955] bw ( KiB/s): min= 2035, max= 6144, per=54.70%, avg=2864.60, stdev=1833.25, samples=5 iops : min= 1, max= 6, avg= 2.60, stdev= 1.95, samples=5 lat (msec) : 100=25.00%, 250=25.00%, 500=8.33%, 750=33.33%, 2000=8.33% cpu : usr=0.07%, sys=19.53%, ctx=259, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182595: Wed Sep 24 15:28:54 2025 read: IOPS=1, BW=1165KiB/s (1193kB/s)(6144KiB/5273msec) clat (msec): min=72, max=834, avg=277.69, stdev=296.94 lat (msec): min=72, max=834, avg=277.69, stdev=296.94 clat percentiles (msec): | 1.00th=[ 73], 5.00th=[ 73], 10.00th=[ 73], 20.00th=[ 84], | 30.00th=[ 84], 40.00th=[ 106], 50.00th=[ 106], 60.00th=[ 184], | 70.00th=[ 388], 80.00th=[ 388], 90.00th=[ 835], 95.00th=[ 835], | 99.00th=[ 835], 99.50th=[ 835], 99.90th=[ 835], 99.95th=[ 835], | 99.99th=[ 835] bw ( KiB/s): min= 2035, max= 4096, per=68.31%, avg=2555.50, stdev=1027.01, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 write: IOPS=1, BW=1165KiB/s (1193kB/s)(6144KiB/5273msec); 0 zone resets clat (msec): min=113, max=1637, avg=574.78, stdev=544.23 lat (msec): min=114, max=1637, avg=574.88, stdev=544.23 clat percentiles (msec): | 1.00th=[ 114], 5.00th=[ 114], 10.00th=[ 114], 20.00th=[ 251], | 30.00th=[ 251], 40.00th=[ 409], 50.00th=[ 409], 60.00th=[ 498], | 70.00th=[ 542], 80.00th=[ 542], 90.00th=[ 1636], 95.00th=[ 1636], | 99.00th=[ 1636], 99.50th=[ 1636], 99.90th=[ 1636], 99.95th=[ 1636], | 99.99th=[ 1636] bw ( KiB/s): min= 1802, max= 4096, per=47.68%, avg=2497.25, stdev=1072.00, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 lat (msec) : 100=16.67%, 250=25.00%, 500=33.33%, 750=8.33%, 1000=8.33% lat (msec) : 2000=8.33% cpu : usr=0.02%, sys=17.15%, ctx=220, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182596: Wed Sep 24 15:28:54 2025 read: IOPS=0, BW=935KiB/s (958kB/s)(5120KiB/5474msec) clat (msec): min=19, max=915, avg=379.05, stdev=369.53 lat (msec): min=19, max=915, avg=379.06, stdev=369.53 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 20], 10.00th=[ 20], 20.00th=[ 20], | 30.00th=[ 110], 40.00th=[ 110], 50.00th=[ 264], 60.00th=[ 264], | 70.00th=[ 592], 80.00th=[ 592], 90.00th=[ 919], 95.00th=[ 919], | 99.00th=[ 919], 99.50th=[ 919], 99.90th=[ 919], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 2043, max= 2048, per=54.70%, avg=2046.33, stdev= 2.89, samples=3 iops : min= 1, max= 2, avg= 1.67, stdev= 0.58, samples=3 write: IOPS=1, BW=1309KiB/s (1341kB/s)(7168KiB/5474msec); 0 zone resets clat (msec): min=20, max=1464, avg=506.53, stdev=537.41 lat (msec): min=20, max=1465, avg=506.61, stdev=537.41 clat percentiles (msec): | 1.00th=[ 21], 5.00th=[ 21], 10.00th=[ 21], 20.00th=[ 53], | 30.00th=[ 61], 40.00th=[ 61], 50.00th=[ 401], 60.00th=[ 667], | 70.00th=[ 667], 80.00th=[ 877], 90.00th=[ 1469], 95.00th=[ 1469], | 99.00th=[ 1469], 99.50th=[ 1469], 99.90th=[ 1469], 99.95th=[ 1469], | 99.99th=[ 1469] bw ( KiB/s): min= 1802, max= 2048, per=37.85%, avg=1982.00, stdev=120.12, samples=4 iops : min= 1, max= 2, avg= 1.25, stdev= 0.50, samples=4 lat (msec) : 20=8.33%, 50=8.33%, 100=16.67%, 250=8.33%, 500=16.67% lat (msec) : 750=16.67%, 1000=16.67%, 2000=8.33% cpu : usr=0.02%, sys=13.58%, ctx=153, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3741KiB/s (3831kB/s), 380KiB/s-1351KiB/s (389kB/s-1384kB/s), io=20.0MiB (20.0MB), run=5273-5474msec WRITE: bw=5238KiB/s (5364kB/s), 965KiB/s-1901KiB/s (988kB/s-1946kB/s), io=28.0MiB (29.4MB), run=5273-5474msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=182602: Wed Sep 24 15:29:06 2025 read: IOPS=1, BW=4955KiB/s (5074kB/s)(12.0MiB/2480msec) clat (msec): min=546, max=1165, avg=808.95, stdev=319.88 lat (msec): min=546, max=1165, avg=808.95, stdev=319.88 clat percentiles (msec): | 1.00th=[ 550], 5.00th=[ 550], 10.00th=[ 550], 20.00th=[ 550], | 30.00th=[ 550], 40.00th=[ 718], 50.00th=[ 718], 60.00th=[ 718], | 70.00th=[ 1167], 80.00th=[ 1167], 90.00th=[ 1167], 95.00th=[ 1167], | 99.00th=[ 1167], 99.50th=[ 1167], 99.90th=[ 1167], 99.95th=[ 1167], | 99.99th=[ 1167] bw ( KiB/s): min= 8175, max= 8192, per=100.00%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=49.42%, ctx=2616, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182603: Wed Sep 24 15:29:06 2025 read: IOPS=0, BW=855KiB/s (875kB/s)(4096KiB/4792msec) clat (nsec): min=553482k, max=553482k, avg=553482343.00, stdev= 0.00 lat (nsec): min=553485k, max=553485k, avg=553485170.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 550], 5.00th=[ 550], 10.00th=[ 550], 20.00th=[ 550], | 30.00th=[ 550], 40.00th=[ 550], 50.00th=[ 550], 60.00th=[ 550], | 70.00th=[ 550], 80.00th=[ 550], 90.00th=[ 550], 95.00th=[ 550], | 99.00th=[ 550], 99.50th=[ 550], 99.90th=[ 550], 99.95th=[ 550], | 99.99th=[ 550] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1710KiB/s (1751kB/s)(8192KiB/4792msec); 0 zone resets clat (msec): min=1640, max=2564, avg=2102.55, stdev=653.71 lat (msec): min=1640, max=2565, avg=2102.92, stdev=653.65 clat percentiles (msec): | 1.00th=[ 1636], 5.00th=[ 1636], 10.00th=[ 1636], 20.00th=[ 1636], | 30.00th=[ 1636], 40.00th=[ 1636], 50.00th=[ 1636], 60.00th=[ 2567], | 70.00th=[ 2567], 80.00th=[ 2567], 90.00th=[ 2567], 95.00th=[ 2567], | 99.00th=[ 2567], 99.50th=[ 2567], 99.90th=[ 2567], 99.95th=[ 2567], | 99.99th=[ 2567] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=28.74%, ctx=809, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182604: Wed Sep 24 15:29:06 2025 write: IOPS=0, BW=2898KiB/s (2968kB/s)(12.0MiB/4240msec); 0 zone resets clat (msec): min=1053, max=2018, avg=1403.71, stdev=534.05 lat (msec): min=1054, max=2018, avg=1404.21, stdev=534.02 clat percentiles (msec): | 1.00th=[ 1053], 5.00th=[ 1053], 10.00th=[ 1053], 20.00th=[ 1053], | 30.00th=[ 1053], 40.00th=[ 1133], 50.00th=[ 1133], 60.00th=[ 1133], | 70.00th=[ 2022], 80.00th=[ 2022], 90.00th=[ 2022], 95.00th=[ 2022], | 99.00th=[ 2022], 99.50th=[ 2022], 99.90th=[ 2022], 99.95th=[ 2022], | 99.99th=[ 2022] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 2000=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=47.63%, ctx=38, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182605: Wed Sep 24 15:29:06 2025 write: IOPS=0, BW=3712KiB/s (3801kB/s)(12.0MiB/3310msec); 0 zone resets clat (msec): min=826, max=1520, avg=1093.14, stdev=373.60 lat (msec): min=827, max=1520, avg=1093.51, stdev=373.61 clat percentiles (msec): | 1.00th=[ 827], 5.00th=[ 827], 10.00th=[ 827], 20.00th=[ 827], | 30.00th=[ 827], 40.00th=[ 936], 50.00th=[ 936], 60.00th=[ 936], | 70.00th=[ 1519], 80.00th=[ 1519], 90.00th=[ 1519], 95.00th=[ 1519], | 99.00th=[ 1519], 99.50th=[ 1519], 99.90th=[ 1519], 99.95th=[ 1519], | 99.99th=[ 1519] bw ( KiB/s): min= 7684, max= 8175, per=100.00%, avg=7929.50, stdev=347.19, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=66.67%, 2000=33.33% cpu : usr=0.00%, sys=65.00%, ctx=197, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3419KiB/s (3501kB/s), 855KiB/s-4955KiB/s (875kB/s-5074kB/s), io=16.0MiB (16.8MB), run=2480-4792msec WRITE: bw=6838KiB/s (7002kB/s), 1710KiB/s-3712KiB/s (1751kB/s-3801kB/s), io=32.0MiB (33.6MB), run=3310-4792msec rand-rw: (groupid=0, jobs=1): err= 0: pid=182607: Wed Sep 24 15:29:06 2025 read: IOPS=0, BW=2612KiB/s (2674kB/s)(12.0MiB/4705msec) clat (msec): min=433, max=2207, avg=1525.40, stdev=955.60 lat (msec): min=433, max=2207, avg=1525.40, stdev=955.60 clat percentiles (msec): | 1.00th=[ 435], 5.00th=[ 435], 10.00th=[ 435], 20.00th=[ 435], | 30.00th=[ 435], 40.00th=[ 1938], 50.00th=[ 1938], 60.00th=[ 1938], | 70.00th=[ 2198], 80.00th=[ 2198], 90.00th=[ 2198], 95.00th=[ 2198], | 99.00th=[ 2198], 99.50th=[ 2198], 99.90th=[ 2198], 99.95th=[ 2198], | 99.99th=[ 2198] bw ( KiB/s): min= 8175, max= 8192, per=100.00%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.04%, sys=6.38%, ctx=86, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182608: Wed Sep 24 15:29:06 2025 read: IOPS=0, BW=832KiB/s (852kB/s)(4096KiB/4924msec) clat (nsec): min=2455.9M, max=2455.9M, avg=2455856004.00, stdev= 0.00 lat (nsec): min=2455.9M, max=2455.9M, avg=2455856982.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 2467], 5.00th=[ 2467], 10.00th=[ 2467], 20.00th=[ 2467], | 30.00th=[ 2467], 40.00th=[ 2467], 50.00th=[ 2467], 60.00th=[ 2467], | 70.00th=[ 2467], 80.00th=[ 2467], 90.00th=[ 2467], 95.00th=[ 2467], | 99.00th=[ 2467], 99.50th=[ 2467], 99.90th=[ 2467], 99.95th=[ 2467], | 99.99th=[ 2467] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1664KiB/s (1704kB/s)(8192KiB/4924msec); 0 zone resets clat (msec): min=606, max=1754, avg=1180.49, stdev=811.65 lat (msec): min=607, max=1754, avg=1180.92, stdev=811.61 clat percentiles (msec): | 1.00th=[ 609], 5.00th=[ 609], 10.00th=[ 609], 20.00th=[ 609], | 30.00th=[ 609], 40.00th=[ 609], 50.00th=[ 609], 60.00th=[ 1754], | 70.00th=[ 1754], 80.00th=[ 1754], 90.00th=[ 1754], 95.00th=[ 1754], | 99.00th=[ 1754], 99.50th=[ 1754], 99.90th=[ 1754], 99.95th=[ 1754], | 99.99th=[ 1754] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.02%, sys=14.14%, ctx=115, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182609: Wed Sep 24 15:29:06 2025 write: IOPS=0, BW=2682KiB/s (2746kB/s)(12.0MiB/4582msec); 0 zone resets clat (msec): min=858, max=1890, avg=1521.85, stdev=575.58 lat (msec): min=859, max=1890, avg=1522.17, stdev=575.48 clat percentiles (msec): | 1.00th=[ 860], 5.00th=[ 860], 10.00th=[ 860], 20.00th=[ 860], | 30.00th=[ 860], 40.00th=[ 1821], 50.00th=[ 1821], 60.00th=[ 1821], | 70.00th=[ 1888], 80.00th=[ 1888], 90.00th=[ 1888], 95.00th=[ 1888], | 99.00th=[ 1888], 99.50th=[ 1888], 99.90th=[ 1888], 99.95th=[ 1888], | 99.99th=[ 1888] bw ( KiB/s): min= 8175, max= 8192, per=100.00%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=20.24%, ctx=276, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=182610: Wed Sep 24 15:29:06 2025 write: IOPS=0, BW=2818KiB/s (2886kB/s)(12.0MiB/4360msec); 0 zone resets clat (msec): min=329, max=2118, avg=1417.88, stdev=955.35 lat (msec): min=330, max=2118, avg=1418.24, stdev=955.29 clat percentiles (msec): | 1.00th=[ 330], 5.00th=[ 330], 10.00th=[ 330], 20.00th=[ 330], | 30.00th=[ 330], 40.00th=[ 1804], 50.00th=[ 1804], 60.00th=[ 1804], | 70.00th=[ 2123], 80.00th=[ 2123], 90.00th=[ 2123], 95.00th=[ 2123], | 99.00th=[ 2123], 99.50th=[ 2123], 99.90th=[ 2123], 99.95th=[ 2123], | 99.99th=[ 2123] bw ( KiB/s): min= 8175, max= 8192, per=100.00%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.09%, sys=14.06%, ctx=79, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3327KiB/s (3407kB/s), 832KiB/s-2612KiB/s (852kB/s-2674kB/s), io=16.0MiB (16.8MB), run=4705-4924msec WRITE: bw=6655KiB/s (6814kB/s), 1664KiB/s-2818KiB/s (1704kB/s-2886kB/s), io=32.0MiB (33.6MB), run=4360-4924msec