== sanity test 398b: DIO and buffer IO race ============== 19:49:02 (1763340542) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 4.27751 s, 11.8 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148992: Sun Nov 16 19:50:58 2025 read: IOPS=15, BW=60.8KiB/s (62.3kB/s)(5992KiB/98482msec) clat (usec): min=93, max=704592, avg=32463.00, stdev=28934.54 lat (usec): min=94, max=704594, avg=32483.74, stdev=28944.48 clat percentiles (usec): | 1.00th=[ 128], 5.00th=[ 159], 10.00th=[ 338], 20.00th=[ 15401], | 30.00th=[ 19792], 40.00th=[ 23987], 50.00th=[ 27919], 60.00th=[ 32375], | 70.00th=[ 39060], 80.00th=[ 49021], 90.00th=[ 62653], 95.00th=[ 71828], | 99.00th=[103285], 99.50th=[111674], 99.90th=[267387], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 14, max= 119, per=25.66%, avg=60.82, stdev=23.53, samples=193 iops : min= 3, max= 29, avg=14.60, stdev= 5.90, samples=193 write: IOPS=15, BW=63.9KiB/s (65.5kB/s)(6296KiB/98482msec); 0 zone resets clat (usec): min=894, max=355134, avg=31599.88, stdev=33082.14 lat (usec): min=895, max=355135, avg=31601.14, stdev=33082.15 clat percentiles (usec): | 1.00th=[ 996], 5.00th=[ 1074], 10.00th=[ 1139], 20.00th=[ 1991], | 30.00th=[ 6652], 40.00th=[ 15139], 50.00th=[ 24249], 60.00th=[ 32637], | 70.00th=[ 41157], 80.00th=[ 53740], 90.00th=[ 74974], 95.00th=[ 89654], | 99.00th=[143655], 99.50th=[177210], 99.90th=[265290], 99.95th=[354419], | 99.99th=[354419] bw ( KiB/s): min= 15, max= 126, per=25.72%, avg=63.80, stdev=23.21, samples=193 iops : min= 3, max= 31, avg=15.36, stdev= 5.87, samples=193 lat (usec) : 100=0.03%, 250=4.33%, 500=0.78%, 1000=0.65% lat (msec) : 2=9.77%, 4=2.38%, 10=5.63%, 20=14.32%, 50=40.95% lat (msec) : 100=18.95%, 250=2.08%, 500=0.10%, 750=0.03% cpu : usr=0.09%, sys=6.49%, ctx=6237, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148993: Sun Nov 16 19:50:58 2025 read: IOPS=14, BW=58.7KiB/s (60.1kB/s)(5940KiB/101163msec) clat (usec): min=104, max=776403, avg=31797.91, stdev=30784.91 lat (usec): min=104, max=776404, avg=31799.25, stdev=30785.06 clat percentiles (usec): | 1.00th=[ 120], 5.00th=[ 159], 10.00th=[ 198], 20.00th=[ 14353], | 30.00th=[ 19268], 40.00th=[ 22938], 50.00th=[ 27395], 60.00th=[ 32375], | 70.00th=[ 39060], 80.00th=[ 48497], 90.00th=[ 60031], 95.00th=[ 70779], | 99.00th=[100140], 99.50th=[156238], 99.90th=[270533], 99.95th=[775947], | 99.99th=[775947] bw ( KiB/s): min= 7, max= 143, per=24.70%, avg=58.53, stdev=24.03, samples=197 iops : min= 1, max= 35, avg=14.03, stdev= 6.08, samples=197 write: IOPS=15, BW=62.8KiB/s (64.3kB/s)(6348KiB/101163msec); 0 zone resets clat (usec): min=872, max=295785, avg=33906.44, stdev=34726.74 lat (usec): min=873, max=295786, avg=33914.53, stdev=34728.17 clat percentiles (usec): | 1.00th=[ 979], 5.00th=[ 1057], 10.00th=[ 1123], 20.00th=[ 2073], | 30.00th=[ 7046], 40.00th=[ 16909], 50.00th=[ 26870], 60.00th=[ 35390], | 70.00th=[ 44827], 80.00th=[ 57934], 90.00th=[ 77071], 95.00th=[101188], | 99.00th=[143655], 99.50th=[183501], 99.90th=[278922], 99.95th=[295699], | 99.99th=[295699] bw ( KiB/s): min= 15, max= 191, per=25.03%, avg=62.09, stdev=26.98, samples=198 iops : min= 3, max= 47, avg=14.92, stdev= 6.80, samples=198 lat (usec) : 250=5.31%, 500=0.88%, 750=0.07%, 1000=0.94% lat (msec) : 2=9.54%, 4=2.70%, 10=4.82%, 20=13.57%, 50=40.07% lat (msec) : 100=19.04%, 250=2.93%, 500=0.10%, 1000=0.03% cpu : usr=0.09%, sys=6.53%, ctx=6206, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148994: Sun Nov 16 19:50:58 2025 read: IOPS=15, BW=60.8KiB/s (62.3kB/s)(6028KiB/99128msec) clat (usec): min=94, max=293439, avg=31771.52, stdev=25552.76 lat (usec): min=94, max=293440, avg=31772.80, stdev=25552.50 clat percentiles (usec): | 1.00th=[ 127], 5.00th=[ 151], 10.00th=[ 192], 20.00th=[ 14746], | 30.00th=[ 19006], 40.00th=[ 22676], 50.00th=[ 27132], 60.00th=[ 32637], | 70.00th=[ 40109], 80.00th=[ 47973], 90.00th=[ 61080], 95.00th=[ 71828], | 99.00th=[115868], 99.50th=[149947], 99.90th=[231736], 99.95th=[291505], | 99.99th=[291505] bw ( KiB/s): min= 7, max= 128, per=25.59%, avg=60.66, stdev=23.88, samples=194 iops : min= 1, max= 32, avg=14.57, stdev= 6.03, samples=194 write: IOPS=15, BW=63.2KiB/s (64.7kB/s)(6260KiB/99128msec); 0 zone resets clat (usec): min=763, max=840117, avg=32680.34, stdev=38512.05 lat (usec): min=763, max=840119, avg=32683.31, stdev=38512.67 clat percentiles (usec): | 1.00th=[ 971], 5.00th=[ 1074], 10.00th=[ 1156], 20.00th=[ 2442], | 30.00th=[ 7242], 40.00th=[ 15795], 50.00th=[ 25035], 60.00th=[ 33817], | 70.00th=[ 43779], 80.00th=[ 55313], 90.00th=[ 72877], 95.00th=[ 91751], | 99.00th=[143655], 99.50th=[193987], 99.90th=[261096], 99.95th=[843056], | 99.99th=[843056] bw ( KiB/s): min= 14, max= 149, per=25.40%, avg=62.99, stdev=23.05, samples=194 iops : min= 3, max= 37, avg=15.15, stdev= 5.80, samples=194 lat (usec) : 100=0.03%, 250=5.66%, 500=0.65%, 750=0.13%, 1000=0.91% lat (msec) : 2=8.63%, 4=2.90%, 10=5.31%, 20=14.75%, 50=40.10% lat (msec) : 100=18.46%, 250=2.38%, 500=0.07%, 1000=0.03% cpu : usr=0.08%, sys=6.47%, ctx=6172, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148995: Sun Nov 16 19:50:58 2025 read: IOPS=15, BW=60.6KiB/s (62.1kB/s)(6052KiB/99875msec) clat (usec): min=99, max=267579, avg=31080.79, stdev=24267.23 lat (usec): min=99, max=267580, avg=31081.80, stdev=24267.24 clat percentiles (usec): | 1.00th=[ 118], 5.00th=[ 153], 10.00th=[ 208], 20.00th=[ 13960], | 30.00th=[ 19006], 40.00th=[ 22676], 50.00th=[ 27132], 60.00th=[ 32113], | 70.00th=[ 38011], 80.00th=[ 46924], 90.00th=[ 60556], 95.00th=[ 70779], | 99.00th=[109577], 99.50th=[125305], 99.90th=[248513], 99.95th=[267387], | 99.99th=[267387] bw ( KiB/s): min= 7, max= 117, per=25.54%, avg=60.54, stdev=23.70, samples=194 iops : min= 1, max= 29, avg=14.53, stdev= 5.99, samples=194 write: IOPS=15, BW=62.4KiB/s (63.9kB/s)(6236KiB/99875msec); 0 zone resets clat (usec): min=901, max=904207, avg=33823.58, stdev=40354.19 lat (usec): min=902, max=904209, avg=33824.95, stdev=40354.25 clat percentiles (usec): | 1.00th=[ 988], 5.00th=[ 1074], 10.00th=[ 1156], 20.00th=[ 2245], | 30.00th=[ 7635], 40.00th=[ 15664], 50.00th=[ 25560], 60.00th=[ 33817], | 70.00th=[ 44303], 80.00th=[ 58983], 90.00th=[ 78119], 95.00th=[ 98042], | 99.00th=[141558], 99.50th=[152044], 99.90th=[350225], 99.95th=[901776], | 99.99th=[901776] bw ( KiB/s): min= 21, max= 135, per=25.07%, avg=62.17, stdev=22.73, samples=195 iops : min= 5, max= 33, avg=14.95, stdev= 5.73, samples=195 lat (usec) : 100=0.03%, 250=5.34%, 500=0.88%, 750=0.07%, 1000=0.75% lat (msec) : 2=9.47%, 4=2.05%, 10=5.96%, 20=14.13%, 50=39.65% lat (msec) : 100=18.72%, 250=2.86%, 500=0.07%, 1000=0.03% cpu : usr=0.08%, sys=6.43%, ctx=6322, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=237KiB/s (243kB/s), 58.7KiB/s-60.8KiB/s (60.1kB/s-62.3kB/s), io=23.4MiB (24.6MB), run=98482-101163msec WRITE: bw=249KiB/s (254kB/s), 62.4KiB/s-63.9KiB/s (63.9kB/s-65.5kB/s), io=24.6MiB (25.7MB), run=98482-101163msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148986: Sun Nov 16 19:53:01 2025 read: IOPS=6, BW=26.8KiB/s (27.4kB/s)(5992KiB/223877msec) clat (msec): min=5, max=827, avg=29.79, stdev=34.36 lat (msec): min=5, max=827, avg=29.79, stdev=34.36 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 26], | 70.00th=[ 33], 80.00th=[ 43], 90.00th=[ 58], 95.00th=[ 75], | 99.00th=[ 138], 99.50th=[ 205], 99.90th=[ 355], 99.95th=[ 827], | 99.99th=[ 827] bw ( KiB/s): min= 7, max= 96, per=26.89%, avg=28.77, stdev=18.23, samples=404 iops : min= 1, max= 24, avg= 6.59, stdev= 4.60, samples=404 write: IOPS=7, BW=28.1KiB/s (28.8kB/s)(6296KiB/223877msec); 0 zone resets clat (msec): min=36, max=453, avg=113.68, stdev=56.03 lat (msec): min=36, max=453, avg=113.68, stdev=56.02 clat percentiles (msec): | 1.00th=[ 45], 5.00th=[ 55], 10.00th=[ 62], 20.00th=[ 71], | 30.00th=[ 81], 40.00th=[ 89], 50.00th=[ 99], 60.00th=[ 112], | 70.00th=[ 126], 80.00th=[ 148], 90.00th=[ 188], 95.00th=[ 220], | 99.00th=[ 330], 99.50th=[ 355], 99.90th=[ 426], 99.95th=[ 456], | 99.99th=[ 456] bw ( KiB/s): min= 7, max= 56, per=24.80%, avg=27.77, stdev=12.05, samples=439 iops : min= 1, max= 14, avg= 6.33, stdev= 3.08, samples=439 lat (msec) : 10=5.05%, 20=20.02%, 50=18.39%, 100=30.63%, 250=24.15% lat (msec) : 500=1.73%, 1000=0.03% cpu : usr=0.05%, sys=1.43%, ctx=5543, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148987: Sun Nov 16 19:53:01 2025 read: IOPS=6, BW=26.6KiB/s (27.3kB/s)(5940KiB/223036msec) clat (msec): min=5, max=882, avg=29.29, stdev=35.78 lat (msec): min=5, max=882, avg=29.30, stdev=35.78 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 33], 80.00th=[ 42], 90.00th=[ 56], 95.00th=[ 73], | 99.00th=[ 144], 99.50th=[ 201], 99.90th=[ 355], 99.95th=[ 885], | 99.99th=[ 885] bw ( KiB/s): min= 7, max= 96, per=26.92%, avg=28.80, stdev=18.92, samples=400 iops : min= 1, max= 24, avg= 6.59, stdev= 4.79, samples=400 write: IOPS=7, BW=28.5KiB/s (29.1kB/s)(6348KiB/223036msec); 0 zone resets clat (msec): min=35, max=454, avg=112.92, stdev=53.81 lat (msec): min=35, max=454, avg=112.92, stdev=53.81 clat percentiles (msec): | 1.00th=[ 45], 5.00th=[ 56], 10.00th=[ 62], 20.00th=[ 73], | 30.00th=[ 82], 40.00th=[ 89], 50.00th=[ 99], 60.00th=[ 110], | 70.00th=[ 123], 80.00th=[ 146], 90.00th=[ 186], 95.00th=[ 220], | 99.00th=[ 309], 99.50th=[ 359], 99.90th=[ 414], 99.95th=[ 456], | 99.99th=[ 456] bw ( KiB/s): min= 7, max= 56, per=25.29%, avg=28.32, stdev=11.87, samples=434 iops : min= 1, max= 14, avg= 6.47, stdev= 3.01, samples=434 lat (msec) : 10=5.34%, 20=19.92%, 50=18.16%, 100=30.44%, 250=24.87% lat (msec) : 500=1.24%, 1000=0.03% cpu : usr=0.04%, sys=1.43%, ctx=5465, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148988: Sun Nov 16 19:53:01 2025 read: IOPS=6, BW=27.1KiB/s (27.8kB/s)(6028KiB/222398msec) clat (msec): min=5, max=317, avg=30.61, stdev=28.27 lat (msec): min=5, max=317, avg=30.62, stdev=28.27 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 21], 60.00th=[ 27], | 70.00th=[ 35], 80.00th=[ 45], 90.00th=[ 61], 95.00th=[ 82], | 99.00th=[ 142], 99.50th=[ 186], 99.90th=[ 264], 99.95th=[ 317], | 99.99th=[ 317] bw ( KiB/s): min= 7, max= 110, per=26.38%, avg=28.22, stdev=19.04, samples=414 iops : min= 1, max= 27, avg= 6.45, stdev= 4.80, samples=414 write: IOPS=7, BW=28.1KiB/s (28.8kB/s)(6260KiB/222398msec); 0 zone resets clat (msec): min=36, max=952, avg=112.42, stdev=56.98 lat (msec): min=36, max=952, avg=112.42, stdev=56.98 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 55], 10.00th=[ 62], 20.00th=[ 71], | 30.00th=[ 81], 40.00th=[ 89], 50.00th=[ 100], 60.00th=[ 112], | 70.00th=[ 125], 80.00th=[ 144], 90.00th=[ 184], 95.00th=[ 215], | 99.00th=[ 305], 99.50th=[ 347], 99.90th=[ 426], 99.95th=[ 953], | 99.99th=[ 953] bw ( KiB/s): min= 7, max= 56, per=24.92%, avg=27.91, stdev=12.00, samples=435 iops : min= 1, max= 14, avg= 6.36, stdev= 3.05, samples=435 lat (msec) : 10=4.49%, 20=19.30%, 50=18.42%, 100=31.35%, 250=25.00% lat (msec) : 500=1.40%, 1000=0.03% cpu : usr=0.04%, sys=1.48%, ctx=5628, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148989: Sun Nov 16 19:53:01 2025 read: IOPS=6, BW=27.4KiB/s (28.0kB/s)(6052KiB/221254msec) clat (msec): min=5, max=316, avg=29.30, stdev=27.12 lat (msec): min=5, max=316, avg=29.30, stdev=27.12 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 33], 80.00th=[ 43], 90.00th=[ 61], 95.00th=[ 78], | 99.00th=[ 120], 99.50th=[ 190], 99.90th=[ 262], 99.95th=[ 317], | 99.99th=[ 317] bw ( KiB/s): min= 7, max= 119, per=27.76%, avg=29.70, stdev=19.47, samples=396 iops : min= 1, max= 29, avg= 6.82, stdev= 4.89, samples=396 write: IOPS=7, BW=28.2KiB/s (28.9kB/s)(6236KiB/221254msec); 0 zone resets clat (msec): min=29, max=1017, avg=113.27, stdev=58.90 lat (msec): min=29, max=1017, avg=113.28, stdev=58.90 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 56], 10.00th=[ 62], 20.00th=[ 72], | 30.00th=[ 81], 40.00th=[ 89], 50.00th=[ 97], 60.00th=[ 109], | 70.00th=[ 124], 80.00th=[ 144], 90.00th=[ 188], 95.00th=[ 226], | 99.00th=[ 309], 99.50th=[ 334], 99.90th=[ 422], 99.95th=[ 1020], | 99.99th=[ 1020] bw ( KiB/s): min= 7, max= 55, per=24.99%, avg=27.99, stdev=11.83, samples=432 iops : min= 1, max= 13, avg= 6.38, stdev= 3.03, samples=432 lat (msec) : 10=4.88%, 20=20.41%, 50=17.74%, 100=32.06%, 250=23.11% lat (msec) : 500=1.76%, 2000=0.03% cpu : usr=0.04%, sys=1.45%, ctx=5530, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=107KiB/s (110kB/s), 26.6KiB/s-27.4KiB/s (27.3kB/s-28.0kB/s), io=23.4MiB (24.6MB), run=221254-223877msec WRITE: bw=112KiB/s (115kB/s), 28.1KiB/s-28.5KiB/s (28.8kB/s-29.1kB/s), io=24.6MiB (25.7MB), run=221254-223877msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=149041: Sun Nov 16 19:53:30 2025 read: IOPS=18, BW=288KiB/s (295kB/s)(5920KiB/20550msec) clat (usec): min=91, max=131813, avg=27644.23, stdev=23290.45 lat (usec): min=92, max=131815, avg=27645.52, stdev=23290.66 clat percentiles (usec): | 1.00th=[ 118], 5.00th=[ 137], 10.00th=[ 151], 20.00th=[ 196], | 30.00th=[ 14091], 40.00th=[ 20317], 50.00th=[ 26084], 60.00th=[ 31589], | 70.00th=[ 38536], 80.00th=[ 45876], 90.00th=[ 56886], 95.00th=[ 64750], | 99.00th=[102237], 99.50th=[116917], 99.90th=[131597], 99.95th=[131597], | 99.99th=[131597] bw ( KiB/s): min= 95, max= 604, per=25.86%, avg=289.67, stdev=106.77, samples=40 iops : min= 5, max= 37, avg=17.32, stdev= 6.65, samples=40 write: IOPS=19, BW=310KiB/s (317kB/s)(6368KiB/20550msec); 0 zone resets clat (usec): min=1291, max=185939, avg=25478.08, stdev=27028.09 lat (usec): min=1291, max=185942, avg=25479.97, stdev=27028.30 clat percentiles (usec): | 1.00th=[ 1385], 5.00th=[ 1532], 10.00th=[ 1631], 20.00th=[ 2409], | 30.00th=[ 7177], 40.00th=[ 11338], 50.00th=[ 16909], 60.00th=[ 23200], | 70.00th=[ 31589], 80.00th=[ 42206], 90.00th=[ 65799], 95.00th=[ 76022], | 99.00th=[119014], 99.50th=[160433], 99.90th=[185598], 99.95th=[185598], | 99.99th=[185598] bw ( KiB/s): min= 95, max= 574, per=26.50%, avg=312.20, stdev=118.54, samples=40 iops : min= 5, max= 35, avg=18.78, stdev= 7.41, samples=40 lat (usec) : 100=0.13%, 250=10.68%, 500=0.91%, 750=0.13%, 1000=0.13% lat (msec) : 2=9.90%, 4=2.99%, 10=7.29%, 20=14.58%, 50=35.94% lat (msec) : 100=16.02%, 250=1.30% cpu : usr=0.12%, sys=7.91%, ctx=1360, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149042: Sun Nov 16 19:53:30 2025 read: IOPS=18, BW=289KiB/s (296kB/s)(5744KiB/19869msec) clat (usec): min=105, max=143772, avg=28355.09, stdev=23575.18 lat (usec): min=106, max=143773, avg=28356.08, stdev=23575.17 clat percentiles (usec): | 1.00th=[ 122], 5.00th=[ 145], 10.00th=[ 157], 20.00th=[ 269], | 30.00th=[ 15795], 40.00th=[ 22152], 50.00th=[ 26870], 60.00th=[ 32375], | 70.00th=[ 40633], 80.00th=[ 46924], 90.00th=[ 56361], 95.00th=[ 62653], | 99.00th=[104334], 99.50th=[141558], 99.90th=[143655], 99.95th=[143655], | 99.99th=[143655] bw ( KiB/s): min= 95, max= 604, per=25.93%, avg=290.42, stdev=96.76, samples=38 iops : min= 5, max= 37, avg=17.39, stdev= 6.08, samples=38 write: IOPS=20, BW=329KiB/s (337kB/s)(6544KiB/19869msec); 0 zone resets clat (usec): min=1303, max=158320, avg=23227.04, stdev=24997.10 lat (usec): min=1305, max=158321, avg=23228.83, stdev=24996.97 clat percentiles (usec): | 1.00th=[ 1401], 5.00th=[ 1549], 10.00th=[ 1631], 20.00th=[ 2114], | 30.00th=[ 4752], 40.00th=[ 8717], 50.00th=[ 13698], 60.00th=[ 20841], | 70.00th=[ 31327], 80.00th=[ 42206], 90.00th=[ 58459], 95.00th=[ 72877], | 99.00th=[101188], 99.50th=[108528], 99.90th=[158335], 99.95th=[158335], | 99.99th=[158335] bw ( KiB/s): min= 125, max= 540, per=27.86%, avg=328.18, stdev=103.88, samples=38 iops : min= 7, max= 33, avg=19.74, stdev= 6.54, samples=38 lat (usec) : 250=8.85%, 500=1.69%, 750=0.26%, 1000=0.26% lat (msec) : 2=10.29%, 4=5.21%, 10=8.33%, 20=13.80%, 50=35.68% lat (msec) : 100=14.32%, 250=1.30% cpu : usr=0.10%, sys=8.19%, ctx=1357, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149043: Sun Nov 16 19:53:30 2025 read: IOPS=18, BW=293KiB/s (300kB/s)(6256KiB/21377msec) clat (usec): min=112, max=260668, avg=28578.39, stdev=26880.21 lat (usec): min=112, max=260670, avg=28590.27, stdev=26873.66 clat percentiles (usec): | 1.00th=[ 118], 5.00th=[ 141], 10.00th=[ 155], 20.00th=[ 208], | 30.00th=[ 13566], 40.00th=[ 20579], 50.00th=[ 25822], 60.00th=[ 31851], | 70.00th=[ 37487], 80.00th=[ 47449], 90.00th=[ 58983], 95.00th=[ 69731], | 99.00th=[104334], 99.50th=[135267], 99.90th=[261096], 99.95th=[261096], | 99.99th=[261096] bw ( KiB/s): min= 95, max= 542, per=25.93%, avg=290.44, stdev=107.28, samples=41 iops : min= 5, max= 33, avg=17.39, stdev= 6.72, samples=41 write: IOPS=17, BW=282KiB/s (289kB/s)(6032KiB/21377msec); 0 zone resets clat (usec): min=1281, max=199412, avg=26897.20, stdev=29916.03 lat (usec): min=1282, max=199414, avg=26899.01, stdev=29916.10 clat percentiles (usec): | 1.00th=[ 1287], 5.00th=[ 1401], 10.00th=[ 1647], 20.00th=[ 2900], | 30.00th=[ 5866], 40.00th=[ 10290], 50.00th=[ 15926], 60.00th=[ 24511], | 70.00th=[ 34866], 80.00th=[ 47449], 90.00th=[ 63701], 95.00th=[ 86508], | 99.00th=[152044], 99.50th=[175113], 99.90th=[200279], 99.95th=[200279], | 99.99th=[200279] bw ( KiB/s): min= 63, max= 507, per=23.21%, avg=273.37, stdev=101.69, samples=41 iops : min= 3, max= 31, avg=16.32, stdev= 6.35, samples=41 lat (usec) : 250=10.81%, 500=1.56%, 750=0.13%, 1000=0.13% lat (msec) : 2=7.94%, 4=5.47%, 10=7.55%, 20=13.28%, 50=35.16% lat (msec) : 100=15.76%, 250=2.08%, 500=0.13% cpu : usr=0.13%, sys=7.89%, ctx=1412, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149044: Sun Nov 16 19:53:30 2025 read: IOPS=19, BW=319KiB/s (327kB/s)(6032KiB/18891msec) clat (usec): min=94, max=106355, avg=24784.80, stdev=20343.92 lat (usec): min=95, max=106356, avg=24785.81, stdev=20343.94 clat percentiles (usec): | 1.00th=[ 111], 5.00th=[ 147], 10.00th=[ 161], 20.00th=[ 251], | 30.00th=[ 12649], 40.00th=[ 18482], 50.00th=[ 24773], 60.00th=[ 29492], | 70.00th=[ 34866], 80.00th=[ 41157], 90.00th=[ 52691], 95.00th=[ 61604], | 99.00th=[ 72877], 99.50th=[ 89654], 99.90th=[106431], 99.95th=[106431], | 99.99th=[106431] bw ( KiB/s): min= 31, max= 636, per=28.75%, avg=322.00, stdev=111.44, samples=36 iops : min= 1, max= 39, avg=19.39, stdev= 6.99, samples=36 write: IOPS=20, BW=331KiB/s (339kB/s)(6256KiB/18891msec); 0 zone resets clat (usec): min=1207, max=159063, avg=24018.85, stdev=26269.79 lat (usec): min=1207, max=159065, avg=24020.75, stdev=26269.83 clat percentiles (usec): | 1.00th=[ 1319], 5.00th=[ 1483], 10.00th=[ 1631], 20.00th=[ 2278], | 30.00th=[ 4817], 40.00th=[ 8717], 50.00th=[ 14353], 60.00th=[ 22152], | 70.00th=[ 32637], 80.00th=[ 43779], 90.00th=[ 57410], 95.00th=[ 73925], | 99.00th=[117965], 99.50th=[147850], 99.90th=[158335], 99.95th=[158335], | 99.99th=[158335] bw ( KiB/s): min= 159, max= 572, per=28.28%, avg=333.08, stdev=101.06, samples=36 iops : min= 9, max= 35, avg=20.08, stdev= 6.30, samples=36 lat (usec) : 100=0.26%, 250=9.64%, 500=2.73%, 750=0.13%, 1000=0.26% lat (msec) : 2=9.24%, 4=5.99%, 10=8.20%, 20=12.89%, 50=37.76% lat (msec) : 100=11.98%, 250=0.91% cpu : usr=0.10%, sys=8.51%, ctx=1339, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1120KiB/s (1147kB/s), 288KiB/s-319KiB/s (295kB/s-327kB/s), io=23.4MiB (24.5MB), run=18891-21377msec WRITE: bw=1179KiB/s (1207kB/s), 282KiB/s-331KiB/s (289kB/s-339kB/s), io=24.6MiB (25.8MB), run=18891-21377msec rand-rw: (groupid=0, jobs=1): err= 0: pid=149046: Sun Nov 16 19:54:02 2025 read: IOPS=7, BW=115KiB/s (118kB/s)(5920KiB/51376msec) clat (msec): min=4, max=206, avg=25.60, stdev=26.71 lat (msec): min=4, max=206, avg=25.61, stdev=26.71 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 23], 80.00th=[ 33], 90.00th=[ 57], 95.00th=[ 87], | 99.00th=[ 136], 99.50th=[ 157], 99.90th=[ 207], 99.95th=[ 207], | 99.99th=[ 207] bw ( KiB/s): min= 31, max= 415, per=26.45%, avg=121.41, stdev=80.95, samples=95 iops : min= 1, max= 25, avg= 6.87, stdev= 5.07, samples=95 write: IOPS=7, BW=124KiB/s (127kB/s)(6368KiB/51376msec); 0 zone resets clat (msec): min=37, max=385, avg=105.13, stdev=52.13 lat (msec): min=37, max=385, avg=105.14, stdev=52.13 clat percentiles (msec): | 1.00th=[ 40], 5.00th=[ 53], 10.00th=[ 58], 20.00th=[ 67], | 30.00th=[ 75], 40.00th=[ 83], 50.00th=[ 92], 60.00th=[ 102], | 70.00th=[ 118], 80.00th=[ 133], 90.00th=[ 167], 95.00th=[ 203], | 99.00th=[ 347], 99.50th=[ 372], 99.90th=[ 384], 99.95th=[ 384], | 99.99th=[ 384] bw ( KiB/s): min= 29, max= 223, per=25.55%, avg=123.41, stdev=54.27, samples=101 iops : min= 1, max= 13, avg= 6.99, stdev= 3.47, samples=101 lat (msec) : 10=7.55%, 20=22.66%, 50=14.45%, 100=31.77%, 250=22.53% lat (msec) : 500=1.04% cpu : usr=0.05%, sys=1.63%, ctx=1380, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149047: Sun Nov 16 19:54:02 2025 read: IOPS=6, BW=110KiB/s (113kB/s)(5744KiB/52087msec) clat (msec): min=6, max=147, avg=25.67, stdev=25.64 lat (msec): min=6, max=147, avg=25.68, stdev=25.65 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 19], | 70.00th=[ 22], 80.00th=[ 33], 90.00th=[ 59], 95.00th=[ 80], | 99.00th=[ 136], 99.50th=[ 144], 99.90th=[ 148], 99.95th=[ 148], | 99.99th=[ 148] bw ( KiB/s): min= 30, max= 384, per=25.81%, avg=118.45, stdev=76.20, samples=95 iops : min= 1, max= 24, avg= 6.71, stdev= 4.91, samples=95 write: IOPS=7, BW=126KiB/s (129kB/s)(6544KiB/52087msec); 0 zone resets clat (msec): min=34, max=381, avg=104.68, stdev=52.78 lat (msec): min=34, max=381, avg=104.69, stdev=52.78 clat percentiles (msec): | 1.00th=[ 40], 5.00th=[ 50], 10.00th=[ 54], 20.00th=[ 64], | 30.00th=[ 74], 40.00th=[ 83], 50.00th=[ 92], 60.00th=[ 103], | 70.00th=[ 116], 80.00th=[ 136], 90.00th=[ 174], 95.00th=[ 215], | 99.00th=[ 279], 99.50th=[ 305], 99.90th=[ 380], 99.95th=[ 380], | 99.99th=[ 380] bw ( KiB/s): min= 30, max= 287, per=25.82%, avg=124.73, stdev=56.61, samples=103 iops : min= 1, max= 17, avg= 7.08, stdev= 3.60, samples=103 lat (msec) : 10=6.64%, 20=24.22%, 50=12.76%, 100=32.42%, 250=22.53% lat (msec) : 500=1.43% cpu : usr=0.05%, sys=1.60%, ctx=1410, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149048: Sun Nov 16 19:54:02 2025 read: IOPS=8, BW=129KiB/s (132kB/s)(6256KiB/48640msec) clat (msec): min=5, max=230, avg=23.82, stdev=25.83 lat (msec): min=5, max=230, avg=23.82, stdev=25.83 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 17], | 70.00th=[ 22], 80.00th=[ 28], 90.00th=[ 50], 95.00th=[ 65], | 99.00th=[ 155], 99.50th=[ 174], 99.90th=[ 230], 99.95th=[ 230], | 99.99th=[ 230] bw ( KiB/s): min= 31, max= 542, per=30.04%, avg=137.88, stdev=99.46, samples=90 iops : min= 1, max= 33, avg= 7.89, stdev= 6.31, samples=90 write: IOPS=7, BW=124KiB/s (127kB/s)(6032KiB/48640msec); 0 zone resets clat (msec): min=31, max=302, avg=104.20, stdev=48.84 lat (msec): min=31, max=302, avg=104.20, stdev=48.84 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 50], 10.00th=[ 54], 20.00th=[ 66], | 30.00th=[ 74], 40.00th=[ 83], 50.00th=[ 90], 60.00th=[ 103], | 70.00th=[ 118], 80.00th=[ 138], 90.00th=[ 174], 95.00th=[ 203], | 99.00th=[ 262], 99.50th=[ 300], 99.90th=[ 305], 99.95th=[ 305], | 99.99th=[ 305] bw ( KiB/s): min= 31, max= 223, per=25.50%, avg=123.17, stdev=51.48, samples=96 iops : min= 1, max= 13, avg= 6.97, stdev= 3.30, samples=96 lat (msec) : 10=7.42%, 20=27.08%, 50=13.80%, 100=29.95%, 250=20.96% lat (msec) : 500=0.78% cpu : usr=0.06%, sys=1.71%, ctx=1426, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149049: Sun Nov 16 19:54:02 2025 read: IOPS=7, BW=118KiB/s (121kB/s)(6032KiB/51129msec) clat (msec): min=5, max=231, avg=24.64, stdev=26.18 lat (msec): min=5, max=231, avg=24.64, stdev=26.18 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 30], 90.00th=[ 56], 95.00th=[ 75], | 99.00th=[ 146], 99.50th=[ 174], 99.90th=[ 232], 99.95th=[ 232], | 99.99th=[ 232] bw ( KiB/s): min= 31, max= 320, per=27.27%, avg=125.16, stdev=79.39, samples=94 iops : min= 1, max= 20, avg= 7.11, stdev= 5.08, samples=94 write: IOPS=7, BW=122KiB/s (125kB/s)(6256KiB/51129msec); 0 zone resets clat (msec): min=36, max=384, avg=106.88, stdev=53.19 lat (msec): min=36, max=384, avg=106.88, stdev=53.19 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 50], 10.00th=[ 56], 20.00th=[ 66], | 30.00th=[ 75], 40.00th=[ 87], 50.00th=[ 96], 60.00th=[ 107], | 70.00th=[ 121], 80.00th=[ 136], 90.00th=[ 169], 95.00th=[ 194], | 99.00th=[ 317], 99.50th=[ 376], 99.90th=[ 384], 99.95th=[ 384], | 99.99th=[ 384] bw ( KiB/s): min= 31, max= 224, per=25.29%, avg=122.16, stdev=51.30, samples=101 iops : min= 1, max= 14, avg= 6.91, stdev= 3.32, samples=101 lat (msec) : 10=8.07%, 20=24.74%, 50=13.15%, 100=29.43%, 250=23.18% lat (msec) : 500=1.43% cpu : usr=0.05%, sys=1.59%, ctx=1410, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=460KiB/s (471kB/s), 110KiB/s-129KiB/s (113kB/s-132kB/s), io=23.4MiB (24.5MB), run=48640-52087msec WRITE: bw=484KiB/s (495kB/s), 122KiB/s-126KiB/s (125kB/s-129kB/s), io=24.6MiB (25.8MB), run=48640-52087msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=149065: Sun Nov 16 19:54:13 2025 read: IOPS=2, BW=2300KiB/s (2356kB/s)(7168KiB/3116msec) clat (msec): min=51, max=309, avg=133.76, stdev=104.24 lat (msec): min=51, max=309, avg=133.76, stdev=104.24 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 52], 10.00th=[ 52], 20.00th=[ 64], | 30.00th=[ 69], 40.00th=[ 69], 50.00th=[ 71], 60.00th=[ 118], | 70.00th=[ 118], 80.00th=[ 253], 90.00th=[ 309], 95.00th=[ 309], | 99.00th=[ 309], 99.50th=[ 309], 99.90th=[ 309], 99.95th=[ 309], | 99.99th=[ 309] bw ( KiB/s): min= 2035, max= 6854, per=90.65%, avg=4444.50, stdev=3407.55, samples=2 iops : min= 1, max= 6, avg= 3.50, stdev= 3.54, samples=2 write: IOPS=1, BW=1643KiB/s (1683kB/s)(5120KiB/3116msec); 0 zone resets clat (msec): min=152, max=1091, avg=424.38, stdev=381.11 lat (msec): min=153, max=1091, avg=424.47, stdev=381.12 clat percentiles (msec): | 1.00th=[ 153], 5.00th=[ 153], 10.00th=[ 153], 20.00th=[ 153], | 30.00th=[ 211], 40.00th=[ 211], 50.00th=[ 334], 60.00th=[ 334], | 70.00th=[ 334], 80.00th=[ 334], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 1370, max= 4079, per=36.08%, avg=2476.33, stdev=1421.07, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 100=33.33%, 250=25.00%, 500=33.33%, 2000=8.33% cpu : usr=0.00%, sys=5.01%, ctx=76, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149066: Sun Nov 16 19:54:13 2025 read: IOPS=0, BW=490KiB/s (502kB/s)(2048KiB/4177msec) clat (msec): min=96, max=793, avg=444.90, stdev=493.13 lat (msec): min=96, max=793, avg=444.90, stdev=493.13 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 96], 10.00th=[ 96], 20.00th=[ 96], | 30.00th=[ 96], 40.00th=[ 96], 50.00th=[ 96], 60.00th=[ 793], | 70.00th=[ 793], 80.00th=[ 793], 90.00th=[ 793], 95.00th=[ 793], | 99.00th=[ 793], 99.50th=[ 793], 99.90th=[ 793], 99.95th=[ 793], | 99.99th=[ 793] bw ( KiB/s): min= 1368, max= 2039, per=34.74%, avg=1703.50, stdev=474.47, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2452KiB/s (2510kB/s)(10.0MiB/4177msec); 0 zone resets clat (msec): min=119, max=1046, avg=326.74, stdev=272.20 lat (msec): min=119, max=1046, avg=326.84, stdev=272.19 clat percentiles (msec): | 1.00th=[ 121], 5.00th=[ 121], 10.00th=[ 121], 20.00th=[ 136], | 30.00th=[ 153], 40.00th=[ 174], 50.00th=[ 253], 60.00th=[ 275], | 70.00th=[ 355], 80.00th=[ 376], 90.00th=[ 388], 95.00th=[ 1045], | 99.00th=[ 1045], 99.50th=[ 1045], 99.90th=[ 1045], 99.95th=[ 1045], | 99.99th=[ 1045] bw ( KiB/s): min= 1980, max= 4106, per=41.40%, avg=2841.60, stdev=1142.18, samples=5 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=5 lat (msec) : 100=8.33%, 250=33.33%, 500=41.67%, 1000=8.33%, 2000=8.33% cpu : usr=0.00%, sys=5.29%, ctx=56, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149067: Sun Nov 16 19:54:13 2025 read: IOPS=1, BW=1765KiB/s (1807kB/s)(6144KiB/3481msec) clat (msec): min=62, max=378, avg=257.15, stdev=117.37 lat (msec): min=62, max=378, avg=257.15, stdev=117.37 clat percentiles (msec): | 1.00th=[ 63], 5.00th=[ 63], 10.00th=[ 63], 20.00th=[ 209], | 30.00th=[ 209], 40.00th=[ 220], 50.00th=[ 220], 60.00th=[ 334], | 70.00th=[ 338], 80.00th=[ 338], 90.00th=[ 380], 95.00th=[ 380], | 99.00th=[ 380], 99.50th=[ 380], 99.90th=[ 380], 99.95th=[ 380], | 99.99th=[ 380] bw ( KiB/s): min= 1980, max= 4087, per=51.53%, avg=2526.50, stdev=1040.62, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=1, BW=1765KiB/s (1807kB/s)(6144KiB/3481msec); 0 zone resets clat (msec): min=130, max=694, avg=302.98, stdev=218.67 lat (msec): min=130, max=694, avg=303.07, stdev=218.67 clat percentiles (msec): | 1.00th=[ 131], 5.00th=[ 131], 10.00th=[ 131], 20.00th=[ 155], | 30.00th=[ 155], 40.00th=[ 205], 50.00th=[ 205], 60.00th=[ 209], | 70.00th=[ 426], 80.00th=[ 426], 90.00th=[ 693], 95.00th=[ 693], | 99.00th=[ 693], 99.50th=[ 693], 99.90th=[ 693], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 2035, max= 4079, per=39.61%, avg=2719.00, stdev=1177.80, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 100=8.33%, 250=50.00%, 500=33.33%, 750=8.33% cpu : usr=0.00%, sys=5.23%, ctx=71, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149068: Sun Nov 16 19:54:13 2025 read: IOPS=1, BW=1454KiB/s (1489kB/s)(5120KiB/3522msec) clat (msec): min=35, max=717, avg=298.41, stdev=295.41 lat (msec): min=35, max=717, avg=298.41, stdev=295.41 clat percentiles (msec): | 1.00th=[ 36], 5.00th=[ 36], 10.00th=[ 36], 20.00th=[ 36], | 30.00th=[ 99], 40.00th=[ 99], 50.00th=[ 144], 60.00th=[ 144], | 70.00th=[ 498], 80.00th=[ 498], 90.00th=[ 718], 95.00th=[ 718], | 99.00th=[ 718], 99.50th=[ 718], 99.90th=[ 718], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 1976, max= 4096, per=55.12%, avg=2702.33, stdev=1207.31, samples=3 iops : min= 1, max= 4, avg= 2.00, stdev= 1.73, samples=3 write: IOPS=1, BW=2035KiB/s (2084kB/s)(7168KiB/3522msec); 0 zone resets clat (msec): min=118, max=663, avg=273.94, stdev=185.78 lat (msec): min=118, max=663, avg=274.03, stdev=185.79 clat percentiles (msec): | 1.00th=[ 120], 5.00th=[ 120], 10.00th=[ 120], 20.00th=[ 165], | 30.00th=[ 199], 40.00th=[ 199], 50.00th=[ 205], 60.00th=[ 220], | 70.00th=[ 220], 80.00th=[ 351], 90.00th=[ 667], 95.00th=[ 667], | 99.00th=[ 667], 99.50th=[ 667], 99.90th=[ 667], 99.95th=[ 667], | 99.99th=[ 667] bw ( KiB/s): min= 2023, max= 4015, per=39.24%, avg=2693.67, stdev=1144.35, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 50=8.33%, 100=8.33%, 250=50.00%, 500=16.67%, 750=16.67% cpu : usr=0.03%, sys=7.24%, ctx=113, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4903KiB/s (5021kB/s), 490KiB/s-2300KiB/s (502kB/s-2356kB/s), io=20.0MiB (20.0MB), run=3116-4177msec WRITE: bw=6864KiB/s (7029kB/s), 1643KiB/s-2452KiB/s (1683kB/s-2510kB/s), io=28.0MiB (29.4MB), run=3116-4177msec rand-rw: (groupid=0, jobs=1): err= 0: pid=149070: Sun Nov 16 19:54:13 2025 read: IOPS=2, BW=2620KiB/s (2683kB/s)(7168KiB/2736msec) clat (msec): min=122, max=321, avg=217.30, stdev=66.08 lat (msec): min=122, max=321, avg=217.30, stdev=66.08 clat percentiles (msec): | 1.00th=[ 124], 5.00th=[ 124], 10.00th=[ 124], 20.00th=[ 176], | 30.00th=[ 178], 40.00th=[ 178], 50.00th=[ 220], 60.00th=[ 241], | 70.00th=[ 241], 80.00th=[ 268], 90.00th=[ 321], 95.00th=[ 321], | 99.00th=[ 321], 99.50th=[ 321], 99.90th=[ 321], 99.95th=[ 321], | 99.99th=[ 321] bw ( KiB/s): min= 2035, max= 4571, per=63.08%, avg=3522.33, stdev=1323.69, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=1, BW=1871KiB/s (1916kB/s)(5120KiB/2736msec); 0 zone resets clat (msec): min=137, max=372, avg=239.22, stdev=97.11 lat (msec): min=137, max=373, avg=239.32, stdev=97.13 clat percentiles (msec): | 1.00th=[ 138], 5.00th=[ 138], 10.00th=[ 138], 20.00th=[ 138], | 30.00th=[ 155], 40.00th=[ 155], 50.00th=[ 247], 60.00th=[ 247], | 70.00th=[ 288], 80.00th=[ 288], 90.00th=[ 372], 95.00th=[ 372], | 99.00th=[ 372], 99.50th=[ 372], 99.90th=[ 372], 99.95th=[ 372], | 99.99th=[ 372] bw ( KiB/s): min= 1523, max= 4047, per=30.83%, avg=2410.00, stdev=1117.70, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 250=66.67%, 500=33.33% cpu : usr=0.04%, sys=26.98%, ctx=475, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149071: Sun Nov 16 19:54:13 2025 read: IOPS=0, BW=558KiB/s (572kB/s)(2048KiB/3667msec) clat (msec): min=147, max=293, avg=220.78, stdev=103.37 lat (msec): min=147, max=293, avg=220.78, stdev=103.37 clat percentiles (msec): | 1.00th=[ 148], 5.00th=[ 148], 10.00th=[ 148], 20.00th=[ 148], | 30.00th=[ 148], 40.00th=[ 148], 50.00th=[ 148], 60.00th=[ 296], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 296], 95.00th=[ 296], | 99.00th=[ 296], 99.50th=[ 296], 99.90th=[ 296], 99.95th=[ 296], | 99.99th=[ 296] bw ( KiB/s): min= 1560, max= 2023, per=32.08%, avg=1791.50, stdev=327.39, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2792KiB/s (2859kB/s)(10.0MiB/3667msec); 0 zone resets clat (msec): min=111, max=821, avg=322.29, stdev=217.73 lat (msec): min=111, max=821, avg=322.38, stdev=217.72 clat percentiles (msec): | 1.00th=[ 112], 5.00th=[ 112], 10.00th=[ 112], 20.00th=[ 171], | 30.00th=[ 213], 40.00th=[ 218], 50.00th=[ 253], 60.00th=[ 257], | 70.00th=[ 268], 80.00th=[ 317], 90.00th=[ 592], 95.00th=[ 818], | 99.00th=[ 818], 99.50th=[ 818], 99.90th=[ 818], 99.95th=[ 818], | 99.99th=[ 818] bw ( KiB/s): min= 2023, max= 5941, per=44.99%, avg=3517.50, stdev=1880.70, samples=4 iops : min= 1, max= 5, avg= 2.50, stdev= 1.91, samples=4 lat (msec) : 250=41.67%, 500=41.67%, 750=8.33%, 1000=8.33% cpu : usr=0.03%, sys=18.17%, ctx=304, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149072: Sun Nov 16 19:54:13 2025 read: IOPS=2, BW=2212KiB/s (2265kB/s)(6144KiB/2778msec) clat (usec): min=917, max=430708, avg=144577.00, stdev=193318.22 lat (usec): min=918, max=430708, avg=144578.51, stdev=193318.16 clat percentiles (usec): | 1.00th=[ 914], 5.00th=[ 914], 10.00th=[ 914], 20.00th=[ 10814], | 30.00th=[ 10814], 40.00th=[ 13173], 50.00th=[ 13173], 60.00th=[ 61604], | 70.00th=[350225], 80.00th=[350225], 90.00th=[429917], 95.00th=[429917], | 99.00th=[429917], 99.50th=[429917], 99.90th=[429917], 99.95th=[429917], | 99.99th=[429917] bw ( KiB/s): min= 2023, max= 6107, per=60.68%, avg=3388.33, stdev=2354.44, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 write: IOPS=2, BW=2212KiB/s (2265kB/s)(6144KiB/2778msec); 0 zone resets clat (msec): min=167, max=713, avg=317.53, stdev=214.34 lat (msec): min=167, max=713, avg=317.60, stdev=214.35 clat percentiles (msec): | 1.00th=[ 167], 5.00th=[ 167], 10.00th=[ 167], 20.00th=[ 201], | 30.00th=[ 201], 40.00th=[ 203], 50.00th=[ 203], 60.00th=[ 203], | 70.00th=[ 418], 80.00th=[ 418], 90.00th=[ 718], 95.00th=[ 718], | 99.00th=[ 718], 99.50th=[ 718], 99.90th=[ 718], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 3195, max= 4071, per=48.34%, avg=3779.00, stdev=505.76, samples=3 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=3 lat (usec) : 1000=8.33% lat (msec) : 20=16.67%, 100=8.33%, 250=33.33%, 500=25.00%, 750=8.33% cpu : usr=0.04%, sys=18.47%, ctx=126, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149073: Sun Nov 16 19:54:13 2025 read: IOPS=1, BW=1608KiB/s (1647kB/s)(5120KiB/3184msec) clat (usec): min=723, max=311648, avg=131668.65, stdev=133929.46 lat (usec): min=724, max=311650, avg=131669.99, stdev=133929.54 clat percentiles (usec): | 1.00th=[ 725], 5.00th=[ 725], 10.00th=[ 725], 20.00th=[ 725], | 30.00th=[ 26870], 40.00th=[ 26870], 50.00th=[ 89654], 60.00th=[ 89654], | 70.00th=[229639], 80.00th=[229639], 90.00th=[312476], 95.00th=[312476], | 99.00th=[312476], 99.50th=[312476], 99.90th=[312476], 99.95th=[312476], | 99.99th=[312476] bw ( KiB/s): min= 4047, max= 4071, per=72.69%, avg=4059.00, stdev=16.97, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2251KiB/s (2305kB/s)(7168KiB/3184msec); 0 zone resets clat (msec): min=141, max=672, avg=359.31, stdev=200.43 lat (msec): min=141, max=673, avg=359.41, stdev=200.44 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 165], | 30.00th=[ 230], 40.00th=[ 230], 50.00th=[ 355], 60.00th=[ 393], | 70.00th=[ 393], 80.00th=[ 558], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 1595, max= 6107, per=43.98%, avg=3438.25, stdev=2085.40, samples=4 iops : min= 1, max= 5, avg= 2.50, stdev= 1.91, samples=4 lat (usec) : 750=8.33% lat (msec) : 50=8.33%, 100=8.33%, 250=33.33%, 500=25.00%, 750=16.67% cpu : usr=0.03%, sys=18.50%, ctx=303, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5585KiB/s (5719kB/s), 558KiB/s-2620KiB/s (572kB/s-2683kB/s), io=20.0MiB (20.0MB), run=2736-3667msec WRITE: bw=7819KiB/s (8007kB/s), 1871KiB/s-2792KiB/s (1916kB/s-2859kB/s), io=28.0MiB (29.4MB), run=2736-3667msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=149080: Sun Nov 16 19:54:25 2025 read: IOPS=1, BW=5178KiB/s (5303kB/s)(12.0MiB/2373msec) clat (msec): min=595, max=944, avg=722.59, stdev=192.61 lat (msec): min=595, max=944, avg=722.60, stdev=192.61 clat percentiles (msec): | 1.00th=[ 592], 5.00th=[ 592], 10.00th=[ 592], 20.00th=[ 592], | 30.00th=[ 592], 40.00th=[ 625], 50.00th=[ 625], 60.00th=[ 625], | 70.00th=[ 944], 80.00th=[ 944], 90.00th=[ 944], 95.00th=[ 944], | 99.00th=[ 944], 99.50th=[ 944], 99.90th=[ 944], 99.95th=[ 944], | 99.99th=[ 944] bw ( KiB/s): min= 8126, max= 8192, per=100.00%, avg=8159.00, stdev=46.67, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 1000=33.33% cpu : usr=0.00%, sys=20.49%, ctx=598, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149082: Sun Nov 16 19:54:25 2025 read: IOPS=0, BW=1630KiB/s (1669kB/s)(4096KiB/2513msec) clat (nsec): min=940562k, max=940562k, avg=940561512.00, stdev= 0.00 lat (nsec): min=940564k, max=940564k, avg=940564101.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 944], 5.00th=[ 944], 10.00th=[ 944], 20.00th=[ 944], | 30.00th=[ 944], 40.00th=[ 944], 50.00th=[ 944], 60.00th=[ 944], | 70.00th=[ 944], 80.00th=[ 944], 90.00th=[ 944], 95.00th=[ 944], | 99.00th=[ 944], 99.50th=[ 944], 99.90th=[ 944], 99.95th=[ 944], | 99.99th=[ 944] bw ( KiB/s): min= 8078, max= 8078, per=100.00%, avg=8078.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3260KiB/s (3338kB/s)(8192KiB/2513msec); 0 zone resets clat (msec): min=422, max=911, avg=666.90, stdev=345.23 lat (msec): min=423, max=911, avg=667.18, stdev=345.31 clat percentiles (msec): | 1.00th=[ 422], 5.00th=[ 422], 10.00th=[ 422], 20.00th=[ 422], | 30.00th=[ 422], 40.00th=[ 422], 50.00th=[ 422], 60.00th=[ 911], | 70.00th=[ 911], 80.00th=[ 911], 90.00th=[ 911], 95.00th=[ 911], | 99.00th=[ 911], 99.50th=[ 911], 99.90th=[ 911], 99.95th=[ 911], | 99.99th=[ 911] bw ( KiB/s): min= 8192, max= 8192, per=70.65%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=66.67% cpu : usr=0.04%, sys=7.64%, ctx=257, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149083: Sun Nov 16 19:54:25 2025 write: IOPS=1, BW=4348KiB/s (4453kB/s)(12.0MiB/2826msec); 0 zone resets clat (msec): min=611, max=1118, avg=865.51, stdev=253.80 lat (msec): min=611, max=1121, avg=866.70, stdev=254.82 clat percentiles (msec): | 1.00th=[ 609], 5.00th=[ 609], 10.00th=[ 609], 20.00th=[ 609], | 30.00th=[ 609], 40.00th=[ 869], 50.00th=[ 869], 60.00th=[ 869], | 70.00th=[ 1116], 80.00th=[ 1116], 90.00th=[ 1116], 95.00th=[ 1116], | 99.00th=[ 1116], 99.50th=[ 1116], 99.90th=[ 1116], 99.95th=[ 1116], | 99.99th=[ 1116] bw ( KiB/s): min= 8078, max= 8192, per=70.16%, avg=8135.00, stdev=80.61, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.07%, sys=9.91%, ctx=59, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149084: Sun Nov 16 19:54:25 2025 write: IOPS=1, BW=5715KiB/s (5853kB/s)(12.0MiB/2150msec); 0 zone resets clat (msec): min=194, max=1480, avg=635.87, stdev=731.73 lat (msec): min=195, max=1480, avg=636.37, stdev=731.68 clat percentiles (msec): | 1.00th=[ 194], 5.00th=[ 194], 10.00th=[ 194], 20.00th=[ 194], | 30.00th=[ 194], 40.00th=[ 232], 50.00th=[ 232], 60.00th=[ 232], | 70.00th=[ 1485], 80.00th=[ 1485], 90.00th=[ 1485], 95.00th=[ 1485], | 99.00th=[ 1485], 99.50th=[ 1485], 99.90th=[ 1485], 99.95th=[ 1485], | 99.99th=[ 1485] bw ( KiB/s): min= 7787, max= 8175, per=68.83%, avg=7981.00, stdev=274.36, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=66.67%, 2000=33.33% cpu : usr=0.00%, sys=7.68%, ctx=90, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6520KiB/s (6676kB/s), 1630KiB/s-5178KiB/s (1669kB/s-5303kB/s), io=16.0MiB (16.8MB), run=2373-2513msec WRITE: bw=11.3MiB/s (11.9MB/s), 3260KiB/s-5715KiB/s (3338kB/s-5853kB/s), io=32.0MiB (33.6MB), run=2150-2826msec rand-rw: (groupid=0, jobs=1): err= 0: pid=149085: Sun Nov 16 19:54:25 2025 read: IOPS=1, BW=4339KiB/s (4443kB/s)(12.0MiB/2832msec) clat (msec): min=441, max=1286, avg=897.28, stdev=426.26 lat (msec): min=441, max=1286, avg=897.28, stdev=426.26 clat percentiles (msec): | 1.00th=[ 443], 5.00th=[ 443], 10.00th=[ 443], 20.00th=[ 443], | 30.00th=[ 443], 40.00th=[ 961], 50.00th=[ 961], 60.00th=[ 961], | 70.00th=[ 1284], 80.00th=[ 1284], 90.00th=[ 1284], 95.00th=[ 1284], | 99.00th=[ 1284], 99.50th=[ 1284], 99.90th=[ 1284], 99.95th=[ 1284], | 99.99th=[ 1284] bw ( KiB/s): min= 8126, max= 8175, per=100.00%, avg=8150.50, stdev=34.65, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=5.26%, ctx=68, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149086: Sun Nov 16 19:54:25 2025 read: IOPS=0, BW=1583KiB/s (1621kB/s)(4096KiB/2587msec) clat (nsec): min=1399.0M, max=1399.0M, avg=1399970505.00, stdev= 0.00 lat (nsec): min=1399.0M, max=1399.0M, avg=1399972455.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1401], 5.00th=[ 1401], 10.00th=[ 1401], 20.00th=[ 1401], | 30.00th=[ 1401], 40.00th=[ 1401], 50.00th=[ 1401], 60.00th=[ 1401], | 70.00th=[ 1401], 80.00th=[ 1401], 90.00th=[ 1401], 95.00th=[ 1401], | 99.00th=[ 1401], 99.50th=[ 1401], 99.90th=[ 1401], 99.95th=[ 1401], | 99.99th=[ 1401] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3167KiB/s (3243kB/s)(8192KiB/2587msec); 0 zone resets clat (msec): min=354, max=670, avg=512.41, stdev=223.17 lat (msec): min=354, max=670, avg=512.83, stdev=223.25 clat percentiles (msec): | 1.00th=[ 355], 5.00th=[ 355], 10.00th=[ 355], 20.00th=[ 355], | 30.00th=[ 355], 40.00th=[ 355], 50.00th=[ 355], 60.00th=[ 667], | 70.00th=[ 667], 80.00th=[ 667], 90.00th=[ 667], 95.00th=[ 667], | 99.00th=[ 667], 99.50th=[ 667], 99.90th=[ 667], 99.95th=[ 667], | 99.99th=[ 667] lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=5.92%, ctx=43, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149087: Sun Nov 16 19:54:25 2025 write: IOPS=1, BW=4496KiB/s (4604kB/s)(12.0MiB/2733msec); 0 zone resets clat (msec): min=710, max=1205, avg=877.07, stdev=284.76 lat (msec): min=711, max=1206, avg=877.36, stdev=284.91 clat percentiles (msec): | 1.00th=[ 709], 5.00th=[ 709], 10.00th=[ 709], 20.00th=[ 709], | 30.00th=[ 709], 40.00th=[ 718], 50.00th=[ 718], 60.00th=[ 718], | 70.00th=[ 1200], 80.00th=[ 1200], 90.00th=[ 1200], 95.00th=[ 1200], | 99.00th=[ 1200], 99.50th=[ 1200], 99.90th=[ 1200], 99.95th=[ 1200], | 99.99th=[ 1200] bw ( KiB/s): min= 8175, max= 8192, per=68.26%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.07%, sys=4.94%, ctx=47, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=149088: Sun Nov 16 19:54:25 2025 write: IOPS=1, BW=6041KiB/s (6186kB/s)(12.0MiB/2034msec); 0 zone resets clat (msec): min=143, max=1510, avg=661.68, stdev=741.02 lat (msec): min=143, max=1510, avg=662.05, stdev=741.04 clat percentiles (msec): | 1.00th=[ 144], 5.00th=[ 144], 10.00th=[ 144], 20.00th=[ 144], | 30.00th=[ 144], 40.00th=[ 330], 50.00th=[ 330], 60.00th=[ 330], | 70.00th=[ 1519], 80.00th=[ 1519], 90.00th=[ 1519], 95.00th=[ 1519], | 99.00th=[ 1519], 99.50th=[ 1519], 99.90th=[ 1519], 99.95th=[ 1519], | 99.99th=[ 1519] bw ( KiB/s): min=12603, max=12603, per=100.00%, avg=12603.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=6.20%, ctx=37, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5785KiB/s (5924kB/s), 1583KiB/s-4339KiB/s (1621kB/s-4443kB/s), io=16.0MiB (16.8MB), run=2587-2832msec WRITE: bw=11.7MiB/s (12.3MB/s), 3167KiB/s-6041KiB/s (3243kB/s-6186kB/s), io=32.0MiB (33.6MB), run=2034-2733msec