== sanity test 398b: DIO and buffer IO race ============== 23:38:15 (1764736695) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.0203 s, 16.7 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=194780: Tue Dec 2 23:39:50 2025 read: IOPS=17, BW=70.3KiB/s (71.0kB/s)(5992KiB/85261msec) clat (usec): min=111, max=769009, avg=31974.81, stdev=28665.09 lat (usec): min=111, max=769010, avg=31977.28, stdev=28663.60 clat percentiles (usec): | 1.00th=[ 151], 5.00th=[ 7177], 10.00th=[ 10421], 20.00th=[ 15139], | 30.00th=[ 19268], 40.00th=[ 22414], 50.00th=[ 26084], 60.00th=[ 31327], | 70.00th=[ 37487], 80.00th=[ 46400], 90.00th=[ 57934], 95.00th=[ 71828], | 99.00th=[105382], 99.50th=[120062], 99.90th=[160433], 99.95th=[767558], | 99.99th=[767558] bw ( KiB/s): min= 6, max= 168, per=24.97%, avg=69.92, stdev=30.32, samples=168 iops : min= 1, max= 42, avg=16.89, stdev= 7.65, samples=168 write: IOPS=18, BW=73.8KiB/s (75.6kB/s)(6296KiB/85261msec); 0 zone resets clat (usec): min=868, max=234821, avg=23554.42, stdev=25583.03 lat (usec): min=870, max=234822, avg=23556.58, stdev=25582.58 clat percentiles (usec): | 1.00th=[ 996], 5.00th=[ 1123], 10.00th=[ 1270], 20.00th=[ 2769], | 30.00th=[ 5604], 40.00th=[ 10421], 50.00th=[ 15664], 60.00th=[ 21627], | 70.00th=[ 29492], 80.00th=[ 39584], 90.00th=[ 56886], 95.00th=[ 73925], | 99.00th=[112722], 99.50th=[141558], 99.90th=[166724], 99.95th=[233833], | 99.99th=[233833] bw ( KiB/s): min= 15, max= 167, per=25.24%, avg=73.95, stdev=34.58, samples=167 iops : min= 3, max= 41, avg=17.89, stdev= 8.70, samples=167 lat (usec) : 250=1.46%, 500=0.39%, 750=0.23%, 1000=0.55% lat (msec) : 2=7.55%, 4=4.82%, 10=9.47%, 20=21.13%, 50=39.36% lat (msec) : 100=13.74%, 250=1.27%, 1000=0.03% cpu : usr=0.11%, sys=9.65%, ctx=6863, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194781: Tue Dec 2 23:39:50 2025 read: IOPS=17, BW=69.6KiB/s (71.3kB/s)(5940KiB/85366msec) clat (usec): min=90, max=177893, avg=32151.66, stdev=23058.83 lat (usec): min=90, max=177895, avg=32152.71, stdev=23058.85 clat percentiles (usec): | 1.00th=[ 141], 5.00th=[ 7570], 10.00th=[ 10814], 20.00th=[ 15008], | 30.00th=[ 18744], 40.00th=[ 22414], 50.00th=[ 26870], 60.00th=[ 31589], | 70.00th=[ 37487], 80.00th=[ 45351], 90.00th=[ 60556], 95.00th=[ 76022], | 99.00th=[113771], 99.50th=[141558], 99.90th=[177210], 99.95th=[177210], | 99.99th=[177210] bw ( KiB/s): min= 6, max= 159, per=24.69%, avg=69.14, stdev=29.11, samples=168 iops : min= 1, max= 39, avg=16.69, stdev= 7.33, samples=168 write: IOPS=18, BW=74.4KiB/s (76.1kB/s)(6348KiB/85366msec); 0 zone resets clat (usec): min=850, max=749907, avg=23548.67, stdev=30487.42 lat (usec): min=852, max=749909, avg=23550.92, stdev=30486.86 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1123], 10.00th=[ 1254], 20.00th=[ 2704], | 30.00th=[ 5473], 40.00th=[ 11207], 50.00th=[ 16450], 60.00th=[ 21890], | 70.00th=[ 28705], 80.00th=[ 38536], 90.00th=[ 55837], 95.00th=[ 69731], | 99.00th=[117965], 99.50th=[135267], 99.90th=[158335], 99.95th=[750781], | 99.99th=[750781] bw ( KiB/s): min= 15, max= 184, per=25.32%, avg=74.20, stdev=36.45, samples=167 iops : min= 3, max= 46, avg=17.96, stdev= 9.18, samples=167 lat (usec) : 100=0.03%, 250=1.40%, 500=0.29%, 750=0.10%, 1000=0.26% lat (msec) : 2=8.04%, 4=5.18%, 10=8.04%, 20=22.27%, 50=40.10% lat (msec) : 100=12.53%, 250=1.73%, 750=0.03% cpu : usr=0.10%, sys=9.80%, ctx=6964, majf=0, minf=20 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194782: Tue Dec 2 23:39:50 2025 read: IOPS=17, BW=70.0KiB/s (72.7kB/s)(6028KiB/84931msec) clat (usec): min=98, max=709148, avg=31671.89, stdev=28410.19 lat (usec): min=99, max=709149, avg=31673.29, stdev=28410.59 clat percentiles (usec): | 1.00th=[ 133], 5.00th=[ 4228], 10.00th=[ 10421], 20.00th=[ 14353], | 30.00th=[ 18220], 40.00th=[ 21365], 50.00th=[ 25297], 60.00th=[ 30278], | 70.00th=[ 36439], 80.00th=[ 46400], 90.00th=[ 61604], 95.00th=[ 76022], | 99.00th=[106431], 99.50th=[132645], 99.90th=[196084], 99.95th=[708838], | 99.99th=[708838] bw ( KiB/s): min= 7, max= 176, per=25.36%, avg=71.01, stdev=32.10, samples=166 iops : min= 1, max= 44, avg=17.16, stdev= 8.07, samples=166 write: IOPS=18, BW=73.7KiB/s (75.5kB/s)(6260KiB/84931msec); 0 zone resets clat (usec): min=882, max=244633, avg=23582.51, stdev=25783.99 lat (usec): min=883, max=244634, avg=23586.04, stdev=25783.01 clat percentiles (usec): | 1.00th=[ 996], 5.00th=[ 1123], 10.00th=[ 1254], 20.00th=[ 2606], | 30.00th=[ 5342], 40.00th=[ 10159], 50.00th=[ 15926], 60.00th=[ 21890], | 70.00th=[ 28967], 80.00th=[ 40633], 90.00th=[ 57934], 95.00th=[ 72877], | 99.00th=[109577], 99.50th=[141558], 99.90th=[200279], 99.95th=[244319], | 99.99th=[244319] bw ( KiB/s): min= 15, max= 159, per=24.97%, avg=73.17, stdev=32.45, samples=167 iops : min= 3, max= 39, avg=17.71, stdev= 8.16, samples=167 lat (usec) : 100=0.03%, 250=1.82%, 500=0.36%, 750=0.10%, 1000=0.55% lat (msec) : 2=8.37%, 4=4.62%, 10=8.89%, 20=21.94%, 50=37.96% lat (msec) : 100=13.93%, 250=1.40%, 750=0.03% cpu : usr=0.11%, sys=9.81%, ctx=7006, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194783: Tue Dec 2 23:39:50 2025 read: IOPS=17, BW=70.6KiB/s (72.3kB/s)(6052KiB/85695msec) clat (usec): min=105, max=275894, avg=31835.07, stdev=25135.25 lat (usec): min=105, max=275895, avg=31836.21, stdev=25135.25 clat percentiles (usec): | 1.00th=[ 141], 5.00th=[ 4817], 10.00th=[ 10028], 20.00th=[ 14091], | 30.00th=[ 17695], 40.00th=[ 21365], 50.00th=[ 25297], 60.00th=[ 30802], | 70.00th=[ 38011], 80.00th=[ 46924], 90.00th=[ 61080], 95.00th=[ 74974], | 99.00th=[119014], 99.50th=[156238], 99.90th=[246416], 99.95th=[274727], | 99.99th=[274727] bw ( KiB/s): min= 7, max= 160, per=25.03%, avg=70.10, stdev=30.85, samples=167 iops : min= 1, max= 40, avg=16.92, stdev= 7.76, samples=167 write: IOPS=18, BW=72.8KiB/s (74.5kB/s)(6236KiB/85695msec); 0 zone resets clat (usec): min=892, max=761585, avg=23900.46, stdev=31131.85 lat (usec): min=893, max=761586, avg=23904.61, stdev=31132.02 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 1123], 10.00th=[ 1237], 20.00th=[ 2606], | 30.00th=[ 5145], 40.00th=[ 10421], 50.00th=[ 16450], 60.00th=[ 22938], | 70.00th=[ 30540], 80.00th=[ 39060], 90.00th=[ 55837], 95.00th=[ 73925], | 99.00th=[107480], 99.50th=[135267], 99.90th=[179307], 99.95th=[759170], | 99.99th=[759170] bw ( KiB/s): min= 6, max= 175, per=24.58%, avg=72.03, stdev=35.18, samples=168 iops : min= 1, max= 43, avg=17.42, stdev= 8.85, samples=168 lat (usec) : 250=1.92%, 500=0.33%, 1000=0.55% lat (msec) : 2=7.94%, 4=5.37%, 10=8.59%, 20=21.35%, 50=38.74% lat (msec) : 100=13.41%, 250=1.73%, 500=0.03%, 1000=0.03% cpu : usr=0.10%, sys=9.54%, ctx=6848, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=280KiB/s (287kB/s), 69.6KiB/s-70.0KiB/s (71.3kB/s-72.7kB/s), io=23.4MiB (24.6MB), run=84931-85695msec WRITE: bw=293KiB/s (300kB/s), 72.8KiB/s-74.4KiB/s (74.5kB/s-76.1kB/s), io=24.6MiB (25.7MB), run=84931-85695msec rand-rw: (groupid=0, jobs=1): err= 0: pid=194785: Tue Dec 2 23:40:09 2025 read: IOPS=14, BW=57.5KiB/s (58.9kB/s)(5992KiB/104244msec) clat (msec): min=4, max=765, avg=27.07, stdev=34.23 lat (msec): min=4, max=765, avg=27.07, stdev=34.23 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 28], 80.00th=[ 40], 90.00th=[ 62], 95.00th=[ 88], | 99.00th=[ 142], 99.50th=[ 165], 99.90th=[ 262], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 7, max= 207, per=25.48%, avg=58.35, stdev=46.39, samples=202 iops : min= 1, max= 51, avg=13.99, stdev=11.65, samples=202 write: IOPS=15, BW=60.4KiB/s (61.8kB/s)(6296KiB/104244msec); 0 zone resets clat (msec): min=6, max=381, avg=40.36, stdev=35.84 lat (msec): min=6, max=381, avg=40.36, stdev=35.84 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 16], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 26], 60.00th=[ 33], | 70.00th=[ 45], 80.00th=[ 63], 90.00th=[ 87], 95.00th=[ 112], | 99.00th=[ 163], 99.50th=[ 182], 99.90th=[ 309], 99.95th=[ 384], | 99.99th=[ 384] bw ( KiB/s): min= 7, max= 192, per=25.16%, avg=60.38, stdev=46.78, samples=205 iops : min= 1, max= 48, avg=14.48, stdev=11.78, samples=205 lat (msec) : 10=14.81%, 20=33.69%, 50=30.66%, 100=15.36%, 250=5.31% lat (msec) : 500=0.13%, 1000=0.03% cpu : usr=0.09%, sys=3.87%, ctx=5030, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194786: Tue Dec 2 23:40:09 2025 read: IOPS=14, BW=56.8KiB/s (58.2kB/s)(5940KiB/104521msec) clat (msec): min=4, max=716, avg=27.61, stdev=34.65 lat (msec): min=4, max=716, avg=27.61, stdev=34.65 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 15], 60.00th=[ 20], | 70.00th=[ 28], 80.00th=[ 42], 90.00th=[ 65], 95.00th=[ 88], | 99.00th=[ 136], 99.50th=[ 174], 99.90th=[ 296], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 7, max= 208, per=24.67%, avg=56.49, stdev=47.44, samples=205 iops : min= 1, max= 52, avg=13.52, stdev=11.92, samples=205 write: IOPS=15, BW=60.7KiB/s (62.2kB/s)(6348KiB/104521msec); 0 zone resets clat (msec): min=5, max=293, avg=39.94, stdev=34.96 lat (msec): min=5, max=293, avg=39.94, stdev=34.96 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 12], 10.00th=[ 13], 20.00th=[ 16], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 25], 60.00th=[ 32], | 70.00th=[ 46], 80.00th=[ 66], 90.00th=[ 91], 95.00th=[ 109], | 99.00th=[ 163], 99.50th=[ 180], 99.90th=[ 257], 99.95th=[ 292], | 99.99th=[ 292] bw ( KiB/s): min= 7, max= 183, per=25.09%, avg=60.21, stdev=47.19, samples=205 iops : min= 1, max= 45, avg=14.44, stdev=11.87, samples=205 lat (msec) : 10=15.23%, 20=35.03%, 50=28.06%, 100=16.34%, 250=5.14% lat (msec) : 500=0.16%, 750=0.03% cpu : usr=0.09%, sys=3.89%, ctx=5061, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194787: Tue Dec 2 23:40:09 2025 read: IOPS=14, BW=57.8KiB/s (59.2kB/s)(6028KiB/104270msec) clat (msec): min=4, max=214, avg=26.86, stdev=28.49 lat (msec): min=4, max=214, avg=26.86, stdev=28.49 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 20], | 70.00th=[ 28], 80.00th=[ 41], 90.00th=[ 64], 95.00th=[ 87], | 99.00th=[ 142], 99.50th=[ 163], 99.90th=[ 209], 99.95th=[ 215], | 99.99th=[ 215] bw ( KiB/s): min= 7, max= 214, per=25.61%, avg=58.65, stdev=47.59, samples=202 iops : min= 1, max= 53, avg=14.06, stdev=11.97, samples=202 write: IOPS=15, BW=60.0KiB/s (61.5kB/s)(6260KiB/104270msec); 0 zone resets clat (msec): min=6, max=720, avg=40.65, stdev=39.98 lat (msec): min=6, max=720, avg=40.65, stdev=39.98 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 12], 10.00th=[ 13], 20.00th=[ 15], | 30.00th=[ 18], 40.00th=[ 20], 50.00th=[ 25], 60.00th=[ 33], | 70.00th=[ 49], 80.00th=[ 64], 90.00th=[ 87], 95.00th=[ 110], | 99.00th=[ 165], 99.50th=[ 209], 99.90th=[ 380], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 7, max= 192, per=24.99%, avg=59.98, stdev=48.48, samples=205 iops : min= 1, max= 48, avg=14.39, stdev=12.19, samples=205 lat (msec) : 10=15.10%, 20=34.90%, 50=27.80%, 100=17.64%, 250=4.36% lat (msec) : 500=0.16%, 750=0.03% cpu : usr=0.07%, sys=3.73%, ctx=4942, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194788: Tue Dec 2 23:40:09 2025 read: IOPS=14, BW=57.9KiB/s (59.3kB/s)(6052KiB/104551msec) clat (msec): min=3, max=288, avg=27.71, stdev=31.37 lat (msec): min=3, max=288, avg=27.71, stdev=31.37 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 21], | 70.00th=[ 28], 80.00th=[ 42], 90.00th=[ 64], 95.00th=[ 87], | 99.00th=[ 165], 99.50th=[ 199], 99.90th=[ 271], 99.95th=[ 288], | 99.99th=[ 288] bw ( KiB/s): min= 7, max= 216, per=25.67%, avg=58.78, stdev=46.82, samples=200 iops : min= 1, max= 54, avg=14.09, stdev=11.77, samples=200 write: IOPS=14, BW=59.6KiB/s (61.1kB/s)(6236KiB/104551msec); 0 zone resets clat (msec): min=7, max=708, avg=40.05, stdev=39.44 lat (msec): min=7, max=708, avg=40.05, stdev=39.44 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 13], 20.00th=[ 15], | 30.00th=[ 18], 40.00th=[ 20], 50.00th=[ 25], 60.00th=[ 33], | 70.00th=[ 46], 80.00th=[ 64], 90.00th=[ 87], 95.00th=[ 108], | 99.00th=[ 171], 99.50th=[ 215], 99.90th=[ 305], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 7, max= 199, per=24.80%, avg=59.52, stdev=47.41, samples=204 iops : min= 1, max= 49, avg=14.27, stdev=11.92, samples=204 lat (msec) : 4=0.03%, 10=16.11%, 20=34.18%, 50=28.16%, 100=16.54% lat (msec) : 250=4.79%, 500=0.16%, 750=0.03% cpu : usr=0.09%, sys=3.71%, ctx=5047, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=230KiB/s (235kB/s), 56.8KiB/s-57.9KiB/s (58.2kB/s-59.3kB/s), io=23.4MiB (24.6MB), run=104244-104551msec WRITE: bw=240KiB/s (246kB/s), 59.6KiB/s-60.7KiB/s (61.1kB/s-62.2kB/s), io=24.6MiB (25.7MB), run=104244-104551msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=194816: Tue Dec 2 23:40:32 2025 read: IOPS=21, BW=348KiB/s (356kB/s)(5920KiB/17014msec) clat (usec): min=125, max=74578, avg=23078.97, stdev=12699.04 lat (usec): min=126, max=74579, avg=23080.04, stdev=12699.05 clat percentiles (usec): | 1.00th=[ 163], 5.00th=[ 408], 10.00th=[10290], 20.00th=[14222], | 30.00th=[16057], 40.00th=[19006], 50.00th=[21365], 60.00th=[24511], | 70.00th=[27657], 80.00th=[31065], 90.00th=[37487], 95.00th=[45351], | 99.00th=[68682], 99.50th=[72877], 99.90th=[74974], 99.95th=[74974], | 99.99th=[74974] bw ( KiB/s): min= 191, max= 508, per=24.55%, avg=345.42, stdev=85.61, samples=33 iops : min= 11, max= 31, avg=20.88, stdev= 5.37, samples=33 write: IOPS=23, BW=374KiB/s (383kB/s)(6368KiB/17014msec); 0 zone resets clat (usec): min=1605, max=136917, avg=20898.74, stdev=18283.95 lat (usec): min=1607, max=136919, avg=20900.74, stdev=18283.99 clat percentiles (usec): | 1.00th=[ 1795], 5.00th=[ 2442], 10.00th=[ 3130], 20.00th=[ 5014], | 30.00th=[ 8160], 40.00th=[ 12518], 50.00th=[ 16057], 60.00th=[ 19792], | 70.00th=[ 27919], 80.00th=[ 34341], 90.00th=[ 44827], 95.00th=[ 53216], | 99.00th=[ 77071], 99.50th=[127402], 99.90th=[137364], 99.95th=[137364], | 99.99th=[137364] bw ( KiB/s): min= 166, max= 766, per=25.44%, avg=376.79, stdev=122.61, samples=33 iops : min= 10, max= 47, avg=22.85, stdev= 7.60, samples=33 lat (usec) : 250=1.43%, 500=1.56%, 1000=0.13% lat (msec) : 2=1.30%, 4=6.51%, 10=11.07%, 20=30.47%, 50=42.32% lat (msec) : 100=4.95%, 250=0.26% cpu : usr=0.13%, sys=17.02%, ctx=2123, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194817: Tue Dec 2 23:40:32 2025 read: IOPS=21, BW=347KiB/s (356kB/s)(5744KiB/16543msec) clat (usec): min=138, max=102738, avg=23699.67, stdev=13842.69 lat (usec): min=139, max=102739, avg=23700.74, stdev=13842.72 clat percentiles (usec): | 1.00th=[ 163], 5.00th=[ 359], 10.00th=[ 10159], 20.00th=[ 14222], | 30.00th=[ 16581], 40.00th=[ 19268], 50.00th=[ 21627], 60.00th=[ 24249], | 70.00th=[ 28181], 80.00th=[ 31851], 90.00th=[ 40633], 95.00th=[ 50070], | 99.00th=[ 73925], 99.50th=[ 80217], 99.90th=[102237], 99.95th=[102237], | 99.99th=[102237] bw ( KiB/s): min= 159, max= 510, per=24.83%, avg=349.34, stdev=88.72, samples=32 iops : min= 9, max= 31, avg=21.12, stdev= 5.53, samples=32 write: IOPS=24, BW=396KiB/s (405kB/s)(6544KiB/16543msec); 0 zone resets clat (usec): min=1698, max=128213, avg=19540.50, stdev=16710.47 lat (usec): min=1701, max=128214, avg=19542.39, stdev=16710.44 clat percentiles (usec): | 1.00th=[ 1795], 5.00th=[ 2278], 10.00th=[ 3425], 20.00th=[ 5145], | 30.00th=[ 7963], 40.00th=[ 12387], 50.00th=[ 15926], 60.00th=[ 19006], | 70.00th=[ 23200], 80.00th=[ 31065], 90.00th=[ 40109], 95.00th=[ 51119], | 99.00th=[ 76022], 99.50th=[ 80217], 99.90th=[128451], 99.95th=[128451], | 99.99th=[128451] bw ( KiB/s): min= 158, max= 541, per=26.55%, avg=393.22, stdev=103.63, samples=32 iops : min= 9, max= 33, avg=23.87, stdev= 6.55, samples=32 lat (usec) : 250=1.43%, 500=1.56%, 1000=0.13% lat (msec) : 2=1.69%, 4=5.86%, 10=11.59%, 20=30.86%, 50=41.80% lat (msec) : 100=4.69%, 250=0.39% cpu : usr=0.13%, sys=17.34%, ctx=2156, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194820: Tue Dec 2 23:40:32 2025 read: IOPS=23, BW=380KiB/s (389kB/s)(6256KiB/16475msec) clat (usec): min=148, max=112715, avg=22876.89, stdev=14366.91 lat (usec): min=150, max=112716, avg=22878.02, stdev=14366.90 clat percentiles (usec): | 1.00th=[ 167], 5.00th=[ 322], 10.00th=[ 8717], 20.00th=[ 13698], | 30.00th=[ 15926], 40.00th=[ 17695], 50.00th=[ 19792], 60.00th=[ 23200], | 70.00th=[ 27395], 80.00th=[ 31851], 90.00th=[ 40633], 95.00th=[ 48497], | 99.00th=[ 78119], 99.50th=[ 81265], 99.90th=[112722], 99.95th=[112722], | 99.99th=[112722] bw ( KiB/s): min= 221, max= 604, per=27.17%, avg=382.25, stdev=89.60, samples=32 iops : min= 13, max= 37, avg=23.19, stdev= 5.64, samples=32 write: IOPS=22, BW=366KiB/s (375kB/s)(6032KiB/16475msec); 0 zone resets clat (usec): min=1558, max=95389, avg=19540.97, stdev=16311.16 lat (usec): min=1559, max=95394, avg=19543.12, stdev=16311.02 clat percentiles (usec): | 1.00th=[ 1991], 5.00th=[ 2606], 10.00th=[ 3195], 20.00th=[ 5342], | 30.00th=[ 9110], 40.00th=[12911], 50.00th=[15533], 60.00th=[18482], | 70.00th=[24249], 80.00th=[31065], 90.00th=[41157], 95.00th=[49546], | 99.00th=[84411], 99.50th=[93848], 99.90th=[94897], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 190, max= 608, per=24.59%, avg=364.16, stdev=113.05, samples=32 iops : min= 11, max= 38, avg=22.06, stdev= 7.10, samples=32 lat (usec) : 250=1.69%, 500=1.82%, 750=0.39% lat (msec) : 2=1.04%, 4=6.25%, 10=10.16%, 20=35.42%, 50=38.80% lat (msec) : 100=4.30%, 250=0.13% cpu : usr=0.12%, sys=17.21%, ctx=2066, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194822: Tue Dec 2 23:40:32 2025 read: IOPS=22, BW=358KiB/s (366kB/s)(6032KiB/16858msec) clat (usec): min=144, max=97028, avg=23986.31, stdev=15254.99 lat (usec): min=145, max=97030, avg=23987.35, stdev=15255.00 clat percentiles (usec): | 1.00th=[ 161], 5.00th=[ 379], 10.00th=[ 9110], 20.00th=[13173], | 30.00th=[16188], 40.00th=[18744], 50.00th=[21627], 60.00th=[24511], | 70.00th=[28705], 80.00th=[33424], 90.00th=[41681], 95.00th=[49546], | 99.00th=[80217], 99.50th=[93848], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 159, max= 542, per=25.02%, avg=352.06, stdev=78.47, samples=33 iops : min= 9, max= 33, avg=21.30, stdev= 4.90, samples=33 write: IOPS=23, BW=371KiB/s (380kB/s)(6256KiB/16858msec); 0 zone resets clat (usec): min=1530, max=117831, avg=19532.59, stdev=16255.72 lat (usec): min=1532, max=117833, avg=19534.59, stdev=16255.78 clat percentiles (usec): | 1.00th=[ 1713], 5.00th=[ 2966], 10.00th=[ 4047], 20.00th=[ 6194], | 30.00th=[ 9503], 40.00th=[ 13829], 50.00th=[ 15795], 60.00th=[ 19006], | 70.00th=[ 22676], 80.00th=[ 28967], 90.00th=[ 37487], 95.00th=[ 49546], | 99.00th=[ 87557], 99.50th=[ 96994], 99.90th=[117965], 99.95th=[117965], | 99.99th=[117965] bw ( KiB/s): min= 191, max= 606, per=24.81%, avg=367.39, stdev=104.29, samples=33 iops : min= 11, max= 37, avg=22.27, stdev= 6.55, samples=33 lat (usec) : 250=1.17%, 500=1.69%, 750=0.39%, 1000=0.26% lat (msec) : 2=1.30%, 4=4.04%, 10=12.24%, 20=33.72%, 50=40.49% lat (msec) : 100=4.56%, 250=0.13% cpu : usr=0.10%, sys=17.19%, ctx=2104, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1408KiB/s (1442kB/s), 347KiB/s-380KiB/s (356kB/s-389kB/s), io=23.4MiB (24.5MB), run=16475-17014msec WRITE: bw=1481KiB/s (1517kB/s), 366KiB/s-396KiB/s (375kB/s-405kB/s), io=24.6MiB (25.8MB), run=16475-17014msec rand-rw: (groupid=0, jobs=1): err= 0: pid=194818: Tue Dec 2 23:40:37 2025 read: IOPS=16, BW=265KiB/s (271kB/s)(5920KiB/22380msec) clat (msec): min=4, max=129, avg=24.52, stdev=17.87 lat (msec): min=4, max=129, avg=24.52, stdev=17.87 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 33], 90.00th=[ 48], 95.00th=[ 65], | 99.00th=[ 88], 99.50th=[ 102], 99.90th=[ 130], 99.95th=[ 130], | 99.99th=[ 130] bw ( KiB/s): min= 95, max= 638, per=24.32%, avg=260.18, stdev=123.38, samples=44 iops : min= 5, max= 39, avg=15.68, stdev= 7.65, samples=44 write: IOPS=17, BW=285KiB/s (291kB/s)(6368KiB/22380msec); 0 zone resets clat (msec): min=7, max=163, avg=33.25, stdev=20.31 lat (msec): min=7, max=163, avg=33.25, stdev=20.31 clat percentiles (msec): | 1.00th=[ 11], 5.00th=[ 14], 10.00th=[ 16], 20.00th=[ 19], | 30.00th=[ 22], 40.00th=[ 25], 50.00th=[ 28], 60.00th=[ 32], | 70.00th=[ 37], 80.00th=[ 45], 90.00th=[ 60], 95.00th=[ 78], | 99.00th=[ 108], 99.50th=[ 138], 99.90th=[ 163], 99.95th=[ 163], | 99.99th=[ 163] bw ( KiB/s): min= 63, max= 540, per=25.12%, avg=282.89, stdev=113.46, samples=44 iops : min= 3, max= 33, avg=17.11, stdev= 7.10, samples=44 lat (msec) : 10=6.38%, 20=32.55%, 50=49.09%, 100=10.94%, 250=1.04% cpu : usr=0.09%, sys=6.14%, ctx=1315, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194819: Tue Dec 2 23:40:37 2025 read: IOPS=16, BW=259KiB/s (265kB/s)(5744KiB/22183msec) clat (usec): min=5014, max=94414, avg=23541.03, stdev=18084.61 lat (usec): min=5015, max=94415, avg=23542.37, stdev=18084.58 clat percentiles (usec): | 1.00th=[ 5538], 5.00th=[ 7701], 10.00th=[ 8586], 20.00th=[10552], | 30.00th=[12256], 40.00th=[13829], 50.00th=[16450], 60.00th=[20055], | 70.00th=[25297], 80.00th=[35390], 90.00th=[53216], 95.00th=[67634], | 99.00th=[87557], 99.50th=[91751], 99.90th=[94897], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 57, max= 662, per=24.25%, avg=259.50, stdev=135.33, samples=44 iops : min= 3, max= 41, avg=15.64, stdev= 8.51, samples=44 write: IOPS=18, BW=295KiB/s (302kB/s)(6544KiB/22183msec); 0 zone resets clat (msec): min=7, max=148, avg=33.13, stdev=20.77 lat (msec): min=7, max=148, avg=33.13, stdev=20.77 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 13], 10.00th=[ 14], 20.00th=[ 17], | 30.00th=[ 21], 40.00th=[ 23], 50.00th=[ 27], 60.00th=[ 31], | 70.00th=[ 40], 80.00th=[ 48], 90.00th=[ 61], 95.00th=[ 73], | 99.00th=[ 102], 99.50th=[ 122], 99.90th=[ 148], 99.95th=[ 148], | 99.99th=[ 148] bw ( KiB/s): min= 126, max= 601, per=26.14%, avg=294.36, stdev=124.14, samples=44 iops : min= 7, max= 37, avg=17.84, stdev= 7.75, samples=44 lat (msec) : 10=8.59%, 20=34.90%, 50=42.58%, 100=13.02%, 250=0.91% cpu : usr=0.10%, sys=6.08%, ctx=1353, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194821: Tue Dec 2 23:40:37 2025 read: IOPS=17, BW=281KiB/s (288kB/s)(6256KiB/22246msec) clat (msec): min=5, max=133, avg=23.50, stdev=18.03 lat (msec): min=5, max=133, avg=23.50, stdev=18.03 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 17], 60.00th=[ 21], | 70.00th=[ 26], 80.00th=[ 34], 90.00th=[ 45], 95.00th=[ 61], | 99.00th=[ 95], 99.50th=[ 102], 99.90th=[ 134], 99.95th=[ 134], | 99.99th=[ 134] bw ( KiB/s): min= 95, max= 734, per=26.31%, avg=281.48, stdev=140.80, samples=44 iops : min= 5, max= 45, avg=17.02, stdev= 8.74, samples=44 write: IOPS=16, BW=271KiB/s (278kB/s)(6032KiB/22246msec); 0 zone resets clat (msec): min=9, max=142, avg=34.17, stdev=20.31 lat (msec): min=9, max=142, avg=34.18, stdev=20.31 clat percentiles (msec): | 1.00th=[ 12], 5.00th=[ 13], 10.00th=[ 15], 20.00th=[ 18], | 30.00th=[ 21], 40.00th=[ 25], 50.00th=[ 30], 60.00th=[ 34], | 70.00th=[ 40], 80.00th=[ 48], 90.00th=[ 61], 95.00th=[ 73], | 99.00th=[ 103], 99.50th=[ 136], 99.90th=[ 144], 99.95th=[ 144], | 99.99th=[ 144] bw ( KiB/s): min= 93, max= 608, per=23.92%, avg=269.32, stdev=118.91, samples=44 iops : min= 5, max= 38, avg=16.25, stdev= 7.44, samples=44 lat (msec) : 10=8.33%, 20=34.51%, 50=44.14%, 100=12.11%, 250=0.91% cpu : usr=0.07%, sys=6.11%, ctx=1375, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194823: Tue Dec 2 23:40:37 2025 read: IOPS=16, BW=270KiB/s (277kB/s)(6032KiB/22303msec) clat (msec): min=5, max=127, avg=23.33, stdev=17.88 lat (msec): min=5, max=127, avg=23.33, stdev=17.89 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 12], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 21], | 70.00th=[ 26], 80.00th=[ 33], 90.00th=[ 52], 95.00th=[ 64], | 99.00th=[ 87], 99.50th=[ 91], 99.90th=[ 128], 99.95th=[ 128], | 99.99th=[ 128] bw ( KiB/s): min= 93, max= 608, per=25.03%, avg=267.86, stdev=133.14, samples=44 iops : min= 5, max= 38, avg=16.16, stdev= 8.30, samples=44 write: IOPS=17, BW=281KiB/s (287kB/s)(6256KiB/22303msec); 0 zone resets clat (msec): min=10, max=163, avg=34.24, stdev=21.63 lat (msec): min=10, max=163, avg=34.25, stdev=21.63 clat percentiles (msec): | 1.00th=[ 12], 5.00th=[ 14], 10.00th=[ 16], 20.00th=[ 19], | 30.00th=[ 21], 40.00th=[ 24], 50.00th=[ 28], 60.00th=[ 32], | 70.00th=[ 39], 80.00th=[ 49], 90.00th=[ 62], 95.00th=[ 77], | 99.00th=[ 120], 99.50th=[ 144], 99.90th=[ 165], 99.95th=[ 165], | 99.99th=[ 165] bw ( KiB/s): min= 94, max= 670, per=24.88%, avg=280.14, stdev=130.31, samples=44 iops : min= 5, max= 41, avg=16.93, stdev= 8.15, samples=44 lat (msec) : 10=9.11%, 20=33.72%, 50=42.45%, 100=13.54%, 250=1.17% cpu : usr=0.10%, sys=6.07%, ctx=1393, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1070KiB/s (1096kB/s), 259KiB/s-281KiB/s (265kB/s-288kB/s), io=23.4MiB (24.5MB), run=22183-22380msec WRITE: bw=1126KiB/s (1153kB/s), 271KiB/s-295KiB/s (278kB/s-302kB/s), io=24.6MiB (25.8MB), run=22183-22380msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=194833: Tue Dec 2 23:40:48 2025 read: IOPS=3, BW=3476KiB/s (3560kB/s)(7168KiB/2062msec) clat (msec): min=60, max=326, avg=149.19, stdev=120.39 lat (msec): min=60, max=326, avg=149.19, stdev=120.39 clat percentiles (msec): | 1.00th=[ 61], 5.00th=[ 61], 10.00th=[ 61], 20.00th=[ 72], | 30.00th=[ 75], 40.00th=[ 75], 50.00th=[ 80], 60.00th=[ 110], | 70.00th=[ 110], 80.00th=[ 321], 90.00th=[ 326], 95.00th=[ 326], | 99.00th=[ 326], 99.50th=[ 326], 99.90th=[ 326], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 2011, max= 5525, per=65.49%, avg=3194.67, stdev=2018.21, samples=3 iops : min= 1, max= 5, avg= 2.67, stdev= 2.08, samples=3 write: IOPS=2, BW=2483KiB/s (2543kB/s)(5120KiB/2062msec); 0 zone resets clat (msec): min=122, max=272, avg=194.45, stdev=61.88 lat (msec): min=122, max=272, avg=194.54, stdev=61.87 clat percentiles (msec): | 1.00th=[ 123], 5.00th=[ 123], 10.00th=[ 123], 20.00th=[ 123], | 30.00th=[ 142], 40.00th=[ 142], 50.00th=[ 211], 60.00th=[ 211], | 70.00th=[ 226], 80.00th=[ 226], 90.00th=[ 271], 95.00th=[ 271], | 99.00th=[ 271], 99.50th=[ 271], 99.90th=[ 271], 99.95th=[ 271], | 99.99th=[ 271] bw ( KiB/s): min= 2011, max= 2048, per=29.72%, avg=2029.50, stdev=26.16, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 100=33.33%, 250=41.67%, 500=25.00% cpu : usr=0.00%, sys=14.17%, ctx=81, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194834: Tue Dec 2 23:40:48 2025 read: IOPS=0, BW=494KiB/s (506kB/s)(2048KiB/4143msec) clat (msec): min=79, max=242, avg=160.64, stdev=115.28 lat (msec): min=79, max=242, avg=160.64, stdev=115.28 clat percentiles (msec): | 1.00th=[ 80], 5.00th=[ 80], 10.00th=[ 80], 20.00th=[ 80], | 30.00th=[ 80], 40.00th=[ 80], 50.00th=[ 80], 60.00th=[ 243], | 70.00th=[ 243], 80.00th=[ 243], 90.00th=[ 243], 95.00th=[ 243], | 99.00th=[ 243], 99.50th=[ 243], 99.90th=[ 243], 99.95th=[ 243], | 99.99th=[ 243] bw ( KiB/s): min= 1747, max= 2039, per=38.81%, avg=1893.00, stdev=206.48, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2472KiB/s (2531kB/s)(10.0MiB/4143msec); 0 zone resets clat (msec): min=41, max=1212, avg=380.33, stdev=348.43 lat (msec): min=41, max=1212, avg=380.43, stdev=348.44 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 42], 10.00th=[ 42], 20.00th=[ 101], | 30.00th=[ 114], 40.00th=[ 114], 50.00th=[ 347], 60.00th=[ 376], | 70.00th=[ 409], 80.00th=[ 485], 90.00th=[ 609], 95.00th=[ 1217], | 99.00th=[ 1217], 99.50th=[ 1217], 99.90th=[ 1217], 99.95th=[ 1217], | 99.99th=[ 1217] bw ( KiB/s): min= 2011, max= 6095, per=45.94%, avg=3137.40, stdev=1770.35, samples=5 iops : min= 1, max= 5, avg= 2.40, stdev= 1.67, samples=5 lat (msec) : 50=8.33%, 100=8.33%, 250=33.33%, 500=33.33%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=11.13%, ctx=106, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194835: Tue Dec 2 23:40:48 2025 read: IOPS=1, BW=1464KiB/s (1499kB/s)(6144KiB/4198msec) clat (msec): min=76, max=496, avg=273.77, stdev=172.97 lat (msec): min=76, max=496, avg=273.77, stdev=172.97 clat percentiles (msec): | 1.00th=[ 78], 5.00th=[ 78], 10.00th=[ 78], 20.00th=[ 127], | 30.00th=[ 127], 40.00th=[ 163], 50.00th=[ 163], 60.00th=[ 384], | 70.00th=[ 397], 80.00th=[ 397], 90.00th=[ 498], 95.00th=[ 498], | 99.00th=[ 498], 99.50th=[ 498], 99.90th=[ 498], 99.95th=[ 498], | 99.99th=[ 498] bw ( KiB/s): min= 2011, max= 4063, per=52.10%, avg=2541.25, stdev=1014.63, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 write: IOPS=1, BW=1464KiB/s (1499kB/s)(6144KiB/4198msec); 0 zone resets clat (msec): min=80, max=1097, avg=408.96, stdev=376.10 lat (msec): min=80, max=1097, avg=409.06, stdev=376.11 clat percentiles (msec): | 1.00th=[ 81], 5.00th=[ 81], 10.00th=[ 81], 20.00th=[ 211], | 30.00th=[ 211], 40.00th=[ 239], 50.00th=[ 239], 60.00th=[ 245], | 70.00th=[ 584], 80.00th=[ 584], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 1903, max= 3567, per=33.97%, avg=2320.00, stdev=699.74, samples=5 iops : min= 1, max= 3, avg= 1.60, stdev= 0.89, samples=5 lat (msec) : 100=16.67%, 250=41.67%, 500=25.00%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=8.15%, ctx=123, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194836: Tue Dec 2 23:40:48 2025 read: IOPS=1, BW=1621KiB/s (1660kB/s)(5120KiB/3158msec) clat (msec): min=84, max=172, avg=130.09, stdev=33.41 lat (msec): min=84, max=172, avg=130.09, stdev=33.41 clat percentiles (msec): | 1.00th=[ 85], 5.00th=[ 85], 10.00th=[ 85], 20.00th=[ 85], | 30.00th=[ 118], 40.00th=[ 118], 50.00th=[ 126], 60.00th=[ 126], | 70.00th=[ 150], 80.00th=[ 150], 90.00th=[ 174], 95.00th=[ 174], | 99.00th=[ 174], 99.50th=[ 174], 99.90th=[ 174], 99.95th=[ 174], | 99.99th=[ 174] bw ( KiB/s): min= 8047, max= 8047, per=100.00%, avg=8047.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=2, BW=2270KiB/s (2324kB/s)(7168KiB/3158msec); 0 zone resets clat (msec): min=77, max=1365, avg=350.24, stdev=461.05 lat (msec): min=77, max=1365, avg=350.34, stdev=461.07 clat percentiles (msec): | 1.00th=[ 78], 5.00th=[ 78], 10.00th=[ 78], 20.00th=[ 87], | 30.00th=[ 100], 40.00th=[ 100], 50.00th=[ 199], 60.00th=[ 236], | 70.00th=[ 236], 80.00th=[ 393], 90.00th=[ 1368], 95.00th=[ 1368], | 99.00th=[ 1368], 99.50th=[ 1368], 99.90th=[ 1368], 99.95th=[ 1368], | 99.99th=[ 1368] bw ( KiB/s): min= 3690, max= 6144, per=67.99%, avg=4643.33, stdev=1315.37, samples=3 iops : min= 3, max= 6, avg= 4.33, stdev= 1.53, samples=3 lat (msec) : 100=33.33%, 250=50.00%, 500=8.33%, 2000=8.33% cpu : usr=0.00%, sys=7.06%, ctx=57, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4879KiB/s (4996kB/s), 494KiB/s-3476KiB/s (506kB/s-3560kB/s), io=20.0MiB (20.0MB), run=2062-4198msec WRITE: bw=6830KiB/s (6994kB/s), 1464KiB/s-2483KiB/s (1499kB/s-2543kB/s), io=28.0MiB (29.4MB), run=2062-4198msec rand-rw: (groupid=0, jobs=1): err= 0: pid=194838: Tue Dec 2 23:40:48 2025 read: IOPS=1, BW=1732KiB/s (1773kB/s)(7168KiB/4139msec) clat (usec): min=852, max=2015.6k, avg=421348.06, stdev=710212.83 lat (usec): min=853, max=2015.6k, avg=421349.65, stdev=710212.66 clat percentiles (usec): | 1.00th=[ 857], 5.00th=[ 857], 10.00th=[ 857], | 20.00th=[ 129500], 30.00th=[ 137364], 40.00th=[ 137364], | 50.00th=[ 143655], 60.00th=[ 177210], 70.00th=[ 177210], | 80.00th=[ 346031], 90.00th=[2021655], 95.00th=[2021655], | 99.00th=[2021655], 99.50th=[2021655], 99.90th=[2021655], | 99.95th=[2021655], 99.99th=[2021655] bw ( KiB/s): min= 1777, max= 6144, per=60.65%, avg=3000.75, stdev=2099.19, samples=4 iops : min= 1, max= 6, avg= 2.25, stdev= 2.50, samples=4 write: IOPS=1, BW=1237KiB/s (1267kB/s)(5120KiB/4139msec); 0 zone resets clat (msec): min=89, max=307, avg=199.05, stdev=89.69 lat (msec): min=89, max=307, avg=199.12, stdev=89.68 clat percentiles (msec): | 1.00th=[ 90], 5.00th=[ 90], 10.00th=[ 90], 20.00th=[ 90], | 30.00th=[ 133], 40.00th=[ 133], 50.00th=[ 205], 60.00th=[ 205], | 70.00th=[ 264], 80.00th=[ 264], 90.00th=[ 309], 95.00th=[ 309], | 99.00th=[ 309], 99.50th=[ 309], 99.90th=[ 309], 99.95th=[ 309], | 99.99th=[ 309] bw ( KiB/s): min= 2048, max= 4087, per=49.19%, avg=3407.33, stdev=1177.22, samples=3 iops : min= 2, max= 3, avg= 2.67, stdev= 0.58, samples=3 lat (usec) : 1000=8.33% lat (msec) : 100=8.33%, 250=50.00%, 500=25.00%, >=2000=8.33% cpu : usr=0.00%, sys=21.53%, ctx=767, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194839: Tue Dec 2 23:40:48 2025 read: IOPS=0, BW=551KiB/s (564kB/s)(2048KiB/3718msec) clat (msec): min=16, max=141, avg=79.04, stdev=88.98 lat (msec): min=16, max=141, avg=79.05, stdev=88.98 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 17], 10.00th=[ 17], 20.00th=[ 17], | 30.00th=[ 17], 40.00th=[ 17], 50.00th=[ 17], 60.00th=[ 142], | 70.00th=[ 142], 80.00th=[ 142], 90.00th=[ 142], 95.00th=[ 142], | 99.00th=[ 142], 99.50th=[ 142], 99.90th=[ 142], 99.95th=[ 142], | 99.99th=[ 142] bw ( KiB/s): min= 1774, max= 2031, per=38.45%, avg=1902.50, stdev=181.73, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2754KiB/s (2820kB/s)(10.0MiB/3718msec); 0 zone resets clat (msec): min=94, max=1144, avg=352.74, stdev=332.03 lat (msec): min=94, max=1144, avg=352.91, stdev=332.02 clat percentiles (msec): | 1.00th=[ 94], 5.00th=[ 94], 10.00th=[ 94], 20.00th=[ 100], | 30.00th=[ 136], 40.00th=[ 155], 50.00th=[ 180], 60.00th=[ 275], | 70.00th=[ 292], 80.00th=[ 514], 90.00th=[ 642], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min= 1774, max= 4071, per=43.46%, avg=3010.17, stdev=1163.26, samples=6 iops : min= 1, max= 3, avg= 2.00, stdev= 1.10, samples=6 lat (msec) : 20=8.33%, 100=16.67%, 250=33.33%, 500=16.67%, 750=16.67% lat (msec) : 2000=8.33% cpu : usr=0.03%, sys=24.56%, ctx=250, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194840: Tue Dec 2 23:40:48 2025 read: IOPS=1, BW=1685KiB/s (1725kB/s)(6144KiB/3647msec) clat (usec): min=638, max=172574, avg=76002.23, stdev=84211.62 lat (usec): min=639, max=172575, avg=76003.51, stdev=84211.30 clat percentiles (usec): | 1.00th=[ 635], 5.00th=[ 635], 10.00th=[ 635], 20.00th=[ 832], | 30.00th=[ 832], 40.00th=[ 906], 50.00th=[ 906], 60.00th=[120062], | 70.00th=[160433], 80.00th=[160433], 90.00th=[173016], 95.00th=[173016], | 99.00th=[173016], 99.50th=[173016], 99.90th=[173016], 99.95th=[173016], | 99.99th=[173016] bw ( KiB/s): min= 2031, max= 6107, per=68.62%, avg=3395.33, stdev=2348.39, samples=3 iops : min= 1, max= 5, avg= 2.67, stdev= 2.08, samples=3 write: IOPS=1, BW=1685KiB/s (1725kB/s)(6144KiB/3647msec); 0 zone resets clat (msec): min=218, max=1206, avg=511.24, stdev=364.25 lat (msec): min=218, max=1206, avg=511.43, stdev=364.28 clat percentiles (msec): | 1.00th=[ 220], 5.00th=[ 220], 10.00th=[ 220], 20.00th=[ 326], | 30.00th=[ 326], 40.00th=[ 342], 50.00th=[ 342], 60.00th=[ 363], | 70.00th=[ 609], 80.00th=[ 609], 90.00th=[ 1200], 95.00th=[ 1200], | 99.00th=[ 1200], 99.50th=[ 1200], 99.90th=[ 1200], 99.95th=[ 1200], | 99.99th=[ 1200] bw ( KiB/s): min= 1825, max= 4079, per=36.04%, avg=2496.75, stdev=1059.77, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (usec) : 750=8.33%, 1000=16.67% lat (msec) : 250=33.33%, 500=25.00%, 750=8.33%, 2000=8.33% cpu : usr=0.03%, sys=21.48%, ctx=347, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194841: Tue Dec 2 23:40:48 2025 read: IOPS=1, BW=1472KiB/s (1507kB/s)(5120KiB/3478msec) clat (usec): min=636, max=183518, avg=73056.62, stdev=98999.73 lat (usec): min=637, max=183520, avg=73057.49, stdev=99000.05 clat percentiles (usec): | 1.00th=[ 635], 5.00th=[ 635], 10.00th=[ 635], 20.00th=[ 635], | 30.00th=[ 685], 40.00th=[ 685], 50.00th=[ 979], 60.00th=[ 979], | 70.00th=[179307], 80.00th=[179307], 90.00th=[183501], 95.00th=[183501], | 99.00th=[183501], 99.50th=[183501], 99.90th=[183501], 99.95th=[183501], | 99.99th=[183501] bw ( KiB/s): min= 8126, max= 8126, per=100.00%, avg=8126.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=2, BW=2061KiB/s (2110kB/s)(7168KiB/3478msec); 0 zone resets clat (msec): min=120, max=1209, avg=419.46, stdev=401.81 lat (msec): min=120, max=1209, avg=419.55, stdev=401.78 clat percentiles (msec): | 1.00th=[ 121], 5.00th=[ 121], 10.00th=[ 121], 20.00th=[ 134], | 30.00th=[ 155], 40.00th=[ 155], 50.00th=[ 182], 60.00th=[ 531], | 70.00th=[ 531], 80.00th=[ 609], 90.00th=[ 1217], 95.00th=[ 1217], | 99.00th=[ 1217], 99.50th=[ 1217], 99.90th=[ 1217], 99.95th=[ 1217], | 99.99th=[ 1217] bw ( KiB/s): min= 2031, max= 4071, per=36.77%, avg=2547.25, stdev=1015.86, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (usec) : 750=16.67%, 1000=8.33% lat (msec) : 250=50.00%, 750=16.67%, 2000=8.33% cpu : usr=0.00%, sys=22.95%, ctx=269, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4948KiB/s (5067kB/s), 551KiB/s-1732KiB/s (564kB/s-1773kB/s), io=20.0MiB (20.0MB), run=3478-4139msec WRITE: bw=6927KiB/s (7094kB/s), 1237KiB/s-2754KiB/s (1267kB/s-2820kB/s), io=28.0MiB (29.4MB), run=3478-4139msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=194848: Tue Dec 2 23:40:57 2025 read: IOPS=1, BW=7054KiB/s (7223kB/s)(12.0MiB/1742msec) clat (msec): min=465, max=655, avg=553.34, stdev=95.80 lat (msec): min=465, max=655, avg=553.34, stdev=95.80 clat percentiles (msec): | 1.00th=[ 468], 5.00th=[ 468], 10.00th=[ 468], 20.00th=[ 468], | 30.00th=[ 468], 40.00th=[ 542], 50.00th=[ 542], 60.00th=[ 542], | 70.00th=[ 659], 80.00th=[ 659], 90.00th=[ 659], 95.00th=[ 659], | 99.00th=[ 659], 99.50th=[ 659], 99.90th=[ 659], 99.95th=[ 659], | 99.99th=[ 659] bw ( KiB/s): min= 6168, max= 8110, per=83.71%, avg=7139.00, stdev=1373.20, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.00%, sys=32.91%, ctx=745, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194849: Tue Dec 2 23:40:57 2025 read: IOPS=0, BW=2132KiB/s (2183kB/s)(4096KiB/1921msec) clat (nsec): min=538446k, max=538446k, avg=538445948.00, stdev= 0.00 lat (nsec): min=538448k, max=538448k, avg=538448253.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 542], 5.00th=[ 542], 10.00th=[ 542], 20.00th=[ 542], | 30.00th=[ 542], 40.00th=[ 542], 50.00th=[ 542], 60.00th=[ 542], | 70.00th=[ 542], 80.00th=[ 542], 90.00th=[ 542], 95.00th=[ 542], | 99.00th=[ 542], 99.50th=[ 542], 99.90th=[ 542], 99.95th=[ 542], | 99.99th=[ 542] bw ( KiB/s): min= 5818, max= 5818, per=68.22%, avg=5818.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4264KiB/s (4367kB/s)(8192KiB/1921msec); 0 zone resets clat (msec): min=188, max=1070, avg=629.61, stdev=623.15 lat (msec): min=189, max=1070, avg=630.10, stdev=623.02 clat percentiles (msec): | 1.00th=[ 190], 5.00th=[ 190], 10.00th=[ 190], 20.00th=[ 190], | 30.00th=[ 190], 40.00th=[ 190], 50.00th=[ 190], 60.00th=[ 1070], | 70.00th=[ 1070], 80.00th=[ 1070], 90.00th=[ 1070], 95.00th=[ 1070], | 99.00th=[ 1070], 99.50th=[ 1070], 99.90th=[ 1070], 99.95th=[ 1070], | 99.99th=[ 1070] bw ( KiB/s): min= 8159, max= 8159, per=60.16%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.05%, sys=10.83%, ctx=286, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194850: Tue Dec 2 23:40:57 2025 write: IOPS=1, BW=5086KiB/s (5208kB/s)(12.0MiB/2416msec); 0 zone resets clat (msec): min=220, max=1377, avg=768.92, stdev=580.68 lat (msec): min=221, max=1378, avg=769.41, stdev=580.84 clat percentiles (msec): | 1.00th=[ 222], 5.00th=[ 222], 10.00th=[ 222], 20.00th=[ 222], | 30.00th=[ 222], 40.00th=[ 709], 50.00th=[ 709], 60.00th=[ 709], | 70.00th=[ 1385], 80.00th=[ 1385], 90.00th=[ 1385], 95.00th=[ 1385], | 99.00th=[ 1385], 99.50th=[ 1385], 99.90th=[ 1385], 99.95th=[ 1385], | 99.99th=[ 1385] bw ( KiB/s): min= 8143, max= 8159, per=60.10%, avg=8151.00, stdev=11.31, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.08%, sys=17.18%, ctx=94, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194851: Tue Dec 2 23:40:57 2025 write: IOPS=3, BW=12.7MiB/s (13.3MB/s)(12.0MiB/944msec); 0 zone resets clat (msec): min=208, max=397, avg=300.69, stdev=94.44 lat (msec): min=210, max=398, avg=301.63, stdev=93.80 clat percentiles (msec): | 1.00th=[ 209], 5.00th=[ 209], 10.00th=[ 209], 20.00th=[ 209], | 30.00th=[ 209], 40.00th=[ 296], 50.00th=[ 296], 60.00th=[ 296], | 70.00th=[ 397], 80.00th=[ 397], 90.00th=[ 397], 95.00th=[ 397], | 99.00th=[ 397], 99.50th=[ 397], 99.90th=[ 397], 99.95th=[ 397], | 99.99th=[ 397] bw ( KiB/s): min=13495, max=13495, per=99.51%, avg=13495.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=66.67% cpu : usr=0.00%, sys=19.94%, ctx=57, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8529KiB/s (8734kB/s), 2132KiB/s-7054KiB/s (2183kB/s-7223kB/s), io=16.0MiB (16.8MB), run=1742-1921msec WRITE: bw=13.2MiB/s (13.9MB/s), 4264KiB/s-12.7MiB/s (4367kB/s-13.3MB/s), io=32.0MiB (33.6MB), run=944-2416msec rand-rw: (groupid=0, jobs=1): err= 0: pid=194853: Tue Dec 2 23:40:57 2025 read: IOPS=2, BW=8275KiB/s (8473kB/s)(12.0MiB/1485msec) clat (msec): min=160, max=931, avg=494.47, stdev=395.93 lat (msec): min=160, max=931, avg=494.47, stdev=395.93 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 161], | 30.00th=[ 161], 40.00th=[ 393], 50.00th=[ 393], 60.00th=[ 393], | 70.00th=[ 936], 80.00th=[ 936], 90.00th=[ 936], 95.00th=[ 936], | 99.00th=[ 936], 99.50th=[ 936], 99.90th=[ 936], 99.95th=[ 936], | 99.99th=[ 936] bw ( KiB/s): min= 5868, max= 5868, per=57.70%, avg=5868.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 1000=33.33% cpu : usr=0.00%, sys=9.64%, ctx=52, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194854: Tue Dec 2 23:40:57 2025 read: IOPS=0, BW=2543KiB/s (2604kB/s)(4096KiB/1611msec) clat (nsec): min=988709k, max=988709k, avg=988708765.00, stdev= 0.00 lat (nsec): min=988711k, max=988711k, avg=988710882.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 986], 5.00th=[ 986], 10.00th=[ 986], 20.00th=[ 986], | 30.00th=[ 986], 40.00th=[ 986], 50.00th=[ 986], 60.00th=[ 986], | 70.00th=[ 986], 80.00th=[ 986], 90.00th=[ 986], 95.00th=[ 986], | 99.00th=[ 986], 99.50th=[ 986], 99.90th=[ 986], 99.95th=[ 986], | 99.99th=[ 986] bw ( KiB/s): min= 8192, max= 8192, per=80.55%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=1, BW=5085KiB/s (5207kB/s)(8192KiB/1611msec); 0 zone resets clat (msec): min=113, max=471, avg=292.44, stdev=253.58 lat (msec): min=113, max=472, avg=292.88, stdev=253.42 clat percentiles (msec): | 1.00th=[ 113], 5.00th=[ 113], 10.00th=[ 113], 20.00th=[ 113], | 30.00th=[ 113], 40.00th=[ 113], 50.00th=[ 113], 60.00th=[ 472], | 70.00th=[ 472], 80.00th=[ 472], 90.00th=[ 472], 95.00th=[ 472], | 99.00th=[ 472], 99.50th=[ 472], 99.90th=[ 472], 99.95th=[ 472], | 99.99th=[ 472] bw ( KiB/s): min= 8192, max= 8192, per=40.88%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 1000=33.33% cpu : usr=0.06%, sys=11.49%, ctx=43, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194855: Tue Dec 2 23:40:57 2025 write: IOPS=1, BW=7516KiB/s (7696kB/s)(12.0MiB/1635msec); 0 zone resets clat (msec): min=198, max=751, avg=544.14, stdev=301.68 lat (msec): min=198, max=752, avg=544.64, stdev=301.65 clat percentiles (msec): | 1.00th=[ 199], 5.00th=[ 199], 10.00th=[ 199], 20.00th=[ 199], | 30.00th=[ 199], 40.00th=[ 684], 50.00th=[ 684], 60.00th=[ 684], | 70.00th=[ 751], 80.00th=[ 751], 90.00th=[ 751], 95.00th=[ 751], | 99.00th=[ 751], 99.50th=[ 751], 99.90th=[ 751], 99.95th=[ 751], | 99.99th=[ 751] bw ( KiB/s): min= 8192, max= 8192, per=40.88%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.06%, sys=10.89%, ctx=44, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194856: Tue Dec 2 23:40:57 2025 write: IOPS=2, BW=9344KiB/s (9569kB/s)(12.0MiB/1315msec); 0 zone resets clat (msec): min=115, max=951, avg=425.99, stdev=457.75 lat (msec): min=115, max=952, avg=426.42, stdev=457.73 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 116], | 30.00th=[ 116], 40.00th=[ 211], 50.00th=[ 211], 60.00th=[ 211], | 70.00th=[ 953], 80.00th=[ 953], 90.00th=[ 953], 95.00th=[ 953], | 99.00th=[ 953], 99.50th=[ 953], 99.90th=[ 953], 99.95th=[ 953], | 99.99th=[ 953] bw ( KiB/s): min= 6159, max= 6159, per=30.73%, avg=6159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, 1000=33.33% cpu : usr=0.00%, sys=12.94%, ctx=63, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=9.93MiB/s (10.4MB/s), 2543KiB/s-8275KiB/s (2604kB/s-8473kB/s), io=16.0MiB (16.8MB), run=1485-1611msec WRITE: bw=19.6MiB/s (20.5MB/s), 5085KiB/s-9344KiB/s (5207kB/s-9569kB/s), io=32.0MiB (33.6MB), run=1315-1635msec