== sanity test 398b: DIO and buffer IO race ============== 21:00:00 (1763344800) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 4.24561 s, 11.9 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148562: Sun Nov 16 21:01:50 2025 read: IOPS=15, BW=62.0KiB/s (63.5kB/s)(5992KiB/96602msec) clat (usec): min=90, max=788311, avg=31176.99, stdev=30404.08 lat (usec): min=90, max=788313, avg=31188.65, stdev=30399.79 clat percentiles (usec): | 1.00th=[ 128], 5.00th=[ 167], 10.00th=[ 281], 20.00th=[ 13829], | 30.00th=[ 17957], 40.00th=[ 22152], 50.00th=[ 26084], 60.00th=[ 31065], | 70.00th=[ 36963], 80.00th=[ 47973], 90.00th=[ 61604], 95.00th=[ 73925], | 99.00th=[102237], 99.50th=[117965], 99.90th=[191890], 99.95th=[784335], | 99.99th=[784335] bw ( KiB/s): min= 7, max= 167, per=24.88%, avg=61.45, stdev=25.27, samples=190 iops : min= 1, max= 41, avg=14.76, stdev= 6.34, samples=190 write: IOPS=16, BW=65.2KiB/s (66.7kB/s)(6296KiB/96602msec); 0 zone resets clat (usec): min=931, max=369492, avg=31437.18, stdev=34248.54 lat (usec): min=932, max=369494, avg=31442.33, stdev=34251.07 clat percentiles (usec): | 1.00th=[ 1012], 5.00th=[ 1139], 10.00th=[ 1270], 20.00th=[ 2835], | 30.00th=[ 6456], 40.00th=[ 13960], 50.00th=[ 22938], 60.00th=[ 32375], | 70.00th=[ 41157], 80.00th=[ 54264], 90.00th=[ 74974], 95.00th=[ 89654], | 99.00th=[135267], 99.50th=[173016], 99.90th=[350225], 99.95th=[371196], | 99.99th=[371196] bw ( KiB/s): min= 7, max= 167, per=25.24%, avg=65.38, stdev=25.67, samples=188 iops : min= 1, max= 41, avg=15.75, stdev= 6.45, samples=188 lat (usec) : 100=0.07%, 250=4.43%, 500=1.17%, 750=0.23%, 1000=0.36% lat (msec) : 2=8.76%, 4=4.04%, 10=6.51%, 20=15.43%, 50=38.28% lat (msec) : 100=18.26%, 250=2.31%, 500=0.13%, 1000=0.03% cpu : usr=0.08%, sys=8.08%, ctx=6590, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148563: Sun Nov 16 21:01:50 2025 read: IOPS=15, BW=61.2KiB/s (62.7kB/s)(5940KiB/96990msec) clat (usec): min=104, max=274224, avg=29347.53, stdev=22687.45 lat (usec): min=104, max=274225, avg=29348.86, stdev=22687.43 clat percentiles (usec): | 1.00th=[ 127], 5.00th=[ 161], 10.00th=[ 221], 20.00th=[ 12911], | 30.00th=[ 17695], 40.00th=[ 21627], 50.00th=[ 25560], 60.00th=[ 30016], | 70.00th=[ 36439], 80.00th=[ 45351], 90.00th=[ 57934], 95.00th=[ 69731], | 99.00th=[ 93848], 99.50th=[117965], 99.90th=[162530], 99.95th=[274727], | 99.99th=[274727] bw ( KiB/s): min= 7, max= 143, per=24.82%, avg=61.30, stdev=25.52, samples=189 iops : min= 1, max= 35, avg=14.72, stdev= 6.42, samples=189 write: IOPS=16, BW=65.4KiB/s (67.0kB/s)(6348KiB/96990msec); 0 zone resets clat (usec): min=822, max=790152, avg=33442.92, stdev=39250.87 lat (usec): min=823, max=790153, avg=33449.51, stdev=39248.84 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1139], 10.00th=[ 1287], 20.00th=[ 3228], | 30.00th=[ 7701], 40.00th=[ 16450], 50.00th=[ 25035], 60.00th=[ 33424], | 70.00th=[ 42730], 80.00th=[ 55837], 90.00th=[ 76022], 95.00th=[ 95945], | 99.00th=[143655], 99.50th=[189793], 99.90th=[308282], 99.95th=[792724], | 99.99th=[792724] bw ( KiB/s): min= 7, max= 183, per=25.02%, avg=64.81, stdev=27.80, samples=191 iops : min= 1, max= 45, avg=15.61, stdev= 6.97, samples=191 lat (usec) : 250=5.14%, 500=1.14%, 750=0.16%, 1000=0.29% lat (msec) : 2=7.91%, 4=3.74%, 10=6.28%, 20=15.49%, 50=39.62% lat (msec) : 100=17.61%, 250=2.44%, 500=0.13%, 1000=0.03% cpu : usr=0.10%, sys=7.76%, ctx=6409, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148564: Sun Nov 16 21:01:50 2025 read: IOPS=15, BW=62.3KiB/s (63.8kB/s)(6028KiB/96713msec) clat (usec): min=94, max=228295, avg=29777.99, stdev=23857.19 lat (usec): min=94, max=228296, avg=29779.36, stdev=23857.20 clat percentiles (usec): | 1.00th=[ 122], 5.00th=[ 161], 10.00th=[ 229], 20.00th=[ 13435], | 30.00th=[ 17433], 40.00th=[ 21103], 50.00th=[ 24511], 60.00th=[ 30016], | 70.00th=[ 36963], 80.00th=[ 45876], 90.00th=[ 58983], 95.00th=[ 69731], | 99.00th=[100140], 99.50th=[141558], 99.90th=[189793], 99.95th=[227541], | 99.99th=[227541] bw ( KiB/s): min= 7, max= 143, per=25.35%, avg=62.62, stdev=25.08, samples=189 iops : min= 1, max= 35, avg=15.05, stdev= 6.32, samples=189 write: IOPS=16, BW=64.7KiB/s (66.3kB/s)(6260KiB/96713msec); 0 zone resets clat (usec): min=888, max=883526, avg=32903.95, stdev=40419.13 lat (usec): min=890, max=883529, avg=32907.73, stdev=40417.74 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 1139], 10.00th=[ 1270], 20.00th=[ 2573], | 30.00th=[ 6325], 40.00th=[ 14091], 50.00th=[ 23462], 60.00th=[ 32113], | 70.00th=[ 42206], 80.00th=[ 56361], 90.00th=[ 76022], 95.00th=[ 99091], | 99.00th=[147850], 99.50th=[181404], 99.90th=[278922], 99.95th=[884999], | 99.99th=[884999] bw ( KiB/s): min= 7, max= 127, per=25.10%, avg=65.01, stdev=25.49, samples=189 iops : min= 1, max= 31, avg=15.66, stdev= 6.37, samples=189 lat (usec) : 100=0.07%, 250=4.92%, 500=1.11%, 750=0.16%, 1000=0.62% lat (msec) : 2=8.01%, 4=4.13%, 10=6.58%, 20=15.95%, 50=38.12% lat (msec) : 100=17.35%, 250=2.93%, 500=0.03%, 1000=0.03% cpu : usr=0.10%, sys=7.65%, ctx=6314, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148565: Sun Nov 16 21:01:50 2025 read: IOPS=15, BW=62.6KiB/s (64.1kB/s)(6052KiB/96639msec) clat (usec): min=93, max=768981, avg=29866.17, stdev=31089.84 lat (usec): min=93, max=768983, avg=29869.26, stdev=31089.67 clat percentiles (usec): | 1.00th=[ 133], 5.00th=[ 167], 10.00th=[ 235], 20.00th=[ 13173], | 30.00th=[ 17957], 40.00th=[ 21627], 50.00th=[ 25035], 60.00th=[ 29492], | 70.00th=[ 36439], 80.00th=[ 45351], 90.00th=[ 57934], 95.00th=[ 69731], | 99.00th=[ 92799], 99.50th=[101188], 99.90th=[522191], 99.95th=[767558], | 99.99th=[767558] bw ( KiB/s): min= 7, max= 142, per=25.56%, avg=63.15, stdev=25.66, samples=186 iops : min= 1, max= 35, avg=15.19, stdev= 6.44, samples=186 write: IOPS=16, BW=64.5KiB/s (66.1kB/s)(6236KiB/96639msec); 0 zone resets clat (usec): min=902, max=945202, avg=32777.25, stdev=40847.29 lat (usec): min=902, max=945204, avg=32781.37, stdev=40845.83 clat percentiles (usec): | 1.00th=[ 1012], 5.00th=[ 1139], 10.00th=[ 1270], 20.00th=[ 2638], | 30.00th=[ 7373], 40.00th=[ 15139], 50.00th=[ 24249], 60.00th=[ 32637], | 70.00th=[ 41681], 80.00th=[ 55837], 90.00th=[ 74974], 95.00th=[ 98042], | 99.00th=[147850], 99.50th=[168821], 99.90th=[358613], 99.95th=[943719], | 99.99th=[943719] bw ( KiB/s): min= 15, max= 141, per=25.05%, avg=64.88, stdev=24.90, samples=187 iops : min= 3, max= 35, avg=15.64, stdev= 6.25, samples=187 lat (usec) : 100=0.03%, 250=5.05%, 500=1.24%, 750=0.10%, 1000=0.39% lat (msec) : 2=8.27%, 4=3.52%, 10=6.48%, 20=15.66%, 50=39.62% lat (msec) : 100=17.12%, 250=2.38%, 500=0.07%, 750=0.03%, 1000=0.07% cpu : usr=0.09%, sys=7.95%, ctx=6497, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=248KiB/s (254kB/s), 61.2KiB/s-62.6KiB/s (62.7kB/s-64.1kB/s), io=23.4MiB (24.6MB), run=96602-96990msec WRITE: bw=259KiB/s (265kB/s), 64.5KiB/s-65.4KiB/s (66.1kB/s-67.0kB/s), io=24.6MiB (25.7MB), run=96602-96990msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148558: Sun Nov 16 21:04:15 2025 read: IOPS=6, BW=24.9KiB/s (25.5kB/s)(5992KiB/240422msec) clat (msec): min=5, max=818, avg=28.45, stdev=31.93 lat (msec): min=5, max=818, avg=28.46, stdev=31.92 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 29], 80.00th=[ 41], 90.00th=[ 57], 95.00th=[ 74], | 99.00th=[ 115], 99.50th=[ 134], 99.90th=[ 376], 99.95th=[ 818], | 99.99th=[ 818] bw ( KiB/s): min= 7, max= 94, per=27.32%, avg=27.04, stdev=16.97, samples=430 iops : min= 1, max= 23, avg= 6.16, stdev= 4.27, samples=430 write: IOPS=6, BW=26.2KiB/s (26.8kB/s)(6296KiB/240422msec); 0 zone resets clat (msec): min=34, max=761, avg=125.44, stdev=53.23 lat (msec): min=34, max=761, avg=125.44, stdev=53.23 clat percentiles (msec): | 1.00th=[ 53], 5.00th=[ 65], 10.00th=[ 73], 20.00th=[ 85], | 30.00th=[ 94], 40.00th=[ 105], 50.00th=[ 115], 60.00th=[ 126], | 70.00th=[ 140], 80.00th=[ 159], 90.00th=[ 192], 95.00th=[ 224], | 99.00th=[ 292], 99.50th=[ 330], 99.90th=[ 558], 99.95th=[ 760], | 99.99th=[ 760] bw ( KiB/s): min= 7, max= 48, per=24.75%, avg=25.74, stdev= 9.77, samples=473 iops : min= 1, max= 12, avg= 5.83, stdev= 2.49, samples=473 lat (msec) : 10=3.94%, 20=22.14%, 50=16.15%, 100=23.44%, 250=32.85% lat (msec) : 500=1.40%, 750=0.03%, 1000=0.07% cpu : usr=0.04%, sys=1.40%, ctx=5269, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148559: Sun Nov 16 21:04:15 2025 read: IOPS=6, BW=24.6KiB/s (25.2kB/s)(5940KiB/241223msec) clat (msec): min=5, max=812, avg=28.64, stdev=34.09 lat (msec): min=5, max=812, avg=28.64, stdev=34.09 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 29], 80.00th=[ 41], 90.00th=[ 61], 95.00th=[ 74], | 99.00th=[ 114], 99.50th=[ 133], 99.90th=[ 592], 99.95th=[ 810], | 99.99th=[ 810] bw ( KiB/s): min= 7, max= 87, per=27.65%, avg=27.37, stdev=17.52, samples=421 iops : min= 1, max= 21, avg= 6.25, stdev= 4.42, samples=421 write: IOPS=6, BW=26.3KiB/s (26.9kB/s)(6348KiB/241223msec); 0 zone resets clat (msec): min=39, max=457, avg=124.98, stdev=50.43 lat (msec): min=39, max=457, avg=124.98, stdev=50.43 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 66], 10.00th=[ 74], 20.00th=[ 86], | 30.00th=[ 94], 40.00th=[ 105], 50.00th=[ 114], 60.00th=[ 126], | 70.00th=[ 140], 80.00th=[ 159], 90.00th=[ 188], 95.00th=[ 218], | 99.00th=[ 300], 99.50th=[ 330], 99.90th=[ 401], 99.95th=[ 460], | 99.99th=[ 460] bw ( KiB/s): min= 7, max= 55, per=24.92%, avg=25.91, stdev= 9.31, samples=474 iops : min= 1, max= 13, avg= 5.87, stdev= 2.39, samples=474 lat (msec) : 10=4.39%, 20=21.00%, 50=16.24%, 100=24.35%, 250=32.32% lat (msec) : 500=1.63%, 750=0.03%, 1000=0.03% cpu : usr=0.05%, sys=1.39%, ctx=5380, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148560: Sun Nov 16 21:04:15 2025 read: IOPS=6, BW=24.0KiB/s (25.5kB/s)(6028KiB/241592msec) clat (msec): min=4, max=353, avg=29.95, stdev=29.31 lat (msec): min=4, max=353, avg=29.96, stdev=29.31 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 31], 80.00th=[ 43], 90.00th=[ 64], 95.00th=[ 82], | 99.00th=[ 121], 99.50th=[ 174], 99.90th=[ 326], 99.95th=[ 355], | 99.99th=[ 355] bw ( KiB/s): min= 7, max= 87, per=27.16%, avg=26.89, stdev=17.28, samples=433 iops : min= 1, max= 21, avg= 6.12, stdev= 4.35, samples=433 write: IOPS=6, BW=25.9KiB/s (26.5kB/s)(6260KiB/241592msec); 0 zone resets clat (msec): min=40, max=872, avg=125.30, stdev=54.02 lat (msec): min=40, max=872, avg=125.31, stdev=54.02 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 65], 10.00th=[ 73], 20.00th=[ 84], | 30.00th=[ 94], 40.00th=[ 104], 50.00th=[ 115], 60.00th=[ 127], | 70.00th=[ 140], 80.00th=[ 161], 90.00th=[ 194], 95.00th=[ 224], | 99.00th=[ 284], 99.50th=[ 305], 99.90th=[ 550], 99.95th=[ 869], | 99.99th=[ 869] bw ( KiB/s): min= 7, max= 48, per=24.60%, avg=25.58, stdev= 9.58, samples=472 iops : min= 1, max= 12, avg= 5.79, stdev= 2.45, samples=472 lat (msec) : 10=3.61%, 20=21.16%, 50=16.86%, 100=25.20%, 250=31.71% lat (msec) : 500=1.40%, 750=0.03%, 1000=0.03% cpu : usr=0.05%, sys=1.40%, ctx=5321, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148561: Sun Nov 16 21:04:15 2025 read: IOPS=6, BW=25.2KiB/s (25.8kB/s)(6052KiB/240036msec) clat (msec): min=5, max=364, avg=28.67, stdev=27.62 lat (msec): min=5, max=364, avg=28.67, stdev=27.62 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 29], 80.00th=[ 42], 90.00th=[ 58], 95.00th=[ 75], | 99.00th=[ 124], 99.50th=[ 167], 99.90th=[ 359], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 7, max= 93, per=28.51%, avg=28.22, stdev=17.23, samples=416 iops : min= 1, max= 23, avg= 6.45, stdev= 4.36, samples=416 write: IOPS=6, BW=25.0KiB/s (26.6kB/s)(6236KiB/240036msec); 0 zone resets clat (msec): min=43, max=932, avg=125.92, stdev=57.03 lat (msec): min=43, max=932, avg=125.92, stdev=57.03 clat percentiles (msec): | 1.00th=[ 53], 5.00th=[ 66], 10.00th=[ 73], 20.00th=[ 86], | 30.00th=[ 94], 40.00th=[ 105], 50.00th=[ 114], 60.00th=[ 125], | 70.00th=[ 140], 80.00th=[ 161], 90.00th=[ 192], 95.00th=[ 224], | 99.00th=[ 305], 99.50th=[ 355], 99.90th=[ 768], 99.95th=[ 936], | 99.99th=[ 936] bw ( KiB/s): min= 7, max= 47, per=24.60%, avg=25.58, stdev= 9.67, samples=471 iops : min= 1, max= 11, avg= 5.79, stdev= 2.48, samples=471 lat (msec) : 10=4.10%, 20=21.84%, 50=16.50%, 100=24.38%, 250=31.67% lat (msec) : 500=1.43%, 1000=0.07% cpu : usr=0.05%, sys=1.38%, ctx=5325, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=99.4KiB/s (102kB/s), 24.6KiB/s-25.2KiB/s (25.2kB/s-25.8kB/s), io=23.4MiB (24.6MB), run=240036-241592msec WRITE: bw=104KiB/s (107kB/s), 25.9KiB/s-26.3KiB/s (26.5kB/s-26.9kB/s), io=24.6MiB (25.7MB), run=240036-241592msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148625: Sun Nov 16 21:04:44 2025 read: IOPS=18, BW=289KiB/s (296kB/s)(5920KiB/20469msec) clat (usec): min=111, max=189667, avg=28595.50, stdev=23979.77 lat (usec): min=111, max=189669, avg=28596.75, stdev=23979.84 clat percentiles (usec): | 1.00th=[ 122], 5.00th=[ 163], 10.00th=[ 212], 20.00th=[ 4948], | 30.00th=[ 17171], 40.00th=[ 21365], 50.00th=[ 25560], 60.00th=[ 29754], | 70.00th=[ 35914], 80.00th=[ 44303], 90.00th=[ 54264], 95.00th=[ 65799], | 99.00th=[107480], 99.50th=[168821], 99.90th=[189793], 99.95th=[189793], | 99.99th=[189793] bw ( KiB/s): min= 126, max= 542, per=25.20%, avg=287.78, stdev=98.30, samples=40 iops : min= 7, max= 33, avg=17.28, stdev= 6.19, samples=40 write: IOPS=19, BW=311KiB/s (319kB/s)(6368KiB/20469msec); 0 zone resets clat (usec): min=1249, max=239809, avg=24722.89, stdev=28413.86 lat (usec): min=1251, max=239812, avg=24725.17, stdev=28413.83 clat percentiles (usec): | 1.00th=[ 1401], 5.00th=[ 1696], 10.00th=[ 1909], 20.00th=[ 3458], | 30.00th=[ 6456], 40.00th=[ 9503], 50.00th=[ 14746], 60.00th=[ 22152], | 70.00th=[ 30016], 80.00th=[ 39060], 90.00th=[ 62653], 95.00th=[ 77071], | 99.00th=[127402], 99.50th=[173016], 99.90th=[240124], 99.95th=[240124], | 99.99th=[240124] bw ( KiB/s): min= 95, max= 637, per=25.87%, avg=310.97, stdev=102.88, samples=40 iops : min= 5, max= 39, avg=18.72, stdev= 6.52, samples=40 lat (usec) : 250=5.99%, 500=2.34%, 750=0.13% lat (msec) : 2=6.25%, 4=6.12%, 10=10.16%, 20=15.23%, 50=39.97% lat (msec) : 100=12.11%, 250=1.69% cpu : usr=0.06%, sys=9.87%, ctx=1572, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148626: Sun Nov 16 21:04:44 2025 read: IOPS=17, BW=274KiB/s (281kB/s)(5744KiB/20961msec) clat (usec): min=106, max=196919, avg=30950.46, stdev=25331.17 lat (usec): min=106, max=196920, avg=30951.78, stdev=25331.16 clat percentiles (usec): | 1.00th=[ 133], 5.00th=[ 157], 10.00th=[ 202], 20.00th=[ 13960], | 30.00th=[ 19530], 40.00th=[ 23200], 50.00th=[ 27395], 60.00th=[ 33162], | 70.00th=[ 38536], 80.00th=[ 46400], 90.00th=[ 58983], 95.00th=[ 72877], | 99.00th=[132645], 99.50th=[156238], 99.90th=[196084], 99.95th=[196084], | 99.99th=[196084] bw ( KiB/s): min= 58, max= 508, per=24.01%, avg=274.24, stdev=99.45, samples=41 iops : min= 3, max= 31, avg=16.41, stdev= 6.22, samples=41 write: IOPS=19, BW=312KiB/s (320kB/s)(6544KiB/20961msec); 0 zone resets clat (usec): min=1220, max=257592, avg=23780.74, stdev=26493.75 lat (usec): min=1221, max=257596, avg=23783.18, stdev=26493.67 clat percentiles (usec): | 1.00th=[ 1434], 5.00th=[ 1614], 10.00th=[ 1926], 20.00th=[ 3752], | 30.00th=[ 6063], 40.00th=[ 9241], 50.00th=[ 15795], 60.00th=[ 21365], | 70.00th=[ 29230], 80.00th=[ 40633], 90.00th=[ 57934], 95.00th=[ 72877], | 99.00th=[100140], 99.50th=[132645], 99.90th=[256902], 99.95th=[256902], | 99.99th=[256902] bw ( KiB/s): min= 95, max= 667, per=26.02%, avg=312.76, stdev=115.18, samples=41 iops : min= 5, max= 41, avg=18.83, stdev= 7.27, samples=41 lat (usec) : 250=5.60%, 500=1.69%, 750=0.13% lat (msec) : 2=5.73%, 4=6.25%, 10=11.46%, 20=13.93%, 50=39.45% lat (msec) : 100=14.06%, 250=1.56%, 500=0.13% cpu : usr=0.09%, sys=9.87%, ctx=1571, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148627: Sun Nov 16 21:04:44 2025 read: IOPS=18, BW=300KiB/s (307kB/s)(6256KiB/20874msec) clat (usec): min=92, max=197696, avg=30039.15, stdev=25707.77 lat (usec): min=92, max=197697, avg=30040.72, stdev=25707.71 clat percentiles (usec): | 1.00th=[ 118], 5.00th=[ 172], 10.00th=[ 202], 20.00th=[ 12911], | 30.00th=[ 19006], 40.00th=[ 22414], 50.00th=[ 26608], 60.00th=[ 30802], | 70.00th=[ 36963], 80.00th=[ 43779], 90.00th=[ 55837], 95.00th=[ 70779], | 99.00th=[143655], 99.50th=[164627], 99.90th=[198181], 99.95th=[198181], | 99.99th=[198181] bw ( KiB/s): min= 126, max= 573, per=26.36%, avg=300.98, stdev=88.75, samples=41 iops : min= 7, max= 35, avg=18.12, stdev= 5.60, samples=41 write: IOPS=18, BW=289KiB/s (296kB/s)(6032KiB/20874msec); 0 zone resets clat (usec): min=1319, max=194277, avg=23914.37, stdev=25127.49 lat (usec): min=1321, max=194278, avg=23919.02, stdev=25125.79 clat percentiles (usec): | 1.00th=[ 1385], 5.00th=[ 1663], 10.00th=[ 1926], 20.00th=[ 3752], | 30.00th=[ 6980], 40.00th=[ 10421], 50.00th=[ 14746], 60.00th=[ 23987], | 70.00th=[ 31589], 80.00th=[ 40633], 90.00th=[ 55837], 95.00th=[ 67634], | 99.00th=[108528], 99.50th=[187696], 99.90th=[193987], 99.95th=[193987], | 99.99th=[193987] bw ( KiB/s): min= 63, max= 528, per=23.80%, avg=286.05, stdev=104.49, samples=41 iops : min= 3, max= 33, avg=17.17, stdev= 6.63, samples=41 lat (usec) : 100=0.13%, 250=6.51%, 500=2.08% lat (msec) : 2=5.86%, 4=4.43%, 10=9.90%, 20=16.28%, 50=41.15% lat (msec) : 100=12.24%, 250=1.43% cpu : usr=0.11%, sys=10.00%, ctx=1607, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148628: Sun Nov 16 21:04:44 2025 read: IOPS=18, BW=293KiB/s (300kB/s)(6032KiB/20595msec) clat (usec): min=102, max=181419, avg=28530.86, stdev=23848.72 lat (usec): min=102, max=181421, avg=28532.20, stdev=23848.59 clat percentiles (usec): | 1.00th=[ 118], 5.00th=[ 161], 10.00th=[ 192], 20.00th=[ 424], | 30.00th=[ 17171], 40.00th=[ 22152], 50.00th=[ 27132], 60.00th=[ 31851], | 70.00th=[ 39060], 80.00th=[ 46400], 90.00th=[ 55313], 95.00th=[ 64226], | 99.00th=[103285], 99.50th=[173016], 99.90th=[181404], 99.95th=[181404], | 99.99th=[181404] bw ( KiB/s): min= 120, max= 480, per=25.58%, avg=292.15, stdev=84.00, samples=40 iops : min= 7, max= 30, avg=17.55, stdev= 5.31, samples=40 write: IOPS=18, BW=304KiB/s (311kB/s)(6256KiB/20595msec); 0 zone resets clat (usec): min=1341, max=295950, avg=24588.51, stdev=30235.04 lat (usec): min=1343, max=295951, avg=24590.78, stdev=30234.93 clat percentiles (usec): | 1.00th=[ 1401], 5.00th=[ 1680], 10.00th=[ 1860], 20.00th=[ 3064], | 30.00th=[ 5342], 40.00th=[ 9372], 50.00th=[ 15008], 60.00th=[ 22152], | 70.00th=[ 29492], 80.00th=[ 43254], 90.00th=[ 58459], 95.00th=[ 72877], | 99.00th=[127402], 99.50th=[254804], 99.90th=[295699], 99.95th=[295699], | 99.99th=[295699] bw ( KiB/s): min= 127, max= 660, per=25.16%, avg=302.42, stdev=116.93, samples=40 iops : min= 7, max= 41, avg=18.20, stdev= 7.41, samples=40 lat (usec) : 250=7.16%, 500=2.99%, 750=0.13%, 1000=0.13% lat (msec) : 2=6.51%, 4=6.90%, 10=9.24%, 20=13.41%, 50=38.54% lat (msec) : 100=13.41%, 250=1.30%, 500=0.26% cpu : usr=0.11%, sys=9.71%, ctx=1489, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1143KiB/s (1170kB/s), 274KiB/s-300KiB/s (281kB/s-307kB/s), io=23.4MiB (24.5MB), run=20469-20961msec WRITE: bw=1202KiB/s (1231kB/s), 289KiB/s-312KiB/s (296kB/s-320kB/s), io=24.6MiB (25.8MB), run=20469-20961msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148630: Sun Nov 16 21:05:26 2025 read: IOPS=5, BW=93.9KiB/s (96.2kB/s)(5920KiB/63015msec) clat (msec): min=6, max=358, avg=32.22, stdev=35.13 lat (msec): min=6, max=358, avg=32.22, stdev=35.13 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 19], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 31], 80.00th=[ 41], 90.00th=[ 69], 95.00th=[ 91], | 99.00th=[ 174], 99.50th=[ 309], 99.90th=[ 359], 99.95th=[ 359], | 99.99th=[ 359] bw ( KiB/s): min= 31, max= 414, per=27.33%, avg=103.57, stdev=66.52, samples=112 iops : min= 1, max= 25, avg= 5.72, stdev= 4.21, samples=112 write: IOPS=6, BW=101KiB/s (103kB/s)(6368KiB/63015msec); 0 zone resets clat (msec): min=55, max=419, avg=127.85, stdev=47.12 lat (msec): min=55, max=419, avg=127.86, stdev=47.12 clat percentiles (msec): | 1.00th=[ 58], 5.00th=[ 73], 10.00th=[ 79], 20.00th=[ 90], | 30.00th=[ 101], 40.00th=[ 111], 50.00th=[ 122], 60.00th=[ 131], | 70.00th=[ 140], 80.00th=[ 159], 90.00th=[ 186], 95.00th=[ 209], | 99.00th=[ 334], 99.50th=[ 359], 99.90th=[ 418], 99.95th=[ 418], | 99.99th=[ 418] bw ( KiB/s): min= 26, max= 192, per=25.22%, avg=100.61, stdev=32.64, samples=124 iops : min= 1, max= 12, avg= 5.55, stdev= 2.12, samples=124 lat (msec) : 10=3.52%, 20=19.53%, 50=17.71%, 100=20.57%, 250=37.63% lat (msec) : 500=1.04% cpu : usr=0.05%, sys=1.58%, ctx=1488, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148631: Sun Nov 16 21:05:26 2025 read: IOPS=5, BW=91.1KiB/s (93.3kB/s)(5744KiB/63040msec) clat (msec): min=6, max=140, avg=28.49, stdev=22.82 lat (msec): min=6, max=140, avg=28.49, stdev=22.82 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 24], | 70.00th=[ 30], 80.00th=[ 38], 90.00th=[ 60], 95.00th=[ 75], | 99.00th=[ 125], 99.50th=[ 134], 99.90th=[ 140], 99.95th=[ 140], | 99.99th=[ 140] bw ( KiB/s): min= 26, max= 316, per=26.56%, avg=100.68, stdev=60.50, samples=111 iops : min= 1, max= 19, avg= 5.56, stdev= 3.80, samples=111 write: IOPS=6, BW=104KiB/s (106kB/s)(6544KiB/63040msec); 0 zone resets clat (msec): min=52, max=434, avg=128.93, stdev=47.40 lat (msec): min=52, max=434, avg=128.94, stdev=47.39 clat percentiles (msec): | 1.00th=[ 59], 5.00th=[ 71], 10.00th=[ 83], 20.00th=[ 94], | 30.00th=[ 102], 40.00th=[ 110], 50.00th=[ 120], 60.00th=[ 129], | 70.00th=[ 144], 80.00th=[ 161], 90.00th=[ 186], 95.00th=[ 215], | 99.00th=[ 300], 99.50th=[ 326], 99.90th=[ 435], 99.95th=[ 435], | 99.99th=[ 435] bw ( KiB/s): min= 31, max= 192, per=25.92%, avg=103.42, stdev=31.91, samples=124 iops : min= 1, max= 12, avg= 5.73, stdev= 2.08, samples=124 lat (msec) : 10=2.86%, 20=20.18%, 50=17.71%, 100=19.79%, 250=38.41% lat (msec) : 500=1.04% cpu : usr=0.05%, sys=1.49%, ctx=1387, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148632: Sun Nov 16 21:05:26 2025 read: IOPS=6, BW=102KiB/s (105kB/s)(6256KiB/61281msec) clat (msec): min=6, max=158, avg=29.38, stdev=22.86 lat (msec): min=6, max=158, avg=29.38, stdev=22.86 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 19], 50.00th=[ 22], 60.00th=[ 26], | 70.00th=[ 30], 80.00th=[ 42], 90.00th=[ 59], 95.00th=[ 79], | 99.00th=[ 129], 99.50th=[ 159], 99.90th=[ 159], 99.95th=[ 159], | 99.99th=[ 159] bw ( KiB/s): min= 26, max= 318, per=27.86%, avg=105.61, stdev=68.76, samples=117 iops : min= 1, max= 19, avg= 5.85, stdev= 4.31, samples=117 write: IOPS=6, BW=98.4KiB/s (101kB/s)(6032KiB/61281msec); 0 zone resets clat (msec): min=63, max=426, avg=131.59, stdev=43.51 lat (msec): min=63, max=426, avg=131.59, stdev=43.51 clat percentiles (msec): | 1.00th=[ 68], 5.00th=[ 79], 10.00th=[ 85], 20.00th=[ 95], | 30.00th=[ 107], 40.00th=[ 114], 50.00th=[ 125], 60.00th=[ 136], | 70.00th=[ 150], 80.00th=[ 163], 90.00th=[ 186], 95.00th=[ 213], | 99.00th=[ 259], 99.50th=[ 292], 99.90th=[ 426], 99.95th=[ 426], | 99.99th=[ 426] bw ( KiB/s): min= 31, max= 160, per=24.57%, avg=98.02, stdev=29.45, samples=121 iops : min= 1, max= 10, avg= 5.37, stdev= 1.87, samples=121 lat (msec) : 10=2.60%, 20=20.96%, 50=20.18%, 100=18.88%, 250=36.59% lat (msec) : 500=0.78% cpu : usr=0.03%, sys=1.59%, ctx=1447, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148633: Sun Nov 16 21:05:26 2025 read: IOPS=6, BW=97.0KiB/s (99.3kB/s)(6032KiB/62178msec) clat (msec): min=6, max=201, avg=28.97, stdev=26.87 lat (msec): min=6, max=201, avg=28.97, stdev=26.87 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 27], 80.00th=[ 36], 90.00th=[ 59], 95.00th=[ 90], | 99.00th=[ 144], 99.50th=[ 186], 99.90th=[ 203], 99.95th=[ 203], | 99.99th=[ 203] bw ( KiB/s): min= 26, max= 318, per=27.80%, avg=105.35, stdev=68.59, samples=111 iops : min= 1, max= 19, avg= 5.82, stdev= 4.26, samples=111 write: IOPS=6, BW=101KiB/s (103kB/s)(6256KiB/62178msec); 0 zone resets clat (msec): min=48, max=356, avg=130.27, stdev=43.48 lat (msec): min=48, max=356, avg=130.28, stdev=43.48 clat percentiles (msec): | 1.00th=[ 58], 5.00th=[ 70], 10.00th=[ 80], 20.00th=[ 93], | 30.00th=[ 105], 40.00th=[ 115], 50.00th=[ 127], 60.00th=[ 134], | 70.00th=[ 146], 80.00th=[ 163], 90.00th=[ 184], 95.00th=[ 209], | 99.00th=[ 268], 99.50th=[ 296], 99.90th=[ 359], 99.95th=[ 359], | 99.99th=[ 359] bw ( KiB/s): min= 31, max= 160, per=25.08%, avg=100.07, stdev=30.88, samples=122 iops : min= 1, max= 10, avg= 5.50, stdev= 2.02, samples=122 lat (msec) : 10=2.73%, 20=22.79%, 50=17.45%, 100=16.80%, 250=39.71% lat (msec) : 500=0.52% cpu : usr=0.06%, sys=1.57%, ctx=1370, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=380KiB/s (389kB/s), 91.1KiB/s-102KiB/s (93.3kB/s-105kB/s), io=23.4MiB (24.5MB), run=61281-63040msec WRITE: bw=400KiB/s (409kB/s), 98.4KiB/s-104KiB/s (101kB/s-106kB/s), io=24.6MiB (25.8MB), run=61281-63040msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148647: Sun Nov 16 21:05:39 2025 read: IOPS=2, BW=2727KiB/s (2792kB/s)(7168KiB/2629msec) clat (usec): min=617, max=316347, avg=190170.45, stdev=103485.94 lat (usec): min=618, max=316348, avg=190171.60, stdev=103486.06 clat percentiles (usec): | 1.00th=[ 619], 5.00th=[ 619], 10.00th=[ 619], 20.00th=[123208], | 30.00th=[181404], 40.00th=[181404], 50.00th=[210764], 60.00th=[240124], | 70.00th=[240124], 80.00th=[258999], 90.00th=[316670], 95.00th=[316670], | 99.00th=[316670], 99.50th=[316670], 99.90th=[316670], 99.95th=[316670], | 99.99th=[316670] bw ( KiB/s): min= 2000, max= 4087, per=52.54%, avg=2984.00, stdev=1140.37, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 write: IOPS=1, BW=1948KiB/s (1994kB/s)(5120KiB/2629msec); 0 zone resets clat (msec): min=127, max=470, avg=231.61, stdev=138.75 lat (msec): min=128, max=471, avg=231.82, stdev=139.03 clat percentiles (msec): | 1.00th=[ 129], 5.00th=[ 129], 10.00th=[ 129], 20.00th=[ 129], | 30.00th=[ 163], 40.00th=[ 163], 50.00th=[ 165], 60.00th=[ 165], | 70.00th=[ 232], 80.00th=[ 232], 90.00th=[ 472], 95.00th=[ 472], | 99.00th=[ 472], 99.50th=[ 472], 99.90th=[ 472], 99.95th=[ 472], | 99.99th=[ 472] bw ( KiB/s): min= 2043, max= 5731, per=48.89%, avg=3887.00, stdev=2607.81, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 lat (usec) : 750=8.33% lat (msec) : 250=66.67%, 500=25.00% cpu : usr=0.04%, sys=20.43%, ctx=274, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148648: Sun Nov 16 21:05:39 2025 read: IOPS=0, BW=568KiB/s (582kB/s)(2048KiB/3606msec) clat (usec): min=692, max=230356, avg=115524.92, stdev=162396.89 lat (usec): min=694, max=230359, avg=115526.75, stdev=162397.54 clat percentiles (usec): | 1.00th=[ 693], 5.00th=[ 693], 10.00th=[ 693], 20.00th=[ 693], | 30.00th=[ 693], 40.00th=[ 693], 50.00th=[ 693], 60.00th=[229639], | 70.00th=[229639], 80.00th=[229639], 90.00th=[229639], 95.00th=[229639], | 99.00th=[229639], 99.50th=[229639], 99.90th=[229639], 99.95th=[229639], | 99.99th=[229639] bw ( KiB/s): min= 1965, max= 2000, per=34.91%, avg=1982.50, stdev=24.75, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2840KiB/s (2908kB/s)(10.0MiB/3606msec); 0 zone resets clat (msec): min=132, max=812, avg=309.15, stdev=198.94 lat (msec): min=132, max=813, avg=309.22, stdev=198.93 clat percentiles (msec): | 1.00th=[ 133], 5.00th=[ 133], 10.00th=[ 133], 20.00th=[ 165], | 30.00th=[ 186], 40.00th=[ 197], 50.00th=[ 228], 60.00th=[ 266], | 70.00th=[ 347], 80.00th=[ 351], 90.00th=[ 414], 95.00th=[ 810], | 99.00th=[ 810], 99.50th=[ 810], 99.90th=[ 810], 99.95th=[ 810], | 99.99th=[ 810] bw ( KiB/s): min= 2000, max= 5774, per=43.44%, avg=3454.25, stdev=1806.06, samples=4 iops : min= 1, max= 5, avg= 2.50, stdev= 1.91, samples=4 lat (usec) : 750=8.33% lat (msec) : 250=50.00%, 500=33.33%, 1000=8.33% cpu : usr=0.00%, sys=17.45%, ctx=156, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148649: Sun Nov 16 21:05:39 2025 read: IOPS=1, BW=1972KiB/s (2019kB/s)(6144KiB/3116msec) clat (usec): min=700, max=303281, avg=164150.00, stdev=129828.79 lat (usec): min=702, max=303282, avg=164151.00, stdev=129828.80 clat percentiles (usec): | 1.00th=[ 701], 5.00th=[ 701], 10.00th=[ 701], 20.00th=[ 2933], | 30.00th=[ 2933], 40.00th=[206570], 50.00th=[206570], 60.00th=[229639], | 70.00th=[244319], 80.00th=[244319], 90.00th=[304088], 95.00th=[304088], | 99.00th=[304088], 99.50th=[304088], 99.90th=[304088], 99.95th=[304088], | 99.99th=[304088] bw ( KiB/s): min= 2000, max= 3849, per=56.75%, avg=3223.00, stdev=1059.25, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=1, BW=1972KiB/s (2019kB/s)(6144KiB/3116msec); 0 zone resets clat (msec): min=154, max=617, avg=339.24, stdev=173.05 lat (msec): min=154, max=617, avg=339.36, stdev=173.13 clat percentiles (msec): | 1.00th=[ 155], 5.00th=[ 155], 10.00th=[ 155], 20.00th=[ 226], | 30.00th=[ 226], 40.00th=[ 255], 50.00th=[ 255], 60.00th=[ 313], | 70.00th=[ 472], 80.00th=[ 472], 90.00th=[ 617], 95.00th=[ 617], | 99.00th=[ 617], 99.50th=[ 617], 99.90th=[ 617], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 1910, max= 3976, per=29.71%, avg=2362.00, stdev=903.22, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 lat (usec) : 750=8.33% lat (msec) : 4=8.33%, 250=41.67%, 500=33.33%, 750=8.33% cpu : usr=0.10%, sys=18.65%, ctx=141, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148651: Sun Nov 16 21:05:39 2025 read: IOPS=1, BW=1950KiB/s (1997kB/s)(5120KiB/2626msec) clat (usec): min=505, max=359578, avg=72806.08, stdev=160313.59 lat (usec): min=505, max=359580, avg=72806.99, stdev=160313.80 clat percentiles (usec): | 1.00th=[ 506], 5.00th=[ 506], 10.00th=[ 506], 20.00th=[ 506], | 30.00th=[ 570], 40.00th=[ 570], 50.00th=[ 660], 60.00th=[ 660], | 70.00th=[ 2737], 80.00th=[ 2737], 90.00th=[358613], 95.00th=[358613], | 99.00th=[358613], 99.50th=[358613], 99.90th=[358613], 99.95th=[358613], | 99.99th=[358613] bw ( KiB/s): min= 7641, max= 7641, per=100.00%, avg=7641.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=2, BW=2730KiB/s (2795kB/s)(7168KiB/2626msec); 0 zone resets clat (msec): min=101, max=584, avg=318.23, stdev=161.35 lat (msec): min=101, max=584, avg=318.30, stdev=161.34 clat percentiles (msec): | 1.00th=[ 102], 5.00th=[ 102], 10.00th=[ 102], 20.00th=[ 197], | 30.00th=[ 213], 40.00th=[ 213], 50.00th=[ 359], 60.00th=[ 376], | 70.00th=[ 376], 80.00th=[ 405], 90.00th=[ 584], 95.00th=[ 584], | 99.00th=[ 584], 99.50th=[ 584], 99.90th=[ 584], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 1910, max= 4000, per=29.83%, avg=2371.40, stdev=911.93, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 lat (usec) : 750=25.00% lat (msec) : 4=8.33%, 250=25.00%, 500=33.33%, 750=8.33% cpu : usr=0.00%, sys=16.61%, ctx=206, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5679KiB/s (5816kB/s), 568KiB/s-2727KiB/s (582kB/s-2792kB/s), io=20.0MiB (20.0MB), run=2626-3606msec WRITE: bw=7951KiB/s (8142kB/s), 1948KiB/s-2840KiB/s (1994kB/s-2908kB/s), io=28.0MiB (29.4MB), run=2626-3606msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148652: Sun Nov 16 21:05:39 2025 read: IOPS=2, BW=2148KiB/s (2200kB/s)(7168KiB/3337msec) clat (msec): min=45, max=994, avg=276.23, stdev=332.07 lat (msec): min=45, max=994, avg=276.23, stdev=332.07 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 46], 10.00th=[ 46], 20.00th=[ 63], | 30.00th=[ 112], 40.00th=[ 112], 50.00th=[ 153], 60.00th=[ 236], | 70.00th=[ 236], 80.00th=[ 334], 90.00th=[ 995], 95.00th=[ 995], | 99.00th=[ 995], 99.50th=[ 995], 99.90th=[ 995], 99.95th=[ 995], | 99.99th=[ 995] bw ( KiB/s): min= 1973, max= 3976, per=52.76%, avg=2909.00, stdev=1066.03, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 write: IOPS=1, BW=1534KiB/s (1571kB/s)(5120KiB/3337msec); 0 zone resets clat (msec): min=148, max=402, avg=242.69, stdev=102.00 lat (msec): min=148, max=402, avg=242.76, stdev=102.00 clat percentiles (msec): | 1.00th=[ 148], 5.00th=[ 148], 10.00th=[ 148], 20.00th=[ 148], | 30.00th=[ 176], 40.00th=[ 176], 50.00th=[ 207], 60.00th=[ 207], | 70.00th=[ 279], 80.00th=[ 279], 90.00th=[ 405], 95.00th=[ 405], | 99.00th=[ 405], 99.50th=[ 405], 99.90th=[ 405], 99.95th=[ 405], | 99.99th=[ 405] bw ( KiB/s): min= 2007, max= 6035, per=52.09%, avg=4021.00, stdev=2848.23, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 lat (msec) : 50=8.33%, 100=8.33%, 250=50.00%, 500=25.00%, 1000=8.33% cpu : usr=0.00%, sys=5.37%, ctx=70, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148653: Sun Nov 16 21:05:39 2025 read: IOPS=0, BW=551KiB/s (565kB/s)(2048KiB/3714msec) clat (msec): min=90, max=546, avg=318.70, stdev=322.47 lat (msec): min=90, max=546, avg=318.70, stdev=322.47 clat percentiles (msec): | 1.00th=[ 91], 5.00th=[ 91], 10.00th=[ 91], 20.00th=[ 91], | 30.00th=[ 91], 40.00th=[ 91], 50.00th=[ 91], 60.00th=[ 550], | 70.00th=[ 550], 80.00th=[ 550], 90.00th=[ 550], 95.00th=[ 550], | 99.00th=[ 550], 99.50th=[ 550], 99.90th=[ 550], 99.95th=[ 550], | 99.99th=[ 550] bw ( KiB/s): min= 2011, max= 2035, per=36.69%, avg=2023.00, stdev=16.97, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2757KiB/s (2823kB/s)(10.0MiB/3714msec); 0 zone resets clat (msec): min=157, max=666, avg=301.16, stdev=173.55 lat (msec): min=157, max=666, avg=301.24, stdev=173.56 clat percentiles (msec): | 1.00th=[ 159], 5.00th=[ 159], 10.00th=[ 159], 20.00th=[ 184], | 30.00th=[ 184], 40.00th=[ 207], 50.00th=[ 247], 60.00th=[ 247], | 70.00th=[ 266], 80.00th=[ 288], 90.00th=[ 567], 95.00th=[ 667], | 99.00th=[ 667], 99.50th=[ 667], 99.90th=[ 667], 99.95th=[ 667], | 99.99th=[ 667] bw ( KiB/s): min= 2007, max= 6095, per=46.03%, avg=3553.00, stdev=1689.95, samples=5 iops : min= 1, max= 5, avg= 2.60, stdev= 1.67, samples=5 lat (msec) : 100=8.33%, 250=50.00%, 500=16.67%, 750=25.00% cpu : usr=0.00%, sys=6.38%, ctx=136, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148654: Sun Nov 16 21:05:39 2025 read: IOPS=1, BW=1881KiB/s (1926kB/s)(6144KiB/3266msec) clat (msec): min=35, max=259, avg=114.12, stdev=82.77 lat (msec): min=35, max=259, avg=114.12, stdev=82.77 clat percentiles (msec): | 1.00th=[ 36], 5.00th=[ 36], 10.00th=[ 36], 20.00th=[ 62], | 30.00th=[ 62], 40.00th=[ 63], 50.00th=[ 63], 60.00th=[ 113], | 70.00th=[ 153], 80.00th=[ 153], 90.00th=[ 259], 95.00th=[ 259], | 99.00th=[ 259], 99.50th=[ 259], 99.90th=[ 259], 99.95th=[ 259], | 99.99th=[ 259] bw ( KiB/s): min= 1988, max= 6035, per=60.63%, avg=3343.33, stdev=2331.07, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 write: IOPS=1, BW=1881KiB/s (1926kB/s)(6144KiB/3266msec); 0 zone resets clat (msec): min=184, max=1012, avg=375.96, stdev=317.25 lat (msec): min=184, max=1012, avg=376.04, stdev=317.25 clat percentiles (msec): | 1.00th=[ 184], 5.00th=[ 184], 10.00th=[ 184], 20.00th=[ 192], | 30.00th=[ 192], 40.00th=[ 239], 50.00th=[ 239], 60.00th=[ 300], | 70.00th=[ 330], 80.00th=[ 330], 90.00th=[ 1011], 95.00th=[ 1011], | 99.00th=[ 1011], 99.50th=[ 1011], 99.90th=[ 1011], 99.95th=[ 1011], | 99.99th=[ 1011] bw ( KiB/s): min= 1988, max= 4071, per=32.64%, avg=2519.25, stdev=1034.55, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 50=8.33%, 100=16.67%, 250=41.67%, 500=25.00%, 2000=8.33% cpu : usr=0.00%, sys=6.55%, ctx=78, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148655: Sun Nov 16 21:05:39 2025 read: IOPS=1, BW=1410KiB/s (1444kB/s)(5120KiB/3631msec) clat (msec): min=42, max=203, avg=121.16, stdev=74.78 lat (msec): min=42, max=203, avg=121.16, stdev=74.78 clat percentiles (msec): | 1.00th=[ 43], 5.00th=[ 43], 10.00th=[ 43], 20.00th=[ 43], | 30.00th=[ 73], 40.00th=[ 73], 50.00th=[ 88], 60.00th=[ 88], | 70.00th=[ 199], 80.00th=[ 199], 90.00th=[ 203], 95.00th=[ 203], | 99.00th=[ 203], 99.50th=[ 203], 99.90th=[ 203], 99.95th=[ 203], | 99.99th=[ 203] bw ( KiB/s): min= 8047, max= 8047, per=100.00%, avg=8047.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=1, BW=1974KiB/s (2021kB/s)(7168KiB/3631msec); 0 zone resets clat (msec): min=141, max=926, avg=393.05, stdev=274.67 lat (msec): min=141, max=926, avg=393.13, stdev=274.67 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 144], | 30.00th=[ 215], 40.00th=[ 215], 50.00th=[ 388], 60.00th=[ 460], | 70.00th=[ 460], 80.00th=[ 477], 90.00th=[ 927], 95.00th=[ 927], | 99.00th=[ 927], 99.50th=[ 927], 99.90th=[ 927], 99.95th=[ 927], | 99.99th=[ 927] bw ( KiB/s): min= 1988, max= 6095, per=45.77%, avg=3533.25, stdev=1951.68, samples=4 iops : min= 1, max= 5, avg= 2.50, stdev= 1.91, samples=4 lat (msec) : 50=8.33%, 100=16.67%, 250=41.67%, 500=25.00%, 1000=8.33% cpu : usr=0.08%, sys=4.35%, ctx=75, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5514KiB/s (5647kB/s), 551KiB/s-2148KiB/s (565kB/s-2200kB/s), io=20.0MiB (20.0MB), run=3266-3714msec WRITE: bw=7720KiB/s (7905kB/s), 1534KiB/s-2757KiB/s (1571kB/s-2823kB/s), io=28.0MiB (29.4MB), run=3266-3714msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148663: Sun Nov 16 21:05:49 2025 read: IOPS=1, BW=5783KiB/s (5921kB/s)(12.0MiB/2125msec) clat (msec): min=596, max=726, avg=674.62, stdev=68.58 lat (msec): min=596, max=726, avg=674.62, stdev=68.58 clat percentiles (msec): | 1.00th=[ 600], 5.00th=[ 600], 10.00th=[ 600], 20.00th=[ 600], | 30.00th=[ 600], 40.00th=[ 701], 50.00th=[ 701], 60.00th=[ 701], | 70.00th=[ 726], 80.00th=[ 726], 90.00th=[ 726], 95.00th=[ 726], | 99.00th=[ 726], 99.50th=[ 726], 99.90th=[ 726], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 8159, max= 8192, per=100.00%, avg=8175.50, stdev=23.33, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=100.00% cpu : usr=0.00%, sys=14.55%, ctx=315, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148664: Sun Nov 16 21:05:49 2025 read: IOPS=0, BW=1704KiB/s (1745kB/s)(4096KiB/2404msec) clat (nsec): min=737088k, max=737088k, avg=737087839.00, stdev= 0.00 lat (nsec): min=737090k, max=737090k, avg=737090217.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 735], 5.00th=[ 735], 10.00th=[ 735], 20.00th=[ 735], | 30.00th=[ 735], 40.00th=[ 735], 50.00th=[ 735], 60.00th=[ 735], | 70.00th=[ 735], 80.00th=[ 735], 90.00th=[ 735], 95.00th=[ 735], | 99.00th=[ 735], 99.50th=[ 735], 99.90th=[ 735], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3408KiB/s (3489kB/s)(8192KiB/2404msec); 0 zone resets clat (msec): min=683, max=939, avg=811.58, stdev=180.72 lat (msec): min=684, max=939, avg=812.02, stdev=180.75 clat percentiles (msec): | 1.00th=[ 684], 5.00th=[ 684], 10.00th=[ 684], 20.00th=[ 684], | 30.00th=[ 684], 40.00th=[ 684], 50.00th=[ 684], 60.00th=[ 936], | 70.00th=[ 936], 80.00th=[ 936], 90.00th=[ 936], 95.00th=[ 936], | 99.00th=[ 936], 99.50th=[ 936], 99.90th=[ 936], 99.95th=[ 936], | 99.99th=[ 936] bw ( KiB/s): min= 7968, max= 7968, per=58.46%, avg=7968.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=66.67%, 1000=33.33% cpu : usr=0.00%, sys=6.91%, ctx=113, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148667: Sun Nov 16 21:05:49 2025 write: IOPS=1, BW=7425KiB/s (7603kB/s)(12.0MiB/1655msec); 0 zone resets clat (msec): min=406, max=660, avg=517.28, stdev=130.50 lat (msec): min=406, max=661, avg=517.58, stdev=130.49 clat percentiles (msec): | 1.00th=[ 405], 5.00th=[ 405], 10.00th=[ 405], 20.00th=[ 405], | 30.00th=[ 405], 40.00th=[ 485], 50.00th=[ 485], 60.00th=[ 485], | 70.00th=[ 659], 80.00th=[ 659], 90.00th=[ 659], 95.00th=[ 659], | 99.00th=[ 659], 99.50th=[ 659], 99.90th=[ 659], 99.95th=[ 659], | 99.99th=[ 659] bw ( KiB/s): min= 6131, max= 8192, per=52.54%, avg=7161.50, stdev=1457.35, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.00%, sys=7.32%, ctx=40, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148668: Sun Nov 16 21:05:49 2025 write: IOPS=1, BW=5774KiB/s (5913kB/s)(12.0MiB/2128msec); 0 zone resets clat (msec): min=247, max=1086, avg=677.50, stdev=419.81 lat (msec): min=247, max=1086, avg=677.79, stdev=419.83 clat percentiles (msec): | 1.00th=[ 247], 5.00th=[ 247], 10.00th=[ 247], 20.00th=[ 247], | 30.00th=[ 247], 40.00th=[ 701], 50.00th=[ 701], 60.00th=[ 701], | 70.00th=[ 1083], 80.00th=[ 1083], 90.00th=[ 1083], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 8159, max= 8159, per=59.86%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=5.74%, ctx=59, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6815KiB/s (6979kB/s), 1704KiB/s-5783KiB/s (1745kB/s-5921kB/s), io=16.0MiB (16.8MB), run=2125-2404msec WRITE: bw=13.3MiB/s (13.0MB/s), 3408KiB/s-7425KiB/s (3489kB/s-7603kB/s), io=32.0MiB (33.6MB), run=1655-2404msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148665: Sun Nov 16 21:05:50 2025 read: IOPS=1, BW=5053KiB/s (5174kB/s)(12.0MiB/2432msec) clat (msec): min=510, max=1008, avg=742.17, stdev=250.55 lat (msec): min=510, max=1008, avg=742.18, stdev=250.55 clat percentiles (msec): | 1.00th=[ 510], 5.00th=[ 510], 10.00th=[ 510], 20.00th=[ 510], | 30.00th=[ 510], 40.00th=[ 709], 50.00th=[ 709], 60.00th=[ 709], | 70.00th=[ 1011], 80.00th=[ 1011], 90.00th=[ 1011], 95.00th=[ 1011], | 99.00th=[ 1011], 99.50th=[ 1011], 99.90th=[ 1011], 99.95th=[ 1011], | 99.99th=[ 1011] bw ( KiB/s): min= 8094, max= 8192, per=100.00%, avg=8143.00, stdev=69.30, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=4.11%, ctx=48, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148666: Sun Nov 16 21:05:50 2025 read: IOPS=0, BW=1424KiB/s (1458kB/s)(4096KiB/2877msec) clat (nsec): min=743058k, max=743058k, avg=743058487.00, stdev= 0.00 lat (nsec): min=743061k, max=743061k, avg=743061447.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 743], 5.00th=[ 743], 10.00th=[ 743], 20.00th=[ 743], | 30.00th=[ 743], 40.00th=[ 743], 50.00th=[ 743], 60.00th=[ 743], | 70.00th=[ 743], 80.00th=[ 743], 90.00th=[ 743], 95.00th=[ 743], | 99.00th=[ 743], 99.50th=[ 743], 99.90th=[ 743], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 8094, max= 8094, per=100.00%, avg=8094.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2847KiB/s (2916kB/s)(8192KiB/2877msec); 0 zone resets clat (msec): min=554, max=1488, avg=1021.62, stdev=660.89 lat (msec): min=554, max=1489, avg=1021.96, stdev=661.00 clat percentiles (msec): | 1.00th=[ 558], 5.00th=[ 558], 10.00th=[ 558], 20.00th=[ 558], | 30.00th=[ 558], 40.00th=[ 558], 50.00th=[ 558], 60.00th=[ 1485], | 70.00th=[ 1485], 80.00th=[ 1485], 90.00th=[ 1485], 95.00th=[ 1485], | 99.00th=[ 1485], 99.50th=[ 1485], 99.90th=[ 1485], 99.95th=[ 1485], | 99.99th=[ 1485] bw ( KiB/s): min= 8000, max= 8000, per=74.01%, avg=8000.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=5.39%, ctx=65, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148669: Sun Nov 16 21:05:50 2025 write: IOPS=0, BW=4054KiB/s (4151kB/s)(12.0MiB/3031msec); 0 zone resets clat (msec): min=284, max=1744, avg=982.36, stdev=732.29 lat (msec): min=284, max=1745, avg=982.82, stdev=732.32 clat percentiles (msec): | 1.00th=[ 284], 5.00th=[ 284], 10.00th=[ 284], 20.00th=[ 284], | 30.00th=[ 284], 40.00th=[ 919], 50.00th=[ 919], 60.00th=[ 919], | 70.00th=[ 1737], 80.00th=[ 1737], 90.00th=[ 1737], 95.00th=[ 1737], | 99.00th=[ 1737], 99.50th=[ 1737], 99.90th=[ 1737], 99.95th=[ 1737], | 99.99th=[ 1737] bw ( KiB/s): min= 8078, max= 8078, per=74.73%, avg=8078.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=3.56%, ctx=49, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148670: Sun Nov 16 21:05:50 2025 write: IOPS=1, BW=6178KiB/s (6326kB/s)(12.0MiB/1989msec); 0 zone resets clat (msec): min=317, max=1014, avg=599.84, stdev=367.03 lat (msec): min=317, max=1015, avg=600.31, stdev=367.02 clat percentiles (msec): | 1.00th=[ 317], 5.00th=[ 317], 10.00th=[ 317], 20.00th=[ 317], | 30.00th=[ 317], 40.00th=[ 468], 50.00th=[ 468], 60.00th=[ 468], | 70.00th=[ 1011], 80.00th=[ 1011], 90.00th=[ 1011], 95.00th=[ 1011], | 99.00th=[ 1011], 99.50th=[ 1011], 99.90th=[ 1011], 99.95th=[ 1011], | 99.99th=[ 1011] bw ( KiB/s): min=16189, max=16189, per=100.00%, avg=16189.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=6.54%, ctx=43, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5695KiB/s (5831kB/s), 1424KiB/s-5053KiB/s (1458kB/s-5174kB/s), io=16.0MiB (16.8MB), run=2432-2877msec WRITE: bw=10.6MiB/s (11.1MB/s), 2847KiB/s-6178KiB/s (2916kB/s-6326kB/s), io=32.0MiB (33.6MB), run=1989-3031msec