== sanity test 398b: DIO and buffer IO race ============== 22:28:54 (1763350134) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 5.15607 s, 9.8 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148762: Sun Nov 16 22:31:03 2025 read: IOPS=13, BW=53.6KiB/s (54.9kB/s)(5992KiB/111749msec) clat (usec): min=104, max=797328, avg=37259.04, stdev=36813.20 lat (usec): min=104, max=797330, avg=37260.50, stdev=36813.21 clat percentiles (usec): | 1.00th=[ 145], 5.00th=[ 184], 10.00th=[ 537], 20.00th=[ 17433], | 30.00th=[ 22414], 40.00th=[ 26870], 50.00th=[ 32375], 60.00th=[ 38011], | 70.00th=[ 45351], 80.00th=[ 53740], 90.00th=[ 68682], 95.00th=[ 80217], | 99.00th=[130548], 99.50th=[173016], 99.90th=[624952], 99.95th=[801113], | 99.99th=[801113] bw ( KiB/s): min= 7, max= 125, per=25.27%, avg=53.56, stdev=22.01, samples=218 iops : min= 1, max= 31, avg=12.75, stdev= 5.55, samples=218 write: IOPS=14, BW=56.3KiB/s (57.7kB/s)(6296KiB/111749msec); 0 zone resets clat (usec): min=852, max=356232, avg=35274.96, stdev=35897.02 lat (usec): min=853, max=356233, avg=35277.26, stdev=35897.06 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 1172], 10.00th=[ 1287], 20.00th=[ 2802], | 30.00th=[ 8717], 40.00th=[ 18482], 50.00th=[ 26870], 60.00th=[ 36439], | 70.00th=[ 46924], 80.00th=[ 61080], 90.00th=[ 81265], 95.00th=[101188], | 99.00th=[147850], 99.50th=[170918], 99.90th=[354419], 99.95th=[354419], | 99.99th=[354419] bw ( KiB/s): min= 15, max= 119, per=25.36%, avg=56.30, stdev=21.29, samples=218 iops : min= 3, max= 29, avg=13.45, stdev= 5.35, samples=218 lat (usec) : 250=4.20%, 500=0.62%, 750=0.16%, 1000=0.23% lat (msec) : 2=9.05%, 4=2.47%, 10=5.34%, 20=11.17%, 50=40.82% lat (msec) : 100=22.46%, 250=3.29%, 500=0.13%, 750=0.03%, 1000=0.03% cpu : usr=0.10%, sys=6.75%, ctx=6454, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148763: Sun Nov 16 22:31:03 2025 read: IOPS=13, BW=52.5KiB/s (53.7kB/s)(5940KiB/113211msec) clat (usec): min=96, max=310879, avg=35835.95, stdev=28644.08 lat (usec): min=96, max=310880, avg=35837.31, stdev=28644.14 clat percentiles (usec): | 1.00th=[ 126], 5.00th=[ 174], 10.00th=[ 343], 20.00th=[ 17171], | 30.00th=[ 21890], 40.00th=[ 25560], 50.00th=[ 30540], 60.00th=[ 35914], | 70.00th=[ 42730], 80.00th=[ 52167], 90.00th=[ 68682], 95.00th=[ 81265], | 99.00th=[126354], 99.50th=[158335], 99.90th=[287310], 99.95th=[312476], | 99.99th=[312476] bw ( KiB/s): min= 7, max= 134, per=24.82%, avg=52.63, stdev=23.66, samples=220 iops : min= 1, max= 33, avg=12.51, stdev= 5.96, samples=220 write: IOPS=14, BW=56.1KiB/s (57.4kB/s)(6348KiB/113211msec); 0 zone resets clat (usec): min=948, max=770170, avg=37542.14, stdev=43424.42 lat (usec): min=949, max=770172, avg=37545.78, stdev=43425.10 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 1156], 10.00th=[ 1287], 20.00th=[ 2769], | 30.00th=[ 8356], 40.00th=[ 17957], 50.00th=[ 26870], 60.00th=[ 35390], | 70.00th=[ 47449], 80.00th=[ 65799], 90.00th=[ 89654], 95.00th=[111674], | 99.00th=[183501], 99.50th=[208667], 99.90th=[413139], 99.95th=[767558], | 99.99th=[767558] bw ( KiB/s): min= 7, max= 198, per=25.38%, avg=56.35, stdev=25.83, samples=220 iops : min= 1, max= 49, avg=13.45, stdev= 6.50, samples=220 lat (usec) : 100=0.03%, 250=4.36%, 500=0.62%, 750=0.03%, 1000=0.29% lat (msec) : 2=9.21%, 4=2.47%, 10=5.21%, 20=11.82%, 50=40.98% lat (msec) : 100=20.05%, 250=4.69%, 500=0.20%, 1000=0.03% cpu : usr=0.09%, sys=6.59%, ctx=6375, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148765: Sun Nov 16 22:31:03 2025 read: IOPS=13, BW=55.5KiB/s (56.8kB/s)(6028KiB/108673msec) clat (usec): min=108, max=708974, avg=34486.49, stdev=29982.51 lat (usec): min=108, max=708975, avg=34487.97, stdev=29982.51 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 174], 10.00th=[ 249], 20.00th=[ 16057], | 30.00th=[ 21365], 40.00th=[ 25560], 50.00th=[ 30278], 60.00th=[ 35390], | 70.00th=[ 42206], 80.00th=[ 51119], 90.00th=[ 65274], 95.00th=[ 77071], | 99.00th=[110625], 99.50th=[129500], 99.90th=[219153], 99.95th=[708838], | 99.99th=[708838] bw ( KiB/s): min= 7, max= 132, per=26.04%, avg=55.20, stdev=22.31, samples=213 iops : min= 1, max= 33, avg=13.16, stdev= 5.64, samples=213 write: IOPS=14, BW=57.6KiB/s (58.0kB/s)(6260KiB/108673msec); 0 zone resets clat (usec): min=874, max=331268, avg=35937.76, stdev=37155.44 lat (usec): min=875, max=331269, avg=35940.18, stdev=37155.37 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 1205], 10.00th=[ 1352], 20.00th=[ 4146], | 30.00th=[ 8848], 40.00th=[ 17957], 50.00th=[ 26608], 60.00th=[ 34866], | 70.00th=[ 46400], 80.00th=[ 60556], 90.00th=[ 83362], 95.00th=[105382], | 99.00th=[175113], 99.50th=[193987], 99.90th=[316670], 99.95th=[329253], | 99.99th=[329253] bw ( KiB/s): min= 7, max= 127, per=25.85%, avg=57.39, stdev=22.49, samples=213 iops : min= 1, max= 31, avg=13.72, stdev= 5.66, samples=213 lat (usec) : 250=4.92%, 500=0.59%, 750=0.10%, 1000=0.39% lat (msec) : 2=7.45%, 4=2.54%, 10=6.61%, 20=12.01%, 50=40.95% lat (msec) : 100=20.67%, 250=3.65%, 500=0.10%, 750=0.03% cpu : usr=0.12%, sys=6.91%, ctx=6297, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148768: Sun Nov 16 22:31:03 2025 read: IOPS=13, BW=54.6KiB/s (55.9kB/s)(6052KiB/110771msec) clat (usec): min=100, max=227542, avg=35387.78, stdev=25296.35 lat (usec): min=101, max=227543, avg=35391.32, stdev=25296.09 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 178], 10.00th=[ 383], 20.00th=[ 17695], | 30.00th=[ 22152], 40.00th=[ 26346], 50.00th=[ 31327], 60.00th=[ 36439], | 70.00th=[ 43779], 80.00th=[ 52691], 90.00th=[ 66847], 95.00th=[ 82314], | 99.00th=[110625], 99.50th=[128451], 99.90th=[212861], 99.95th=[227541], | 99.99th=[227541] bw ( KiB/s): min= 7, max= 133, per=25.87%, avg=54.85, stdev=22.54, samples=215 iops : min= 1, max= 33, avg=13.08, stdev= 5.67, samples=215 write: IOPS=14, BW=56.3KiB/s (57.6kB/s)(6236KiB/110771msec); 0 zone resets clat (usec): min=933, max=814183, avg=36427.21, stdev=43764.80 lat (usec): min=934, max=814185, avg=36431.90, stdev=43763.38 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 1172], 10.00th=[ 1287], 20.00th=[ 2769], | 30.00th=[ 7635], 40.00th=[ 16319], 50.00th=[ 26346], 60.00th=[ 36963], | 70.00th=[ 47449], 80.00th=[ 61604], 90.00th=[ 83362], 95.00th=[108528], | 99.00th=[168821], 99.50th=[196084], 99.90th=[566232], 99.95th=[817890], | 99.99th=[817890] bw ( KiB/s): min= 7, max= 118, per=25.32%, avg=56.21, stdev=21.84, samples=216 iops : min= 1, max= 29, avg=13.42, stdev= 5.53, samples=216 lat (usec) : 250=4.33%, 500=0.81%, 750=0.03%, 1000=0.36% lat (msec) : 2=8.89%, 4=3.12%, 10=5.40%, 20=11.52%, 50=40.66% lat (msec) : 100=20.70%, 250=4.07%, 500=0.03%, 750=0.03%, 1000=0.03% cpu : usr=0.11%, sys=6.59%, ctx=6297, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=212KiB/s (217kB/s), 52.5KiB/s-55.5KiB/s (53.7kB/s-56.8kB/s), io=23.4MiB (24.6MB), run=108673-113211msec WRITE: bw=222KiB/s (227kB/s), 56.1KiB/s-57.6KiB/s (57.4kB/s-58.0kB/s), io=24.6MiB (25.7MB), run=108673-113211msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148764: Sun Nov 16 22:33:33 2025 read: IOPS=5, BW=22.8KiB/s (23.4kB/s)(5992KiB/262362msec) clat (msec): min=5, max=813, avg=35.22, stdev=40.66 lat (msec): min=5, max=813, avg=35.22, stdev=40.66 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 25], 60.00th=[ 30], | 70.00th=[ 38], 80.00th=[ 51], 90.00th=[ 67], 95.00th=[ 86], | 99.00th=[ 159], 99.50th=[ 259], 99.90th=[ 575], 99.95th=[ 810], | 99.99th=[ 810] bw ( KiB/s): min= 7, max= 103, per=27.50%, avg=25.02, stdev=16.49, samples=463 iops : min= 1, max= 25, avg= 5.63, stdev= 4.17, samples=463 write: IOPS=5, BW=23.0KiB/s (24.6kB/s)(6296KiB/262362msec); 0 zone resets clat (msec): min=37, max=450, avg=132.92, stdev=57.85 lat (msec): min=37, max=450, avg=132.92, stdev=57.85 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 67], 10.00th=[ 77], 20.00th=[ 87], | 30.00th=[ 97], 40.00th=[ 107], 50.00th=[ 118], 60.00th=[ 132], | 70.00th=[ 150], 80.00th=[ 174], 90.00th=[ 211], 95.00th=[ 251], | 99.00th=[ 317], 99.50th=[ 355], 99.90th=[ 414], 99.95th=[ 451], | 99.99th=[ 451] bw ( KiB/s): min= 7, max= 55, per=24.81%, avg=23.57, stdev= 9.84, samples=515 iops : min= 1, max= 13, avg= 5.25, stdev= 2.52, samples=515 lat (msec) : 10=2.08%, 20=16.86%, 50=20.28%, 100=25.23%, 250=32.58% lat (msec) : 500=2.86%, 750=0.07%, 1000=0.03% cpu : usr=0.05%, sys=1.41%, ctx=5577, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148766: Sun Nov 16 22:33:33 2025 read: IOPS=5, BW=22.7KiB/s (23.2kB/s)(5940KiB/261760msec) clat (msec): min=5, max=691, avg=33.28, stdev=31.85 lat (msec): min=5, max=691, avg=33.28, stdev=31.85 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 24], 60.00th=[ 29], | 70.00th=[ 36], 80.00th=[ 47], 90.00th=[ 67], 95.00th=[ 84], | 99.00th=[ 134], 99.50th=[ 159], 99.90th=[ 292], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 7, max= 88, per=28.01%, avg=25.49, stdev=16.08, samples=450 iops : min= 1, max= 22, avg= 5.74, stdev= 4.07, samples=450 write: IOPS=6, BW=24.3KiB/s (24.8kB/s)(6348KiB/261760msec); 0 zone resets clat (msec): min=33, max=591, avg=133.55, stdev=61.20 lat (msec): min=33, max=591, avg=133.55, stdev=61.20 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 67], 10.00th=[ 74], 20.00th=[ 86], | 30.00th=[ 96], 40.00th=[ 108], 50.00th=[ 120], 60.00th=[ 132], | 70.00th=[ 148], 80.00th=[ 176], 90.00th=[ 215], 95.00th=[ 253], | 99.00th=[ 334], 99.50th=[ 388], 99.90th=[ 550], 99.95th=[ 592], | 99.99th=[ 592] bw ( KiB/s): min= 7, max= 47, per=25.11%, avg=23.86, stdev= 9.69, samples=512 iops : min= 1, max= 11, avg= 5.33, stdev= 2.46, samples=512 lat (msec) : 10=2.05%, 20=16.70%, 50=21.35%, 100=24.22%, 250=32.88% lat (msec) : 500=2.70%, 750=0.10% cpu : usr=0.05%, sys=1.41%, ctx=5547, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148767: Sun Nov 16 22:33:33 2025 read: IOPS=5, BW=22.9KiB/s (23.5kB/s)(6028KiB/262907msec) clat (msec): min=5, max=354, avg=35.51, stdev=31.88 lat (msec): min=5, max=354, avg=35.52, stdev=31.87 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 13], 20.00th=[ 16], | 30.00th=[ 18], 40.00th=[ 22], 50.00th=[ 26], 60.00th=[ 32], | 70.00th=[ 40], 80.00th=[ 51], 90.00th=[ 68], 95.00th=[ 88], | 99.00th=[ 155], 99.50th=[ 255], 99.90th=[ 321], 99.95th=[ 355], | 99.99th=[ 355] bw ( KiB/s): min= 7, max= 111, per=27.34%, avg=24.88, stdev=16.50, samples=468 iops : min= 1, max= 27, avg= 5.59, stdev= 4.15, samples=468 write: IOPS=5, BW=23.8KiB/s (24.4kB/s)(6260KiB/262907msec); 0 zone resets clat (msec): min=35, max=774, avg=133.52, stdev=62.21 lat (msec): min=35, max=774, avg=133.52, stdev=62.21 clat percentiles (msec): | 1.00th=[ 53], 5.00th=[ 67], 10.00th=[ 74], 20.00th=[ 87], | 30.00th=[ 100], 40.00th=[ 110], 50.00th=[ 121], 60.00th=[ 132], | 70.00th=[ 148], 80.00th=[ 171], 90.00th=[ 207], 95.00th=[ 243], | 99.00th=[ 347], 99.50th=[ 401], 99.90th=[ 642], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 7, max= 55, per=24.70%, avg=23.47, stdev= 9.78, samples=514 iops : min= 1, max= 13, avg= 5.23, stdev= 2.51, samples=514 lat (msec) : 10=2.12%, 20=15.92%, 50=21.19%, 100=23.83%, 250=34.28% lat (msec) : 500=2.54%, 750=0.10%, 1000=0.03% cpu : usr=0.06%, sys=1.42%, ctx=5666, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148769: Sun Nov 16 22:33:33 2025 read: IOPS=5, BW=23.2KiB/s (23.7kB/s)(6052KiB/261377msec) clat (msec): min=5, max=326, avg=33.73, stdev=29.38 lat (msec): min=5, max=326, avg=33.74, stdev=29.38 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 29], | 70.00th=[ 38], 80.00th=[ 50], 90.00th=[ 68], 95.00th=[ 87], | 99.00th=[ 148], 99.50th=[ 192], 99.90th=[ 247], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 7, max= 119, per=28.31%, avg=25.76, stdev=16.88, samples=454 iops : min= 1, max= 29, avg= 5.80, stdev= 4.27, samples=454 write: IOPS=5, BW=23.9KiB/s (24.4kB/s)(6236KiB/261377msec); 0 zone resets clat (msec): min=36, max=800, avg=134.65, stdev=63.36 lat (msec): min=36, max=800, avg=134.65, stdev=63.36 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 69], 10.00th=[ 77], 20.00th=[ 87], | 30.00th=[ 97], 40.00th=[ 108], 50.00th=[ 120], 60.00th=[ 133], | 70.00th=[ 150], 80.00th=[ 176], 90.00th=[ 213], 95.00th=[ 255], | 99.00th=[ 342], 99.50th=[ 418], 99.90th=[ 567], 99.95th=[ 802], | 99.99th=[ 802] bw ( KiB/s): min= 7, max= 47, per=24.60%, avg=23.37, stdev= 9.88, samples=514 iops : min= 1, max= 11, avg= 5.20, stdev= 2.52, samples=514 lat (msec) : 10=2.73%, 20=17.71%, 50=19.50%, 100=24.71%, 250=32.55% lat (msec) : 500=2.73%, 750=0.03%, 1000=0.03% cpu : usr=0.05%, sys=1.42%, ctx=5670, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=91.3KiB/s (93.5kB/s), 22.7KiB/s-23.2KiB/s (23.2kB/s-23.7kB/s), io=23.4MiB (24.6MB), run=261377-262907msec WRITE: bw=95.6KiB/s (97.9kB/s), 23.8KiB/s-24.3KiB/s (24.4kB/s-24.8kB/s), io=24.6MiB (25.7MB), run=261377-262907msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148827: Sun Nov 16 22:34:02 2025 read: IOPS=18, BW=302KiB/s (309kB/s)(5920KiB/19612msec) clat (usec): min=103, max=173009, avg=27237.15, stdev=23501.08 lat (usec): min=104, max=173009, avg=27238.90, stdev=23501.18 clat percentiles (usec): | 1.00th=[ 123], 5.00th=[ 161], 10.00th=[ 184], 20.00th=[ 297], | 30.00th=[ 16057], 40.00th=[ 21365], 50.00th=[ 25822], 60.00th=[ 30278], | 70.00th=[ 36439], 80.00th=[ 44827], 90.00th=[ 54264], 95.00th=[ 61080], | 99.00th=[104334], 99.50th=[160433], 99.90th=[173016], 99.95th=[173016], | 99.99th=[173016] bw ( KiB/s): min= 126, max= 614, per=26.25%, avg=301.13, stdev=112.83, samples=38 iops : min= 7, max= 38, avg=18.13, stdev= 7.15, samples=38 write: IOPS=20, BW=325KiB/s (332kB/s)(6368KiB/19612msec); 0 zone resets clat (usec): min=1332, max=137792, avg=23600.18, stdev=25196.69 lat (usec): min=1335, max=137795, avg=23602.54, stdev=25196.65 clat percentiles (usec): | 1.00th=[ 1532], 5.00th=[ 1680], 10.00th=[ 1991], 20.00th=[ 3458], | 30.00th=[ 5604], 40.00th=[ 8586], 50.00th=[ 13435], 60.00th=[ 20841], | 70.00th=[ 29492], 80.00th=[ 42206], 90.00th=[ 60031], 95.00th=[ 72877], | 99.00th=[108528], 99.50th=[119014], 99.90th=[137364], 99.95th=[137364], | 99.99th=[137364] bw ( KiB/s): min= 158, max= 638, per=27.06%, avg=326.63, stdev=112.33, samples=38 iops : min= 9, max= 39, avg=19.74, stdev= 7.11, samples=38 lat (usec) : 250=7.81%, 500=3.52%, 750=0.26% lat (msec) : 2=5.86%, 4=7.55%, 10=10.68%, 20=11.72%, 50=38.02% lat (msec) : 100=13.28%, 250=1.30% cpu : usr=0.12%, sys=10.79%, ctx=1568, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148828: Sun Nov 16 22:34:02 2025 read: IOPS=17, BW=275KiB/s (282kB/s)(5744KiB/20866msec) clat (usec): min=113, max=131778, avg=27055.72, stdev=22573.36 lat (usec): min=114, max=131779, avg=27057.27, stdev=22573.42 clat percentiles (usec): | 1.00th=[ 126], 5.00th=[ 169], 10.00th=[ 202], 20.00th=[ 334], | 30.00th=[ 15139], 40.00th=[ 21890], 50.00th=[ 25035], 60.00th=[ 31065], | 70.00th=[ 36963], 80.00th=[ 44303], 90.00th=[ 52167], 95.00th=[ 63177], | 99.00th=[111674], 99.50th=[129500], 99.90th=[131597], 99.95th=[131597], | 99.99th=[131597] bw ( KiB/s): min= 26, max= 535, per=24.07%, avg=276.12, stdev=116.79, samples=41 iops : min= 1, max= 33, avg=16.59, stdev= 7.40, samples=41 write: IOPS=19, BW=314KiB/s (321kB/s)(6544KiB/20866msec); 0 zone resets clat (usec): min=1276, max=262821, avg=26870.16, stdev=30044.27 lat (usec): min=1277, max=262824, avg=26872.44, stdev=30044.39 clat percentiles (usec): | 1.00th=[ 1450], 5.00th=[ 1729], 10.00th=[ 1958], 20.00th=[ 3425], | 30.00th=[ 6587], 40.00th=[ 10552], 50.00th=[ 17171], 60.00th=[ 26608], | 70.00th=[ 35390], 80.00th=[ 46924], 90.00th=[ 61080], 95.00th=[ 76022], | 99.00th=[145753], 99.50th=[179307], 99.90th=[263193], 99.95th=[263193], | 99.99th=[263193] bw ( KiB/s): min= 95, max= 542, per=25.98%, avg=313.56, stdev=115.80, samples=41 iops : min= 5, max= 33, avg=18.90, stdev= 7.34, samples=41 lat (usec) : 250=7.55%, 500=2.99%, 750=0.52%, 1000=0.13% lat (msec) : 2=6.38%, 4=6.38%, 10=8.85%, 20=13.02%, 50=39.45% lat (msec) : 100=12.89%, 250=1.69%, 500=0.13% cpu : usr=0.11%, sys=9.82%, ctx=1494, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148829: Sun Nov 16 22:34:02 2025 read: IOPS=19, BW=306KiB/s (313kB/s)(6256KiB/20456msec) clat (usec): min=110, max=115939, avg=27529.63, stdev=21498.32 lat (usec): min=111, max=115940, avg=27531.14, stdev=21498.22 clat percentiles (usec): | 1.00th=[ 125], 5.00th=[ 176], 10.00th=[ 212], 20.00th=[ 2737], | 30.00th=[ 16909], 40.00th=[ 20579], 50.00th=[ 25560], 60.00th=[ 30802], | 70.00th=[ 36439], 80.00th=[ 43254], 90.00th=[ 53216], 95.00th=[ 64750], | 99.00th=[ 95945], 99.50th=[107480], 99.90th=[115868], 99.95th=[115868], | 99.99th=[115868] bw ( KiB/s): min= 126, max= 548, per=26.81%, avg=307.50, stdev=79.53, samples=40 iops : min= 7, max= 34, avg=18.55, stdev= 5.09, samples=40 write: IOPS=18, BW=295KiB/s (302kB/s)(6032KiB/20456msec); 0 zone resets clat (usec): min=1320, max=212506, avg=25530.37, stdev=27037.87 lat (usec): min=1322, max=212509, avg=25544.23, stdev=27034.52 clat percentiles (usec): | 1.00th=[ 1483], 5.00th=[ 1778], 10.00th=[ 1991], 20.00th=[ 3949], | 30.00th=[ 6587], 40.00th=[ 10421], 50.00th=[ 15795], 60.00th=[ 23462], | 70.00th=[ 32375], 80.00th=[ 46924], 90.00th=[ 60031], 95.00th=[ 77071], | 99.00th=[109577], 99.50th=[141558], 99.90th=[212861], 99.95th=[212861], | 99.99th=[212861] bw ( KiB/s): min= 94, max= 510, per=24.33%, avg=293.70, stdev=104.93, samples=40 iops : min= 5, max= 31, avg=17.68, stdev= 6.60, samples=40 lat (usec) : 250=7.03%, 500=2.73%, 1000=0.26% lat (msec) : 2=5.21%, 4=6.12%, 10=8.98%, 20=16.02%, 50=38.28% lat (msec) : 100=14.19%, 250=1.17% cpu : usr=0.18%, sys=10.15%, ctx=1527, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148830: Sun Nov 16 22:34:02 2025 read: IOPS=18, BW=304KiB/s (311kB/s)(6032KiB/19849msec) clat (usec): min=106, max=192086, avg=26663.81, stdev=24359.80 lat (usec): min=107, max=192087, avg=26665.15, stdev=24359.80 clat percentiles (usec): | 1.00th=[ 116], 5.00th=[ 157], 10.00th=[ 176], 20.00th=[ 247], | 30.00th=[ 12256], 40.00th=[ 19268], 50.00th=[ 25297], 60.00th=[ 30540], | 70.00th=[ 36963], 80.00th=[ 44827], 90.00th=[ 56361], 95.00th=[ 66323], | 99.00th=[100140], 99.50th=[158335], 99.90th=[191890], 99.95th=[191890], | 99.99th=[191890] bw ( KiB/s): min= 94, max= 540, per=27.22%, avg=312.24, stdev=112.97, samples=38 iops : min= 5, max= 33, avg=18.84, stdev= 7.15, samples=38 write: IOPS=19, BW=315KiB/s (323kB/s)(6256KiB/19849msec); 0 zone resets clat (usec): min=1341, max=192420, avg=24533.39, stdev=28734.71 lat (usec): min=1342, max=192423, avg=24538.56, stdev=28733.48 clat percentiles (usec): | 1.00th=[ 1401], 5.00th=[ 1713], 10.00th=[ 1926], 20.00th=[ 3392], | 30.00th=[ 5538], 40.00th=[ 8225], 50.00th=[ 14746], 60.00th=[ 21627], | 70.00th=[ 30016], 80.00th=[ 40633], 90.00th=[ 57934], 95.00th=[ 76022], | 99.00th=[141558], 99.50th=[162530], 99.90th=[191890], 99.95th=[191890], | 99.99th=[191890] bw ( KiB/s): min= 63, max= 692, per=26.20%, avg=316.21, stdev=131.08, samples=39 iops : min= 3, max= 43, avg=19.10, stdev= 8.26, samples=39 lat (usec) : 250=10.03%, 500=2.60%, 750=0.39%, 1000=0.13% lat (msec) : 2=5.86%, 4=7.03%, 10=10.55%, 20=13.28%, 50=35.81% lat (msec) : 100=12.11%, 250=2.21% cpu : usr=0.12%, sys=10.11%, ctx=1445, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1148KiB/s (1175kB/s), 275KiB/s-306KiB/s (282kB/s-313kB/s), io=23.4MiB (24.5MB), run=19612-20866msec WRITE: bw=1208KiB/s (1237kB/s), 295KiB/s-325KiB/s (302kB/s-332kB/s), io=24.6MiB (25.8MB), run=19612-20866msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148822: Sun Nov 16 22:34:39 2025 read: IOPS=6, BW=103KiB/s (106kB/s)(5920KiB/57393msec) clat (msec): min=6, max=162, avg=28.57, stdev=21.76 lat (msec): min=6, max=162, avg=28.58, stdev=21.76 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 19], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 29], 80.00th=[ 39], 90.00th=[ 58], 95.00th=[ 75], | 99.00th=[ 115], 99.50th=[ 136], 99.90th=[ 163], 99.95th=[ 163], | 99.99th=[ 163] bw ( KiB/s): min= 31, max= 382, per=26.45%, avg=107.91, stdev=62.98, samples=107 iops : min= 1, max= 23, avg= 6.04, stdev= 4.00, samples=107 write: IOPS=6, BW=111KiB/s (114kB/s)(6368KiB/57393msec); 0 zone resets clat (msec): min=47, max=298, avg=117.27, stdev=38.22 lat (msec): min=47, max=298, avg=117.27, stdev=38.22 clat percentiles (msec): | 1.00th=[ 58], 5.00th=[ 69], 10.00th=[ 77], 20.00th=[ 86], | 30.00th=[ 93], 40.00th=[ 103], 50.00th=[ 114], 60.00th=[ 123], | 70.00th=[ 132], 80.00th=[ 144], 90.00th=[ 161], 95.00th=[ 182], | 99.00th=[ 271], 99.50th=[ 284], 99.90th=[ 300], 99.95th=[ 300], | 99.99th=[ 300] bw ( KiB/s): min= 30, max= 192, per=25.72%, avg=110.32, stdev=33.41, samples=113 iops : min= 1, max= 12, avg= 6.19, stdev= 2.20, samples=113 lat (msec) : 10=2.21%, 20=19.92%, 50=20.18%, 100=24.61%, 250=32.42% lat (msec) : 500=0.65% cpu : usr=0.06%, sys=1.85%, ctx=1385, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148823: Sun Nov 16 22:34:39 2025 read: IOPS=6, BW=97.9KiB/s (100kB/s)(5744KiB/58652msec) clat (msec): min=8, max=262, avg=30.32, stdev=29.94 lat (msec): min=8, max=262, avg=30.34, stdev=29.94 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 24], | 70.00th=[ 28], 80.00th=[ 37], 90.00th=[ 63], 95.00th=[ 91], | 99.00th=[ 161], 99.50th=[ 192], 99.90th=[ 264], 99.95th=[ 264], | 99.99th=[ 264] bw ( KiB/s): min= 30, max= 352, per=25.31%, avg=103.27, stdev=65.84, samples=109 iops : min= 1, max= 22, avg= 5.73, stdev= 4.22, samples=109 write: IOPS=6, BW=112KiB/s (114kB/s)(6544KiB/58652msec); 0 zone resets clat (msec): min=46, max=272, avg=116.45, stdev=38.39 lat (msec): min=46, max=272, avg=116.45, stdev=38.39 clat percentiles (msec): | 1.00th=[ 55], 5.00th=[ 67], 10.00th=[ 73], 20.00th=[ 83], | 30.00th=[ 94], 40.00th=[ 103], 50.00th=[ 112], 60.00th=[ 122], | 70.00th=[ 128], 80.00th=[ 146], 90.00th=[ 167], 95.00th=[ 188], | 99.00th=[ 247], 99.50th=[ 255], 99.90th=[ 275], 99.95th=[ 275], | 99.99th=[ 275] bw ( KiB/s): min= 31, max= 223, per=25.89%, avg=111.07, stdev=37.11, samples=116 iops : min= 1, max= 13, avg= 6.24, stdev= 2.42, samples=116 lat (msec) : 10=1.82%, 20=21.35%, 50=17.58%, 100=23.57%, 250=35.03% lat (msec) : 500=0.65% cpu : usr=0.05%, sys=1.88%, ctx=1407, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148824: Sun Nov 16 22:34:39 2025 read: IOPS=6, BW=112KiB/s (115kB/s)(6256KiB/55901msec) clat (msec): min=6, max=304, avg=28.66, stdev=29.26 lat (msec): min=6, max=304, avg=28.66, stdev=29.26 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 34], 90.00th=[ 57], 95.00th=[ 78], | 99.00th=[ 167], 99.50th=[ 262], 99.90th=[ 305], 99.95th=[ 305], | 99.99th=[ 305] bw ( KiB/s): min= 31, max= 447, per=28.08%, avg=114.56, stdev=76.13, samples=108 iops : min= 1, max= 27, avg= 6.44, stdev= 4.81, samples=108 write: IOPS=6, BW=108KiB/s (110kB/s)(6032KiB/55901msec); 0 zone resets clat (msec): min=50, max=290, avg=118.42, stdev=39.38 lat (msec): min=50, max=290, avg=118.42, stdev=39.37 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 66], 10.00th=[ 74], 20.00th=[ 86], | 30.00th=[ 95], 40.00th=[ 103], 50.00th=[ 112], 60.00th=[ 123], | 70.00th=[ 133], 80.00th=[ 146], 90.00th=[ 171], 95.00th=[ 192], | 99.00th=[ 251], 99.50th=[ 284], 99.90th=[ 292], 99.95th=[ 292], | 99.99th=[ 292] bw ( KiB/s): min= 31, max= 222, per=25.23%, avg=108.22, stdev=34.16, samples=109 iops : min= 1, max= 13, avg= 6.05, stdev= 2.17, samples=109 lat (msec) : 10=1.95%, 20=24.48%, 50=18.49%, 100=22.14%, 250=32.29% lat (msec) : 500=0.65% cpu : usr=0.06%, sys=1.95%, ctx=1426, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148825: Sun Nov 16 22:34:39 2025 read: IOPS=6, BW=105KiB/s (107kB/s)(6032KiB/57663msec) clat (msec): min=7, max=305, avg=29.93, stdev=31.38 lat (msec): min=7, max=305, avg=29.93, stdev=31.38 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 19], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 28], 80.00th=[ 35], 90.00th=[ 56], 95.00th=[ 92], | 99.00th=[ 167], 99.50th=[ 218], 99.90th=[ 305], 99.95th=[ 305], | 99.99th=[ 305] bw ( KiB/s): min= 31, max= 349, per=26.79%, avg=109.29, stdev=71.95, samples=108 iops : min= 1, max= 21, avg= 6.14, stdev= 4.53, samples=108 write: IOPS=6, BW=108KiB/s (111kB/s)(6256KiB/57663msec); 0 zone resets clat (msec): min=48, max=252, avg=118.03, stdev=38.18 lat (msec): min=48, max=252, avg=118.04, stdev=38.18 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 67], 10.00th=[ 74], 20.00th=[ 86], | 30.00th=[ 94], 40.00th=[ 104], 50.00th=[ 114], 60.00th=[ 123], | 70.00th=[ 131], 80.00th=[ 150], 90.00th=[ 167], 95.00th=[ 199], | 99.00th=[ 228], 99.50th=[ 239], 99.90th=[ 253], 99.95th=[ 253], | 99.99th=[ 253] bw ( KiB/s): min= 31, max= 192, per=25.22%, avg=108.21, stdev=35.49, samples=114 iops : min= 1, max= 12, avg= 6.06, stdev= 2.30, samples=114 lat (msec) : 10=2.21%, 20=23.18%, 50=17.71%, 100=22.40%, 250=34.24% lat (msec) : 500=0.26% cpu : usr=0.05%, sys=1.93%, ctx=1415, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=408KiB/s (418kB/s), 97.9KiB/s-112KiB/s (100kB/s-115kB/s), io=23.4MiB (24.5MB), run=55901-58652msec WRITE: bw=430KiB/s (440kB/s), 108KiB/s-112KiB/s (110kB/s-114kB/s), io=24.6MiB (25.8MB), run=55901-58652msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148848: Sun Nov 16 22:34:53 2025 read: IOPS=1, BW=1933KiB/s (1979kB/s)(7168KiB/3709msec) clat (msec): min=137, max=295, avg=224.13, stdev=63.82 lat (msec): min=137, max=295, avg=224.13, stdev=63.82 clat percentiles (msec): | 1.00th=[ 138], 5.00th=[ 138], 10.00th=[ 138], 20.00th=[ 157], | 30.00th=[ 188], 40.00th=[ 188], 50.00th=[ 249], 60.00th=[ 251], | 70.00th=[ 251], 80.00th=[ 292], 90.00th=[ 296], 95.00th=[ 296], | 99.00th=[ 296], 99.50th=[ 296], 99.90th=[ 296], 99.95th=[ 296], | 99.99th=[ 296] bw ( KiB/s): min= 1961, max= 4090, per=57.59%, avg=2529.25, stdev=1041.06, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=1, BW=1380KiB/s (1414kB/s)(5120KiB/3709msec); 0 zone resets clat (msec): min=131, max=956, avg=406.53, stdev=328.60 lat (msec): min=131, max=956, avg=406.61, stdev=328.61 clat percentiles (msec): | 1.00th=[ 132], 5.00th=[ 132], 10.00th=[ 132], 20.00th=[ 132], | 30.00th=[ 180], 40.00th=[ 180], 50.00th=[ 363], 60.00th=[ 363], | 70.00th=[ 405], 80.00th=[ 405], 90.00th=[ 961], 95.00th=[ 961], | 99.00th=[ 961], 99.50th=[ 961], 99.90th=[ 961], 99.95th=[ 961], | 99.99th=[ 961] bw ( KiB/s): min= 1961, max= 2043, per=32.80%, avg=2017.50, stdev=38.28, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 lat (msec) : 250=50.00%, 500=41.67%, 1000=8.33% cpu : usr=0.00%, sys=18.61%, ctx=228, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148849: Sun Nov 16 22:34:53 2025 read: IOPS=0, BW=439KiB/s (450kB/s)(2048KiB/4662msec) clat (msec): min=221, max=1991, avg=1106.22, stdev=1251.29 lat (msec): min=221, max=1991, avg=1106.22, stdev=1251.29 clat percentiles (msec): | 1.00th=[ 222], 5.00th=[ 222], 10.00th=[ 222], 20.00th=[ 222], | 30.00th=[ 222], 40.00th=[ 222], 50.00th=[ 222], 60.00th=[ 1989], | 70.00th=[ 1989], 80.00th=[ 1989], 90.00th=[ 1989], 95.00th=[ 1989], | 99.00th=[ 1989], 99.50th=[ 1989], 99.90th=[ 1989], 99.95th=[ 1989], | 99.99th=[ 1989] bw ( KiB/s): min= 2000, max= 2043, per=46.03%, avg=2021.50, stdev=30.41, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2196KiB/s (2249kB/s)(10.0MiB/4662msec); 0 zone resets clat (msec): min=101, max=435, avg=229.41, stdev=102.67 lat (msec): min=101, max=435, avg=229.65, stdev=102.64 clat percentiles (msec): | 1.00th=[ 102], 5.00th=[ 102], 10.00th=[ 102], 20.00th=[ 159], | 30.00th=[ 174], 40.00th=[ 176], 50.00th=[ 178], 60.00th=[ 207], | 70.00th=[ 209], 80.00th=[ 296], 90.00th=[ 363], 95.00th=[ 435], | 99.00th=[ 435], 99.50th=[ 435], 99.90th=[ 435], 99.95th=[ 435], | 99.99th=[ 435] bw ( KiB/s): min= 2000, max= 5885, per=64.48%, avg=3965.75, stdev=1587.19, samples=4 iops : min= 1, max= 5, avg= 3.00, stdev= 1.63, samples=4 lat (msec) : 250=66.67%, 500=25.00%, 2000=8.33% cpu : usr=0.11%, sys=15.55%, ctx=307, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148850: Sun Nov 16 22:34:53 2025 read: IOPS=2, BW=2394KiB/s (2452kB/s)(6144KiB/2566msec) clat (usec): min=664, max=429208, avg=123293.06, stdev=167055.49 lat (usec): min=666, max=429209, avg=123294.56, stdev=167054.90 clat percentiles (usec): | 1.00th=[ 668], 5.00th=[ 668], 10.00th=[ 668], 20.00th=[ 668], | 30.00th=[ 668], 40.00th=[ 4948], 50.00th=[ 4948], 60.00th=[141558], | 70.00th=[162530], 80.00th=[162530], 90.00th=[429917], 95.00th=[429917], | 99.00th=[429917], 99.50th=[429917], 99.90th=[429917], 99.95th=[429917], | 99.99th=[429917] bw ( KiB/s): min= 2043, max= 6119, per=92.92%, avg=4081.00, stdev=2882.17, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=2, BW=2394KiB/s (2452kB/s)(6144KiB/2566msec); 0 zone resets clat (msec): min=160, max=691, avg=287.51, stdev=205.44 lat (msec): min=160, max=692, avg=287.59, stdev=205.45 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 161], | 30.00th=[ 161], 40.00th=[ 174], 50.00th=[ 174], 60.00th=[ 243], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 693], 95.00th=[ 693], | 99.00th=[ 693], 99.50th=[ 693], 99.90th=[ 693], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 2027, max= 2716, per=36.76%, avg=2260.67, stdev=394.38, samples=3 iops : min= 1, max= 2, avg= 1.33, stdev= 0.58, samples=3 lat (usec) : 750=16.67% lat (msec) : 10=8.33%, 250=50.00%, 500=16.67%, 750=8.33% cpu : usr=0.00%, sys=22.85%, ctx=104, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148851: Sun Nov 16 22:34:53 2025 read: IOPS=1, BW=1393KiB/s (1427kB/s)(5120KiB/3675msec) clat (msec): min=2, max=458, avg=181.80, stdev=189.76 lat (msec): min=2, max=458, avg=181.80, stdev=189.76 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 3], 20.00th=[ 3], | 30.00th=[ 9], 40.00th=[ 9], 50.00th=[ 186], 60.00th=[ 186], | 70.00th=[ 255], 80.00th=[ 255], 90.00th=[ 460], 95.00th=[ 460], | 99.00th=[ 460], 99.50th=[ 460], 99.90th=[ 460], 99.95th=[ 460], | 99.99th=[ 460] bw ( KiB/s): min= 2039, max= 6131, per=93.01%, avg=4085.00, stdev=2893.48, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=1, BW=1950KiB/s (1997kB/s)(7168KiB/3675msec); 0 zone resets clat (msec): min=130, max=770, avg=391.87, stdev=266.97 lat (msec): min=130, max=770, avg=391.99, stdev=266.93 clat percentiles (msec): | 1.00th=[ 131], 5.00th=[ 131], 10.00th=[ 131], 20.00th=[ 136], | 30.00th=[ 194], 40.00th=[ 194], 50.00th=[ 309], 60.00th=[ 523], | 70.00th=[ 523], 80.00th=[ 684], 90.00th=[ 768], 95.00th=[ 768], | 99.00th=[ 768], 99.50th=[ 768], 99.90th=[ 768], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 1380, max= 4055, per=38.35%, avg=2358.75, stdev=1168.43, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 4=8.33%, 10=8.33%, 250=33.33%, 500=25.00%, 750=16.67% lat (msec) : 1000=8.33% cpu : usr=0.11%, sys=17.31%, ctx=218, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4393KiB/s (4498kB/s), 439KiB/s-2394KiB/s (450kB/s-2452kB/s), io=20.0MiB (20.0MB), run=2566-4662msec WRITE: bw=6150KiB/s (6298kB/s), 1380KiB/s-2394KiB/s (1414kB/s-2452kB/s), io=28.0MiB (29.4MB), run=2566-4662msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148852: Sun Nov 16 22:34:53 2025 read: IOPS=1, BW=1587KiB/s (1625kB/s)(7168KiB/4518msec) clat (msec): min=70, max=917, avg=255.80, stdev=306.63 lat (msec): min=70, max=917, avg=255.81, stdev=306.63 clat percentiles (msec): | 1.00th=[ 71], 5.00th=[ 71], 10.00th=[ 71], 20.00th=[ 79], | 30.00th=[ 113], 40.00th=[ 113], 50.00th=[ 115], 60.00th=[ 150], | 70.00th=[ 150], 80.00th=[ 351], 90.00th=[ 919], 95.00th=[ 919], | 99.00th=[ 919], 99.50th=[ 919], 99.90th=[ 919], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 2035, max= 4079, per=86.09%, avg=3376.33, stdev=1162.07, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=1, BW=1133KiB/s (1160kB/s)(5120KiB/4518msec); 0 zone resets clat (msec): min=227, max=956, avg=483.53, stdev=334.22 lat (msec): min=227, max=956, avg=483.63, stdev=334.21 clat percentiles (msec): | 1.00th=[ 228], 5.00th=[ 228], 10.00th=[ 228], 20.00th=[ 228], | 30.00th=[ 230], 40.00th=[ 230], 50.00th=[ 288], 60.00th=[ 288], | 70.00th=[ 718], 80.00th=[ 718], 90.00th=[ 953], 95.00th=[ 953], | 99.00th=[ 953], 99.50th=[ 953], 99.90th=[ 953], 99.95th=[ 953], | 99.99th=[ 953] bw ( KiB/s): min= 2011, max= 4055, per=55.24%, avg=3033.00, stdev=1445.33, samples=2 iops : min= 1, max= 3, avg= 2.00, stdev= 1.41, samples=2 lat (msec) : 100=16.67%, 250=41.67%, 500=16.67%, 750=8.33%, 1000=16.67% cpu : usr=0.04%, sys=5.58%, ctx=82, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148853: Sun Nov 16 22:34:53 2025 read: IOPS=0, BW=392KiB/s (402kB/s)(2048KiB/5221msec) clat (msec): min=92, max=2132, avg=1112.64, stdev=1442.66 lat (msec): min=92, max=2132, avg=1112.64, stdev=1442.66 clat percentiles (msec): | 1.00th=[ 93], 5.00th=[ 93], 10.00th=[ 93], 20.00th=[ 93], | 30.00th=[ 93], 40.00th=[ 93], 50.00th=[ 93], 60.00th=[ 2140], | 70.00th=[ 2140], 80.00th=[ 2140], 90.00th=[ 2140], 95.00th=[ 2140], | 99.00th=[ 2140], 99.50th=[ 2140], 99.90th=[ 2140], 99.95th=[ 2140], | 99.99th=[ 2140] bw ( KiB/s): min= 2011, max= 2039, per=51.63%, avg=2025.00, stdev=19.80, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1961KiB/s (2008kB/s)(10.0MiB/5221msec); 0 zone resets clat (msec): min=81, max=914, avg=266.87, stdev=239.26 lat (msec): min=81, max=914, avg=266.96, stdev=239.26 clat percentiles (msec): | 1.00th=[ 82], 5.00th=[ 82], 10.00th=[ 82], 20.00th=[ 109], | 30.00th=[ 155], 40.00th=[ 192], 50.00th=[ 197], 60.00th=[ 199], | 70.00th=[ 222], 80.00th=[ 257], 90.00th=[ 342], 95.00th=[ 919], | 99.00th=[ 919], 99.50th=[ 919], 99.90th=[ 919], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 1906, max= 6119, per=73.24%, avg=4021.75, stdev=1720.09, samples=4 iops : min= 1, max= 5, avg= 3.00, stdev= 1.63, samples=4 lat (msec) : 100=16.67%, 250=50.00%, 500=16.67%, 1000=8.33%, >=2000=8.33% cpu : usr=0.13%, sys=3.98%, ctx=75, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148854: Sun Nov 16 22:34:53 2025 read: IOPS=1, BW=1198KiB/s (1227kB/s)(6144KiB/5127msec) clat (msec): min=72, max=206, avg=122.82, stdev=51.09 lat (msec): min=72, max=206, avg=122.82, stdev=51.09 clat percentiles (msec): | 1.00th=[ 73], 5.00th=[ 73], 10.00th=[ 73], 20.00th=[ 89], | 30.00th=[ 89], 40.00th=[ 101], 50.00th=[ 101], 60.00th=[ 108], | 70.00th=[ 163], 80.00th=[ 163], 90.00th=[ 207], 95.00th=[ 207], | 99.00th=[ 207], 99.50th=[ 207], 99.90th=[ 207], 99.95th=[ 207], | 99.99th=[ 207] bw ( KiB/s): min= 2039, max= 4055, per=85.85%, avg=3367.00, stdev=1150.33, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=1, BW=1198KiB/s (1227kB/s)(6144KiB/5127msec); 0 zone resets clat (msec): min=100, max=2921, avg=706.33, stdev=1094.68 lat (msec): min=100, max=2921, avg=706.44, stdev=1094.68 clat percentiles (msec): | 1.00th=[ 101], 5.00th=[ 101], 10.00th=[ 101], 20.00th=[ 121], | 30.00th=[ 121], 40.00th=[ 241], 50.00th=[ 241], 60.00th=[ 422], | 70.00th=[ 435], 80.00th=[ 435], 90.00th=[ 2937], 95.00th=[ 2937], | 99.00th=[ 2937], 99.50th=[ 2937], 99.90th=[ 2937], 99.95th=[ 2937], | 99.99th=[ 2937] bw ( KiB/s): min= 2027, max= 4007, per=54.12%, avg=2971.50, stdev=1086.59, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 lat (msec) : 100=25.00%, 250=50.00%, 500=16.67%, >=2000=8.33% cpu : usr=0.06%, sys=4.35%, ctx=92, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148855: Sun Nov 16 22:34:53 2025 read: IOPS=0, BW=1007KiB/s (1032kB/s)(5120KiB/5082msec) clat (msec): min=98, max=404, avg=223.14, stdev=126.91 lat (msec): min=98, max=404, avg=223.14, stdev=126.91 clat percentiles (msec): | 1.00th=[ 100], 5.00th=[ 100], 10.00th=[ 100], 20.00th=[ 100], | 30.00th=[ 122], 40.00th=[ 122], 50.00th=[ 197], 60.00th=[ 197], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 405], 95.00th=[ 405], | 99.00th=[ 405], 99.50th=[ 405], 99.90th=[ 405], 99.95th=[ 405], | 99.99th=[ 405] bw ( KiB/s): min= 4055, max= 4079, per=100.00%, avg=4067.00, stdev=16.97, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1410KiB/s (1444kB/s)(7168KiB/5082msec); 0 zone resets clat (msec): min=159, max=1245, avg=559.05, stdev=405.41 lat (msec): min=159, max=1245, avg=559.17, stdev=405.41 clat percentiles (msec): | 1.00th=[ 159], 5.00th=[ 159], 10.00th=[ 159], 20.00th=[ 199], | 30.00th=[ 245], 40.00th=[ 245], 50.00th=[ 550], 60.00th=[ 600], | 70.00th=[ 600], 80.00th=[ 911], 90.00th=[ 1250], 95.00th=[ 1250], | 99.00th=[ 1250], 99.50th=[ 1250], 99.90th=[ 1250], 99.95th=[ 1250], | 99.99th=[ 1250] bw ( KiB/s): min= 1906, max= 2048, per=36.65%, avg=2012.60, stdev=59.89, samples=5 iops : min= 1, max= 2, avg= 1.20, stdev= 0.45, samples=5 lat (msec) : 100=8.33%, 250=41.67%, 500=16.67%, 750=16.67%, 1000=8.33% lat (msec) : 2000=8.33% cpu : usr=0.08%, sys=3.72%, ctx=77, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3923KiB/s (4017kB/s), 392KiB/s-1587KiB/s (402kB/s-1625kB/s), io=20.0MiB (20.0MB), run=4518-5221msec WRITE: bw=5492KiB/s (5623kB/s), 1133KiB/s-1961KiB/s (1160kB/s-2008kB/s), io=28.0MiB (29.4MB), run=4518-5221msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148864: Sun Nov 16 22:35:05 2025 read: IOPS=1, BW=4717KiB/s (4830kB/s)(12.0MiB/2605msec) clat (msec): min=682, max=897, avg=784.88, stdev=107.92 lat (msec): min=682, max=897, avg=784.89, stdev=107.91 clat percentiles (msec): | 1.00th=[ 684], 5.00th=[ 684], 10.00th=[ 684], 20.00th=[ 684], | 30.00th=[ 684], 40.00th=[ 776], 50.00th=[ 776], 60.00th=[ 776], | 70.00th=[ 894], 80.00th=[ 894], 90.00th=[ 894], 95.00th=[ 894], | 99.00th=[ 894], 99.50th=[ 894], 99.90th=[ 894], 99.95th=[ 894], | 99.99th=[ 894] bw ( KiB/s): min= 7953, max= 8031, per=100.00%, avg=7992.00, stdev=55.15, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=66.67% cpu : usr=0.00%, sys=14.06%, ctx=266, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148866: Sun Nov 16 22:35:05 2025 read: IOPS=0, BW=1357KiB/s (1390kB/s)(4096KiB/3018msec) clat (nsec): min=673710k, max=673710k, avg=673710146.00, stdev= 0.00 lat (nsec): min=673713k, max=673713k, avg=673713337.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 676], 5.00th=[ 676], 10.00th=[ 676], 20.00th=[ 676], | 30.00th=[ 676], 40.00th=[ 676], 50.00th=[ 676], 60.00th=[ 676], | 70.00th=[ 676], 80.00th=[ 676], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 8143, max= 8143, per=100.00%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2714KiB/s (2780kB/s)(8192KiB/3018msec); 0 zone resets clat (msec): min=445, max=1705, avg=1075.63, stdev=890.47 lat (msec): min=446, max=1705, avg=1075.95, stdev=890.37 clat percentiles (msec): | 1.00th=[ 447], 5.00th=[ 447], 10.00th=[ 447], 20.00th=[ 447], | 30.00th=[ 447], 40.00th=[ 447], 50.00th=[ 447], 60.00th=[ 1703], | 70.00th=[ 1703], 80.00th=[ 1703], 90.00th=[ 1703], 95.00th=[ 1703], | 99.00th=[ 1703], 99.50th=[ 1703], 99.90th=[ 1703], 99.95th=[ 1703], | 99.99th=[ 1703] bw ( KiB/s): min= 7861, max= 7861, per=72.40%, avg=7861.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.03%, sys=6.17%, ctx=170, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148869: Sun Nov 16 22:35:05 2025 write: IOPS=2, BW=11.0MiB/s (11.5MB/s)(12.0MiB/1090msec); 0 zone resets clat (msec): min=250, max=454, avg=322.85, stdev=114.07 lat (msec): min=251, max=454, avg=323.27, stdev=114.07 clat percentiles (msec): | 1.00th=[ 251], 5.00th=[ 251], 10.00th=[ 251], 20.00th=[ 251], | 30.00th=[ 251], 40.00th=[ 264], 50.00th=[ 264], 60.00th=[ 264], | 70.00th=[ 456], 80.00th=[ 456], 90.00th=[ 456], 95.00th=[ 456], | 99.00th=[ 456], 99.50th=[ 456], 99.90th=[ 456], 99.95th=[ 456], | 99.99th=[ 456] bw ( KiB/s): min= 7393, max= 8031, per=71.03%, avg=7712.00, stdev=451.13, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=100.00% cpu : usr=0.28%, sys=11.57%, ctx=39, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148870: Sun Nov 16 22:35:05 2025 write: IOPS=1, BW=6403KiB/s (6557kB/s)(12.0MiB/1919msec); 0 zone resets clat (msec): min=409, max=827, avg=571.74, stdev=223.84 lat (msec): min=409, max=827, avg=572.19, stdev=223.82 clat percentiles (msec): | 1.00th=[ 409], 5.00th=[ 409], 10.00th=[ 409], 20.00th=[ 409], | 30.00th=[ 409], 40.00th=[ 481], 50.00th=[ 481], 60.00th=[ 481], | 70.00th=[ 827], 80.00th=[ 827], 90.00th=[ 827], 95.00th=[ 827], | 99.00th=[ 827], 99.50th=[ 827], 99.90th=[ 827], 99.95th=[ 827], | 99.99th=[ 827] bw ( KiB/s): min= 5649, max= 8143, per=63.52%, avg=6896.00, stdev=1763.52, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, 1000=33.33% cpu : usr=0.10%, sys=5.63%, ctx=45, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5429KiB/s (5559kB/s), 1357KiB/s-4717KiB/s (1390kB/s-4830kB/s), io=16.0MiB (16.8MB), run=2605-3018msec WRITE: bw=10.6MiB/s (11.1MB/s), 2714KiB/s-11.0MiB/s (2780kB/s-11.5MB/s), io=32.0MiB (33.6MB), run=1090-3018msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148863: Sun Nov 16 22:35:05 2025 read: IOPS=1, BW=4607KiB/s (4718kB/s)(12.0MiB/2667msec) clat (msec): min=772, max=884, avg=840.90, stdev=59.93 lat (msec): min=772, max=884, avg=840.90, stdev=59.93 clat percentiles (msec): | 1.00th=[ 776], 5.00th=[ 776], 10.00th=[ 776], 20.00th=[ 776], | 30.00th=[ 776], 40.00th=[ 869], 50.00th=[ 869], 60.00th=[ 869], | 70.00th=[ 885], 80.00th=[ 885], 90.00th=[ 885], 95.00th=[ 885], | 99.00th=[ 885], 99.50th=[ 885], 99.90th=[ 885], 99.95th=[ 885], | 99.99th=[ 885] bw ( KiB/s): min= 7787, max= 8062, per=100.00%, avg=7924.50, stdev=194.45, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=100.00% cpu : usr=0.00%, sys=3.23%, ctx=48, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148865: Sun Nov 16 22:35:05 2025 read: IOPS=0, BW=1533KiB/s (1570kB/s)(4096KiB/2672msec) clat (nsec): min=992890k, max=992890k, avg=992890466.00, stdev= 0.00 lat (nsec): min=992894k, max=992894k, avg=992893793.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 995], 5.00th=[ 995], 10.00th=[ 995], 20.00th=[ 995], | 30.00th=[ 995], 40.00th=[ 995], 50.00th=[ 995], 60.00th=[ 995], | 70.00th=[ 995], 80.00th=[ 995], 90.00th=[ 995], 95.00th=[ 995], | 99.00th=[ 995], 99.50th=[ 995], 99.90th=[ 995], 99.95th=[ 995], | 99.99th=[ 995] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3066KiB/s (3139kB/s)(8192KiB/2672msec); 0 zone resets clat (msec): min=473, max=981, avg=727.58, stdev=359.51 lat (msec): min=473, max=982, avg=727.97, stdev=359.56 clat percentiles (msec): | 1.00th=[ 472], 5.00th=[ 472], 10.00th=[ 472], 20.00th=[ 472], | 30.00th=[ 472], 40.00th=[ 472], 50.00th=[ 472], 60.00th=[ 986], | 70.00th=[ 986], 80.00th=[ 986], 90.00th=[ 986], 95.00th=[ 986], | 99.00th=[ 986], 99.50th=[ 986], 99.90th=[ 986], 99.95th=[ 986], | 99.99th=[ 986] bw ( KiB/s): min= 8062, max= 8062, per=78.29%, avg=8062.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=66.67% cpu : usr=0.00%, sys=4.53%, ctx=43, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148867: Sun Nov 16 22:35:05 2025 write: IOPS=0, BW=3862KiB/s (3954kB/s)(12.0MiB/3182msec); 0 zone resets clat (msec): min=247, max=1951, avg=965.55, stdev=883.02 lat (msec): min=247, max=1951, avg=966.57, stdev=882.78 clat percentiles (msec): | 1.00th=[ 249], 5.00th=[ 249], 10.00th=[ 249], 20.00th=[ 249], | 30.00th=[ 249], 40.00th=[ 701], 50.00th=[ 701], 60.00th=[ 701], | 70.00th=[ 1955], 80.00th=[ 1955], 90.00th=[ 1955], 95.00th=[ 1955], | 99.00th=[ 1955], 99.50th=[ 1955], 99.90th=[ 1955], 99.95th=[ 1955], | 99.99th=[ 1955] bw ( KiB/s): min= 8175, max= 8175, per=79.39%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=5.91%, ctx=58, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148868: Sun Nov 16 22:35:05 2025 write: IOPS=1, BW=7095KiB/s (7265kB/s)(12.0MiB/1732msec); 0 zone resets clat (msec): min=345, max=850, avg=570.78, stdev=256.96 lat (msec): min=345, max=851, avg=571.39, stdev=257.23 clat percentiles (msec): | 1.00th=[ 347], 5.00th=[ 347], 10.00th=[ 347], 20.00th=[ 347], | 30.00th=[ 347], 40.00th=[ 518], 50.00th=[ 518], 60.00th=[ 518], | 70.00th=[ 852], 80.00th=[ 852], 90.00th=[ 852], 95.00th=[ 852], | 99.00th=[ 852], 99.50th=[ 852], 99.90th=[ 852], 99.95th=[ 852], | 99.99th=[ 852] bw ( KiB/s): min= 7433, max= 7787, per=73.91%, avg=7610.00, stdev=250.32, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=4.56%, ctx=42, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6132KiB/s (6279kB/s), 1533KiB/s-4607KiB/s (1570kB/s-4718kB/s), io=16.0MiB (16.8MB), run=2667-2672msec WRITE: bw=10.1MiB/s (10.5MB/s), 3066KiB/s-7095KiB/s (3139kB/s-7265kB/s), io=32.0MiB (33.6MB), run=1732-3182msec