== sanity test 398b: DIO and buffer IO race ============== 21:01:01 (1763085661) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.09181 s, 16.3 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=183448: Thu Nov 13 21:02:13 2025 read: IOPS=24, BW=96.4KiB/s (98.7kB/s)(5992KiB/62175msec) clat (usec): min=178, max=704477, avg=22389.46, stdev=20772.12 lat (usec): min=179, max=704478, avg=22390.24, stdev=20772.14 clat percentiles (usec): | 1.00th=[ 635], 5.00th=[ 7635], 10.00th=[ 9634], 20.00th=[ 13042], | 30.00th=[ 15401], 40.00th=[ 17433], 50.00th=[ 20317], 60.00th=[ 23200], | 70.00th=[ 26608], 80.00th=[ 30278], 90.00th=[ 36439], 95.00th=[ 41681], | 99.00th=[ 53740], 99.50th=[ 62129], 99.90th=[ 67634], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 23, max= 168, per=25.14%, avg=96.53, stdev=29.70, samples=122 iops : min= 5, max= 42, avg=23.70, stdev= 7.58, samples=122 write: IOPS=25, BW=101KiB/s (104kB/s)(6296KiB/62175msec); 0 zone resets clat (usec): min=705, max=82367, avg=18163.19, stdev=13355.01 lat (usec): min=705, max=82367, avg=18164.32, stdev=13354.91 clat percentiles (usec): | 1.00th=[ 1254], 5.00th=[ 2638], 10.00th=[ 4015], 20.00th=[ 6259], | 30.00th=[ 8848], 40.00th=[11994], 50.00th=[15401], 60.00th=[18744], | 70.00th=[22676], 80.00th=[28705], 90.00th=[36439], 95.00th=[43779], | 99.00th=[61604], 99.50th=[65799], 99.90th=[77071], 99.95th=[82314], | 99.99th=[82314] bw ( KiB/s): min= 32, max= 183, per=25.15%, avg=101.11, stdev=32.07, samples=122 iops : min= 8, max= 45, avg=24.85, stdev= 8.07, samples=122 lat (usec) : 250=0.10%, 500=0.26%, 750=0.39%, 1000=0.52% lat (msec) : 2=0.81%, 4=4.00%, 10=16.67%, 20=33.46%, 50=41.18% lat (msec) : 100=2.57%, 750=0.03% cpu : usr=0.09%, sys=26.97%, ctx=9927, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183449: Thu Nov 13 21:02:13 2025 read: IOPS=24, BW=96.4KiB/s (98.7kB/s)(5940KiB/61628msec) clat (usec): min=180, max=725867, avg=22612.19, stdev=21344.67 lat (usec): min=182, max=725868, avg=22613.02, stdev=21344.69 clat percentiles (usec): | 1.00th=[ 758], 5.00th=[ 8455], 10.00th=[ 10159], 20.00th=[ 12649], | 30.00th=[ 15401], 40.00th=[ 17695], 50.00th=[ 20317], 60.00th=[ 23200], | 70.00th=[ 26608], 80.00th=[ 30278], 90.00th=[ 36963], 95.00th=[ 42730], | 99.00th=[ 54789], 99.50th=[ 61604], 99.90th=[ 92799], 99.95th=[725615], | 99.99th=[725615] bw ( KiB/s): min= 40, max= 208, per=25.19%, avg=96.74, stdev=30.21, samples=121 iops : min= 10, max= 52, avg=23.75, stdev= 7.69, samples=121 write: IOPS=25, BW=103KiB/s (105kB/s)(6348KiB/61628msec); 0 zone resets clat (usec): min=613, max=95208, avg=17640.21, stdev=12938.83 lat (usec): min=613, max=95208, avg=17641.23, stdev=12938.86 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 2573], 10.00th=[ 3752], 20.00th=[ 5735], | 30.00th=[ 8291], 40.00th=[11994], 50.00th=[15533], 60.00th=[19006], | 70.00th=[22414], 80.00th=[27132], 90.00th=[35390], 95.00th=[41681], | 99.00th=[56361], 99.50th=[69731], 99.90th=[89654], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 39, max= 224, per=25.65%, avg=103.11, stdev=32.69, samples=121 iops : min= 9, max= 56, avg=25.36, stdev= 8.27, samples=121 lat (usec) : 250=0.03%, 500=0.20%, 750=0.46%, 1000=0.42% lat (msec) : 2=1.04%, 4=4.17%, 10=15.79%, 20=33.89%, 50=41.96% lat (msec) : 100=2.02%, 750=0.03% cpu : usr=0.11%, sys=27.32%, ctx=10026, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183450: Thu Nov 13 21:02:13 2025 read: IOPS=24, BW=98.5KiB/s (101kB/s)(6028KiB/61211msec) clat (usec): min=112, max=84276, avg=21647.64, stdev=10687.50 lat (usec): min=113, max=84277, avg=21648.58, stdev=10687.51 clat percentiles (usec): | 1.00th=[ 578], 5.00th=[ 7898], 10.00th=[10028], 20.00th=[12780], | 30.00th=[15270], 40.00th=[17433], 50.00th=[19792], 60.00th=[22938], | 70.00th=[25560], 80.00th=[30278], 90.00th=[35914], 95.00th=[40633], | 99.00th=[53740], 99.50th=[59507], 99.90th=[65799], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 39, max= 183, per=25.78%, avg=98.99, stdev=29.88, samples=120 iops : min= 9, max= 45, avg=24.33, stdev= 7.54, samples=120 write: IOPS=25, BW=102KiB/s (105kB/s)(6260KiB/61211msec); 0 zone resets clat (usec): min=659, max=736224, avg=18237.47, stdev=22359.17 lat (usec): min=660, max=736226, avg=18238.62, stdev=22359.16 clat percentiles (usec): | 1.00th=[ 1106], 5.00th=[ 2638], 10.00th=[ 3687], 20.00th=[ 5932], | 30.00th=[ 8586], 40.00th=[ 12125], 50.00th=[ 15401], 60.00th=[ 18744], | 70.00th=[ 22676], 80.00th=[ 27132], 90.00th=[ 35390], 95.00th=[ 42730], | 99.00th=[ 61080], 99.50th=[ 68682], 99.90th=[ 79168], 99.95th=[734004], | 99.99th=[734004] bw ( KiB/s): min= 31, max= 224, per=25.50%, avg=102.52, stdev=36.28, samples=120 iops : min= 7, max= 56, avg=25.21, stdev= 9.16, samples=120 lat (usec) : 250=0.07%, 500=0.36%, 750=0.49%, 1000=0.39% lat (msec) : 2=0.88%, 4=4.85%, 10=15.30%, 20=34.93%, 50=40.79% lat (msec) : 100=1.92%, 750=0.03% cpu : usr=0.10%, sys=27.10%, ctx=9958, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183451: Thu Nov 13 21:02:13 2025 read: IOPS=24, BW=96.0KiB/s (99.3kB/s)(6052KiB/62395msec) clat (usec): min=181, max=92440, avg=22189.03, stdev=11637.93 lat (usec): min=182, max=92441, avg=22189.89, stdev=11637.95 clat percentiles (usec): | 1.00th=[ 529], 5.00th=[ 7963], 10.00th=[10028], 20.00th=[12911], | 30.00th=[15270], 40.00th=[17695], 50.00th=[20055], 60.00th=[23200], | 70.00th=[26346], 80.00th=[30278], 90.00th=[36963], 95.00th=[42730], | 99.00th=[63177], 99.50th=[68682], 99.90th=[86508], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 47, max= 192, per=25.32%, avg=97.24, stdev=29.84, samples=123 iops : min= 11, max= 48, avg=23.89, stdev= 7.62, samples=123 write: IOPS=24, BW=99.9KiB/s (102kB/s)(6236KiB/62395msec); 0 zone resets clat (usec): min=610, max=733516, avg=18442.13, stdev=22193.53 lat (usec): min=611, max=733518, avg=18444.27, stdev=22194.37 clat percentiles (usec): | 1.00th=[ 1287], 5.00th=[ 2704], 10.00th=[ 4047], 20.00th=[ 5932], | 30.00th=[ 8848], 40.00th=[ 11994], 50.00th=[ 15401], 60.00th=[ 18744], | 70.00th=[ 22938], 80.00th=[ 28967], 90.00th=[ 36439], 95.00th=[ 43254], | 99.00th=[ 54264], 99.50th=[ 59507], 99.90th=[ 84411], 99.95th=[734004], | 99.99th=[734004] bw ( KiB/s): min= 47, max= 192, per=24.92%, avg=100.17, stdev=30.71, samples=123 iops : min= 11, max= 48, avg=24.62, stdev= 7.75, samples=123 lat (usec) : 250=0.03%, 500=0.46%, 750=0.42%, 1000=0.33% lat (msec) : 2=0.81%, 4=3.97%, 10=15.63%, 20=35.12%, 50=41.08% lat (msec) : 100=2.12%, 750=0.03% cpu : usr=0.10%, sys=26.94%, ctx=10098, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=385KiB/s (394kB/s), 96.4KiB/s-98.5KiB/s (98.7kB/s-101kB/s), io=23.4MiB (24.6MB), run=61211-62395msec WRITE: bw=403KiB/s (413kB/s), 99.9KiB/s-103KiB/s (102kB/s-105kB/s), io=24.6MiB (25.7MB), run=61211-62395msec rand-rw: (groupid=0, jobs=1): err= 0: pid=183443: Thu Nov 13 21:02:19 2025 read: IOPS=21, BW=87.3KiB/s (89.4kB/s)(5992KiB/68645msec) clat (msec): min=2, max=756, avg=20.91, stdev=24.61 lat (msec): min=2, max=756, avg=20.91, stdev=24.61 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 13], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 32], 90.00th=[ 44], 95.00th=[ 54], | 99.00th=[ 72], 99.50th=[ 80], 99.90th=[ 128], 99.95th=[ 760], | 99.99th=[ 760] bw ( KiB/s): min= 23, max= 312, per=24.97%, avg=87.16, stdev=58.89, samples=135 iops : min= 5, max= 78, avg=21.38, stdev=14.82, samples=135 write: IOPS=22, BW=91.7KiB/s (93.9kB/s)(6296KiB/68645msec); 0 zone resets clat (msec): min=4, max=115, avg=23.58, stdev=16.20 lat (msec): min=4, max=115, avg=23.58, stdev=16.20 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 12], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 25], | 70.00th=[ 30], 80.00th=[ 37], 90.00th=[ 46], 95.00th=[ 55], | 99.00th=[ 72], 99.50th=[ 85], 99.90th=[ 114], 99.95th=[ 115], | 99.99th=[ 115] bw ( KiB/s): min= 31, max= 376, per=24.97%, avg=91.13, stdev=62.28, samples=135 iops : min= 7, max= 94, avg=22.37, stdev=15.68, samples=135 lat (msec) : 4=0.55%, 10=26.92%, 20=29.10%, 50=36.56%, 100=6.67% lat (msec) : 250=0.16%, 1000=0.03% cpu : usr=0.09%, sys=13.70%, ctx=6271, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183444: Thu Nov 13 21:02:19 2025 read: IOPS=21, BW=86.4KiB/s (88.5kB/s)(5940KiB/68721msec) clat (msec): min=2, max=780, avg=19.92, stdev=24.75 lat (msec): min=2, max=780, avg=19.92, stdev=24.75 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 10], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 29], 90.00th=[ 42], 95.00th=[ 52], | 99.00th=[ 70], 99.50th=[ 73], 99.90th=[ 101], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 16, max= 352, per=24.65%, avg=86.03, stdev=63.65, samples=135 iops : min= 4, max= 88, avg=21.10, stdev=16.03, samples=135 write: IOPS=23, BW=92.4KiB/s (94.6kB/s)(6348KiB/68721msec); 0 zone resets clat (msec): min=3, max=109, avg=24.54, stdev=17.38 lat (msec): min=3, max=109, avg=24.54, stdev=17.38 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 12], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 26], | 70.00th=[ 31], 80.00th=[ 39], 90.00th=[ 50], 95.00th=[ 57], | 99.00th=[ 80], 99.50th=[ 93], 99.90th=[ 108], 99.95th=[ 110], | 99.99th=[ 110] bw ( KiB/s): min= 32, max= 328, per=24.98%, avg=91.18, stdev=60.72, samples=135 iops : min= 8, max= 82, avg=22.39, stdev=15.28, samples=135 lat (msec) : 4=0.98%, 10=26.79%, 20=30.14%, 50=34.54%, 100=7.39% lat (msec) : 250=0.13%, 1000=0.03% cpu : usr=0.08%, sys=13.74%, ctx=6273, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183445: Thu Nov 13 21:02:19 2025 read: IOPS=21, BW=87.0KiB/s (90.1kB/s)(6028KiB/68527msec) clat (usec): min=3288, max=98002, avg=19940.05, stdev=15586.43 lat (usec): min=3289, max=98003, avg=19941.09, stdev=15586.55 clat percentiles (usec): | 1.00th=[ 4015], 5.00th=[ 4948], 10.00th=[ 5604], 20.00th=[ 7242], | 30.00th=[ 9634], 40.00th=[12649], 50.00th=[15533], 60.00th=[18220], | 70.00th=[22676], 80.00th=[29492], 90.00th=[42206], 95.00th=[53216], | 99.00th=[76022], 99.50th=[84411], 99.90th=[93848], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 24, max= 352, per=25.24%, avg=88.08, stdev=63.20, samples=135 iops : min= 6, max= 88, avg=21.61, stdev=15.89, samples=135 write: IOPS=22, BW=91.4KiB/s (93.5kB/s)(6260KiB/68527msec); 0 zone resets clat (msec): min=4, max=786, avg=24.46, stdev=25.01 lat (msec): min=4, max=786, avg=24.46, stdev=25.01 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 13], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 25], | 70.00th=[ 31], 80.00th=[ 37], 90.00th=[ 47], 95.00th=[ 55], | 99.00th=[ 75], 99.50th=[ 81], 99.90th=[ 103], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 24, max= 320, per=25.05%, avg=91.42, stdev=60.24, samples=135 iops : min= 6, max= 80, avg=22.44, stdev=15.17, samples=135 lat (msec) : 4=0.46%, 10=26.56%, 20=30.21%, 50=36.00%, 100=6.67% lat (msec) : 250=0.07%, 1000=0.03% cpu : usr=0.08%, sys=13.63%, ctx=6339, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183446: Thu Nov 13 21:02:19 2025 read: IOPS=22, BW=88.2KiB/s (90.4kB/s)(6052KiB/68579msec) clat (msec): min=3, max=117, avg=20.21, stdev=15.71 lat (msec): min=3, max=117, avg=20.21, stdev=15.71 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 31], 90.00th=[ 43], 95.00th=[ 53], | 99.00th=[ 70], 99.50th=[ 87], 99.90th=[ 103], 99.95th=[ 118], | 99.99th=[ 118] bw ( KiB/s): min= 15, max= 336, per=25.21%, avg=87.99, stdev=60.27, samples=135 iops : min= 3, max= 84, avg=21.59, stdev=15.16, samples=135 write: IOPS=22, BW=90.9KiB/s (93.1kB/s)(6236KiB/68579msec); 0 zone resets clat (msec): min=4, max=785, avg=24.25, stdev=25.30 lat (msec): min=4, max=785, avg=24.25, stdev=25.30 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 12], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 25], | 70.00th=[ 31], 80.00th=[ 37], 90.00th=[ 46], 95.00th=[ 55], | 99.00th=[ 77], 99.50th=[ 87], 99.90th=[ 117], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 31, max= 336, per=24.90%, avg=90.88, stdev=62.81, samples=135 iops : min= 7, max= 84, avg=22.31, stdev=15.83, samples=135 lat (msec) : 4=0.36%, 10=27.64%, 20=28.29%, 50=37.14%, 100=6.35% lat (msec) : 250=0.20%, 1000=0.03% cpu : usr=0.08%, sys=13.70%, ctx=6264, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=349KiB/s (358kB/s), 86.4KiB/s-88.2KiB/s (88.5kB/s-90.4kB/s), io=23.4MiB (24.6MB), run=68527-68721msec WRITE: bw=366KiB/s (375kB/s), 90.9KiB/s-92.4KiB/s (93.1kB/s-94.6kB/s), io=24.6MiB (25.7MB), run=68527-68721msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=183477: Thu Nov 13 21:02:41 2025 read: IOPS=23, BW=377KiB/s (386kB/s)(5920KiB/15719msec) clat (usec): min=113, max=97232, avg=20574.99, stdev=11468.12 lat (usec): min=113, max=97234, avg=20577.07, stdev=11467.32 clat percentiles (usec): | 1.00th=[ 355], 5.00th=[ 5211], 10.00th=[10552], 20.00th=[12518], | 30.00th=[14615], 40.00th=[16450], 50.00th=[18482], 60.00th=[21103], | 70.00th=[23725], 80.00th=[26870], 90.00th=[33817], 95.00th=[40633], | 99.00th=[68682], 99.50th=[72877], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 190, max= 576, per=24.89%, avg=376.61, stdev=107.84, samples=31 iops : min= 11, max= 36, avg=23.06, stdev= 6.83, samples=31 write: IOPS=25, BW=405KiB/s (415kB/s)(6368KiB/15719msec); 0 zone resets clat (usec): min=1235, max=71134, avg=20019.60, stdev=12327.46 lat (usec): min=1235, max=71136, avg=20021.05, stdev=12327.53 clat percentiles (usec): | 1.00th=[ 1696], 5.00th=[ 4228], 10.00th=[ 5735], 20.00th=[ 9110], | 30.00th=[12125], 40.00th=[14353], 50.00th=[17695], 60.00th=[21365], | 70.00th=[24511], 80.00th=[30016], 90.00th=[37487], 95.00th=[43779], | 99.00th=[58459], 99.50th=[62129], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 191, max= 699, per=25.54%, avg=406.55, stdev=110.48, samples=31 iops : min= 11, max= 43, avg=24.94, stdev= 6.90, samples=31 lat (usec) : 250=0.26%, 500=1.30%, 750=0.39%, 1000=0.26% lat (msec) : 2=0.78%, 4=1.43%, 10=11.20%, 20=40.36%, 50=41.93% lat (msec) : 100=2.08% cpu : usr=0.09%, sys=28.38%, ctx=2943, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183478: Thu Nov 13 21:02:41 2025 read: IOPS=23, BW=377KiB/s (386kB/s)(5744KiB/15256msec) clat (usec): min=276, max=64306, avg=21622.97, stdev=10132.78 lat (usec): min=278, max=64307, avg=21623.81, stdev=10132.82 clat percentiles (usec): | 1.00th=[ 363], 5.00th=[ 9372], 10.00th=[11469], 20.00th=[13960], | 30.00th=[16057], 40.00th=[17957], 50.00th=[20055], 60.00th=[22414], | 70.00th=[24773], 80.00th=[28443], 90.00th=[34341], 95.00th=[40109], | 99.00th=[55313], 99.50th=[63177], 99.90th=[64226], 99.95th=[64226], | 99.99th=[64226] bw ( KiB/s): min= 159, max= 573, per=24.77%, avg=374.73, stdev=104.12, samples=30 iops : min= 9, max= 35, avg=22.97, stdev= 6.55, samples=30 write: IOPS=26, BW=429KiB/s (439kB/s)(6544KiB/15256msec); 0 zone resets clat (usec): min=1494, max=70423, avg=18253.83, stdev=12503.45 lat (usec): min=1495, max=70425, avg=18255.25, stdev=12503.52 clat percentiles (usec): | 1.00th=[ 3294], 5.00th=[ 4686], 10.00th=[ 5342], 20.00th=[ 7504], | 30.00th=[10159], 40.00th=[12780], 50.00th=[15008], 60.00th=[17957], | 70.00th=[21627], 80.00th=[26870], 90.00th=[36439], 95.00th=[44827], | 99.00th=[56886], 99.50th=[61080], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 286, max= 650, per=26.70%, avg=425.00, stdev=116.15, samples=30 iops : min= 17, max= 40, avg=26.10, stdev= 7.37, samples=30 lat (usec) : 500=0.78%, 750=0.39% lat (msec) : 2=0.26%, 4=1.30%, 10=15.10%, 20=40.76%, 50=39.19% lat (msec) : 100=2.21% cpu : usr=0.12%, sys=29.41%, ctx=2862, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183479: Thu Nov 13 21:02:41 2025 read: IOPS=24, BW=395KiB/s (405kB/s)(6256KiB/15824msec) clat (usec): min=110, max=77828, avg=20994.38, stdev=10989.23 lat (usec): min=111, max=77829, avg=20995.33, stdev=10989.29 clat percentiles (usec): | 1.00th=[ 281], 5.00th=[ 7439], 10.00th=[10290], 20.00th=[13042], | 30.00th=[15008], 40.00th=[17957], 50.00th=[19792], 60.00th=[21365], | 70.00th=[24249], 80.00th=[27657], 90.00th=[34341], 95.00th=[41681], | 99.00th=[54789], 99.50th=[69731], 99.90th=[78119], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 190, max= 608, per=26.25%, avg=397.23, stdev=111.42, samples=31 iops : min= 11, max= 38, avg=24.35, stdev= 7.04, samples=31 write: IOPS=23, BW=381KiB/s (390kB/s)(6032KiB/15824msec); 0 zone resets clat (usec): min=1722, max=70914, avg=19999.41, stdev=13057.24 lat (usec): min=1723, max=70915, avg=20006.82, stdev=13056.22 clat percentiles (usec): | 1.00th=[ 3359], 5.00th=[ 4948], 10.00th=[ 5932], 20.00th=[ 8717], | 30.00th=[11207], 40.00th=[14484], 50.00th=[16581], 60.00th=[20579], | 70.00th=[24249], 80.00th=[29754], 90.00th=[38536], 95.00th=[45876], | 99.00th=[60556], 99.50th=[61604], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 160, max= 606, per=23.56%, avg=375.06, stdev=102.51, samples=31 iops : min= 10, max= 37, avg=22.97, stdev= 6.43, samples=31 lat (usec) : 250=0.13%, 500=1.43%, 750=0.65% lat (msec) : 2=0.26%, 4=0.78%, 10=13.93%, 20=37.63%, 50=42.32% lat (msec) : 100=2.86% cpu : usr=0.09%, sys=28.49%, ctx=2957, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183480: Thu Nov 13 21:02:41 2025 read: IOPS=24, BW=389KiB/s (398kB/s)(6032KiB/15525msec) clat (usec): min=202, max=70260, avg=21467.16, stdev=12137.46 lat (usec): min=202, max=70260, avg=21468.03, stdev=12137.52 clat percentiles (usec): | 1.00th=[ 314], 5.00th=[ 603], 10.00th=[ 9634], 20.00th=[12256], | 30.00th=[14091], 40.00th=[17171], 50.00th=[19792], 60.00th=[22938], | 70.00th=[26084], 80.00th=[30278], 90.00th=[35914], 95.00th=[45351], | 99.00th=[61080], 99.50th=[66323], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 190, max= 608, per=25.22%, avg=381.57, stdev=96.55, samples=30 iops : min= 11, max= 38, avg=23.40, stdev= 6.11, samples=30 write: IOPS=25, BW=403KiB/s (413kB/s)(6256KiB/15525msec); 0 zone resets clat (usec): min=1241, max=68781, avg=18700.92, stdev=12718.78 lat (usec): min=1243, max=68783, avg=18702.38, stdev=12718.83 clat percentiles (usec): | 1.00th=[ 1614], 5.00th=[ 4293], 10.00th=[ 5211], 20.00th=[ 7373], | 30.00th=[10159], 40.00th=[12780], 50.00th=[15664], 60.00th=[19268], | 70.00th=[22676], 80.00th=[28967], 90.00th=[36963], 95.00th=[43779], | 99.00th=[55837], 99.50th=[62129], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 159, max= 608, per=25.28%, avg=402.43, stdev=130.89, samples=30 iops : min= 9, max= 38, avg=24.70, stdev= 8.21, samples=30 lat (usec) : 250=0.26%, 500=1.56%, 750=1.04%, 1000=0.13% lat (msec) : 2=0.91%, 4=1.30%, 10=15.23%, 20=36.33%, 50=40.62% lat (msec) : 100=2.60% cpu : usr=0.09%, sys=28.16%, ctx=2947, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1514KiB/s (1550kB/s), 377KiB/s-395KiB/s (386kB/s-405kB/s), io=23.4MiB (24.5MB), run=15256-15824msec WRITE: bw=1593KiB/s (1631kB/s), 381KiB/s-429KiB/s (390kB/s-439kB/s), io=24.6MiB (25.8MB), run=15256-15824msec rand-rw: (groupid=0, jobs=1): err= 0: pid=183473: Thu Nov 13 21:02:43 2025 read: IOPS=20, BW=335KiB/s (343kB/s)(5920KiB/17678msec) clat (usec): min=2972, max=96927, avg=20171.72, stdev=14511.42 lat (usec): min=2973, max=96928, avg=20172.64, stdev=14511.25 clat percentiles (usec): | 1.00th=[ 3392], 5.00th=[ 6849], 10.00th=[ 7832], 20.00th=[ 9503], | 30.00th=[10814], 40.00th=[12518], 50.00th=[15139], 60.00th=[18744], | 70.00th=[21890], 80.00th=[28443], 90.00th=[42206], 95.00th=[50070], | 99.00th=[66847], 99.50th=[80217], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 127, max= 960, per=24.64%, avg=333.60, stdev=171.89, samples=35 iops : min= 7, max= 60, avg=20.34, stdev=10.90, samples=35 write: IOPS=22, BW=360KiB/s (369kB/s)(6368KiB/17678msec); 0 zone resets clat (usec): min=4206, max=89187, avg=25403.20, stdev=15951.12 lat (usec): min=4208, max=89189, avg=25404.82, stdev=15951.07 clat percentiles (usec): | 1.00th=[ 4948], 5.00th=[ 8291], 10.00th=[ 9634], 20.00th=[11207], | 30.00th=[13042], 40.00th=[17171], 50.00th=[21103], 60.00th=[25822], | 70.00th=[31851], 80.00th=[39060], 90.00th=[47973], 95.00th=[56361], | 99.00th=[80217], 99.50th=[83362], 99.90th=[89654], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 191, max= 928, per=25.24%, avg=359.63, stdev=180.77, samples=35 iops : min= 11, max= 58, avg=21.97, stdev=11.50, samples=35 lat (msec) : 4=1.04%, 10=16.67%, 20=38.02%, 50=37.24%, 100=7.03% cpu : usr=0.09%, sys=13.59%, ctx=1583, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183474: Thu Nov 13 21:02:43 2025 read: IOPS=20, BW=326KiB/s (334kB/s)(5744KiB/17627msec) clat (usec): min=3933, max=88272, avg=20368.02, stdev=14045.75 lat (usec): min=3934, max=88273, avg=20368.79, stdev=14045.74 clat percentiles (usec): | 1.00th=[ 6390], 5.00th=[ 7504], 10.00th=[ 8160], 20.00th=[ 9634], | 30.00th=[10945], 40.00th=[12649], 50.00th=[14484], 60.00th=[17957], | 70.00th=[22938], 80.00th=[30278], 90.00th=[42206], 95.00th=[49021], | 99.00th=[65799], 99.50th=[69731], 99.90th=[88605], 99.95th=[88605], | 99.99th=[88605] bw ( KiB/s): min= 127, max= 800, per=23.25%, avg=314.85, stdev=130.82, samples=34 iops : min= 7, max= 50, avg=19.15, stdev= 8.32, samples=34 write: IOPS=23, BW=371KiB/s (380kB/s)(6544KiB/17627msec); 0 zone resets clat (usec): min=6572, max=80697, avg=25041.16, stdev=16016.47 lat (usec): min=6574, max=80698, avg=25046.71, stdev=16012.57 clat percentiles (usec): | 1.00th=[ 7111], 5.00th=[ 8848], 10.00th=[ 9503], 20.00th=[11076], | 30.00th=[12911], 40.00th=[16057], 50.00th=[20055], 60.00th=[25035], | 70.00th=[30802], 80.00th=[36963], 90.00th=[51119], 95.00th=[57410], | 99.00th=[72877], 99.50th=[73925], 99.90th=[80217], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 191, max= 960, per=24.98%, avg=355.97, stdev=178.00, samples=34 iops : min= 11, max= 60, avg=21.74, stdev=11.30, samples=34 lat (msec) : 4=0.13%, 10=17.06%, 20=39.84%, 50=35.42%, 100=7.55% cpu : usr=0.14%, sys=13.47%, ctx=1540, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183475: Thu Nov 13 21:02:43 2025 read: IOPS=22, BW=357KiB/s (365kB/s)(6256KiB/17535msec) clat (usec): min=5664, max=76497, avg=20237.33, stdev=13690.47 lat (usec): min=5665, max=76498, avg=20238.18, stdev=13690.44 clat percentiles (usec): | 1.00th=[ 7177], 5.00th=[ 8160], 10.00th=[ 8848], 20.00th=[10159], | 30.00th=[11338], 40.00th=[13173], 50.00th=[14877], 60.00th=[17695], | 70.00th=[21890], 80.00th=[28705], 90.00th=[40109], 95.00th=[51643], | 99.00th=[68682], 99.50th=[72877], 99.90th=[76022], 99.95th=[76022], | 99.99th=[76022] bw ( KiB/s): min= 160, max= 992, per=26.06%, avg=352.79, stdev=189.29, samples=34 iops : min= 10, max= 62, avg=21.53, stdev=12.02, samples=34 write: IOPS=21, BW=344KiB/s (352kB/s)(6032KiB/17535msec); 0 zone resets clat (usec): min=7035, max=83412, avg=25462.34, stdev=15106.69 lat (usec): min=7037, max=83414, avg=25463.97, stdev=15106.69 clat percentiles (usec): | 1.00th=[ 8094], 5.00th=[ 9110], 10.00th=[10159], 20.00th=[12256], | 30.00th=[14746], 40.00th=[17433], 50.00th=[21627], 60.00th=[25035], | 70.00th=[30540], 80.00th=[37487], 90.00th=[49546], 95.00th=[56886], | 99.00th=[69731], 99.50th=[73925], 99.90th=[83362], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 64, max= 608, per=23.15%, avg=329.91, stdev=117.73, samples=34 iops : min= 4, max= 38, avg=20.12, stdev= 7.48, samples=34 lat (msec) : 10=13.93%, 20=42.19%, 50=36.46%, 100=7.42% cpu : usr=0.07%, sys=13.84%, ctx=1627, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183476: Thu Nov 13 21:02:43 2025 read: IOPS=21, BW=342KiB/s (351kB/s)(6032KiB/17614msec) clat (usec): min=4457, max=71834, avg=19446.70, stdev=13271.60 lat (usec): min=4457, max=71835, avg=19447.74, stdev=13271.68 clat percentiles (usec): | 1.00th=[ 5407], 5.00th=[ 7308], 10.00th=[ 8029], 20.00th=[ 9110], | 30.00th=[10028], 40.00th=[12125], 50.00th=[14484], 60.00th=[16909], | 70.00th=[21890], 80.00th=[28967], 90.00th=[39584], 95.00th=[47449], | 99.00th=[63177], 99.50th=[69731], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min= 128, max= 896, per=24.35%, avg=329.74, stdev=171.94, samples=34 iops : min= 8, max= 56, avg=20.09, stdev=10.92, samples=34 write: IOPS=22, BW=355KiB/s (364kB/s)(6256KiB/17614msec); 0 zone resets clat (usec): min=6616, max=83100, avg=26162.11, stdev=16202.48 lat (usec): min=6618, max=83101, avg=26163.86, stdev=16202.36 clat percentiles (usec): | 1.00th=[ 7373], 5.00th=[ 8979], 10.00th=[ 9503], 20.00th=[11469], | 30.00th=[13566], 40.00th=[18220], 50.00th=[22414], 60.00th=[26608], | 70.00th=[32375], 80.00th=[39060], 90.00th=[50594], 95.00th=[58459], | 99.00th=[72877], 99.50th=[80217], 99.90th=[83362], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 222, max= 896, per=24.18%, avg=344.53, stdev=146.32, samples=34 iops : min= 13, max= 56, avg=21.03, stdev= 9.25, samples=34 lat (msec) : 10=21.35%, 20=34.11%, 50=38.02%, 100=6.51% cpu : usr=0.08%, sys=13.79%, ctx=1597, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1355KiB/s (1387kB/s), 326KiB/s-357KiB/s (334kB/s-365kB/s), io=23.4MiB (24.5MB), run=17535-17678msec WRITE: bw=1426KiB/s (1460kB/s), 344KiB/s-371KiB/s (352kB/s-380kB/s), io=24.6MiB (25.8MB), run=17535-17678msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=183489: Thu Nov 13 21:02:52 2025 read: IOPS=2, BW=2747KiB/s (2813kB/s)(7168KiB/2609msec) clat (usec): min=749, max=492100, avg=155836.66, stdev=163715.36 lat (usec): min=750, max=492101, avg=155838.18, stdev=163715.25 clat percentiles (usec): | 1.00th=[ 750], 5.00th=[ 750], 10.00th=[ 750], 20.00th=[ 13304], | 30.00th=[111674], 40.00th=[111674], 50.00th=[137364], 60.00th=[149947], | 70.00th=[149947], 80.00th=[187696], 90.00th=[492831], 95.00th=[492831], | 99.00th=[492831], 99.50th=[492831], 99.90th=[492831], 99.95th=[492831], | 99.99th=[492831] bw ( KiB/s): min= 4087, max= 5077, per=73.10%, avg=4582.00, stdev=700.04, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1962KiB/s (2010kB/s)(5120KiB/2609msec); 0 zone resets clat (msec): min=175, max=545, avg=301.35, stdev=144.07 lat (msec): min=175, max=545, avg=301.41, stdev=144.06 clat percentiles (msec): | 1.00th=[ 176], 5.00th=[ 176], 10.00th=[ 176], 20.00th=[ 176], | 30.00th=[ 222], 40.00th=[ 222], 50.00th=[ 268], 60.00th=[ 268], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 550], 95.00th=[ 550], | 99.00th=[ 550], 99.50th=[ 550], 99.90th=[ 550], 99.95th=[ 550], | 99.99th=[ 550] bw ( KiB/s): min= 2048, max= 4087, per=34.95%, avg=3067.50, stdev=1441.79, samples=2 iops : min= 2, max= 3, avg= 2.50, stdev= 0.71, samples=2 lat (usec) : 750=8.33% lat (msec) : 20=8.33%, 250=50.00%, 500=25.00%, 750=8.33% cpu : usr=0.04%, sys=47.85%, ctx=715, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183490: Thu Nov 13 21:02:52 2025 read: IOPS=0, BW=627KiB/s (642kB/s)(2048KiB/3267msec) clat (msec): min=173, max=215, avg=194.30, stdev=29.68 lat (msec): min=173, max=215, avg=194.30, stdev=29.68 clat percentiles (msec): | 1.00th=[ 174], 5.00th=[ 174], 10.00th=[ 174], 20.00th=[ 174], | 30.00th=[ 174], 40.00th=[ 174], 50.00th=[ 174], 60.00th=[ 215], | 70.00th=[ 215], 80.00th=[ 215], 90.00th=[ 215], 95.00th=[ 215], | 99.00th=[ 215], 99.50th=[ 215], 99.90th=[ 215], 99.95th=[ 215], | 99.99th=[ 215] bw ( KiB/s): min= 1695, max= 2043, per=29.82%, avg=1869.00, stdev=246.07, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3134KiB/s (3210kB/s)(10.0MiB/3267msec); 0 zone resets clat (msec): min=126, max=648, avg=283.23, stdev=189.54 lat (msec): min=126, max=648, avg=283.30, stdev=189.54 clat percentiles (msec): | 1.00th=[ 127], 5.00th=[ 127], 10.00th=[ 127], 20.00th=[ 134], | 30.00th=[ 169], 40.00th=[ 188], 50.00th=[ 190], 60.00th=[ 234], | 70.00th=[ 253], 80.00th=[ 279], 90.00th=[ 617], 95.00th=[ 651], | 99.00th=[ 651], 99.50th=[ 651], 99.90th=[ 651], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 2007, max= 5595, per=40.63%, avg=3565.60, stdev=1534.54, samples=5 iops : min= 1, max= 5, avg= 2.80, stdev= 1.79, samples=5 lat (msec) : 250=66.67%, 500=16.67%, 750=16.67% cpu : usr=0.06%, sys=49.02%, ctx=697, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183491: Thu Nov 13 21:02:52 2025 read: IOPS=2, BW=2341KiB/s (2397kB/s)(6144KiB/2625msec) clat (usec): min=826, max=197939, avg=81882.27, stdev=87480.85 lat (usec): min=827, max=197939, avg=81883.18, stdev=87480.76 clat percentiles (usec): | 1.00th=[ 824], 5.00th=[ 824], 10.00th=[ 824], 20.00th=[ 1020], | 30.00th=[ 1020], 40.00th=[ 11863], 50.00th=[ 11863], 60.00th=[133694], | 70.00th=[145753], 80.00th=[145753], 90.00th=[198181], 95.00th=[198181], | 99.00th=[198181], 99.50th=[198181], 99.90th=[198181], 99.95th=[198181], | 99.99th=[198181] bw ( KiB/s): min= 2007, max= 4087, per=54.14%, avg=3393.67, stdev=1200.89, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=2, BW=2341KiB/s (2397kB/s)(6144KiB/2625msec); 0 zone resets clat (msec): min=179, max=586, avg=342.73, stdev=156.92 lat (msec): min=179, max=586, avg=342.79, stdev=156.93 clat percentiles (msec): | 1.00th=[ 180], 5.00th=[ 180], 10.00th=[ 180], 20.00th=[ 218], | 30.00th=[ 218], 40.00th=[ 262], 50.00th=[ 262], 60.00th=[ 347], | 70.00th=[ 464], 80.00th=[ 464], 90.00th=[ 584], 95.00th=[ 584], | 99.00th=[ 584], 99.50th=[ 584], 99.90th=[ 584], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 2043, max= 4096, per=34.75%, avg=3049.25, stdev=1162.39, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.50, samples=4 lat (usec) : 1000=8.33% lat (msec) : 2=8.33%, 20=8.33%, 250=41.67%, 500=25.00%, 750=8.33% cpu : usr=0.04%, sys=50.57%, ctx=633, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183492: Thu Nov 13 21:02:52 2025 read: IOPS=1, BW=1568KiB/s (1606kB/s)(5120KiB/3265msec) clat (usec): min=740, max=239565, avg=119661.35, stdev=90808.29 lat (usec): min=741, max=239565, avg=119662.17, stdev=90808.24 clat percentiles (usec): | 1.00th=[ 742], 5.00th=[ 742], 10.00th=[ 742], 20.00th=[ 742], | 30.00th=[ 76022], 40.00th=[ 76022], 50.00th=[111674], 60.00th=[111674], | 70.00th=[170918], 80.00th=[170918], 90.00th=[240124], 95.00th=[240124], | 99.00th=[240124], 99.50th=[240124], 99.90th=[240124], 99.95th=[240124], | 99.99th=[240124] bw ( KiB/s): min= 4015, max= 4087, per=64.63%, avg=4051.00, stdev=50.91, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2195KiB/s (2248kB/s)(7168KiB/3265msec); 0 zone resets clat (msec): min=173, max=649, avg=359.40, stdev=170.94 lat (msec): min=173, max=649, avg=359.47, stdev=170.93 clat percentiles (msec): | 1.00th=[ 174], 5.00th=[ 174], 10.00th=[ 174], 20.00th=[ 207], | 30.00th=[ 268], 40.00th=[ 268], 50.00th=[ 326], 60.00th=[ 384], | 70.00th=[ 384], 80.00th=[ 510], 90.00th=[ 651], 95.00th=[ 651], | 99.00th=[ 651], 99.50th=[ 651], 99.90th=[ 651], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 2043, max= 4096, per=33.93%, avg=2978.00, stdev=1089.94, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.50, samples=4 lat (usec) : 750=8.33% lat (msec) : 100=8.33%, 250=41.67%, 500=25.00%, 750=16.67% cpu : usr=0.00%, sys=40.90%, ctx=569, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6269KiB/s (6419kB/s), 627KiB/s-2747KiB/s (642kB/s-2813kB/s), io=20.0MiB (20.0MB), run=2609-3267msec WRITE: bw=8776KiB/s (8987kB/s), 1962KiB/s-3134KiB/s (2010kB/s-3210kB/s), io=28.0MiB (29.4MB), run=2609-3267msec rand-rw: (groupid=0, jobs=1): err= 0: pid=183493: Thu Nov 13 21:02:53 2025 read: IOPS=2, BW=2353KiB/s (2410kB/s)(7168KiB/3046msec) clat (msec): min=71, max=267, avg=178.11, stdev=63.06 lat (msec): min=71, max=267, avg=178.11, stdev=63.06 clat percentiles (msec): | 1.00th=[ 72], 5.00th=[ 72], 10.00th=[ 72], 20.00th=[ 138], | 30.00th=[ 163], 40.00th=[ 163], 50.00th=[ 184], 60.00th=[ 203], | 70.00th=[ 203], 80.00th=[ 224], 90.00th=[ 268], 95.00th=[ 268], | 99.00th=[ 268], 99.50th=[ 268], 99.90th=[ 268], 99.95th=[ 268], | 99.99th=[ 268] bw ( KiB/s): min= 4063, max= 5585, per=81.13%, avg=4824.00, stdev=1076.22, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=1, BW=1681KiB/s (1721kB/s)(5120KiB/3046msec); 0 zone resets clat (msec): min=136, max=567, avg=355.61, stdev=186.87 lat (msec): min=136, max=567, avg=355.75, stdev=186.83 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 136], | 30.00th=[ 257], 40.00th=[ 257], 50.00th=[ 284], 60.00th=[ 284], | 70.00th=[ 535], 80.00th=[ 535], 90.00th=[ 567], 95.00th=[ 567], | 99.00th=[ 567], 99.50th=[ 567], 99.90th=[ 567], 99.95th=[ 567], | 99.99th=[ 567] bw ( KiB/s): min= 1950, max= 2007, per=23.82%, avg=1983.00, stdev=29.55, samples=3 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=3 lat (msec) : 100=8.33%, 250=50.00%, 500=25.00%, 750=16.67% cpu : usr=0.03%, sys=20.10%, ctx=122, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183494: Thu Nov 13 21:02:53 2025 read: IOPS=0, BW=605KiB/s (620kB/s)(2048KiB/3385msec) clat (msec): min=63, max=257, avg=160.93, stdev=137.12 lat (msec): min=63, max=257, avg=160.93, stdev=137.12 clat percentiles (msec): | 1.00th=[ 65], 5.00th=[ 65], 10.00th=[ 65], 20.00th=[ 65], | 30.00th=[ 65], 40.00th=[ 65], 50.00th=[ 65], 60.00th=[ 257], | 70.00th=[ 257], 80.00th=[ 257], 90.00th=[ 257], 95.00th=[ 257], | 99.00th=[ 257], 99.50th=[ 257], 99.90th=[ 257], 99.95th=[ 257], | 99.99th=[ 257] bw ( KiB/s): min= 1875, max= 2007, per=32.64%, avg=1941.00, stdev=93.34, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=3025KiB/s (3098kB/s)(10.0MiB/3385msec); 0 zone resets clat (msec): min=89, max=780, avg=302.72, stdev=238.99 lat (msec): min=89, max=780, avg=302.79, stdev=238.98 clat percentiles (msec): | 1.00th=[ 89], 5.00th=[ 89], 10.00th=[ 89], 20.00th=[ 94], | 30.00th=[ 102], 40.00th=[ 184], 50.00th=[ 249], 60.00th=[ 268], | 70.00th=[ 284], 80.00th=[ 305], 90.00th=[ 676], 95.00th=[ 776], | 99.00th=[ 776], 99.50th=[ 776], 99.90th=[ 776], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 2007, max= 4063, per=41.39%, avg=3446.00, stdev=969.80, samples=4 iops : min= 1, max= 3, avg= 2.50, stdev= 1.00, samples=4 lat (msec) : 100=25.00%, 250=25.00%, 500=33.33%, 750=8.33%, 1000=8.33% cpu : usr=0.03%, sys=23.82%, ctx=323, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183495: Thu Nov 13 21:02:53 2025 read: IOPS=1, BW=1784KiB/s (1827kB/s)(6144KiB/3444msec) clat (msec): min=23, max=576, avg=242.15, stdev=198.46 lat (msec): min=23, max=576, avg=242.15, stdev=198.46 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 24], 10.00th=[ 24], 20.00th=[ 97], | 30.00th=[ 97], 40.00th=[ 163], 50.00th=[ 163], 60.00th=[ 251], | 70.00th=[ 342], 80.00th=[ 342], 90.00th=[ 575], 95.00th=[ 575], | 99.00th=[ 575], 99.50th=[ 575], 99.90th=[ 575], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 1865, max= 2031, per=33.19%, avg=1973.75, stdev=74.26, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 write: IOPS=1, BW=1784KiB/s (1827kB/s)(6144KiB/3444msec); 0 zone resets clat (msec): min=19, max=460, avg=324.39, stdev=162.25 lat (msec): min=19, max=460, avg=324.44, stdev=162.25 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 20], 10.00th=[ 20], 20.00th=[ 313], | 30.00th=[ 313], 40.00th=[ 330], 50.00th=[ 330], 60.00th=[ 368], | 70.00th=[ 460], 80.00th=[ 460], 90.00th=[ 460], 95.00th=[ 460], | 99.00th=[ 460], 99.50th=[ 460], 99.90th=[ 460], 99.95th=[ 460], | 99.99th=[ 460] bw ( KiB/s): min= 1872, max= 2031, per=23.60%, avg=1965.00, stdev=70.70, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 lat (msec) : 20=8.33%, 50=8.33%, 100=8.33%, 250=8.33%, 500=58.33% lat (msec) : 750=8.33% cpu : usr=0.03%, sys=14.49%, ctx=122, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183496: Thu Nov 13 21:02:53 2025 read: IOPS=1, BW=1758KiB/s (1800kB/s)(5120KiB/2912msec) clat (msec): min=82, max=283, avg=156.02, stdev=84.62 lat (msec): min=82, max=283, avg=156.03, stdev=84.62 clat percentiles (msec): | 1.00th=[ 84], 5.00th=[ 84], 10.00th=[ 84], 20.00th=[ 84], | 30.00th=[ 89], 40.00th=[ 89], 50.00th=[ 129], 60.00th=[ 129], | 70.00th=[ 197], 80.00th=[ 197], 90.00th=[ 284], 95.00th=[ 284], | 99.00th=[ 284], 99.50th=[ 284], 99.90th=[ 284], 99.95th=[ 284], | 99.99th=[ 284] bw ( KiB/s): min= 7968, max= 7968, per=100.00%, avg=7968.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=2, BW=2462KiB/s (2521kB/s)(7168KiB/2912msec); 0 zone resets clat (msec): min=113, max=458, avg=283.22, stdev=125.10 lat (msec): min=113, max=458, avg=283.30, stdev=125.09 clat percentiles (msec): | 1.00th=[ 114], 5.00th=[ 114], 10.00th=[ 114], 20.00th=[ 163], | 30.00th=[ 211], 40.00th=[ 211], 50.00th=[ 300], 60.00th=[ 359], | 70.00th=[ 359], 80.00th=[ 380], 90.00th=[ 460], 95.00th=[ 460], | 99.00th=[ 460], 99.50th=[ 460], 99.90th=[ 460], 99.95th=[ 460], | 99.99th=[ 460] bw ( KiB/s): min= 1896, max= 4063, per=29.78%, avg=2479.00, stdev=1056.97, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 100=16.67%, 250=41.67%, 500=41.67% cpu : usr=0.03%, sys=19.24%, ctx=126, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5947KiB/s (6089kB/s), 605KiB/s-2353KiB/s (620kB/s-2410kB/s), io=20.0MiB (20.0MB), run=2912-3444msec WRITE: bw=8325KiB/s (8525kB/s), 1681KiB/s-3025KiB/s (1721kB/s-3098kB/s), io=28.0MiB (29.4MB), run=2912-3444msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=183501: Thu Nov 13 21:03:01 2025 read: IOPS=1, BW=6614KiB/s (6772kB/s)(12.0MiB/1858msec) clat (msec): min=463, max=768, avg=618.17, stdev=152.28 lat (msec): min=463, max=768, avg=618.18, stdev=152.28 clat percentiles (msec): | 1.00th=[ 464], 5.00th=[ 464], 10.00th=[ 464], 20.00th=[ 464], | 30.00th=[ 464], 40.00th=[ 625], 50.00th=[ 625], 60.00th=[ 625], | 70.00th=[ 768], 80.00th=[ 768], 90.00th=[ 768], 95.00th=[ 768], | 99.00th=[ 768], 99.50th=[ 768], 99.90th=[ 768], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 8192, max= 8192, per=92.90%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=15.83%, ctx=66, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183502: Thu Nov 13 21:03:01 2025 read: IOPS=0, BW=2227KiB/s (2281kB/s)(4096KiB/1839msec) clat (nsec): min=134939k, max=134939k, avg=134939494.00, stdev= 0.00 lat (nsec): min=134942k, max=134942k, avg=134941684.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 136], | 30.00th=[ 136], 40.00th=[ 136], 50.00th=[ 136], 60.00th=[ 136], | 70.00th=[ 136], 80.00th=[ 136], 90.00th=[ 136], 95.00th=[ 136], | 99.00th=[ 136], 99.50th=[ 136], 99.90th=[ 136], 99.95th=[ 136], | 99.99th=[ 136] bw ( KiB/s): min= 7314, max= 7314, per=82.94%, avg=7314.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4455KiB/s (4562kB/s)(8192KiB/1839msec); 0 zone resets clat (msec): min=323, max=1367, avg=845.31, stdev=737.89 lat (msec): min=323, max=1367, avg=845.58, stdev=737.87 clat percentiles (msec): | 1.00th=[ 326], 5.00th=[ 326], 10.00th=[ 326], 20.00th=[ 326], | 30.00th=[ 326], 40.00th=[ 326], 50.00th=[ 326], 60.00th=[ 1368], | 70.00th=[ 1368], 80.00th=[ 1368], 90.00th=[ 1368], 95.00th=[ 1368], | 99.00th=[ 1368], 99.50th=[ 1368], 99.90th=[ 1368], 99.95th=[ 1368], | 99.99th=[ 1368] bw ( KiB/s): min= 7314, max= 7314, per=41.05%, avg=7314.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.05%, sys=17.14%, ctx=86, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183503: Thu Nov 13 21:03:01 2025 write: IOPS=1, BW=7367KiB/s (7544kB/s)(12.0MiB/1668msec); 0 zone resets clat (msec): min=311, max=818, avg=547.39, stdev=254.91 lat (msec): min=312, max=818, avg=547.68, stdev=254.92 clat percentiles (msec): | 1.00th=[ 313], 5.00th=[ 313], 10.00th=[ 313], 20.00th=[ 313], | 30.00th=[ 313], 40.00th=[ 514], 50.00th=[ 514], 60.00th=[ 514], | 70.00th=[ 818], 80.00th=[ 818], 90.00th=[ 818], 95.00th=[ 818], | 99.00th=[ 818], 99.50th=[ 818], 99.90th=[ 818], 99.95th=[ 818], | 99.99th=[ 818] bw ( KiB/s): min= 7314, max= 8192, per=43.51%, avg=7753.00, stdev=620.84, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.06%, sys=21.18%, ctx=78, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183504: Thu Nov 13 21:03:01 2025 write: IOPS=2, BW=8253KiB/s (8451kB/s)(12.0MiB/1489msec); 0 zone resets clat (msec): min=141, max=1173, avg=495.79, stdev=587.38 lat (msec): min=141, max=1174, avg=496.01, stdev=587.45 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 142], | 30.00th=[ 142], 40.00th=[ 174], 50.00th=[ 174], 60.00th=[ 174], | 70.00th=[ 1167], 80.00th=[ 1167], 90.00th=[ 1167], 95.00th=[ 1167], | 99.00th=[ 1167], 99.50th=[ 1167], 99.90th=[ 1167], 99.95th=[ 1167], | 99.99th=[ 1167] bw ( KiB/s): min=15312, max=15312, per=85.94%, avg=15312.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, 2000=33.33% cpu : usr=0.00%, sys=17.67%, ctx=49, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8818KiB/s (9030kB/s), 2227KiB/s-6614KiB/s (2281kB/s-6772kB/s), io=16.0MiB (16.8MB), run=1839-1858msec WRITE: bw=17.4MiB/s (18.2MB/s), 4455KiB/s-8253KiB/s (4562kB/s-8451kB/s), io=32.0MiB (33.6MB), run=1489-1839msec rand-rw: (groupid=0, jobs=1): err= 0: pid=183506: Thu Nov 13 21:03:01 2025 read: IOPS=2, BW=9563KiB/s (9792kB/s)(12.0MiB/1285msec) clat (msec): min=391, max=460, avg=427.81, stdev=34.48 lat (msec): min=391, max=460, avg=427.82, stdev=34.48 clat percentiles (msec): | 1.00th=[ 393], 5.00th=[ 393], 10.00th=[ 393], 20.00th=[ 393], | 30.00th=[ 393], 40.00th=[ 430], 50.00th=[ 430], 60.00th=[ 430], | 70.00th=[ 460], 80.00th=[ 460], 90.00th=[ 460], 95.00th=[ 460], | 99.00th=[ 460], 99.50th=[ 460], 99.90th=[ 460], 99.95th=[ 460], | 99.99th=[ 460] bw ( KiB/s): min= 5793, max= 8192, per=73.75%, avg=6992.50, stdev=1696.35, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=100.00% cpu : usr=0.00%, sys=60.05%, ctx=3018, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183507: Thu Nov 13 21:03:01 2025 read: IOPS=0, BW=2370KiB/s (2427kB/s)(4096KiB/1728msec) clat (nsec): min=375437k, max=375437k, avg=375436804.00, stdev= 0.00 lat (nsec): min=375438k, max=375438k, avg=375438366.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 376], 5.00th=[ 376], 10.00th=[ 376], 20.00th=[ 376], | 30.00th=[ 376], 40.00th=[ 376], 50.00th=[ 376], 60.00th=[ 376], | 70.00th=[ 376], 80.00th=[ 376], 90.00th=[ 376], 95.00th=[ 376], | 99.00th=[ 376], 99.50th=[ 376], 99.90th=[ 376], 99.95th=[ 376], | 99.99th=[ 376] bw ( KiB/s): min= 6291, max= 6291, per=66.35%, avg=6291.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4741KiB/s (4855kB/s)(8192KiB/1728msec); 0 zone resets clat (msec): min=517, max=832, avg=675.25, stdev=222.86 lat (msec): min=517, max=832, avg=675.38, stdev=222.86 clat percentiles (msec): | 1.00th=[ 518], 5.00th=[ 518], 10.00th=[ 518], 20.00th=[ 518], | 30.00th=[ 518], 40.00th=[ 518], 50.00th=[ 518], 60.00th=[ 835], | 70.00th=[ 835], 80.00th=[ 835], 90.00th=[ 835], 95.00th=[ 835], | 99.00th=[ 835], 99.50th=[ 835], 99.90th=[ 835], 99.95th=[ 835], | 99.99th=[ 835] bw ( KiB/s): min= 8192, max= 8192, per=43.73%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=16.50%, ctx=897, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183508: Thu Nov 13 21:03:01 2025 write: IOPS=1, BW=7026KiB/s (7194kB/s)(12.0MiB/1749msec); 0 zone resets clat (msec): min=141, max=903, avg=582.25, stdev=394.53 lat (msec): min=142, max=903, avg=582.51, stdev=394.50 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 142], | 30.00th=[ 142], 40.00th=[ 701], 50.00th=[ 701], 60.00th=[ 701], | 70.00th=[ 902], 80.00th=[ 902], 90.00th=[ 902], 95.00th=[ 902], | 99.00th=[ 902], 99.50th=[ 902], 99.90th=[ 902], 99.95th=[ 902], | 99.99th=[ 902] bw ( KiB/s): min= 8192, max= 8192, per=43.73%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.11%, sys=15.16%, ctx=45, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183509: Thu Nov 13 21:03:01 2025 write: IOPS=1, BW=8010KiB/s (8203kB/s)(12.0MiB/1534msec); 0 zone resets clat (msec): min=186, max=772, avg=510.57, stdev=298.35 lat (msec): min=186, max=773, avg=510.80, stdev=298.28 clat percentiles (msec): | 1.00th=[ 186], 5.00th=[ 186], 10.00th=[ 186], 20.00th=[ 186], | 30.00th=[ 186], 40.00th=[ 575], 50.00th=[ 575], 60.00th=[ 575], | 70.00th=[ 776], 80.00th=[ 776], 90.00th=[ 776], 95.00th=[ 776], | 99.00th=[ 776], 99.50th=[ 776], 99.90th=[ 776], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 5818, max= 5818, per=31.05%, avg=5818.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=17.81%, ctx=55, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=9481KiB/s (9709kB/s), 2370KiB/s-9563KiB/s (2427kB/s-9792kB/s), io=16.0MiB (16.8MB), run=1285-1728msec WRITE: bw=18.3MiB/s (19.2MB/s), 4741KiB/s-8010KiB/s (4855kB/s-8203kB/s), io=32.0MiB (33.6MB), run=1534-1749msec