== sanity test 398b: DIO and buffer IO race ============== 19:24:46 (1755041086) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.98759 s, 16.8 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=153940: Tue Aug 12 19:25:45 2025 read: IOPS=30, BW=124KiB/s (127kB/s)(5992KiB/48350msec) clat (usec): min=79, max=972216, avg=17008.05, stdev=26762.32 lat (usec): min=79, max=972217, avg=17008.72, stdev=26762.34 clat percentiles (usec): | 1.00th=[ 194], 5.00th=[ 4228], 10.00th=[ 6718], 20.00th=[ 8586], | 30.00th=[ 10290], 40.00th=[ 12387], 50.00th=[ 14615], 60.00th=[ 16581], | 70.00th=[ 19530], 80.00th=[ 22938], 90.00th=[ 28967], 95.00th=[ 34341], | 99.00th=[ 52691], 99.50th=[ 61604], 99.90th=[ 88605], 99.95th=[968885], | 99.99th=[968885] bw ( KiB/s): min= 13, max= 247, per=25.44%, avg=124.89, stdev=37.10, samples=95 iops : min= 3, max= 61, avg=30.96, stdev= 9.26, samples=95 write: IOPS=32, BW=130KiB/s (133kB/s)(6296KiB/48350msec); 0 zone resets clat (usec): min=609, max=157693, avg=14399.37, stdev=12854.01 lat (usec): min=610, max=157695, avg=14400.49, stdev=12854.00 clat percentiles (usec): | 1.00th=[ 766], 5.00th=[ 1893], 10.00th=[ 2474], 20.00th=[ 3589], | 30.00th=[ 5669], 40.00th=[ 8455], 50.00th=[ 11207], 60.00th=[ 14615], | 70.00th=[ 18482], 80.00th=[ 22414], 90.00th=[ 30016], 95.00th=[ 37487], | 99.00th=[ 57934], 99.50th=[ 65799], 99.90th=[ 98042], 99.95th=[158335], | 99.99th=[158335] bw ( KiB/s): min= 48, max= 247, per=25.84%, avg=132.83, stdev=41.42, samples=94 iops : min= 12, max= 61, avg=32.94, stdev=10.39, samples=94 lat (usec) : 100=0.16%, 250=0.49%, 500=1.14%, 750=0.81%, 1000=0.59% lat (msec) : 2=1.92%, 4=8.92%, 10=23.27%, 20=35.29%, 50=25.85% lat (msec) : 100=1.50%, 250=0.03%, 1000=0.03% cpu : usr=0.10%, sys=27.09%, ctx=7950, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153941: Tue Aug 12 19:25:45 2025 read: IOPS=30, BW=123KiB/s (126kB/s)(5940KiB/48225msec) clat (usec): min=73, max=108216, avg=16168.54, stdev=10355.99 lat (usec): min=73, max=108217, avg=16169.19, stdev=10356.06 clat percentiles (usec): | 1.00th=[ 258], 5.00th=[ 5538], 10.00th=[ 6915], 20.00th=[ 8586], | 30.00th=[ 10159], 40.00th=[ 12125], 50.00th=[ 13829], 60.00th=[ 16319], | 70.00th=[ 19006], 80.00th=[ 22414], 90.00th=[ 28443], 95.00th=[ 33817], | 99.00th=[ 51643], 99.50th=[ 65274], 99.90th=[ 99091], 99.95th=[108528], | 99.99th=[108528] bw ( KiB/s): min= 6, max= 215, per=25.39%, avg=124.66, stdev=41.78, samples=95 iops : min= 1, max= 53, avg=30.89, stdev=10.46, samples=95 write: IOPS=32, BW=132KiB/s (135kB/s)(6348KiB/48225msec); 0 zone resets clat (usec): min=634, max=968333, avg=15121.71, stdev=27371.70 lat (usec): min=634, max=968333, avg=15122.57, stdev=27371.70 clat percentiles (usec): | 1.00th=[ 832], 5.00th=[ 1696], 10.00th=[ 2376], 20.00th=[ 3556], | 30.00th=[ 5932], 40.00th=[ 8717], 50.00th=[ 11600], 60.00th=[ 14484], | 70.00th=[ 17957], 80.00th=[ 22938], 90.00th=[ 30278], 95.00th=[ 38536], | 99.00th=[ 62653], 99.50th=[ 83362], 99.90th=[129500], 99.95th=[968885], | 99.99th=[968885] bw ( KiB/s): min= 64, max= 223, per=26.19%, avg=134.60, stdev=38.18, samples=94 iops : min= 16, max= 55, avg=33.38, stdev= 9.57, samples=94 lat (usec) : 100=0.13%, 250=0.33%, 500=0.98%, 750=0.65%, 1000=0.72% lat (msec) : 2=2.60%, 4=7.91%, 10=23.86%, 20=36.85%, 50=24.22% lat (msec) : 100=1.60%, 250=0.13%, 1000=0.03% cpu : usr=0.09%, sys=27.13%, ctx=7756, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153942: Tue Aug 12 19:25:45 2025 read: IOPS=31, BW=124KiB/s (127kB/s)(6028KiB/48546msec) clat (usec): min=80, max=127209, avg=16587.07, stdev=10658.60 lat (usec): min=80, max=127210, avg=16587.82, stdev=10658.66 clat percentiles (usec): | 1.00th=[ 176], 5.00th=[ 1516], 10.00th=[ 6849], 20.00th=[ 8848], | 30.00th=[ 10552], 40.00th=[ 12649], 50.00th=[ 14615], 60.00th=[ 16909], | 70.00th=[ 19530], 80.00th=[ 23200], 90.00th=[ 29230], 95.00th=[ 35390], | 99.00th=[ 53216], 99.50th=[ 60556], 99.90th=[ 86508], 99.95th=[127402], | 99.99th=[127402] bw ( KiB/s): min= 64, max= 208, per=25.68%, avg=126.09, stdev=36.03, samples=94 iops : min= 16, max= 52, avg=31.26, stdev= 8.99, samples=94 write: IOPS=32, BW=129KiB/s (132kB/s)(6260KiB/48546msec); 0 zone resets clat (usec): min=519, max=952737, avg=14913.54, stdev=26717.57 lat (usec): min=520, max=952738, avg=14914.44, stdev=26717.58 clat percentiles (usec): | 1.00th=[ 832], 5.00th=[ 1876], 10.00th=[ 2540], 20.00th=[ 3916], | 30.00th=[ 6128], 40.00th=[ 8979], 50.00th=[ 11600], 60.00th=[ 14484], | 70.00th=[ 17957], 80.00th=[ 22152], 90.00th=[ 29230], 95.00th=[ 35390], | 99.00th=[ 56361], 99.50th=[ 77071], 99.90th=[112722], 99.95th=[952108], | 99.99th=[952108] bw ( KiB/s): min= 6, max= 215, per=25.12%, avg=129.14, stdev=37.13, samples=95 iops : min= 1, max= 53, avg=32.01, stdev= 9.31, samples=95 lat (usec) : 100=0.07%, 250=0.78%, 500=1.24%, 750=0.49%, 1000=0.62% lat (msec) : 2=2.21%, 4=7.68%, 10=22.43%, 20=38.22%, 50=24.74% lat (msec) : 100=1.43%, 250=0.07%, 1000=0.03% cpu : usr=0.08%, sys=27.25%, ctx=7975, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153943: Tue Aug 12 19:25:45 2025 read: IOPS=30, BW=124KiB/s (127kB/s)(6052KiB/48901msec) clat (usec): min=76, max=99702, avg=16253.39, stdev=10625.28 lat (usec): min=76, max=99703, avg=16254.18, stdev=10625.39 clat percentiles (usec): | 1.00th=[ 239], 5.00th=[ 3163], 10.00th=[ 6521], 20.00th=[ 8160], | 30.00th=[ 10028], 40.00th=[ 12256], 50.00th=[ 14222], 60.00th=[ 16319], | 70.00th=[ 19268], 80.00th=[ 22676], 90.00th=[ 28705], 95.00th=[ 35390], | 99.00th=[ 54789], 99.50th=[ 62129], 99.90th=[ 82314], 99.95th=[100140], | 99.99th=[100140] bw ( KiB/s): min= 47, max= 239, per=25.51%, avg=125.27, stdev=38.19, samples=95 iops : min= 11, max= 59, avg=31.05, stdev= 9.59, samples=95 write: IOPS=31, BW=128KiB/s (131kB/s)(6236KiB/48901msec); 0 zone resets clat (usec): min=571, max=982547, avg=15457.80, stdev=28044.74 lat (usec): min=571, max=982549, avg=15458.64, stdev=28044.77 clat percentiles (usec): | 1.00th=[ 734], 5.00th=[ 1745], 10.00th=[ 2442], 20.00th=[ 3490], | 30.00th=[ 5669], 40.00th=[ 8586], 50.00th=[ 11469], 60.00th=[ 14746], | 70.00th=[ 18220], 80.00th=[ 23200], 90.00th=[ 32113], 95.00th=[ 39584], | 99.00th=[ 63701], 99.50th=[ 76022], 99.90th=[145753], 99.95th=[985662], | 99.99th=[985662] bw ( KiB/s): min= 56, max= 248, per=25.19%, avg=129.47, stdev=37.33, samples=95 iops : min= 14, max= 62, avg=32.11, stdev= 9.34, samples=95 lat (usec) : 100=0.13%, 250=0.42%, 500=1.04%, 750=1.20%, 1000=0.68% lat (msec) : 2=2.28%, 4=8.50%, 10=22.95%, 20=35.71%, 50=25.33% lat (msec) : 100=1.66%, 250=0.07%, 1000=0.03% cpu : usr=0.11%, sys=26.54%, ctx=8051, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=491KiB/s (503kB/s), 123KiB/s-124KiB/s (126kB/s-127kB/s), io=23.4MiB (24.6MB), run=48225-48901msec WRITE: bw=514KiB/s (526kB/s), 128KiB/s-132KiB/s (131kB/s-135kB/s), io=24.6MiB (25.7MB), run=48225-48901msec rand-rw: (groupid=0, jobs=1): err= 0: pid=153945: Tue Aug 12 19:26:11 2025 read: IOPS=20, BW=80.5KiB/s (82.5kB/s)(5992KiB/74398msec) clat (msec): min=2, max=986, avg=14.85, stdev=28.82 lat (msec): min=2, max=986, avg=14.85, stdev=28.82 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 10], 60.00th=[ 13], | 70.00th=[ 16], 80.00th=[ 21], 90.00th=[ 32], 95.00th=[ 43], | 99.00th=[ 68], 99.50th=[ 85], 99.90th=[ 136], 99.95th=[ 986], | 99.99th=[ 986] bw ( KiB/s): min= 8, max= 216, per=25.15%, avg=80.99, stdev=49.31, samples=147 iops : min= 2, max= 54, avg=20.01, stdev=12.45, samples=147 write: IOPS=21, BW=84.6KiB/s (86.7kB/s)(6296KiB/74398msec); 0 zone resets clat (msec): min=10, max=229, avg=33.04, stdev=18.79 lat (msec): min=10, max=229, avg=33.04, stdev=18.79 clat percentiles (msec): | 1.00th=[ 14], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 20], | 30.00th=[ 22], 40.00th=[ 24], 50.00th=[ 27], 60.00th=[ 31], | 70.00th=[ 38], 80.00th=[ 46], 90.00th=[ 58], 95.00th=[ 71], | 99.00th=[ 96], 99.50th=[ 114], 99.90th=[ 157], 99.95th=[ 230], | 99.99th=[ 230] bw ( KiB/s): min= 8, max= 192, per=25.42%, avg=85.66, stdev=44.18, samples=146 iops : min= 2, max= 48, avg=21.18, stdev=11.17, samples=146 lat (msec) : 4=6.80%, 10=17.58%, 20=26.76%, 50=39.10%, 100=9.15% lat (msec) : 250=0.59%, 1000=0.03% cpu : usr=0.08%, sys=9.58%, ctx=4152, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153946: Tue Aug 12 19:26:11 2025 read: IOPS=19, BW=79.8KiB/s (81.8kB/s)(5940KiB/74389msec) clat (msec): min=2, max=955, avg=15.06, stdev=28.15 lat (msec): min=2, max=955, avg=15.06, stdev=28.15 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 16], 80.00th=[ 22], 90.00th=[ 32], 95.00th=[ 43], | 99.00th=[ 67], 99.50th=[ 78], 99.90th=[ 148], 99.95th=[ 953], | 99.99th=[ 953] bw ( KiB/s): min= 7, max= 216, per=24.95%, avg=80.34, stdev=49.23, samples=147 iops : min= 1, max= 54, avg=19.85, stdev=12.43, samples=147 write: IOPS=21, BW=85.3KiB/s (87.4kB/s)(6348KiB/74389msec); 0 zone resets clat (msec): min=12, max=156, avg=32.69, stdev=18.04 lat (msec): min=12, max=156, avg=32.69, stdev=18.04 clat percentiles (msec): | 1.00th=[ 14], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 20], | 30.00th=[ 22], 40.00th=[ 24], 50.00th=[ 27], 60.00th=[ 30], | 70.00th=[ 37], 80.00th=[ 45], 90.00th=[ 57], 95.00th=[ 68], | 99.00th=[ 97], 99.50th=[ 120], 99.90th=[ 136], 99.95th=[ 157], | 99.99th=[ 157] bw ( KiB/s): min= 24, max= 184, per=25.63%, avg=86.37, stdev=43.39, samples=146 iops : min= 6, max= 46, avg=21.36, stdev=10.97, samples=146 lat (msec) : 4=6.32%, 10=17.58%, 20=26.20%, 50=40.79%, 100=8.46% lat (msec) : 250=0.62%, 1000=0.03% cpu : usr=0.09%, sys=9.67%, ctx=4184, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153947: Tue Aug 12 19:26:11 2025 read: IOPS=20, BW=81.2KiB/s (83.2kB/s)(6028KiB/74202msec) clat (msec): min=2, max=136, avg=14.51, stdev=14.48 lat (msec): min=2, max=136, avg=14.51, stdev=14.48 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 16], 80.00th=[ 21], 90.00th=[ 32], 95.00th=[ 42], | 99.00th=[ 70], 99.50th=[ 94], 99.90th=[ 116], 99.95th=[ 138], | 99.99th=[ 138] bw ( KiB/s): min= 15, max= 224, per=25.59%, avg=82.39, stdev=49.32, samples=145 iops : min= 3, max= 56, avg=20.36, stdev=12.45, samples=145 write: IOPS=21, BW=84.4KiB/s (86.4kB/s)(6260KiB/74202msec); 0 zone resets clat (msec): min=12, max=960, avg=33.35, stdev=30.16 lat (msec): min=12, max=960, avg=33.35, stdev=30.16 clat percentiles (msec): | 1.00th=[ 14], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 20], | 30.00th=[ 22], 40.00th=[ 24], 50.00th=[ 26], 60.00th=[ 30], | 70.00th=[ 37], 80.00th=[ 46], 90.00th=[ 57], 95.00th=[ 69], | 99.00th=[ 97], 99.50th=[ 121], 99.90th=[ 230], 99.95th=[ 961], | 99.99th=[ 961] bw ( KiB/s): min= 16, max= 192, per=25.14%, avg=84.73, stdev=44.97, samples=146 iops : min= 4, max= 48, avg=20.95, stdev=11.37, samples=146 lat (msec) : 4=6.28%, 10=18.03%, 20=27.15%, 50=38.90%, 100=8.98% lat (msec) : 250=0.62%, 1000=0.03% cpu : usr=0.08%, sys=9.64%, ctx=4182, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153948: Tue Aug 12 19:26:11 2025 read: IOPS=20, BW=81.6KiB/s (83.5kB/s)(6052KiB/74185msec) clat (msec): min=2, max=122, avg=14.14, stdev=13.88 lat (msec): min=2, max=122, avg=14.14, stdev=13.88 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 16], 80.00th=[ 21], 90.00th=[ 31], 95.00th=[ 41], | 99.00th=[ 66], 99.50th=[ 86], 99.90th=[ 120], 99.95th=[ 123], | 99.99th=[ 123] bw ( KiB/s): min= 15, max= 240, per=25.64%, avg=82.55, stdev=49.81, samples=145 iops : min= 3, max= 60, avg=20.40, stdev=12.57, samples=145 write: IOPS=21, BW=84.1KiB/s (86.1kB/s)(6236KiB/74185msec); 0 zone resets clat (msec): min=11, max=977, avg=33.77, stdev=30.34 lat (msec): min=11, max=977, avg=33.77, stdev=30.34 clat percentiles (msec): | 1.00th=[ 14], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 20], | 30.00th=[ 22], 40.00th=[ 24], 50.00th=[ 27], 60.00th=[ 31], | 70.00th=[ 38], 80.00th=[ 47], 90.00th=[ 59], 95.00th=[ 70], | 99.00th=[ 101], 99.50th=[ 121], 99.90th=[ 157], 99.95th=[ 978], | 99.99th=[ 978] bw ( KiB/s): min= 8, max= 184, per=25.06%, avg=84.45, stdev=44.43, samples=146 iops : min= 2, max= 46, avg=20.88, stdev=11.24, samples=146 lat (msec) : 4=7.23%, 10=17.29%, 20=26.50%, 50=39.39%, 100=8.95% lat (msec) : 250=0.62%, 1000=0.03% cpu : usr=0.08%, sys=9.55%, ctx=4205, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=323KiB/s (330kB/s), 79.8KiB/s-81.6KiB/s (81.8kB/s-83.5kB/s), io=23.4MiB (24.6MB), run=74185-74398msec WRITE: bw=338KiB/s (346kB/s), 84.1KiB/s-85.3KiB/s (86.1kB/s-87.4kB/s), io=24.6MiB (25.7MB), run=74185-74398msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=153970: Tue Aug 12 19:26:29 2025 read: IOPS=29, BW=471KiB/s (482kB/s)(5920KiB/12581msec) clat (usec): min=186, max=96540, avg=16858.64, stdev=11045.78 lat (usec): min=187, max=96541, avg=16859.32, stdev=11045.80 clat percentiles (usec): | 1.00th=[ 273], 5.00th=[ 429], 10.00th=[ 799], 20.00th=[ 9896], | 30.00th=[11731], 40.00th=[13435], 50.00th=[15139], 60.00th=[18220], | 70.00th=[20317], 80.00th=[22938], 90.00th=[29230], 95.00th=[34341], | 99.00th=[57410], 99.50th=[71828], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 191, max= 704, per=24.05%, avg=457.58, stdev=153.19, samples=24 iops : min= 11, max= 44, avg=28.17, stdev= 9.88, samples=24 write: IOPS=31, BW=506KiB/s (518kB/s)(6368KiB/12581msec); 0 zone resets clat (usec): min=932, max=107124, avg=15780.10, stdev=13096.85 lat (usec): min=934, max=107125, avg=15781.25, stdev=13096.99 clat percentiles (usec): | 1.00th=[ 1254], 5.00th=[ 3228], 10.00th=[ 4293], 20.00th=[ 5080], | 30.00th=[ 6652], 40.00th=[ 9634], 50.00th=[ 12780], 60.00th=[ 15926], | 70.00th=[ 19530], 80.00th=[ 23987], 90.00th=[ 32113], 95.00th=[ 39060], | 99.00th=[ 58983], 99.50th=[ 94897], 99.90th=[107480], 99.95th=[107480], | 99.99th=[107480] bw ( KiB/s): min= 256, max= 800, per=24.44%, avg=489.50, stdev=143.93, samples=24 iops : min= 16, max= 50, avg=30.17, stdev= 9.05, samples=24 lat (usec) : 250=0.39%, 500=3.12%, 750=1.17%, 1000=0.26% lat (msec) : 2=1.30%, 4=2.99%, 10=22.14%, 20=38.54%, 50=28.52% lat (msec) : 100=1.43%, 250=0.13% cpu : usr=0.13%, sys=32.51%, ctx=2528, majf=0, minf=19 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153971: Tue Aug 12 19:26:29 2025 read: IOPS=28, BW=459KiB/s (470kB/s)(5744KiB/12519msec) clat (usec): min=244, max=75335, avg=18216.08, stdev=12098.06 lat (usec): min=244, max=75335, avg=18216.91, stdev=12098.09 clat percentiles (usec): | 1.00th=[ 322], 5.00th=[ 469], 10.00th=[ 7308], 20.00th=[10159], | 30.00th=[11731], 40.00th=[13435], 50.00th=[15795], 60.00th=[18220], | 70.00th=[21365], 80.00th=[25297], 90.00th=[33424], 95.00th=[41681], | 99.00th=[64226], 99.50th=[67634], 99.90th=[74974], 99.95th=[74974], | 99.99th=[74974] bw ( KiB/s): min= 159, max= 608, per=23.85%, avg=453.96, stdev=121.30, samples=24 iops : min= 9, max= 38, avg=27.96, stdev= 7.81, samples=24 write: IOPS=32, BW=523KiB/s (535kB/s)(6544KiB/12519msec); 0 zone resets clat (usec): min=1009, max=74795, avg=14505.72, stdev=11284.02 lat (usec): min=1009, max=74796, avg=14506.87, stdev=11284.03 clat percentiles (usec): | 1.00th=[ 1172], 5.00th=[ 3425], 10.00th=[ 4228], 20.00th=[ 5211], | 30.00th=[ 6063], 40.00th=[ 9110], 50.00th=[11600], 60.00th=[14353], | 70.00th=[17695], 80.00th=[22676], 90.00th=[28181], 95.00th=[34866], | 99.00th=[58459], 99.50th=[65799], 99.90th=[74974], 99.95th=[74974], | 99.99th=[74974] bw ( KiB/s): min= 255, max= 800, per=25.58%, avg=512.29, stdev=185.78, samples=24 iops : min= 15, max= 50, avg=31.58, stdev=11.88, samples=24 lat (usec) : 250=0.13%, 500=2.47%, 750=0.52%, 1000=0.13% lat (msec) : 2=1.56%, 4=3.26%, 10=23.57%, 20=39.32%, 50=27.08% lat (msec) : 100=1.95% cpu : usr=0.06%, sys=32.68%, ctx=2516, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153972: Tue Aug 12 19:26:29 2025 read: IOPS=31, BW=503KiB/s (515kB/s)(6256KiB/12436msec) clat (usec): min=314, max=60905, avg=16892.36, stdev=10413.33 lat (usec): min=315, max=60906, avg=16893.15, stdev=10413.34 clat percentiles (usec): | 1.00th=[ 351], 5.00th=[ 478], 10.00th=[ 2933], 20.00th=[ 9503], | 30.00th=[12125], 40.00th=[13960], 50.00th=[15270], 60.00th=[17171], | 70.00th=[20055], 80.00th=[23987], 90.00th=[29230], 95.00th=[34866], | 99.00th=[55837], 99.50th=[60556], 99.90th=[61080], 99.95th=[61080], | 99.99th=[61080] bw ( KiB/s): min= 191, max= 892, per=26.30%, avg=500.50, stdev=168.21, samples=24 iops : min= 11, max= 55, avg=30.87, stdev=10.67, samples=24 write: IOPS=30, BW=485KiB/s (497kB/s)(6032KiB/12436msec); 0 zone resets clat (usec): min=1218, max=82553, avg=15410.34, stdev=11484.78 lat (usec): min=1219, max=82554, avg=15411.59, stdev=11484.82 clat percentiles (usec): | 1.00th=[ 2671], 5.00th=[ 4080], 10.00th=[ 4883], 20.00th=[ 5669], | 30.00th=[ 7701], 40.00th=[10159], 50.00th=[12125], 60.00th=[13829], | 70.00th=[18482], 80.00th=[23725], 90.00th=[31065], 95.00th=[36439], | 99.00th=[54789], 99.50th=[62653], 99.90th=[82314], 99.95th=[82314], | 99.99th=[82314] bw ( KiB/s): min= 222, max= 736, per=23.53%, avg=471.25, stdev=136.39, samples=24 iops : min= 13, max= 46, avg=29.04, stdev= 8.83, samples=24 lat (usec) : 500=2.60%, 750=1.95%, 1000=0.52% lat (msec) : 2=0.26%, 4=1.95%, 10=22.79%, 20=41.80%, 50=26.69% lat (msec) : 100=1.43% cpu : usr=0.12%, sys=32.67%, ctx=2664, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153973: Tue Aug 12 19:26:29 2025 read: IOPS=30, BW=491KiB/s (503kB/s)(6032KiB/12277msec) clat (usec): min=179, max=114907, avg=16989.90, stdev=12325.64 lat (usec): min=180, max=114908, avg=16990.71, stdev=12325.65 clat percentiles (usec): | 1.00th=[ 306], 5.00th=[ 375], 10.00th=[ 515], 20.00th=[ 9241], | 30.00th=[ 11600], 40.00th=[ 13698], 50.00th=[ 15795], 60.00th=[ 17957], | 70.00th=[ 20055], 80.00th=[ 23200], 90.00th=[ 29492], 95.00th=[ 36439], | 99.00th=[ 56886], 99.50th=[ 91751], 99.90th=[114820], 99.95th=[114820], | 99.99th=[114820] bw ( KiB/s): min= 191, max= 736, per=25.48%, avg=484.96, stdev=153.58, samples=24 iops : min= 11, max= 46, avg=29.87, stdev= 9.79, samples=24 write: IOPS=31, BW=510KiB/s (522kB/s)(6256KiB/12277msec); 0 zone resets clat (usec): min=1033, max=99790, avg=14917.22, stdev=11883.60 lat (usec): min=1034, max=99791, avg=14920.68, stdev=11884.27 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 3589], 10.00th=[ 4293], 20.00th=[ 5276], | 30.00th=[ 7439], 40.00th=[ 10028], 50.00th=[ 11731], 60.00th=[ 14615], | 70.00th=[ 17695], 80.00th=[ 21890], 90.00th=[ 30016], 95.00th=[ 38011], | 99.00th=[ 65799], 99.50th=[ 70779], 99.90th=[100140], 99.95th=[100140], | 99.99th=[100140] bw ( KiB/s): min= 287, max= 736, per=25.06%, avg=501.96, stdev=133.00, samples=24 iops : min= 17, max= 46, avg=30.96, stdev= 8.54, samples=24 lat (usec) : 250=0.13%, 500=4.30%, 750=2.08% lat (msec) : 2=0.91%, 4=3.12%, 10=20.83%, 20=41.41%, 50=25.78% lat (msec) : 100=1.30%, 250=0.13% cpu : usr=0.06%, sys=32.59%, ctx=2468, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1904KiB/s (1950kB/s), 459KiB/s-503KiB/s (470kB/s-515kB/s), io=23.4MiB (24.5MB), run=12277-12581msec WRITE: bw=2003KiB/s (2051kB/s), 485KiB/s-523KiB/s (497kB/s-535kB/s), io=24.6MiB (25.8MB), run=12277-12581msec rand-rw: (groupid=0, jobs=1): err= 0: pid=153975: Tue Aug 12 19:26:36 2025 read: IOPS=19, BW=304KiB/s (311kB/s)(5920KiB/19464msec) clat (msec): min=2, max=121, avg=15.89, stdev=15.04 lat (msec): min=2, max=121, avg=15.89, stdev=15.04 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 9], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 18], 80.00th=[ 22], 90.00th=[ 36], 95.00th=[ 46], | 99.00th=[ 67], 99.50th=[ 112], 99.90th=[ 122], 99.95th=[ 122], | 99.99th=[ 122] bw ( KiB/s): min= 32, max= 736, per=23.95%, avg=294.53, stdev=150.11, samples=38 iops : min= 2, max= 46, avg=18.08, stdev= 9.54, samples=38 write: IOPS=20, BW=327KiB/s (335kB/s)(6368KiB/19464msec); 0 zone resets clat (msec): min=12, max=194, avg=33.99, stdev=19.51 lat (msec): min=12, max=194, avg=33.99, stdev=19.51 clat percentiles (msec): | 1.00th=[ 14], 5.00th=[ 18], 10.00th=[ 19], 20.00th=[ 22], | 30.00th=[ 24], 40.00th=[ 26], 50.00th=[ 28], 60.00th=[ 31], | 70.00th=[ 34], 80.00th=[ 44], 90.00th=[ 59], 95.00th=[ 72], | 99.00th=[ 109], 99.50th=[ 157], 99.90th=[ 194], 99.95th=[ 194], | 99.99th=[ 194] bw ( KiB/s): min= 64, max= 640, per=24.85%, avg=321.53, stdev=151.01, samples=38 iops : min= 4, max= 40, avg=19.79, stdev= 9.57, samples=38 lat (msec) : 4=4.30%, 10=17.71%, 20=21.35%, 50=46.61%, 100=9.11% lat (msec) : 250=0.91% cpu : usr=0.08%, sys=11.62%, ctx=1206, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153976: Tue Aug 12 19:26:36 2025 read: IOPS=18, BW=299KiB/s (306kB/s)(5744KiB/19215msec) clat (usec): min=3051, max=77793, avg=13927.96, stdev=11509.41 lat (usec): min=3052, max=77794, avg=13928.57, stdev=11509.39 clat percentiles (usec): | 1.00th=[ 3228], 5.00th=[ 3752], 10.00th=[ 4359], 20.00th=[ 5276], | 30.00th=[ 6456], 40.00th=[ 8160], 50.00th=[10159], 60.00th=[12649], | 70.00th=[15533], 80.00th=[19792], 90.00th=[28967], 95.00th=[38536], | 99.00th=[57934], 99.50th=[65274], 99.90th=[78119], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 58, max= 832, per=24.26%, avg=298.37, stdev=179.59, samples=38 iops : min= 3, max= 52, avg=18.32, stdev=11.46, samples=38 write: IOPS=21, BW=341KiB/s (349kB/s)(6544KiB/19215msec); 0 zone resets clat (msec): min=13, max=121, avg=34.61, stdev=18.13 lat (msec): min=13, max=121, avg=34.61, stdev=18.13 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 18], 10.00th=[ 19], 20.00th=[ 22], | 30.00th=[ 24], 40.00th=[ 26], 50.00th=[ 29], 60.00th=[ 33], | 70.00th=[ 37], 80.00th=[ 45], 90.00th=[ 59], 95.00th=[ 75], | 99.00th=[ 96], 99.50th=[ 110], 99.90th=[ 122], 99.95th=[ 122], | 99.99th=[ 122] bw ( KiB/s): min= 191, max= 638, per=26.26%, avg=339.76, stdev=139.37, samples=38 iops : min= 11, max= 39, avg=20.92, stdev= 8.84, samples=38 lat (msec) : 4=2.86%, 10=20.18%, 20=22.53%, 50=45.96%, 100=7.94% lat (msec) : 250=0.52% cpu : usr=0.09%, sys=10.99%, ctx=1103, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153977: Tue Aug 12 19:26:36 2025 read: IOPS=20, BW=326KiB/s (334kB/s)(6256KiB/19184msec) clat (msec): min=2, max=111, avg=16.20, stdev=15.24 lat (msec): min=2, max=111, avg=16.20, stdev=15.24 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 9], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 18], 80.00th=[ 24], 90.00th=[ 33], 95.00th=[ 49], | 99.00th=[ 73], 99.50th=[ 102], 99.90th=[ 111], 99.95th=[ 111], | 99.99th=[ 111] bw ( KiB/s): min= 94, max= 960, per=26.69%, avg=328.26, stdev=183.53, samples=38 iops : min= 5, max= 60, avg=20.21, stdev=11.56, samples=38 write: IOPS=19, BW=314KiB/s (322kB/s)(6032KiB/19184msec); 0 zone resets clat (msec): min=14, max=149, avg=33.94, stdev=17.63 lat (msec): min=14, max=149, avg=33.94, stdev=17.63 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 18], 10.00th=[ 20], 20.00th=[ 22], | 30.00th=[ 25], 40.00th=[ 27], 50.00th=[ 29], 60.00th=[ 32], | 70.00th=[ 37], 80.00th=[ 43], 90.00th=[ 55], 95.00th=[ 63], | 99.00th=[ 110], 99.50th=[ 125], 99.90th=[ 150], 99.95th=[ 150], | 99.99th=[ 150] bw ( KiB/s): min= 126, max= 640, per=24.19%, avg=313.03, stdev=145.24, samples=38 iops : min= 7, max= 40, avg=19.24, stdev= 9.27, samples=38 lat (msec) : 4=3.78%, 10=18.88%, 20=21.09%, 50=47.53%, 100=7.81% lat (msec) : 250=0.91% cpu : usr=0.07%, sys=11.38%, ctx=1218, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153978: Tue Aug 12 19:26:36 2025 read: IOPS=19, BW=311KiB/s (319kB/s)(6032KiB/19386msec) clat (msec): min=2, max=121, avg=16.34, stdev=15.95 lat (msec): min=2, max=121, avg=16.34, stdev=15.95 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 8], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 23], 90.00th=[ 35], 95.00th=[ 44], | 99.00th=[ 93], 99.50th=[ 97], 99.90th=[ 122], 99.95th=[ 122], | 99.99th=[ 122] bw ( KiB/s): min= 126, max= 704, per=24.84%, avg=305.50, stdev=140.53, samples=38 iops : min= 7, max= 44, avg=18.76, stdev= 8.91, samples=38 write: IOPS=20, BW=323KiB/s (330kB/s)(6256KiB/19386msec); 0 zone resets clat (msec): min=14, max=142, avg=33.72, stdev=17.07 lat (msec): min=14, max=142, avg=33.72, stdev=17.07 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 18], 10.00th=[ 19], 20.00th=[ 22], | 30.00th=[ 24], 40.00th=[ 26], 50.00th=[ 29], 60.00th=[ 32], | 70.00th=[ 36], 80.00th=[ 43], 90.00th=[ 57], 95.00th=[ 69], | 99.00th=[ 100], 99.50th=[ 116], 99.90th=[ 142], 99.95th=[ 142], | 99.99th=[ 142] bw ( KiB/s): min= 127, max= 640, per=24.71%, avg=319.76, stdev=149.56, samples=38 iops : min= 7, max= 40, avg=19.66, stdev= 9.54, samples=38 lat (msec) : 4=2.08%, 10=19.01%, 20=23.70%, 50=46.22%, 100=8.46% lat (msec) : 250=0.52% cpu : usr=0.09%, sys=11.30%, ctx=1179, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1231KiB/s (1260kB/s), 299KiB/s-326KiB/s (306kB/s-334kB/s), io=23.4MiB (24.5MB), run=19184-19464msec WRITE: bw=1295KiB/s (1326kB/s), 314KiB/s-341KiB/s (322kB/s-349kB/s), io=24.6MiB (25.8MB), run=19184-19464msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=153986: Tue Aug 12 19:26:45 2025 read: IOPS=2, BW=2698KiB/s (2763kB/s)(7168KiB/2657msec) clat (msec): min=96, max=839, avg=242.10, stdev=269.13 lat (msec): min=96, max=839, avg=242.10, stdev=269.13 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 96], 10.00th=[ 96], 20.00th=[ 109], | 30.00th=[ 113], 40.00th=[ 113], 50.00th=[ 128], 60.00th=[ 150], | 70.00th=[ 150], 80.00th=[ 262], 90.00th=[ 844], 95.00th=[ 844], | 99.00th=[ 844], 99.50th=[ 844], 99.90th=[ 844], 99.95th=[ 844], | 99.99th=[ 844] bw ( KiB/s): min= 2000, max= 6119, per=59.59%, avg=3893.33, stdev=2079.51, samples=3 iops : min= 1, max= 5, avg= 3.00, stdev= 2.00, samples=3 write: IOPS=1, BW=1927KiB/s (1973kB/s)(5120KiB/2657msec); 0 zone resets clat (msec): min=86, max=300, avg=185.35, stdev=87.78 lat (msec): min=86, max=300, avg=185.40, stdev=87.79 clat percentiles (msec): | 1.00th=[ 87], 5.00th=[ 87], 10.00th=[ 87], 20.00th=[ 87], | 30.00th=[ 115], 40.00th=[ 115], 50.00th=[ 186], 60.00th=[ 186], | 70.00th=[ 241], 80.00th=[ 241], 90.00th=[ 300], 95.00th=[ 300], | 99.00th=[ 300], 99.50th=[ 300], 99.90th=[ 300], 99.95th=[ 300], | 99.99th=[ 300] bw ( KiB/s): min= 2000, max= 6047, per=36.75%, avg=3362.00, stdev=2325.36, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 lat (msec) : 100=16.67%, 250=58.33%, 500=16.67%, 1000=8.33% cpu : usr=0.04%, sys=44.62%, ctx=1367, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153987: Tue Aug 12 19:26:45 2025 read: IOPS=0, BW=719KiB/s (736kB/s)(2048KiB/2848msec) clat (usec): min=827, max=93935, avg=47381.51, stdev=65837.44 lat (usec): min=828, max=93937, avg=47382.71, stdev=65837.76 clat percentiles (usec): | 1.00th=[ 832], 5.00th=[ 832], 10.00th=[ 832], 20.00th=[ 832], | 30.00th=[ 832], 40.00th=[ 832], 50.00th=[ 832], 60.00th=[93848], | 70.00th=[93848], 80.00th=[93848], 90.00th=[93848], 95.00th=[93848], | 99.00th=[93848], 99.50th=[93848], 99.90th=[93848], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 1750, max= 2015, per=28.81%, avg=1882.50, stdev=187.38, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3596KiB/s (3682kB/s)(10.0MiB/2848msec); 0 zone resets clat (msec): min=94, max=1097, avg=274.11, stdev=313.84 lat (msec): min=94, max=1097, avg=274.17, stdev=313.85 clat percentiles (msec): | 1.00th=[ 95], 5.00th=[ 95], 10.00th=[ 95], 20.00th=[ 97], | 30.00th=[ 104], 40.00th=[ 111], 50.00th=[ 126], 60.00th=[ 161], | 70.00th=[ 203], 80.00th=[ 255], 90.00th=[ 498], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 2015, max= 8192, per=66.34%, avg=6069.00, stdev=3512.18, samples=3 iops : min= 1, max= 8, avg= 5.33, stdev= 3.79, samples=3 lat (usec) : 1000=8.33% lat (msec) : 100=25.00%, 250=41.67%, 500=16.67%, 2000=8.33% cpu : usr=0.00%, sys=37.37%, ctx=389, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153988: Tue Aug 12 19:26:45 2025 read: IOPS=1, BW=1960KiB/s (2007kB/s)(6144KiB/3134msec) clat (usec): min=623, max=240381, avg=117464.30, stdev=100461.22 lat (usec): min=623, max=240382, avg=117465.13, stdev=100461.25 clat percentiles (usec): | 1.00th=[ 627], 5.00th=[ 627], 10.00th=[ 627], 20.00th=[ 685], | 30.00th=[ 685], 40.00th=[119014], 50.00th=[119014], 60.00th=[139461], | 70.00th=[204473], 80.00th=[204473], 90.00th=[240124], 95.00th=[240124], | 99.00th=[240124], 99.50th=[240124], 99.90th=[240124], 99.95th=[240124], | 99.99th=[240124] bw ( KiB/s): min= 2035, max= 4031, per=51.35%, avg=3355.33, stdev=1143.55, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=1, BW=1960KiB/s (2007kB/s)(6144KiB/3134msec); 0 zone resets clat (msec): min=142, max=1243, avg=398.72, stdev=416.46 lat (msec): min=142, max=1243, avg=398.77, stdev=416.47 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 220], | 30.00th=[ 220], 40.00th=[ 257], 50.00th=[ 257], 60.00th=[ 266], | 70.00th=[ 266], 80.00th=[ 266], 90.00th=[ 1250], 95.00th=[ 1250], | 99.00th=[ 1250], 99.50th=[ 1250], 99.90th=[ 1250], 99.95th=[ 1250], | 99.99th=[ 1250] bw ( KiB/s): min= 3500, max= 4071, per=42.28%, avg=3867.33, stdev=318.75, samples=3 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=3 lat (usec) : 750=16.67% lat (msec) : 250=50.00%, 500=25.00%, 2000=8.33% cpu : usr=0.03%, sys=29.62%, ctx=474, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153989: Tue Aug 12 19:26:45 2025 read: IOPS=2, BW=2246KiB/s (2300kB/s)(5120KiB/2280msec) clat (usec): min=861, max=170822, avg=70037.05, stdev=68781.69 lat (usec): min=862, max=170823, avg=70037.95, stdev=68781.61 clat percentiles (usec): | 1.00th=[ 865], 5.00th=[ 865], 10.00th=[ 865], 20.00th=[ 865], | 30.00th=[ 10552], 40.00th=[ 10552], 50.00th=[ 79168], 60.00th=[ 79168], | 70.00th=[ 89654], 80.00th=[ 89654], 90.00th=[170918], 95.00th=[170918], | 99.00th=[170918], 99.50th=[170918], 99.90th=[170918], 99.95th=[170918], | 99.99th=[170918] bw ( KiB/s): min= 4031, max= 4079, per=62.06%, avg=4055.00, stdev=33.94, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3144KiB/s (3219kB/s)(7168KiB/2280msec); 0 zone resets clat (msec): min=63, max=925, avg=269.15, stdev=300.13 lat (msec): min=63, max=925, avg=269.20, stdev=300.13 clat percentiles (msec): | 1.00th=[ 64], 5.00th=[ 64], 10.00th=[ 64], 20.00th=[ 97], | 30.00th=[ 122], 40.00th=[ 122], 50.00th=[ 144], 60.00th=[ 259], | 70.00th=[ 259], 80.00th=[ 275], 90.00th=[ 927], 95.00th=[ 927], | 99.00th=[ 927], 99.50th=[ 927], 99.90th=[ 927], 99.95th=[ 927], | 99.99th=[ 927] bw ( KiB/s): min= 1771, max= 6119, per=36.09%, avg=3301.67, stdev=2442.93, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 lat (usec) : 1000=8.33% lat (msec) : 20=8.33%, 100=33.33%, 250=25.00%, 500=16.67%, 1000=8.33% cpu : usr=0.00%, sys=39.32%, ctx=603, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6535KiB/s (6692kB/s), 719KiB/s-2698KiB/s (736kB/s-2763kB/s), io=20.0MiB (20.0MB), run=2280-3134msec WRITE: bw=9149KiB/s (9368kB/s), 1927KiB/s-3596KiB/s (1973kB/s-3682kB/s), io=28.0MiB (29.4MB), run=2280-3134msec rand-rw: (groupid=0, jobs=1): err= 0: pid=153991: Tue Aug 12 19:26:45 2025 read: IOPS=2, BW=2475KiB/s (2535kB/s)(7168KiB/2896msec) clat (msec): min=54, max=1150, avg=299.94, stdev=389.91 lat (msec): min=54, max=1150, avg=299.95, stdev=389.91 clat percentiles (msec): | 1.00th=[ 55], 5.00th=[ 55], 10.00th=[ 55], 20.00th=[ 62], | 30.00th=[ 65], 40.00th=[ 65], 50.00th=[ 171], 60.00th=[ 292], | 70.00th=[ 292], 80.00th=[ 305], 90.00th=[ 1150], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min= 1957, max= 3915, per=38.15%, avg=2488.75, stdev=951.68, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 write: IOPS=1, BW=1768KiB/s (1810kB/s)(5120KiB/2896msec); 0 zone resets clat (msec): min=82, max=302, avg=147.51, stdev=88.98 lat (msec): min=82, max=302, avg=147.58, stdev=88.98 clat percentiles (msec): | 1.00th=[ 83], 5.00th=[ 83], 10.00th=[ 83], 20.00th=[ 83], | 30.00th=[ 106], 40.00th=[ 106], 50.00th=[ 109], 60.00th=[ 109], | 70.00th=[ 140], 80.00th=[ 140], 90.00th=[ 305], 95.00th=[ 305], | 99.00th=[ 305], 99.50th=[ 305], 99.90th=[ 305], 99.95th=[ 305], | 99.99th=[ 305] bw ( KiB/s): min= 2035, max= 3915, per=32.57%, avg=2975.00, stdev=1329.36, samples=2 iops : min= 1, max= 3, avg= 2.00, stdev= 1.41, samples=2 lat (msec) : 100=33.33%, 250=33.33%, 500=25.00%, 2000=8.33% cpu : usr=0.00%, sys=15.85%, ctx=163, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153992: Tue Aug 12 19:26:45 2025 read: IOPS=0, BW=686KiB/s (702kB/s)(2048KiB/2986msec) clat (msec): min=250, max=282, avg=266.83, stdev=22.47 lat (msec): min=250, max=282, avg=266.83, stdev=22.47 clat percentiles (msec): | 1.00th=[ 251], 5.00th=[ 251], 10.00th=[ 251], 20.00th=[ 251], | 30.00th=[ 251], 40.00th=[ 251], 50.00th=[ 251], 60.00th=[ 284], | 70.00th=[ 284], 80.00th=[ 284], 90.00th=[ 284], 95.00th=[ 284], | 99.00th=[ 284], 99.50th=[ 284], 99.90th=[ 284], 99.95th=[ 284], | 99.99th=[ 284] bw ( KiB/s): min= 1957, max= 1988, per=30.23%, avg=1972.50, stdev=21.92, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3429KiB/s (3512kB/s)(10.0MiB/2986msec); 0 zone resets clat (msec): min=48, max=1315, avg=241.10, stdev=379.61 lat (msec): min=48, max=1316, avg=241.15, stdev=379.61 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 48], 10.00th=[ 48], 20.00th=[ 89], | 30.00th=[ 112], 40.00th=[ 114], 50.00th=[ 123], 60.00th=[ 128], | 70.00th=[ 142], 80.00th=[ 148], 90.00th=[ 194], 95.00th=[ 1318], | 99.00th=[ 1318], 99.50th=[ 1318], 99.90th=[ 1318], 99.95th=[ 1318], | 99.99th=[ 1318] bw ( KiB/s): min= 1988, max= 6144, per=43.96%, avg=4015.67, stdev=2079.83, samples=3 iops : min= 1, max= 6, avg= 3.33, stdev= 2.52, samples=3 lat (msec) : 50=8.33%, 100=8.33%, 250=58.33%, 500=16.67%, 2000=8.33% cpu : usr=0.00%, sys=17.45%, ctx=585, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153993: Tue Aug 12 19:26:45 2025 read: IOPS=1, BW=1957KiB/s (2004kB/s)(6144KiB/3139msec) clat (msec): min=15, max=394, avg=104.07, stdev=147.04 lat (msec): min=15, max=394, avg=104.07, stdev=147.04 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 16], 10.00th=[ 16], 20.00th=[ 20], | 30.00th=[ 20], 40.00th=[ 30], 50.00th=[ 30], 60.00th=[ 50], | 70.00th=[ 117], 80.00th=[ 117], 90.00th=[ 397], 95.00th=[ 397], | 99.00th=[ 397], 99.50th=[ 397], 99.90th=[ 397], 99.95th=[ 397], | 99.99th=[ 397] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=1, BW=1957KiB/s (2004kB/s)(6144KiB/3139msec); 0 zone resets clat (msec): min=43, max=1970, avg=411.59, stdev=765.68 lat (msec): min=43, max=1970, avg=411.65, stdev=765.68 clat percentiles (msec): | 1.00th=[ 44], 5.00th=[ 44], 10.00th=[ 44], 20.00th=[ 62], | 30.00th=[ 62], 40.00th=[ 75], 50.00th=[ 75], 60.00th=[ 130], | 70.00th=[ 190], 80.00th=[ 190], 90.00th=[ 1972], 95.00th=[ 1972], | 99.00th=[ 1972], 99.50th=[ 1972], 99.90th=[ 1972], 99.95th=[ 1972], | 99.99th=[ 1972] bw ( KiB/s): min= 1957, max= 4096, per=36.46%, avg=3330.33, stdev=1191.96, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 lat (msec) : 20=16.67%, 50=25.00%, 100=16.67%, 250=25.00%, 500=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=11.22%, ctx=130, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153994: Tue Aug 12 19:26:45 2025 read: IOPS=1, BW=1659KiB/s (1698kB/s)(5120KiB/3087msec) clat (msec): min=40, max=132, avg=82.49, stdev=34.97 lat (msec): min=40, max=132, avg=82.49, stdev=34.97 clat percentiles (msec): | 1.00th=[ 41], 5.00th=[ 41], 10.00th=[ 41], 20.00th=[ 41], | 30.00th=[ 67], 40.00th=[ 67], 50.00th=[ 74], 60.00th=[ 74], | 70.00th=[ 99], 80.00th=[ 99], 90.00th=[ 133], 95.00th=[ 133], | 99.00th=[ 133], 99.50th=[ 133], 99.90th=[ 133], 99.95th=[ 133], | 99.99th=[ 133] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=2, BW=2322KiB/s (2378kB/s)(7168KiB/3087msec); 0 zone resets clat (msec): min=42, max=1884, avg=375.38, stdev=670.76 lat (msec): min=42, max=1884, avg=375.46, stdev=670.76 clat percentiles (msec): | 1.00th=[ 43], 5.00th=[ 43], 10.00th=[ 43], 20.00th=[ 56], | 30.00th=[ 61], 40.00th=[ 61], 50.00th=[ 131], 60.00th=[ 171], | 70.00th=[ 171], 80.00th=[ 284], 90.00th=[ 1888], 95.00th=[ 1888], | 99.00th=[ 1888], 99.50th=[ 1888], 99.90th=[ 1888], 99.95th=[ 1888], | 99.99th=[ 1888] bw ( KiB/s): min= 1996, max= 5873, per=43.66%, avg=3988.33, stdev=1940.74, samples=3 iops : min= 1, max= 5, avg= 3.33, stdev= 2.08, samples=3 lat (msec) : 50=16.67%, 100=41.67%, 250=25.00%, 500=8.33%, 2000=8.33% cpu : usr=0.03%, sys=12.70%, ctx=102, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6524KiB/s (6681kB/s), 686KiB/s-2475KiB/s (702kB/s-2535kB/s), io=20.0MiB (20.0MB), run=2896-3139msec WRITE: bw=9134KiB/s (9353kB/s), 1768KiB/s-3429KiB/s (1810kB/s-3512kB/s), io=28.0MiB (29.4MB), run=2896-3139msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=154000: Tue Aug 12 19:26:53 2025 read: IOPS=1, BW=5038KiB/s (5159kB/s)(12.0MiB/2439msec) clat (msec): min=448, max=1143, avg=804.15, stdev=348.04 lat (msec): min=448, max=1143, avg=804.15, stdev=348.04 clat percentiles (msec): | 1.00th=[ 447], 5.00th=[ 447], 10.00th=[ 447], 20.00th=[ 447], | 30.00th=[ 447], 40.00th=[ 818], 50.00th=[ 818], 60.00th=[ 818], | 70.00th=[ 1150], 80.00th=[ 1150], 90.00th=[ 1150], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=13.25%, ctx=41, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154001: Tue Aug 12 19:26:53 2025 read: IOPS=0, BW=1938KiB/s (1985kB/s)(4096KiB/2113msec) clat (nsec): min=659897k, max=659897k, avg=659896775.00, stdev= 0.00 lat (nsec): min=659898k, max=659898k, avg=659897913.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 659], 5.00th=[ 659], 10.00th=[ 659], 20.00th=[ 659], | 30.00th=[ 659], 40.00th=[ 659], 50.00th=[ 659], 60.00th=[ 659], | 70.00th=[ 659], 80.00th=[ 659], 90.00th=[ 659], 95.00th=[ 659], | 99.00th=[ 659], 99.50th=[ 659], 99.90th=[ 659], 99.95th=[ 659], | 99.99th=[ 659] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3877KiB/s (3970kB/s)(8192KiB/2113msec); 0 zone resets clat (msec): min=347, max=1078, avg=712.84, stdev=517.26 lat (msec): min=347, max=1078, avg=713.02, stdev=517.16 clat percentiles (msec): | 1.00th=[ 347], 5.00th=[ 347], 10.00th=[ 347], 20.00th=[ 347], | 30.00th=[ 347], 40.00th=[ 347], 50.00th=[ 347], 60.00th=[ 1083], | 70.00th=[ 1083], 80.00th=[ 1083], 90.00th=[ 1083], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 8192, max= 8192, per=52.83%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=15.86%, ctx=70, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154002: Tue Aug 12 19:26:53 2025 write: IOPS=2, BW=8623KiB/s (8830kB/s)(12.0MiB/1425msec); 0 zone resets clat (msec): min=172, max=691, avg=465.60, stdev=266.04 lat (msec): min=172, max=691, avg=465.76, stdev=265.95 clat percentiles (msec): | 1.00th=[ 174], 5.00th=[ 174], 10.00th=[ 174], 20.00th=[ 174], | 30.00th=[ 174], 40.00th=[ 535], 50.00th=[ 535], 60.00th=[ 535], | 70.00th=[ 693], 80.00th=[ 693], 90.00th=[ 693], 95.00th=[ 693], | 99.00th=[ 693], 99.50th=[ 693], 99.90th=[ 693], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 7173, max= 8192, per=49.54%, avg=7682.50, stdev=720.54, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 750=66.67% cpu : usr=0.00%, sys=22.96%, ctx=117, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154003: Tue Aug 12 19:26:53 2025 write: IOPS=1, BW=5896KiB/s (6038kB/s)(12.0MiB/2084msec); 0 zone resets clat (msec): min=156, max=1759, avg=691.53, stdev=925.06 lat (msec): min=156, max=1759, avg=691.74, stdev=924.97 clat percentiles (msec): | 1.00th=[ 157], 5.00th=[ 157], 10.00th=[ 157], 20.00th=[ 157], | 30.00th=[ 157], 40.00th=[ 159], 50.00th=[ 159], 60.00th=[ 159], | 70.00th=[ 1754], 80.00th=[ 1754], 90.00th=[ 1754], 95.00th=[ 1754], | 99.00th=[ 1754], 99.50th=[ 1754], 99.90th=[ 1754], 99.95th=[ 1754], | 99.99th=[ 1754] bw ( KiB/s): min=14840, max=14840, per=95.70%, avg=14840.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, 2000=33.33% cpu : usr=0.00%, sys=16.66%, ctx=55, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6718KiB/s (6879kB/s), 1938KiB/s-5038KiB/s (1985kB/s-5159kB/s), io=16.0MiB (16.8MB), run=2113-2439msec WRITE: bw=15.1MiB/s (15.9MB/s), 3877KiB/s-8623KiB/s (3970kB/s-8830kB/s), io=32.0MiB (33.6MB), run=1425-2113msec rand-rw: (groupid=0, jobs=1): err= 0: pid=154005: Tue Aug 12 19:26:53 2025 read: IOPS=1, BW=7802KiB/s (7989kB/s)(12.0MiB/1575msec) clat (msec): min=450, max=631, avg=524.61, stdev=95.25 lat (msec): min=450, max=631, avg=524.61, stdev=95.25 clat percentiles (msec): | 1.00th=[ 451], 5.00th=[ 451], 10.00th=[ 451], 20.00th=[ 451], | 30.00th=[ 451], 40.00th=[ 493], 50.00th=[ 493], 60.00th=[ 493], | 70.00th=[ 634], 80.00th=[ 634], 90.00th=[ 634], 95.00th=[ 634], | 99.00th=[ 634], 99.50th=[ 634], 99.90th=[ 634], 99.95th=[ 634], | 99.99th=[ 634] bw ( KiB/s): min= 5512, max= 8094, per=100.00%, avg=6803.00, stdev=1825.75, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.00%, sys=48.35%, ctx=1211, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154006: Tue Aug 12 19:26:53 2025 read: IOPS=0, BW=1553KiB/s (1590kB/s)(4096KiB/2638msec) clat (nsec): min=478088k, max=478088k, avg=478087922.00, stdev= 0.00 lat (nsec): min=478089k, max=478089k, avg=478089160.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 477], 5.00th=[ 477], 10.00th=[ 477], 20.00th=[ 477], | 30.00th=[ 477], 40.00th=[ 477], 50.00th=[ 477], 60.00th=[ 477], | 70.00th=[ 477], 80.00th=[ 477], 90.00th=[ 477], 95.00th=[ 477], | 99.00th=[ 477], 99.50th=[ 477], 99.90th=[ 477], 99.95th=[ 477], | 99.99th=[ 477] bw ( KiB/s): min= 5618, max= 5618, per=90.47%, avg=5618.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3105KiB/s (3180kB/s)(8192KiB/2638msec); 0 zone resets clat (msec): min=731, max=1424, avg=1077.81, stdev=490.27 lat (msec): min=731, max=1424, avg=1078.05, stdev=490.25 clat percentiles (msec): | 1.00th=[ 735], 5.00th=[ 735], 10.00th=[ 735], 20.00th=[ 735], | 30.00th=[ 735], 40.00th=[ 735], 50.00th=[ 735], 60.00th=[ 1418], | 70.00th=[ 1418], 80.00th=[ 1418], 90.00th=[ 1418], 95.00th=[ 1418], | 99.00th=[ 1418], 99.50th=[ 1418], 99.90th=[ 1418], 99.95th=[ 1418], | 99.99th=[ 1418] bw ( KiB/s): min= 8175, max= 8175, per=67.49%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=32.16%, ctx=248, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154007: Tue Aug 12 19:26:53 2025 write: IOPS=1, BW=4543KiB/s (4652kB/s)(12.0MiB/2705msec); 0 zone resets clat (msec): min=401, max=1383, avg=899.62, stdev=490.93 lat (msec): min=402, max=1383, avg=899.88, stdev=490.93 clat percentiles (msec): | 1.00th=[ 401], 5.00th=[ 401], 10.00th=[ 401], 20.00th=[ 401], | 30.00th=[ 401], 40.00th=[ 911], 50.00th=[ 911], 60.00th=[ 911], | 70.00th=[ 1385], 80.00th=[ 1385], 90.00th=[ 1385], 95.00th=[ 1385], | 99.00th=[ 1385], 99.50th=[ 1385], 99.90th=[ 1385], 99.95th=[ 1385], | 99.99th=[ 1385] bw ( KiB/s): min= 8192, max= 8192, per=67.63%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=45.56%, ctx=146, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154008: Tue Aug 12 19:26:53 2025 write: IOPS=1, BW=5283KiB/s (5410kB/s)(12.0MiB/2326msec); 0 zone resets clat (msec): min=558, max=888, avg=774.67, stdev=187.01 lat (msec): min=559, max=888, avg=774.91, stdev=187.03 clat percentiles (msec): | 1.00th=[ 558], 5.00th=[ 558], 10.00th=[ 558], 20.00th=[ 558], | 30.00th=[ 558], 40.00th=[ 877], 50.00th=[ 877], 60.00th=[ 877], | 70.00th=[ 885], 80.00th=[ 885], 90.00th=[ 885], 95.00th=[ 885], | 99.00th=[ 885], 99.50th=[ 885], 99.90th=[ 885], 99.95th=[ 885], | 99.99th=[ 885] bw ( KiB/s): min= 8015, max= 8192, per=66.90%, avg=8103.50, stdev=125.16, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 1000=66.67% cpu : usr=0.04%, sys=53.38%, ctx=171, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6211KiB/s (6360kB/s), 1553KiB/s-7802KiB/s (1590kB/s-7989kB/s), io=16.0MiB (16.8MB), run=1575-2638msec WRITE: bw=11.8MiB/s (12.4MB/s), 3105KiB/s-5283KiB/s (3180kB/s-5410kB/s), io=32.0MiB (33.6MB), run=2326-2705msec