== sanity test 398b: DIO and buffer IO race ============== 15:05:18 (1758740718) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.46612 s, 14.5 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147764: Wed Sep 24 15:06:33 2025 read: IOPS=23, BW=93.9KiB/s (96.2kB/s)(5992KiB/63808msec) clat (usec): min=114, max=725338, avg=21174.83, stdev=24591.00 lat (usec): min=114, max=725339, avg=21175.85, stdev=24591.00 clat percentiles (usec): | 1.00th=[ 141], 5.00th=[ 245], 10.00th=[ 619], 20.00th=[ 10552], | 30.00th=[ 12780], 40.00th=[ 14877], 50.00th=[ 17433], 60.00th=[ 20841], | 70.00th=[ 24773], 80.00th=[ 29754], 90.00th=[ 37487], 95.00th=[ 44827], | 99.00th=[ 88605], 99.50th=[115868], 99.90th=[198181], 99.95th=[725615], | 99.99th=[725615] bw ( KiB/s): min= 15, max= 175, per=24.87%, avg=93.50, stdev=29.44, samples=126 iops : min= 3, max= 43, avg=22.77, stdev= 7.42, samples=126 write: IOPS=24, BW=98.7KiB/s (101kB/s)(6296KiB/63808msec); 0 zone resets clat (usec): min=867, max=146974, avg=20210.62, stdev=18254.73 lat (usec): min=868, max=146975, avg=20211.90, stdev=18254.70 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1270], 10.00th=[ 1795], 20.00th=[ 2868], | 30.00th=[ 5604], 40.00th=[ 12125], 50.00th=[ 16450], 60.00th=[ 21627], | 70.00th=[ 27132], 80.00th=[ 34866], 90.00th=[ 44303], 95.00th=[ 54264], | 99.00th=[ 77071], 99.50th=[ 92799], 99.90th=[110625], 99.95th=[147850], | 99.99th=[147850] bw ( KiB/s): min= 7, max= 198, per=25.24%, avg=99.18, stdev=33.30, samples=125 iops : min= 1, max= 49, avg=24.19, stdev= 8.34, samples=125 lat (usec) : 250=2.54%, 500=2.31%, 750=0.10%, 1000=0.36% lat (msec) : 2=5.99%, 4=6.93%, 10=8.92%, 20=30.37%, 50=37.43% lat (msec) : 100=4.65%, 250=0.36%, 750=0.03% cpu : usr=0.13%, sys=13.08%, ctx=7034, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147766: Wed Sep 24 15:06:33 2025 read: IOPS=24, BW=96.8KiB/s (99.1kB/s)(5940KiB/61384msec) clat (usec): min=105, max=770119, avg=20021.99, stdev=24470.46 lat (usec): min=105, max=770122, avg=20022.99, stdev=24470.51 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 208], 10.00th=[ 347], 20.00th=[ 9765], | 30.00th=[ 11863], 40.00th=[ 13960], 50.00th=[ 16188], 60.00th=[ 19268], | 70.00th=[ 23987], 80.00th=[ 30016], 90.00th=[ 37487], 95.00th=[ 45351], | 99.00th=[ 65274], 99.50th=[ 86508], 99.90th=[137364], 99.95th=[767558], | 99.99th=[767558] bw ( KiB/s): min= 7, max= 215, per=25.85%, avg=97.21, stdev=32.13, samples=121 iops : min= 1, max= 53, avg=23.69, stdev= 8.09, samples=121 write: IOPS=25, BW=103KiB/s (106kB/s)(6348KiB/61384msec); 0 zone resets clat (usec): min=637, max=150146, avg=19767.91, stdev=19361.32 lat (usec): min=637, max=150147, avg=19769.99, stdev=19361.25 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 1254], 10.00th=[ 1680], 20.00th=[ 2704], | 30.00th=[ 4752], 40.00th=[ 10421], 50.00th=[ 15533], 60.00th=[ 20055], | 70.00th=[ 25560], 80.00th=[ 33817], 90.00th=[ 45351], 95.00th=[ 55837], | 99.00th=[ 85459], 99.50th=[103285], 99.90th=[133694], 99.95th=[149947], | 99.99th=[149947] bw ( KiB/s): min= 39, max= 224, per=26.67%, avg=104.80, stdev=32.65, samples=120 iops : min= 9, max= 56, avg=25.60, stdev= 8.21, samples=120 lat (usec) : 250=2.86%, 500=2.54%, 750=0.20%, 1000=0.26% lat (msec) : 2=7.26%, 4=6.80%, 10=10.32%, 20=30.66%, 50=33.69% lat (msec) : 100=4.92%, 250=0.46%, 1000=0.03% cpu : usr=0.12%, sys=13.28%, ctx=6671, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147769: Wed Sep 24 15:06:33 2025 read: IOPS=23, BW=95.8KiB/s (98.1kB/s)(6028KiB/62945msec) clat (usec): min=106, max=765979, avg=19676.26, stdev=23625.05 lat (usec): min=107, max=765980, avg=19677.61, stdev=23624.98 clat percentiles (usec): | 1.00th=[ 145], 5.00th=[ 255], 10.00th=[ 416], 20.00th=[ 10159], | 30.00th=[ 12518], 40.00th=[ 14615], 50.00th=[ 16712], 60.00th=[ 19268], | 70.00th=[ 23462], 80.00th=[ 28443], 90.00th=[ 35914], 95.00th=[ 43254], | 99.00th=[ 59507], 99.50th=[ 68682], 99.90th=[202376], 99.95th=[767558], | 99.99th=[767558] bw ( KiB/s): min= 23, max= 183, per=25.53%, avg=95.99, stdev=31.73, samples=124 iops : min= 5, max= 45, avg=23.40, stdev= 7.97, samples=124 write: IOPS=24, BW=99.5KiB/s (102kB/s)(6260KiB/62945msec); 0 zone resets clat (usec): min=878, max=217232, avg=21106.68, stdev=20119.69 lat (usec): min=880, max=217234, avg=21108.88, stdev=20120.02 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 1287], 10.00th=[ 1811], 20.00th=[ 2900], | 30.00th=[ 6128], 40.00th=[ 12780], 50.00th=[ 17433], 60.00th=[ 21365], | 70.00th=[ 27657], 80.00th=[ 35390], 90.00th=[ 47449], 95.00th=[ 55313], | 99.00th=[ 81265], 99.50th=[111674], 99.90th=[193987], 99.95th=[217056], | 99.99th=[217056] bw ( KiB/s): min= 7, max= 199, per=25.15%, avg=98.86, stdev=31.29, samples=125 iops : min= 1, max= 49, avg=24.11, stdev= 7.84, samples=125 lat (usec) : 250=2.41%, 500=2.67%, 750=0.26%, 1000=0.23% lat (msec) : 2=6.15%, 4=6.45%, 10=9.47%, 20=31.67%, 50=35.58% lat (msec) : 100=4.72%, 250=0.36%, 1000=0.03% cpu : usr=0.12%, sys=13.27%, ctx=7026, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147771: Wed Sep 24 15:06:33 2025 read: IOPS=24, BW=96.6KiB/s (98.0kB/s)(6052KiB/62629msec) clat (usec): min=104, max=773151, avg=19892.53, stdev=23604.99 lat (usec): min=104, max=773151, avg=19893.59, stdev=23604.98 clat percentiles (usec): | 1.00th=[ 143], 5.00th=[ 231], 10.00th=[ 660], 20.00th=[ 10028], | 30.00th=[ 12780], 40.00th=[ 14877], 50.00th=[ 17171], 60.00th=[ 20055], | 70.00th=[ 23725], 80.00th=[ 28967], 90.00th=[ 35390], 95.00th=[ 43254], | 99.00th=[ 56361], 99.50th=[ 66323], 99.90th=[166724], 99.95th=[775947], | 99.99th=[775947] bw ( KiB/s): min= 7, max= 168, per=25.53%, avg=95.98, stdev=27.59, samples=124 iops : min= 1, max= 42, avg=23.40, stdev= 6.96, samples=124 write: IOPS=24, BW=99.6KiB/s (102kB/s)(6236KiB/62629msec); 0 zone resets clat (usec): min=954, max=236364, avg=20699.32, stdev=20418.33 lat (usec): min=955, max=236365, avg=20701.71, stdev=20418.63 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 1270], 10.00th=[ 1827], 20.00th=[ 2999], | 30.00th=[ 5800], 40.00th=[ 11994], 50.00th=[ 16909], 60.00th=[ 21365], | 70.00th=[ 26346], 80.00th=[ 34341], 90.00th=[ 45876], 95.00th=[ 54789], | 99.00th=[ 82314], 99.50th=[123208], 99.90th=[212861], 99.95th=[235930], | 99.99th=[235930] bw ( KiB/s): min= 7, max= 183, per=25.16%, avg=98.90, stdev=31.87, samples=124 iops : min= 1, max= 45, avg=24.12, stdev= 8.01, samples=124 lat (usec) : 250=2.73%, 500=2.05%, 750=0.26%, 1000=0.36% lat (msec) : 2=5.96%, 4=6.67%, 10=10.16%, 20=30.18%, 50=36.69% lat (msec) : 100=4.56%, 250=0.36%, 1000=0.03% cpu : usr=0.13%, sys=13.18%, ctx=7017, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=376KiB/s (385kB/s), 93.9KiB/s-96.8KiB/s (96.2kB/s-99.1kB/s), io=23.4MiB (24.6MB), run=61384-63808msec WRITE: bw=394KiB/s (403kB/s), 98.7KiB/s-103KiB/s (101kB/s-106kB/s), io=24.6MiB (25.7MB), run=61384-63808msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147765: Wed Sep 24 15:07:53 2025 read: IOPS=10, BW=41.9KiB/s (42.9kB/s)(5992KiB/143011msec) clat (msec): min=4, max=754, avg=17.61, stdev=25.31 lat (msec): min=4, max=754, avg=17.61, stdev=25.31 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 24], 90.00th=[ 34], 95.00th=[ 44], | 99.00th=[ 94], 99.50th=[ 126], 99.90th=[ 203], 99.95th=[ 751], | 99.99th=[ 751] bw ( KiB/s): min= 7, max= 135, per=25.66%, avg=42.60, stdev=25.90, samples=276 iops : min= 1, max= 33, avg=10.12, stdev= 6.51, samples=276 write: IOPS=11, BW=44.0KiB/s (45.1kB/s)(6296KiB/143011msec); 0 zone resets clat (msec): min=24, max=348, avg=73.92, stdev=30.94 lat (msec): min=24, max=348, avg=73.92, stdev=30.94 clat percentiles (msec): | 1.00th=[ 32], 5.00th=[ 39], 10.00th=[ 43], 20.00th=[ 50], | 30.00th=[ 56], 40.00th=[ 62], 50.00th=[ 68], 60.00th=[ 75], | 70.00th=[ 84], 80.00th=[ 94], 90.00th=[ 113], 95.00th=[ 132], | 99.00th=[ 180], 99.50th=[ 209], 99.90th=[ 271], 99.95th=[ 347], | 99.99th=[ 347] bw ( KiB/s): min= 7, max= 88, per=25.14%, avg=43.75, stdev=15.71, samples=282 iops : min= 1, max= 22, avg=10.41, stdev= 3.96, samples=282 lat (msec) : 10=19.47%, 20=17.55%, 50=20.70%, 100=34.08%, 250=8.11% lat (msec) : 500=0.07%, 1000=0.03% cpu : usr=0.06%, sys=2.46%, ctx=5522, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147767: Wed Sep 24 15:07:53 2025 read: IOPS=10, BW=41.3KiB/s (42.3kB/s)(5940KiB/143856msec) clat (msec): min=4, max=772, avg=17.93, stdev=25.58 lat (msec): min=4, max=772, avg=17.94, stdev=25.58 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 24], 90.00th=[ 37], 95.00th=[ 46], | 99.00th=[ 79], 99.50th=[ 120], 99.90th=[ 209], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 7, max= 128, per=25.27%, avg=41.94, stdev=25.13, samples=278 iops : min= 1, max= 32, avg= 9.96, stdev= 6.29, samples=278 write: IOPS=11, BW=44.1KiB/s (45.2kB/s)(6348KiB/143856msec); 0 zone resets clat (msec): min=26, max=350, avg=73.68, stdev=30.54 lat (msec): min=26, max=350, avg=73.69, stdev=30.54 clat percentiles (msec): | 1.00th=[ 33], 5.00th=[ 40], 10.00th=[ 43], 20.00th=[ 50], | 30.00th=[ 56], 40.00th=[ 61], 50.00th=[ 67], 60.00th=[ 75], | 70.00th=[ 84], 80.00th=[ 93], 90.00th=[ 113], 95.00th=[ 130], | 99.00th=[ 176], 99.50th=[ 203], 99.90th=[ 271], 99.95th=[ 351], | 99.99th=[ 351] bw ( KiB/s): min= 15, max= 87, per=25.21%, avg=43.86, stdev=15.77, samples=284 iops : min= 3, max= 21, avg=10.44, stdev= 3.96, samples=284 lat (msec) : 10=18.33%, 20=18.10%, 50=21.09%, 100=34.31%, 250=8.07% lat (msec) : 500=0.07%, 1000=0.03% cpu : usr=0.07%, sys=2.47%, ctx=5493, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147768: Wed Sep 24 15:07:53 2025 read: IOPS=10, BW=42.1KiB/s (43.1kB/s)(6028KiB/143074msec) clat (msec): min=4, max=177, avg=17.62, stdev=15.74 lat (msec): min=4, max=177, avg=17.62, stdev=15.74 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 15], | 70.00th=[ 18], 80.00th=[ 25], 90.00th=[ 35], 95.00th=[ 45], | 99.00th=[ 80], 99.50th=[ 104], 99.90th=[ 161], 99.95th=[ 178], | 99.99th=[ 178] bw ( KiB/s): min= 7, max= 128, per=25.60%, avg=42.49, stdev=25.83, samples=278 iops : min= 1, max= 32, avg=10.09, stdev= 6.49, samples=278 write: IOPS=10, BW=43.8KiB/s (44.8kB/s)(6260KiB/143074msec); 0 zone resets clat (msec): min=24, max=821, avg=74.27, stdev=35.49 lat (msec): min=24, max=821, avg=74.27, stdev=35.49 clat percentiles (msec): | 1.00th=[ 31], 5.00th=[ 39], 10.00th=[ 43], 20.00th=[ 51], | 30.00th=[ 56], 40.00th=[ 62], 50.00th=[ 67], 60.00th=[ 74], | 70.00th=[ 83], 80.00th=[ 94], 90.00th=[ 114], 95.00th=[ 133], | 99.00th=[ 180], 99.50th=[ 201], 99.90th=[ 239], 99.95th=[ 818], | 99.99th=[ 818] bw ( KiB/s): min= 8, max= 87, per=24.89%, avg=43.31, stdev=15.82, samples=283 iops : min= 2, max= 21, avg=10.30, stdev= 3.96, samples=283 lat (msec) : 10=17.94%, 20=18.42%, 50=21.22%, 100=33.72%, 250=8.66% lat (msec) : 1000=0.03% cpu : usr=0.07%, sys=2.51%, ctx=5568, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147770: Wed Sep 24 15:07:53 2025 read: IOPS=10, BW=42.5KiB/s (43.5kB/s)(6052KiB/142462msec) clat (msec): min=4, max=156, avg=17.17, stdev=15.27 lat (msec): min=4, max=156, avg=17.17, stdev=15.28 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 18], 80.00th=[ 24], 90.00th=[ 35], 95.00th=[ 46], | 99.00th=[ 78], 99.50th=[ 93], 99.90th=[ 155], 99.95th=[ 157], | 99.99th=[ 157] bw ( KiB/s): min= 7, max= 135, per=26.09%, avg=43.31, stdev=25.58, samples=274 iops : min= 1, max= 33, avg=10.30, stdev= 6.38, samples=274 write: IOPS=10, BW=43.8KiB/s (44.8kB/s)(6236KiB/142462msec); 0 zone resets clat (msec): min=25, max=825, avg=74.54, stdev=36.25 lat (msec): min=25, max=825, avg=74.54, stdev=36.25 clat percentiles (msec): | 1.00th=[ 33], 5.00th=[ 38], 10.00th=[ 44], 20.00th=[ 51], | 30.00th=[ 55], 40.00th=[ 62], 50.00th=[ 67], 60.00th=[ 74], | 70.00th=[ 84], 80.00th=[ 95], 90.00th=[ 113], 95.00th=[ 132], | 99.00th=[ 169], 99.50th=[ 220], 99.90th=[ 275], 99.95th=[ 827], | 99.99th=[ 827] bw ( KiB/s): min= 8, max= 87, per=24.90%, avg=43.33, stdev=15.91, samples=282 iops : min= 2, max= 21, avg=10.30, stdev= 4.03, samples=282 lat (msec) : 10=18.88%, 20=17.90%, 50=20.44%, 100=34.02%, 250=8.63% lat (msec) : 500=0.10%, 1000=0.03% cpu : usr=0.07%, sys=2.46%, ctx=5495, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=167KiB/s (171kB/s), 41.3KiB/s-42.5KiB/s (42.3kB/s-43.5kB/s), io=23.4MiB (24.6MB), run=142462-143856msec WRITE: bw=175KiB/s (179kB/s), 43.8KiB/s-44.1KiB/s (44.8kB/s-45.2kB/s), io=24.6MiB (25.7MB), run=142462-143856msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147813: Wed Sep 24 15:08:13 2025 read: IOPS=27, BW=437KiB/s (447kB/s)(5920KiB/13552msec) clat (usec): min=105, max=230864, avg=18920.50, stdev=19399.05 lat (usec): min=106, max=230865, avg=18922.10, stdev=19399.13 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 174], 10.00th=[ 239], 20.00th=[ 889], | 30.00th=[ 11600], 40.00th=[ 14877], 50.00th=[ 17433], 60.00th=[ 20055], | 70.00th=[ 23987], 80.00th=[ 28181], 90.00th=[ 34341], 95.00th=[ 42206], | 99.00th=[ 76022], 99.50th=[168821], 99.90th=[231736], 99.95th=[231736], | 99.99th=[231736] bw ( KiB/s): min= 192, max= 638, per=25.91%, avg=429.85, stdev=126.08, samples=26 iops : min= 12, max= 39, avg=26.12, stdev= 7.83, samples=26 write: IOPS=29, BW=470KiB/s (481kB/s)(6368KiB/13552msec); 0 zone resets clat (usec): min=1382, max=117167, avg=16314.72, stdev=15636.90 lat (usec): min=1384, max=117171, avg=16316.65, stdev=15636.89 clat percentiles (usec): | 1.00th=[ 1467], 5.00th=[ 1926], 10.00th=[ 2507], 20.00th=[ 3949], | 30.00th=[ 5276], 40.00th=[ 7177], 50.00th=[ 11469], 60.00th=[ 16057], | 70.00th=[ 21365], 80.00th=[ 26870], 90.00th=[ 38011], 95.00th=[ 44827], | 99.00th=[ 78119], 99.50th=[ 90702], 99.90th=[116917], 99.95th=[116917], | 99.99th=[116917] bw ( KiB/s): min= 192, max= 793, per=26.85%, avg=468.81, stdev=152.00, samples=26 iops : min= 12, max= 49, avg=28.58, stdev= 9.52, samples=26 lat (usec) : 250=4.95%, 500=3.91%, 750=0.39%, 1000=0.52% lat (msec) : 2=3.39%, 4=8.46%, 10=15.63%, 20=26.95%, 50=33.20% lat (msec) : 100=2.21%, 250=0.39% cpu : usr=0.20%, sys=17.09%, ctx=1574, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147816: Wed Sep 24 15:08:13 2025 read: IOPS=24, BW=398KiB/s (408kB/s)(5744KiB/14431msec) clat (usec): min=135, max=134195, avg=19192.96, stdev=16592.80 lat (usec): min=136, max=134196, avg=19194.05, stdev=16592.73 clat percentiles (usec): | 1.00th=[ 147], 5.00th=[ 190], 10.00th=[ 285], 20.00th=[ 1811], | 30.00th=[ 11994], 40.00th=[ 15008], 50.00th=[ 17695], 60.00th=[ 20317], | 70.00th=[ 23725], 80.00th=[ 28181], 90.00th=[ 34866], 95.00th=[ 44303], | 99.00th=[ 88605], 99.50th=[110625], 99.90th=[133694], 99.95th=[133694], | 99.99th=[133694] bw ( KiB/s): min= 29, max= 633, per=23.63%, avg=392.07, stdev=171.91, samples=28 iops : min= 1, max= 39, avg=23.79, stdev=10.77, samples=28 write: IOPS=28, BW=453KiB/s (464kB/s)(6544KiB/14431msec); 0 zone resets clat (usec): min=1358, max=191538, avg=17943.15, stdev=22301.73 lat (usec): min=1359, max=191540, avg=17945.28, stdev=22301.72 clat percentiles (usec): | 1.00th=[ 1516], 5.00th=[ 1811], 10.00th=[ 2409], 20.00th=[ 3654], | 30.00th=[ 4752], 40.00th=[ 6587], 50.00th=[ 10028], 60.00th=[ 15008], | 70.00th=[ 20055], 80.00th=[ 28443], 90.00th=[ 39584], 95.00th=[ 57410], | 99.00th=[114820], 99.50th=[129500], 99.90th=[191890], 99.95th=[191890], | 99.99th=[191890] bw ( KiB/s): min= 119, max= 637, per=25.24%, avg=440.61, stdev=135.20, samples=28 iops : min= 7, max= 39, avg=26.86, stdev= 8.46, samples=28 lat (usec) : 250=3.78%, 500=4.69%, 750=0.39% lat (msec) : 2=4.17%, 4=9.64%, 10=15.36%, 20=26.69%, 50=29.82% lat (msec) : 100=4.30%, 250=1.17% cpu : usr=0.15%, sys=15.75%, ctx=1617, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147817: Wed Sep 24 15:08:13 2025 read: IOPS=28, BW=454KiB/s (465kB/s)(6256KiB/13786msec) clat (usec): min=108, max=101227, avg=18303.64, stdev=14120.91 lat (usec): min=108, max=101228, avg=18306.04, stdev=14121.17 clat percentiles (usec): | 1.00th=[ 143], 5.00th=[ 176], 10.00th=[ 269], 20.00th=[ 1631], | 30.00th=[ 11863], 40.00th=[ 15008], 50.00th=[ 16909], 60.00th=[ 19530], | 70.00th=[ 23462], 80.00th=[ 27395], 90.00th=[ 35914], 95.00th=[ 43254], | 99.00th=[ 66323], 99.50th=[ 74974], 99.90th=[101188], 99.95th=[101188], | 99.99th=[101188] bw ( KiB/s): min= 288, max= 860, per=27.60%, avg=457.89, stdev=124.54, samples=27 iops : min= 18, max= 53, avg=27.93, stdev= 7.72, samples=27 write: IOPS=27, BW=438KiB/s (448kB/s)(6032KiB/13786msec); 0 zone resets clat (usec): min=1368, max=137527, avg=17094.16, stdev=18165.51 lat (usec): min=1370, max=137529, avg=17096.77, stdev=18165.98 clat percentiles (usec): | 1.00th=[ 1631], 5.00th=[ 1942], 10.00th=[ 2769], 20.00th=[ 4047], | 30.00th=[ 5211], 40.00th=[ 6587], 50.00th=[ 9634], 60.00th=[ 14222], | 70.00th=[ 20841], 80.00th=[ 29230], 90.00th=[ 40633], 95.00th=[ 53740], | 99.00th=[ 84411], 99.50th=[104334], 99.90th=[137364], 99.95th=[137364], | 99.99th=[137364] bw ( KiB/s): min= 250, max= 768, per=24.88%, avg=434.44, stdev=114.62, samples=27 iops : min= 15, max= 48, avg=26.44, stdev= 7.22, samples=27 lat (usec) : 250=4.95%, 500=4.04%, 750=0.13%, 1000=0.65% lat (msec) : 2=3.39%, 4=6.90%, 10=17.32%, 20=27.21%, 50=31.38% lat (msec) : 100=3.65%, 250=0.39% cpu : usr=0.18%, sys=17.16%, ctx=1659, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147818: Wed Sep 24 15:08:13 2025 read: IOPS=26, BW=427KiB/s (437kB/s)(6032KiB/14129msec) clat (usec): min=102, max=174802, avg=18651.14, stdev=15495.71 lat (usec): min=102, max=174803, avg=18652.48, stdev=15495.86 clat percentiles (usec): | 1.00th=[ 125], 5.00th=[ 167], 10.00th=[ 281], 20.00th=[ 1876], | 30.00th=[ 12256], 40.00th=[ 14746], 50.00th=[ 17171], 60.00th=[ 20579], | 70.00th=[ 23725], 80.00th=[ 28967], 90.00th=[ 35390], 95.00th=[ 42730], | 99.00th=[ 62129], 99.50th=[ 66323], 99.90th=[175113], 99.95th=[175113], | 99.99th=[175113] bw ( KiB/s): min= 190, max= 701, per=25.74%, avg=427.00, stdev=122.49, samples=28 iops : min= 11, max= 43, avg=25.96, stdev= 7.63, samples=28 write: IOPS=27, BW=443KiB/s (453kB/s)(6256KiB/14129msec); 0 zone resets clat (usec): min=1386, max=116830, avg=17786.73, stdev=16901.98 lat (usec): min=1387, max=116832, avg=17788.65, stdev=16901.95 clat percentiles (usec): | 1.00th=[ 1500], 5.00th=[ 1909], 10.00th=[ 2409], 20.00th=[ 3654], | 30.00th=[ 5211], 40.00th=[ 7832], 50.00th=[ 12780], 60.00th=[ 18482], | 70.00th=[ 23200], 80.00th=[ 30540], 90.00th=[ 41681], 95.00th=[ 48497], | 99.00th=[ 70779], 99.50th=[110625], 99.90th=[116917], 99.95th=[116917], | 99.99th=[116917] bw ( KiB/s): min= 223, max= 701, per=25.41%, avg=443.71, stdev=137.18, samples=28 iops : min= 13, max= 43, avg=27.04, stdev= 8.55, samples=28 lat (usec) : 250=4.30%, 500=4.43%, 750=0.52%, 1000=0.13% lat (msec) : 2=3.52%, 4=9.11%, 10=12.76%, 20=26.43%, 50=35.42% lat (msec) : 100=2.99%, 250=0.39% cpu : usr=0.14%, sys=16.56%, ctx=1671, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1660KiB/s (1700kB/s), 398KiB/s-454KiB/s (408kB/s-465kB/s), io=23.4MiB (24.5MB), run=13552-14431msec WRITE: bw=1746KiB/s (1788kB/s), 438KiB/s-470KiB/s (448kB/s-481kB/s), io=24.6MiB (25.8MB), run=13552-14431msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147811: Wed Sep 24 15:08:34 2025 read: IOPS=10, BW=169KiB/s (173kB/s)(5920KiB/35062msec) clat (msec): min=5, max=148, avg=18.00, stdev=18.13 lat (msec): min=5, max=148, avg=18.00, stdev=18.13 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 36], 95.00th=[ 55], | 99.00th=[ 101], 99.50th=[ 123], 99.90th=[ 148], 99.95th=[ 148], | 99.99th=[ 148] bw ( KiB/s): min= 31, max= 480, per=25.33%, avg=170.24, stdev=99.63, samples=68 iops : min= 1, max= 30, avg= 9.99, stdev= 6.41, samples=68 write: IOPS=11, BW=182KiB/s (186kB/s)(6368KiB/35062msec); 0 zone resets clat (msec): min=31, max=217, avg=70.92, stdev=28.54 lat (msec): min=31, max=217, avg=70.93, stdev=28.54 clat percentiles (msec): | 1.00th=[ 34], 5.00th=[ 39], 10.00th=[ 42], 20.00th=[ 48], | 30.00th=[ 55], 40.00th=[ 59], 50.00th=[ 65], 60.00th=[ 73], | 70.00th=[ 79], 80.00th=[ 88], 90.00th=[ 108], 95.00th=[ 130], | 99.00th=[ 180], 99.50th=[ 207], 99.90th=[ 218], 99.95th=[ 218], | 99.99th=[ 218] bw ( KiB/s): min= 31, max= 288, per=25.59%, avg=181.19, stdev=64.96, samples=69 iops : min= 1, max= 18, avg=10.67, stdev= 4.16, samples=69 lat (msec) : 10=18.49%, 20=18.75%, 50=20.96%, 100=35.29%, 250=6.51% cpu : usr=0.07%, sys=3.09%, ctx=1460, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147812: Wed Sep 24 15:08:34 2025 read: IOPS=10, BW=161KiB/s (165kB/s)(5744KiB/35593msec) clat (msec): min=4, max=159, avg=16.98, stdev=17.32 lat (msec): min=4, max=159, avg=16.98, stdev=17.32 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 16], 80.00th=[ 21], 90.00th=[ 35], 95.00th=[ 52], | 99.00th=[ 92], 99.50th=[ 114], 99.90th=[ 159], 99.95th=[ 159], | 99.99th=[ 159] bw ( KiB/s): min= 29, max= 510, per=25.25%, avg=169.68, stdev=100.70, samples=66 iops : min= 1, max= 31, avg= 9.94, stdev= 6.40, samples=66 write: IOPS=11, BW=184KiB/s (188kB/s)(6544KiB/35593msec); 0 zone resets clat (msec): min=27, max=258, avg=71.79, stdev=30.66 lat (msec): min=27, max=258, avg=71.79, stdev=30.66 clat percentiles (msec): | 1.00th=[ 35], 5.00th=[ 40], 10.00th=[ 43], 20.00th=[ 48], | 30.00th=[ 54], 40.00th=[ 58], 50.00th=[ 65], 60.00th=[ 71], | 70.00th=[ 81], 80.00th=[ 92], 90.00th=[ 112], 95.00th=[ 140], | 99.00th=[ 186], 99.50th=[ 188], 99.90th=[ 259], 99.95th=[ 259], | 99.99th=[ 259] bw ( KiB/s): min= 31, max= 288, per=25.73%, avg=182.14, stdev=63.42, samples=70 iops : min= 1, max= 18, avg=10.73, stdev= 4.09, samples=70 lat (msec) : 10=20.18%, 20=16.93%, 50=20.44%, 100=34.51%, 250=7.81% lat (msec) : 500=0.13% cpu : usr=0.06%, sys=2.90%, ctx=1456, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147814: Wed Sep 24 15:08:34 2025 read: IOPS=11, BW=182KiB/s (187kB/s)(6256KiB/34333msec) clat (msec): min=4, max=155, avg=17.32, stdev=16.93 lat (msec): min=4, max=155, avg=17.33, stdev=16.93 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 16], 80.00th=[ 21], 90.00th=[ 32], 95.00th=[ 47], | 99.00th=[ 86], 99.50th=[ 129], 99.90th=[ 157], 99.95th=[ 157], | 99.99th=[ 157] bw ( KiB/s): min= 31, max= 448, per=27.22%, avg=182.90, stdev=115.74, samples=68 iops : min= 1, max= 28, avg=10.78, stdev= 7.39, samples=68 write: IOPS=10, BW=176KiB/s (180kB/s)(6032KiB/34333msec); 0 zone resets clat (msec): min=27, max=200, avg=72.97, stdev=29.71 lat (msec): min=27, max=200, avg=72.98, stdev=29.70 clat percentiles (msec): | 1.00th=[ 31], 5.00th=[ 40], 10.00th=[ 44], 20.00th=[ 50], | 30.00th=[ 55], 40.00th=[ 60], 50.00th=[ 67], 60.00th=[ 72], | 70.00th=[ 81], 80.00th=[ 92], 90.00th=[ 116], 95.00th=[ 134], | 99.00th=[ 184], 99.50th=[ 188], 99.90th=[ 201], 99.95th=[ 201], | 99.99th=[ 201] bw ( KiB/s): min= 63, max= 287, per=24.70%, avg=174.88, stdev=59.42, samples=68 iops : min= 3, max= 17, avg=10.28, stdev= 3.86, samples=68 lat (msec) : 10=17.71%, 20=22.14%, 50=18.75%, 100=32.81%, 250=8.59% cpu : usr=0.07%, sys=3.03%, ctx=1467, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147815: Wed Sep 24 15:08:34 2025 read: IOPS=10, BW=173KiB/s (177kB/s)(6032KiB/34862msec) clat (msec): min=4, max=129, avg=18.08, stdev=18.12 lat (msec): min=4, max=129, avg=18.09, stdev=18.12 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 13], | 70.00th=[ 16], 80.00th=[ 23], 90.00th=[ 39], 95.00th=[ 58], | 99.00th=[ 93], 99.50th=[ 130], 99.90th=[ 130], 99.95th=[ 130], | 99.99th=[ 130] bw ( KiB/s): min= 31, max= 448, per=26.21%, avg=176.12, stdev=88.83, samples=67 iops : min= 1, max= 28, avg=10.36, stdev= 5.70, samples=67 write: IOPS=11, BW=179KiB/s (184kB/s)(6256KiB/34862msec); 0 zone resets clat (msec): min=29, max=255, avg=71.31, stdev=28.55 lat (msec): min=29, max=255, avg=71.31, stdev=28.55 clat percentiles (msec): | 1.00th=[ 32], 5.00th=[ 41], 10.00th=[ 44], 20.00th=[ 50], | 30.00th=[ 55], 40.00th=[ 61], 50.00th=[ 66], 60.00th=[ 71], | 70.00th=[ 78], 80.00th=[ 86], 90.00th=[ 107], 95.00th=[ 124], | 99.00th=[ 169], 99.50th=[ 220], 99.90th=[ 257], 99.95th=[ 257], | 99.99th=[ 257] bw ( KiB/s): min= 63, max= 286, per=25.32%, avg=179.26, stdev=63.19, samples=69 iops : min= 3, max= 17, avg=10.55, stdev= 4.08, samples=69 lat (msec) : 10=17.97%, 20=20.05%, 50=18.10%, 100=36.72%, 250=7.03% lat (msec) : 500=0.13% cpu : usr=0.08%, sys=3.00%, ctx=1476, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=673KiB/s (689kB/s), 161KiB/s-182KiB/s (165kB/s-187kB/s), io=23.4MiB (24.5MB), run=34333-35593msec WRITE: bw=708KiB/s (725kB/s), 176KiB/s-184KiB/s (180kB/s-188kB/s), io=24.6MiB (25.8MB), run=34333-35593msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147829: Wed Sep 24 15:08:44 2025 read: IOPS=1, BW=1920KiB/s (1966kB/s)(7168KiB/3734msec) clat (usec): min=780, max=2031.7k, avg=393454.88, stdev=729387.85 lat (usec): min=781, max=2031.7k, avg=393456.43, stdev=729387.93 clat percentiles (usec): | 1.00th=[ 783], 5.00th=[ 783], 10.00th=[ 783], | 20.00th=[ 4359], 30.00th=[ 121111], 40.00th=[ 121111], | 50.00th=[ 147850], 60.00th=[ 149947], 70.00th=[ 149947], | 80.00th=[ 299893], 90.00th=[2038432], 95.00th=[2038432], | 99.00th=[2038432], 99.50th=[2038432], 99.90th=[2038432], | 99.95th=[2038432], 99.99th=[2038432] bw ( KiB/s): min= 2023, max= 4087, per=53.58%, avg=2938.50, stdev=1066.23, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 write: IOPS=1, BW=1371KiB/s (1404kB/s)(5120KiB/3734msec); 0 zone resets clat (msec): min=76, max=283, avg=166.68, stdev=84.50 lat (msec): min=76, max=283, avg=166.75, stdev=84.49 clat percentiles (msec): | 1.00th=[ 78], 5.00th=[ 78], 10.00th=[ 78], 20.00th=[ 78], | 30.00th=[ 113], 40.00th=[ 113], 50.00th=[ 138], 60.00th=[ 138], | 70.00th=[ 222], 80.00th=[ 222], 90.00th=[ 284], 95.00th=[ 284], | 99.00th=[ 284], 99.50th=[ 284], 99.90th=[ 284], 99.95th=[ 284], | 99.99th=[ 284] bw ( KiB/s): min= 2023, max= 8159, per=66.31%, avg=5091.00, stdev=4338.81, samples=2 iops : min= 1, max= 7, avg= 4.00, stdev= 4.24, samples=2 lat (usec) : 1000=8.33% lat (msec) : 10=8.33%, 100=8.33%, 250=50.00%, 500=16.67%, >=2000=8.33% cpu : usr=0.00%, sys=26.79%, ctx=777, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147831: Wed Sep 24 15:08:44 2025 read: IOPS=0, BW=591KiB/s (605kB/s)(2048KiB/3465msec) clat (usec): min=872, max=128073, avg=64473.26, stdev=89944.36 lat (usec): min=874, max=128076, avg=64475.30, stdev=89945.22 clat percentiles (usec): | 1.00th=[ 873], 5.00th=[ 873], 10.00th=[ 873], 20.00th=[ 873], | 30.00th=[ 873], 40.00th=[ 873], 50.00th=[ 873], 60.00th=[128451], | 70.00th=[128451], 80.00th=[128451], 90.00th=[128451], 95.00th=[128451], | 99.00th=[128451], 99.50th=[128451], 99.90th=[128451], 99.95th=[128451], | 99.99th=[128451] bw ( KiB/s): min= 1783, max= 2048, per=34.93%, avg=1915.50, stdev=187.38, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2955KiB/s (3026kB/s)(10.0MiB/3465msec); 0 zone resets clat (msec): min=65, max=1232, avg=326.83, stdev=343.54 lat (msec): min=65, max=1232, avg=326.92, stdev=343.53 clat percentiles (msec): | 1.00th=[ 66], 5.00th=[ 66], 10.00th=[ 66], 20.00th=[ 94], | 30.00th=[ 153], 40.00th=[ 157], 50.00th=[ 218], 60.00th=[ 226], | 70.00th=[ 284], 80.00th=[ 330], 90.00th=[ 514], 95.00th=[ 1234], | 99.00th=[ 1234], 99.50th=[ 1234], 99.90th=[ 1234], 99.95th=[ 1234], | 99.99th=[ 1234] bw ( KiB/s): min= 4087, max= 6144, per=70.93%, avg=5446.00, stdev=1177.07, samples=3 iops : min= 3, max= 6, avg= 4.67, stdev= 1.53, samples=3 lat (usec) : 1000=8.33% lat (msec) : 100=16.67%, 250=41.67%, 500=16.67%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=29.79%, ctx=190, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147833: Wed Sep 24 15:08:44 2025 read: IOPS=1, BW=1843KiB/s (1888kB/s)(6144KiB/3333msec) clat (usec): min=856, max=226743, avg=125843.94, stdev=78210.30 lat (usec): min=857, max=226745, avg=125845.10, stdev=78210.39 clat percentiles (usec): | 1.00th=[ 857], 5.00th=[ 857], 10.00th=[ 857], 20.00th=[101188], | 30.00th=[101188], 40.00th=[117965], 50.00th=[117965], 60.00th=[119014], | 70.00th=[189793], 80.00th=[189793], 90.00th=[227541], 95.00th=[227541], | 99.00th=[227541], 99.50th=[227541], 99.90th=[227541], 99.95th=[227541], | 99.99th=[227541] bw ( KiB/s): min= 2043, max= 6144, per=62.18%, avg=3410.00, stdev=2367.71, samples=3 iops : min= 1, max= 6, avg= 2.67, stdev= 2.89, samples=3 write: IOPS=1, BW=1843KiB/s (1888kB/s)(6144KiB/3333msec); 0 zone resets clat (msec): min=154, max=1095, avg=425.31, stdev=357.24 lat (msec): min=154, max=1095, avg=425.39, stdev=357.23 clat percentiles (msec): | 1.00th=[ 155], 5.00th=[ 155], 10.00th=[ 155], 20.00th=[ 167], | 30.00th=[ 167], 40.00th=[ 209], 50.00th=[ 209], 60.00th=[ 460], | 70.00th=[ 464], 80.00th=[ 464], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 2043, max= 3488, per=31.33%, avg=2405.50, stdev=721.67, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (usec) : 1000=8.33% lat (msec) : 250=66.67%, 500=16.67%, 2000=8.33% cpu : usr=0.06%, sys=21.43%, ctx=160, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147834: Wed Sep 24 15:08:44 2025 read: IOPS=1, BW=1542KiB/s (1579kB/s)(5120KiB/3321msec) clat (usec): min=737, max=229821, avg=144595.10, stdev=93104.10 lat (usec): min=739, max=229822, avg=144596.27, stdev=93104.10 clat percentiles (usec): | 1.00th=[ 742], 5.00th=[ 742], 10.00th=[ 742], 20.00th=[ 742], | 30.00th=[107480], 40.00th=[107480], 50.00th=[173016], 60.00th=[173016], | 70.00th=[210764], 80.00th=[210764], 90.00th=[229639], 95.00th=[229639], | 99.00th=[229639], 99.50th=[229639], 99.90th=[229639], 99.95th=[229639], | 99.99th=[229639] bw ( KiB/s): min= 4071, max= 4096, per=74.46%, avg=4083.50, stdev=17.68, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2158KiB/s (2210kB/s)(7168KiB/3321msec); 0 zone resets clat (msec): min=140, max=1163, avg=366.81, stdev=373.91 lat (msec): min=140, max=1163, avg=366.88, stdev=373.89 clat percentiles (msec): | 1.00th=[ 140], 5.00th=[ 140], 10.00th=[ 140], 20.00th=[ 148], | 30.00th=[ 155], 40.00th=[ 155], 50.00th=[ 199], 60.00th=[ 257], | 70.00th=[ 257], 80.00th=[ 506], 90.00th=[ 1167], 95.00th=[ 1167], | 99.00th=[ 1167], 99.50th=[ 1167], 99.90th=[ 1167], 99.95th=[ 1167], | 99.99th=[ 1167] bw ( KiB/s): min= 1818, max= 4087, per=39.20%, avg=3010.00, stdev=1247.15, samples=4 iops : min= 1, max= 3, avg= 2.25, stdev= 0.96, samples=4 lat (usec) : 750=8.33% lat (msec) : 250=66.67%, 500=8.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=31.42%, ctx=333, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5485KiB/s (5616kB/s), 591KiB/s-1920KiB/s (605kB/s-1966kB/s), io=20.0MiB (20.0MB), run=3321-3734msec WRITE: bw=7679KiB/s (7863kB/s), 1371KiB/s-2955KiB/s (1404kB/s-3026kB/s), io=28.0MiB (29.4MB), run=3321-3734msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147830: Wed Sep 24 15:08:44 2025 read: IOPS=1, BW=1835KiB/s (1879kB/s)(7168KiB/3907msec) clat (msec): min=38, max=1331, avg=364.18, stdev=479.43 lat (msec): min=38, max=1331, avg=364.18, stdev=479.43 clat percentiles (msec): | 1.00th=[ 40], 5.00th=[ 40], 10.00th=[ 40], 20.00th=[ 41], | 30.00th=[ 69], 40.00th=[ 69], 50.00th=[ 171], 60.00th=[ 234], | 70.00th=[ 234], 80.00th=[ 667], 90.00th=[ 1334], 95.00th=[ 1334], | 99.00th=[ 1334], 99.50th=[ 1334], 99.90th=[ 1334], 99.95th=[ 1334], | 99.99th=[ 1334] bw ( KiB/s): min= 2015, max= 4000, per=48.17%, avg=2524.50, stdev=983.76, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 write: IOPS=1, BW=1310KiB/s (1342kB/s)(5120KiB/3907msec); 0 zone resets clat (msec): min=66, max=528, avg=210.15, stdev=190.76 lat (msec): min=66, max=528, avg=210.27, stdev=190.72 clat percentiles (msec): | 1.00th=[ 67], 5.00th=[ 67], 10.00th=[ 67], 20.00th=[ 67], | 30.00th=[ 68], 40.00th=[ 68], 50.00th=[ 155], 60.00th=[ 155], | 70.00th=[ 232], 80.00th=[ 232], 90.00th=[ 527], 95.00th=[ 527], | 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527], | 99.99th=[ 527] bw ( KiB/s): min= 1992, max= 2000, per=27.20%, avg=1996.00, stdev= 5.66, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 50=16.67%, 100=25.00%, 250=33.33%, 750=16.67%, 2000=8.33% cpu : usr=0.05%, sys=8.01%, ctx=105, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147832: Wed Sep 24 15:08:44 2025 read: IOPS=0, BW=547KiB/s (561kB/s)(2048KiB/3741msec) clat (msec): min=56, max=343, avg=200.17, stdev=202.64 lat (msec): min=56, max=343, avg=200.17, stdev=202.64 clat percentiles (msec): | 1.00th=[ 57], 5.00th=[ 57], 10.00th=[ 57], 20.00th=[ 57], | 30.00th=[ 57], 40.00th=[ 57], 50.00th=[ 57], 60.00th=[ 342], | 70.00th=[ 342], 80.00th=[ 342], 90.00th=[ 342], 95.00th=[ 342], | 99.00th=[ 342], 99.50th=[ 342], 99.90th=[ 342], 99.95th=[ 342], | 99.99th=[ 342] bw ( KiB/s): min= 1965, max= 2035, per=38.16%, avg=2000.00, stdev=49.50, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2737KiB/s (2803kB/s)(10.0MiB/3741msec); 0 zone resets clat (msec): min=49, max=1383, avg=330.80, stdev=406.60 lat (msec): min=49, max=1383, avg=330.90, stdev=406.59 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 50], 20.00th=[ 68], | 30.00th=[ 78], 40.00th=[ 146], 50.00th=[ 150], 60.00th=[ 199], | 70.00th=[ 305], 80.00th=[ 317], 90.00th=[ 617], 95.00th=[ 1385], | 99.00th=[ 1385], 99.50th=[ 1385], 99.90th=[ 1385], 99.95th=[ 1385], | 99.99th=[ 1385] bw ( KiB/s): min= 2043, max= 6144, per=55.63%, avg=4082.25, stdev=1674.28, samples=4 iops : min= 1, max= 6, avg= 3.25, stdev= 2.06, samples=4 lat (msec) : 50=8.33%, 100=25.00%, 250=25.00%, 500=25.00%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.08%, sys=11.26%, ctx=106, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147835: Wed Sep 24 15:08:44 2025 read: IOPS=1, BW=1634KiB/s (1673kB/s)(6144KiB/3760msec) clat (msec): min=36, max=105, avg=67.85, stdev=27.39 lat (msec): min=36, max=105, avg=67.86, stdev=27.39 clat percentiles (msec): | 1.00th=[ 37], 5.00th=[ 37], 10.00th=[ 37], 20.00th=[ 40], | 30.00th=[ 40], 40.00th=[ 63], 50.00th=[ 63], 60.00th=[ 72], | 70.00th=[ 91], 80.00th=[ 91], 90.00th=[ 106], 95.00th=[ 106], | 99.00th=[ 106], 99.50th=[ 106], 99.90th=[ 106], 99.95th=[ 106], | 99.99th=[ 106] bw ( KiB/s): min= 1992, max= 4071, per=64.45%, avg=3378.00, stdev=1200.31, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=1, BW=1634KiB/s (1673kB/s)(6144KiB/3760msec); 0 zone resets clat (msec): min=118, max=1259, avg=506.85, stdev=436.26 lat (msec): min=118, max=1259, avg=506.95, stdev=436.25 clat percentiles (msec): | 1.00th=[ 120], 5.00th=[ 120], 10.00th=[ 120], 20.00th=[ 142], | 30.00th=[ 142], 40.00th=[ 255], 50.00th=[ 255], 60.00th=[ 575], | 70.00th=[ 693], 80.00th=[ 693], 90.00th=[ 1267], 95.00th=[ 1267], | 99.00th=[ 1267], 99.50th=[ 1267], 99.90th=[ 1267], 99.95th=[ 1267], | 99.99th=[ 1267] bw ( KiB/s): min= 1992, max= 2043, per=27.59%, avg=2024.80, stdev=20.30, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (msec) : 50=16.67%, 100=25.00%, 250=25.00%, 500=8.33%, 750=16.67% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=9.55%, ctx=103, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147836: Wed Sep 24 15:08:44 2025 read: IOPS=1, BW=1356KiB/s (1389kB/s)(5120KiB/3775msec) clat (msec): min=39, max=410, avg=134.00, stdev=157.27 lat (msec): min=39, max=410, avg=134.00, stdev=157.27 clat percentiles (msec): | 1.00th=[ 40], 5.00th=[ 40], 10.00th=[ 40], 20.00th=[ 40], | 30.00th=[ 45], 40.00th=[ 45], 50.00th=[ 63], 60.00th=[ 63], | 70.00th=[ 112], 80.00th=[ 112], 90.00th=[ 409], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 2048, max= 6107, per=77.80%, avg=4077.50, stdev=2870.15, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 write: IOPS=1, BW=1899KiB/s (1944kB/s)(7168KiB/3775msec); 0 zone resets clat (msec): min=135, max=1400, avg=435.04, stdev=455.32 lat (msec): min=135, max=1400, avg=435.14, stdev=455.35 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 153], | 30.00th=[ 188], 40.00th=[ 188], 50.00th=[ 194], 60.00th=[ 397], | 70.00th=[ 397], 80.00th=[ 584], 90.00th=[ 1401], 95.00th=[ 1401], | 99.00th=[ 1401], 99.50th=[ 1401], 99.90th=[ 1401], 99.95th=[ 1401], | 99.99th=[ 1401] bw ( KiB/s): min= 1992, max= 6107, per=45.94%, avg=3371.33, stdev=2369.18, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 lat (msec) : 50=16.67%, 100=8.33%, 250=41.67%, 500=16.67%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.03%, sys=12.80%, ctx=123, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5242KiB/s (5368kB/s), 547KiB/s-1835KiB/s (561kB/s-1879kB/s), io=20.0MiB (20.0MB), run=3741-3907msec WRITE: bw=7339KiB/s (7515kB/s), 1310KiB/s-2737KiB/s (1342kB/s-2803kB/s), io=28.0MiB (29.4MB), run=3741-3907msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147841: Wed Sep 24 15:08:53 2025 read: IOPS=1, BW=6029KiB/s (6174kB/s)(12.0MiB/2038msec) clat (msec): min=490, max=746, avg=658.90, stdev=145.48 lat (msec): min=490, max=747, avg=658.90, stdev=145.48 clat percentiles (msec): | 1.00th=[ 493], 5.00th=[ 493], 10.00th=[ 493], 20.00th=[ 493], | 30.00th=[ 493], 40.00th=[ 743], 50.00th=[ 743], 60.00th=[ 743], | 70.00th=[ 751], 80.00th=[ 751], 90.00th=[ 751], 95.00th=[ 751], | 99.00th=[ 751], 99.50th=[ 751], 99.90th=[ 751], 99.95th=[ 751], | 99.99th=[ 751] bw ( KiB/s): min= 8094, max= 8175, per=100.00%, avg=8134.50, stdev=57.28, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.00%, sys=40.60%, ctx=817, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147842: Wed Sep 24 15:08:53 2025 read: IOPS=0, BW=1402KiB/s (1436kB/s)(4096KiB/2921msec) clat (nsec): min=488496k, max=488496k, avg=488495590.00, stdev= 0.00 lat (nsec): min=488498k, max=488498k, avg=488497755.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 489], 5.00th=[ 489], 10.00th=[ 489], 20.00th=[ 489], | 30.00th=[ 489], 40.00th=[ 489], 50.00th=[ 489], 60.00th=[ 489], | 70.00th=[ 489], 80.00th=[ 489], 90.00th=[ 489], 95.00th=[ 489], | 99.00th=[ 489], 99.50th=[ 489], 99.90th=[ 489], 99.95th=[ 489], | 99.99th=[ 489] bw ( KiB/s): min= 8094, max= 8094, per=100.00%, avg=8094.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2805KiB/s (2872kB/s)(8192KiB/2921msec); 0 zone resets clat (msec): min=739, max=1666, avg=1203.04, stdev=655.12 lat (msec): min=740, max=1667, avg=1204.09, stdev=656.01 clat percentiles (msec): | 1.00th=[ 743], 5.00th=[ 743], 10.00th=[ 743], 20.00th=[ 743], | 30.00th=[ 743], 40.00th=[ 743], 50.00th=[ 743], 60.00th=[ 1670], | 70.00th=[ 1670], 80.00th=[ 1670], 90.00th=[ 1670], 95.00th=[ 1670], | 99.00th=[ 1670], 99.50th=[ 1670], 99.90th=[ 1670], 99.95th=[ 1670], | 99.99th=[ 1670] bw ( KiB/s): min= 8159, max= 8159, per=72.73%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.03%, sys=27.19%, ctx=195, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147843: Wed Sep 24 15:08:53 2025 write: IOPS=1, BW=4289KiB/s (4392kB/s)(12.0MiB/2865msec); 0 zone resets clat (msec): min=703, max=1420, avg=946.43, stdev=410.55 lat (msec): min=703, max=1420, avg=946.87, stdev=410.52 clat percentiles (msec): | 1.00th=[ 701], 5.00th=[ 701], 10.00th=[ 701], 20.00th=[ 701], | 30.00th=[ 701], 40.00th=[ 718], 50.00th=[ 718], 60.00th=[ 718], | 70.00th=[ 1418], 80.00th=[ 1418], 90.00th=[ 1418], 95.00th=[ 1418], | 99.00th=[ 1418], 99.50th=[ 1418], 99.90th=[ 1418], 99.95th=[ 1418], | 99.99th=[ 1418] bw ( KiB/s): min= 8094, max= 8175, per=72.51%, avg=8134.50, stdev=57.28, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=38.64%, ctx=244, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147844: Wed Sep 24 15:08:53 2025 write: IOPS=1, BW=4919KiB/s (5037kB/s)(12.0MiB/2498msec); 0 zone resets clat (msec): min=565, max=1112, avg=820.98, stdev=275.41 lat (msec): min=565, max=1112, avg=821.38, stdev=275.42 clat percentiles (msec): | 1.00th=[ 567], 5.00th=[ 567], 10.00th=[ 567], 20.00th=[ 567], | 30.00th=[ 567], 40.00th=[ 785], 50.00th=[ 785], 60.00th=[ 785], | 70.00th=[ 1116], 80.00th=[ 1116], 90.00th=[ 1116], 95.00th=[ 1116], | 99.00th=[ 1116], 99.50th=[ 1116], 99.90th=[ 1116], 99.95th=[ 1116], | 99.99th=[ 1116] bw ( KiB/s): min= 7937, max= 8094, per=71.45%, avg=8015.50, stdev=111.02, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=48.90%, ctx=118, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5609KiB/s (5744kB/s), 1402KiB/s-6029KiB/s (1436kB/s-6174kB/s), io=16.0MiB (16.8MB), run=2038-2921msec WRITE: bw=10.0MiB/s (11.5MB/s), 2805KiB/s-4919KiB/s (2872kB/s-5037kB/s), io=32.0MiB (33.6MB), run=2498-2921msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147846: Wed Sep 24 15:08:54 2025 read: IOPS=0, BW=3920KiB/s (4014kB/s)(12.0MiB/3135msec) clat (msec): min=981, max=1139, avg=1038.51, stdev=88.06 lat (msec): min=981, max=1139, avg=1038.51, stdev=88.06 clat percentiles (msec): | 1.00th=[ 986], 5.00th=[ 986], 10.00th=[ 986], 20.00th=[ 986], | 30.00th=[ 986], 40.00th=[ 995], 50.00th=[ 995], 60.00th=[ 995], | 70.00th=[ 1133], 80.00th=[ 1133], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 8110, max= 8192, per=100.00%, avg=8151.00, stdev=57.98, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 1000=66.67%, 2000=33.33% cpu : usr=0.00%, sys=7.08%, ctx=261, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147847: Wed Sep 24 15:08:54 2025 read: IOPS=0, BW=1140KiB/s (1167kB/s)(4096KiB/3594msec) clat (nsec): min=1286.2M, max=1286.2M, avg=1286189687.00, stdev= 0.00 lat (nsec): min=1286.2M, max=1286.2M, avg=1286195873.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1284], 5.00th=[ 1284], 10.00th=[ 1284], 20.00th=[ 1284], | 30.00th=[ 1284], 40.00th=[ 1284], 50.00th=[ 1284], 60.00th=[ 1284], | 70.00th=[ 1284], 80.00th=[ 1284], 90.00th=[ 1284], 95.00th=[ 1284], | 99.00th=[ 1284], 99.50th=[ 1284], 99.90th=[ 1284], 99.95th=[ 1284], | 99.99th=[ 1284] bw ( KiB/s): min= 8015, max= 8015, per=100.00%, avg=8015.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2279KiB/s (2334kB/s)(8192KiB/3594msec); 0 zone resets clat (msec): min=147, max=2078, avg=1112.90, stdev=1364.86 lat (msec): min=148, max=2078, avg=1113.70, stdev=1364.83 clat percentiles (msec): | 1.00th=[ 148], 5.00th=[ 148], 10.00th=[ 148], 20.00th=[ 148], | 30.00th=[ 148], 40.00th=[ 148], 50.00th=[ 148], 60.00th=[ 2072], | 70.00th=[ 2072], 80.00th=[ 2072], 90.00th=[ 2072], 95.00th=[ 2072], | 99.00th=[ 2072], 99.50th=[ 2072], 99.90th=[ 2072], 99.95th=[ 2072], | 99.99th=[ 2072] bw ( KiB/s): min= 8192, max= 8192, per=89.85%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.03%, sys=7.15%, ctx=72, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147848: Wed Sep 24 15:08:54 2025 write: IOPS=0, BW=3715KiB/s (3804kB/s)(12.0MiB/3308msec); 0 zone resets clat (msec): min=977, max=1138, avg=1079.05, stdev=88.48 lat (msec): min=977, max=1139, avg=1079.42, stdev=88.34 clat percentiles (msec): | 1.00th=[ 978], 5.00th=[ 978], 10.00th=[ 978], 20.00th=[ 978], | 30.00th=[ 978], 40.00th=[ 1116], 50.00th=[ 1116], 60.00th=[ 1116], | 70.00th=[ 1133], 80.00th=[ 1133], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 8015, max= 8110, per=88.43%, avg=8062.50, stdev=67.18, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=7.22%, ctx=77, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147849: Wed Sep 24 15:08:54 2025 write: IOPS=0, BW=3475KiB/s (3559kB/s)(12.0MiB/3536msec); 0 zone resets clat (msec): min=370, max=2283, avg=1162.24, stdev=998.33 lat (msec): min=371, max=2284, avg=1162.87, stdev=998.47 clat percentiles (msec): | 1.00th=[ 372], 5.00th=[ 372], 10.00th=[ 372], 20.00th=[ 372], | 30.00th=[ 372], 40.00th=[ 835], 50.00th=[ 835], 60.00th=[ 835], | 70.00th=[ 2299], 80.00th=[ 2299], 90.00th=[ 2299], 95.00th=[ 2299], | 99.00th=[ 2299], 99.50th=[ 2299], 99.90th=[ 2299], 99.95th=[ 2299], | 99.99th=[ 2299] bw ( KiB/s): min= 8110, max= 8110, per=88.95%, avg=8110.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=33.33%, >=2000=33.33% cpu : usr=0.03%, sys=13.07%, ctx=131, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4559KiB/s (4668kB/s), 1140KiB/s-3920KiB/s (1167kB/s-4014kB/s), io=16.0MiB (16.8MB), run=3135-3594msec WRITE: bw=9117KiB/s (9336kB/s), 2279KiB/s-3715KiB/s (2334kB/s-3804kB/s), io=32.0MiB (33.6MB), run=3308-3594msec