== sanity test 398b: DIO and buffer IO race ============== 03:34:55 (1759822495) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.39651 s, 14.8 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189322: Tue Oct 7 03:36:08 2025 read: IOPS=23, BW=94.8KiB/s (97.1kB/s)(5992KiB/63218msec) clat (usec): min=103, max=768547, avg=23118.17, stdev=22623.81 lat (usec): min=104, max=768547, avg=23118.91, stdev=22623.85 clat percentiles (usec): | 1.00th=[ 408], 5.00th=[ 8356], 10.00th=[ 10421], 20.00th=[ 13304], | 30.00th=[ 15926], 40.00th=[ 17957], 50.00th=[ 20579], 60.00th=[ 23200], | 70.00th=[ 26608], 80.00th=[ 30802], 90.00th=[ 36963], 95.00th=[ 43779], | 99.00th=[ 62129], 99.50th=[ 69731], 99.90th=[127402], 99.95th=[767558], | 99.99th=[767558] bw ( KiB/s): min= 7, max= 198, per=24.82%, avg=93.81, stdev=31.07, samples=125 iops : min= 1, max= 49, avg=22.94, stdev= 7.80, samples=125 write: IOPS=24, BW=99.6KiB/s (102kB/s)(6296KiB/63218msec); 0 zone resets clat (usec): min=641, max=97255, avg=18025.57, stdev=12927.78 lat (usec): min=641, max=97256, avg=18026.45, stdev=12927.84 clat percentiles (usec): | 1.00th=[ 1123], 5.00th=[ 2933], 10.00th=[ 3818], 20.00th=[ 6128], | 30.00th=[ 8979], 40.00th=[11863], 50.00th=[14877], 60.00th=[19530], | 70.00th=[23462], 80.00th=[28967], 90.00th=[36439], 95.00th=[42206], | 99.00th=[54789], 99.50th=[61080], 99.90th=[81265], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 8, max= 183, per=25.00%, avg=99.01, stdev=33.00, samples=124 iops : min= 2, max= 45, avg=24.24, stdev= 8.26, samples=124 lat (usec) : 250=0.10%, 500=0.42%, 750=0.49%, 1000=0.39% lat (msec) : 2=0.78%, 4=4.49%, 10=14.75%, 20=33.30%, 50=42.97% lat (msec) : 100=2.25%, 250=0.03%, 1000=0.03% cpu : usr=0.07%, sys=27.17%, ctx=10551, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189324: Tue Oct 7 03:36:08 2025 read: IOPS=23, BW=94.6KiB/s (96.8kB/s)(5940KiB/62812msec) clat (usec): min=171, max=93301, avg=22645.03, stdev=11127.82 lat (usec): min=172, max=93302, avg=22645.76, stdev=11127.93 clat percentiles (usec): | 1.00th=[ 734], 5.00th=[ 8848], 10.00th=[10945], 20.00th=[13304], | 30.00th=[15664], 40.00th=[17957], 50.00th=[20579], 60.00th=[23462], | 70.00th=[26870], 80.00th=[31327], 90.00th=[36963], 95.00th=[42730], | 99.00th=[56361], 99.50th=[63701], 99.90th=[73925], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 7, max= 167, per=25.14%, avg=95.02, stdev=27.09, samples=124 iops : min= 1, max= 41, avg=23.23, stdev= 6.78, samples=124 write: IOPS=25, BW=101KiB/s (103kB/s)(6348KiB/62812msec); 0 zone resets clat (usec): min=662, max=762406, avg=18254.94, stdev=22568.00 lat (usec): min=663, max=762406, avg=18255.87, stdev=22568.01 clat percentiles (usec): | 1.00th=[ 1975], 5.00th=[ 3195], 10.00th=[ 4293], 20.00th=[ 6063], | 30.00th=[ 9634], 40.00th=[ 12649], 50.00th=[ 15664], 60.00th=[ 18744], | 70.00th=[ 22152], 80.00th=[ 26870], 90.00th=[ 34866], 95.00th=[ 41681], | 99.00th=[ 58459], 99.50th=[ 66323], 99.90th=[107480], 99.95th=[759170], | 99.99th=[759170] bw ( KiB/s): min= 39, max= 207, per=25.65%, avg=101.59, stdev=35.12, samples=124 iops : min= 9, max= 51, avg=24.89, stdev= 8.79, samples=124 lat (usec) : 250=0.07%, 500=0.10%, 750=0.36%, 1000=0.23% lat (msec) : 2=0.49%, 4=4.36%, 10=14.39%, 20=35.35%, 50=42.68% lat (msec) : 100=1.92%, 250=0.03%, 1000=0.03% cpu : usr=0.10%, sys=27.33%, ctx=10340, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189326: Tue Oct 7 03:36:08 2025 read: IOPS=23, BW=95.5KiB/s (97.8kB/s)(6028KiB/63090msec) clat (usec): min=95, max=94612, avg=22370.39, stdev=11605.43 lat (usec): min=95, max=94613, avg=22371.24, stdev=11605.48 clat percentiles (usec): | 1.00th=[ 611], 5.00th=[ 8717], 10.00th=[10290], 20.00th=[13304], | 30.00th=[15664], 40.00th=[17695], 50.00th=[19792], 60.00th=[22938], | 70.00th=[26084], 80.00th=[30540], 90.00th=[37487], 95.00th=[43254], | 99.00th=[60031], 99.50th=[70779], 99.90th=[83362], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 8, max= 176, per=25.31%, avg=95.68, stdev=29.33, samples=124 iops : min= 2, max= 44, avg=23.40, stdev= 7.34, samples=124 write: IOPS=24, BW=99.2KiB/s (102kB/s)(6260KiB/63090msec); 0 zone resets clat (usec): min=731, max=745649, avg=18636.42, stdev=22357.20 lat (usec): min=732, max=745651, avg=18637.60, stdev=22357.39 clat percentiles (usec): | 1.00th=[ 1663], 5.00th=[ 3097], 10.00th=[ 3982], 20.00th=[ 6063], | 30.00th=[ 9503], 40.00th=[ 12780], 50.00th=[ 15795], 60.00th=[ 19268], | 70.00th=[ 22938], 80.00th=[ 28443], 90.00th=[ 35914], 95.00th=[ 41681], | 99.00th=[ 57410], 99.50th=[ 63701], 99.90th=[ 80217], 99.95th=[742392], | 99.99th=[742392] bw ( KiB/s): min= 24, max= 184, per=25.03%, avg=99.14, stdev=31.64, samples=124 iops : min= 6, max= 46, avg=24.27, stdev= 7.95, samples=124 lat (usec) : 100=0.07%, 250=0.03%, 500=0.29%, 750=0.33%, 1000=0.29% lat (msec) : 2=0.46%, 4=4.62%, 10=14.06%, 20=36.17%, 50=41.21% lat (msec) : 100=2.44%, 750=0.03% cpu : usr=0.07%, sys=26.99%, ctx=10453, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189328: Tue Oct 7 03:36:08 2025 read: IOPS=23, BW=95.5KiB/s (97.8kB/s)(6052KiB/63362msec) clat (usec): min=154, max=83084, avg=22374.13, stdev=11481.31 lat (usec): min=155, max=83085, avg=22375.56, stdev=11482.04 clat percentiles (usec): | 1.00th=[ 437], 5.00th=[ 8225], 10.00th=[10159], 20.00th=[12911], | 30.00th=[15270], 40.00th=[17695], 50.00th=[20317], 60.00th=[23462], | 70.00th=[26608], 80.00th=[30802], 90.00th=[37487], 95.00th=[43254], | 99.00th=[57410], 99.50th=[69731], 99.90th=[79168], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 24, max= 216, per=25.39%, avg=95.99, stdev=30.75, samples=125 iops : min= 6, max= 54, avg=23.49, stdev= 7.74, samples=125 write: IOPS=24, BW=98.4KiB/s (101kB/s)(6236KiB/63362msec); 0 zone resets clat (usec): min=731, max=759840, avg=18762.34, stdev=22862.37 lat (usec): min=732, max=759841, avg=18764.95, stdev=22861.61 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 2999], 10.00th=[ 3785], 20.00th=[ 6259], | 30.00th=[ 9241], 40.00th=[ 12780], 50.00th=[ 15533], 60.00th=[ 19530], | 70.00th=[ 23462], 80.00th=[ 28705], 90.00th=[ 35914], 95.00th=[ 43254], | 99.00th=[ 57410], 99.50th=[ 67634], 99.90th=[ 87557], 99.95th=[759170], | 99.99th=[759170] bw ( KiB/s): min= 32, max= 208, per=24.95%, avg=98.79, stdev=35.93, samples=125 iops : min= 8, max= 52, avg=24.19, stdev= 9.01, samples=125 lat (usec) : 250=0.10%, 500=0.49%, 750=0.23%, 1000=0.39% lat (msec) : 2=0.81%, 4=4.43%, 10=14.42%, 20=34.08%, 50=42.90% lat (msec) : 100=2.12%, 1000=0.03% cpu : usr=0.08%, sys=27.09%, ctx=10385, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=379KiB/s (388kB/s), 94.6KiB/s-95.5KiB/s (96.8kB/s-97.8kB/s), io=23.4MiB (24.6MB), run=62812-63362msec WRITE: bw=397KiB/s (406kB/s), 98.4KiB/s-101KiB/s (101kB/s-103kB/s), io=24.6MiB (25.7MB), run=62812-63362msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189321: Tue Oct 7 03:36:14 2025 read: IOPS=21, BW=86.8KiB/s (88.8kB/s)(5992KiB/69061msec) clat (msec): min=2, max=764, avg=20.41, stdev=24.30 lat (msec): min=2, max=764, avg=20.42, stdev=24.30 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 31], 90.00th=[ 41], 95.00th=[ 51], | 99.00th=[ 69], 99.50th=[ 78], 99.90th=[ 113], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 23, max= 328, per=24.89%, avg=86.38, stdev=59.52, samples=136 iops : min= 5, max= 82, avg=21.14, stdev=15.01, samples=136 write: IOPS=22, BW=91.2KiB/s (93.4kB/s)(6296KiB/69061msec); 0 zone resets clat (msec): min=3, max=149, avg=24.32, stdev=17.02 lat (msec): min=3, max=149, avg=24.32, stdev=17.02 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 13], 40.00th=[ 17], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 31], 80.00th=[ 37], 90.00th=[ 47], 95.00th=[ 57], | 99.00th=[ 80], 99.50th=[ 88], 99.90th=[ 130], 99.95th=[ 150], | 99.99th=[ 150] bw ( KiB/s): min= 8, max= 392, per=24.83%, avg=90.39, stdev=61.67, samples=136 iops : min= 2, max= 98, avg=22.14, stdev=15.55, samples=136 lat (msec) : 4=1.04%, 10=24.58%, 20=31.02%, 50=36.82%, 100=6.35% lat (msec) : 250=0.16%, 1000=0.03% cpu : usr=0.07%, sys=13.54%, ctx=6322, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189323: Tue Oct 7 03:36:14 2025 read: IOPS=21, BW=86.4KiB/s (88.5kB/s)(5940KiB/68712msec) clat (msec): min=2, max=734, avg=20.16, stdev=23.45 lat (msec): min=2, max=734, avg=20.16, stdev=23.45 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 23], 80.00th=[ 30], 90.00th=[ 41], 95.00th=[ 50], | 99.00th=[ 65], 99.50th=[ 71], 99.90th=[ 121], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 8, max= 376, per=24.91%, avg=86.42, stdev=61.58, samples=135 iops : min= 2, max= 94, avg=21.15, stdev=15.52, samples=135 write: IOPS=23, BW=92.4KiB/s (94.6kB/s)(6348KiB/68712msec); 0 zone resets clat (msec): min=4, max=132, avg=24.30, stdev=16.52 lat (msec): min=4, max=132, avg=24.30, stdev=16.52 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 31], 80.00th=[ 37], 90.00th=[ 47], 95.00th=[ 55], | 99.00th=[ 80], 99.50th=[ 89], 99.90th=[ 125], 99.95th=[ 133], | 99.99th=[ 133] bw ( KiB/s): min= 23, max= 368, per=25.26%, avg=91.94, stdev=61.57, samples=135 iops : min= 5, max= 92, avg=22.53, stdev=15.52, samples=135 lat (msec) : 4=0.75%, 10=23.93%, 20=31.64%, 50=37.53%, 100=5.92% lat (msec) : 250=0.20%, 750=0.03% cpu : usr=0.08%, sys=13.48%, ctx=6364, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189325: Tue Oct 7 03:36:14 2025 read: IOPS=21, BW=87.6KiB/s (89.7kB/s)(6028KiB/68848msec) clat (msec): min=2, max=121, avg=19.92, stdev=15.21 lat (msec): min=2, max=121, avg=19.92, stdev=15.21 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 23], 80.00th=[ 31], 90.00th=[ 43], 95.00th=[ 52], | 99.00th=[ 68], 99.50th=[ 77], 99.90th=[ 92], 99.95th=[ 123], | 99.99th=[ 123] bw ( KiB/s): min= 15, max= 360, per=25.30%, avg=87.79, stdev=62.00, samples=136 iops : min= 3, max= 90, avg=21.49, stdev=15.63, samples=136 write: IOPS=22, BW=90.9KiB/s (93.1kB/s)(6260KiB/68848msec); 0 zone resets clat (msec): min=4, max=734, avg=24.67, stdev=24.31 lat (msec): min=4, max=734, avg=24.67, stdev=24.31 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 30], 80.00th=[ 36], 90.00th=[ 47], 95.00th=[ 56], | 99.00th=[ 79], 99.50th=[ 89], 99.90th=[ 130], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 32, max= 336, per=25.04%, avg=91.15, stdev=58.97, samples=136 iops : min= 8, max= 84, avg=22.33, stdev=14.88, samples=136 lat (msec) : 4=0.68%, 10=24.22%, 20=31.58%, 50=37.08%, 100=6.28% lat (msec) : 250=0.13%, 750=0.03% cpu : usr=0.09%, sys=13.55%, ctx=6486, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189327: Tue Oct 7 03:36:14 2025 read: IOPS=21, BW=87.7KiB/s (89.8kB/s)(6052KiB/69013msec) clat (msec): min=2, max=124, avg=20.46, stdev=15.36 lat (msec): min=2, max=124, avg=20.46, stdev=15.36 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 31], 90.00th=[ 43], 95.00th=[ 52], | 99.00th=[ 71], 99.50th=[ 74], 99.90th=[ 118], 99.95th=[ 125], | 99.99th=[ 125] bw ( KiB/s): min= 23, max= 360, per=25.12%, avg=87.15, stdev=58.80, samples=136 iops : min= 5, max= 90, avg=21.33, stdev=14.83, samples=136 write: IOPS=22, BW=90.4KiB/s (92.5kB/s)(6236KiB/69013msec); 0 zone resets clat (msec): min=3, max=739, avg=24.28, stdev=24.21 lat (msec): min=3, max=739, avg=24.28, stdev=24.21 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 13], 40.00th=[ 17], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 31], 80.00th=[ 37], 90.00th=[ 46], 95.00th=[ 53], | 99.00th=[ 78], 99.50th=[ 90], 99.90th=[ 118], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 31, max= 336, per=24.75%, avg=90.08, stdev=62.78, samples=136 iops : min= 7, max= 84, avg=22.07, stdev=15.84, samples=136 lat (msec) : 4=1.24%, 10=24.41%, 20=29.98%, 50=38.44%, 100=5.73% lat (msec) : 250=0.16%, 750=0.03% cpu : usr=0.07%, sys=13.32%, ctx=6393, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=348KiB/s (356kB/s), 86.4KiB/s-87.7KiB/s (88.5kB/s-89.8kB/s), io=23.4MiB (24.6MB), run=68712-69061msec WRITE: bw=364KiB/s (373kB/s), 90.4KiB/s-92.4KiB/s (92.5kB/s-94.6kB/s), io=24.6MiB (25.7MB), run=68712-69061msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189356: Tue Oct 7 03:36:36 2025 read: IOPS=23, BW=383KiB/s (392kB/s)(5920KiB/15463msec) clat (usec): min=239, max=68616, avg=21206.54, stdev=10930.60 lat (usec): min=240, max=68616, avg=21207.12, stdev=10930.63 clat percentiles (usec): | 1.00th=[ 392], 5.00th=[ 898], 10.00th=[10028], 20.00th=[12387], | 30.00th=[15270], 40.00th=[17433], 50.00th=[19530], 60.00th=[22152], | 70.00th=[25297], 80.00th=[30278], 90.00th=[35914], 95.00th=[40633], | 99.00th=[51119], 99.50th=[60031], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 192, max= 704, per=25.53%, avg=378.87, stdev=114.41, samples=30 iops : min= 12, max= 44, avg=23.17, stdev= 7.19, samples=30 write: IOPS=25, BW=412KiB/s (422kB/s)(6368KiB/15463msec); 0 zone resets clat (usec): min=1160, max=59433, avg=19063.48, stdev=11868.82 lat (usec): min=1161, max=59434, avg=19093.11, stdev=11862.70 clat percentiles (usec): | 1.00th=[ 3064], 5.00th=[ 4817], 10.00th=[ 5932], 20.00th=[ 9110], | 30.00th=[11600], 40.00th=[13566], 50.00th=[16319], 60.00th=[18744], | 70.00th=[22414], 80.00th=[28967], 90.00th=[35914], 95.00th=[43779], | 99.00th=[57410], 99.50th=[58459], 99.90th=[59507], 99.95th=[59507], | 99.99th=[59507] bw ( KiB/s): min= 221, max= 638, per=26.36%, avg=411.80, stdev=121.23, samples=30 iops : min= 13, max= 39, avg=25.23, stdev= 7.57, samples=30 lat (usec) : 250=0.13%, 500=1.04%, 750=0.91%, 1000=0.39% lat (msec) : 2=0.39%, 4=0.91%, 10=12.50%, 20=41.93%, 50=40.23% lat (msec) : 100=1.56% cpu : usr=0.06%, sys=30.25%, ctx=3059, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189357: Tue Oct 7 03:36:36 2025 read: IOPS=22, BW=356KiB/s (365kB/s)(5744KiB/16130msec) clat (usec): min=346, max=69772, avg=22337.75, stdev=11407.28 lat (usec): min=346, max=69773, avg=22338.35, stdev=11407.33 clat percentiles (usec): | 1.00th=[ 537], 5.00th=[ 8717], 10.00th=[11076], 20.00th=[13829], | 30.00th=[15664], 40.00th=[17695], 50.00th=[20579], 60.00th=[22676], | 70.00th=[25560], 80.00th=[31065], 90.00th=[38536], 95.00th=[44303], | 99.00th=[58459], 99.50th=[65274], 99.90th=[69731], 99.95th=[69731], | 99.99th=[69731] bw ( KiB/s): min= 158, max= 512, per=23.82%, avg=353.47, stdev=95.43, samples=32 iops : min= 9, max= 32, avg=21.59, stdev= 6.04, samples=32 write: IOPS=25, BW=406KiB/s (415kB/s)(6544KiB/16130msec); 0 zone resets clat (usec): min=1103, max=90919, avg=19772.62, stdev=13191.23 lat (usec): min=1104, max=90920, avg=19774.01, stdev=13191.29 clat percentiles (usec): | 1.00th=[ 2474], 5.00th=[ 4686], 10.00th=[ 5669], 20.00th=[ 8848], | 30.00th=[11731], 40.00th=[13829], 50.00th=[16188], 60.00th=[19530], | 70.00th=[23462], 80.00th=[29754], 90.00th=[39060], 95.00th=[45351], | 99.00th=[61604], 99.50th=[65274], 99.90th=[90702], 99.95th=[90702], | 99.99th=[90702] bw ( KiB/s): min= 159, max= 800, per=25.92%, avg=404.94, stdev=140.11, samples=32 iops : min= 9, max= 50, avg=24.81, stdev= 8.82, samples=32 lat (usec) : 500=0.39%, 750=0.78%, 1000=0.65% lat (msec) : 2=0.65%, 4=1.04%, 10=11.85%, 20=39.32%, 50=43.10% lat (msec) : 100=2.21% cpu : usr=0.12%, sys=28.96%, ctx=3101, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189358: Tue Oct 7 03:36:36 2025 read: IOPS=24, BW=393KiB/s (402kB/s)(6256KiB/15916msec) clat (usec): min=290, max=76772, avg=22164.78, stdev=11884.87 lat (usec): min=290, max=76773, avg=22165.48, stdev=11884.91 clat percentiles (usec): | 1.00th=[ 490], 5.00th=[ 2474], 10.00th=[10683], 20.00th=[13435], | 30.00th=[15926], 40.00th=[17957], 50.00th=[19792], 60.00th=[22938], | 70.00th=[26084], 80.00th=[30016], 90.00th=[36439], 95.00th=[43254], | 99.00th=[68682], 99.50th=[74974], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 223, max= 637, per=26.38%, avg=391.48, stdev=97.30, samples=31 iops : min= 13, max= 39, avg=23.97, stdev= 6.12, samples=31 write: IOPS=23, BW=379KiB/s (388kB/s)(6032KiB/15916msec); 0 zone resets clat (usec): min=1113, max=79145, avg=19191.61, stdev=11995.00 lat (usec): min=1113, max=79147, avg=19192.80, stdev=11995.04 clat percentiles (usec): | 1.00th=[ 2114], 5.00th=[ 4948], 10.00th=[ 5604], 20.00th=[ 8455], | 30.00th=[11600], 40.00th=[14484], 50.00th=[17433], 60.00th=[20317], | 70.00th=[23725], 80.00th=[28705], 90.00th=[33817], 95.00th=[40633], | 99.00th=[53740], 99.50th=[78119], 99.90th=[79168], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 159, max= 577, per=23.73%, avg=370.68, stdev=107.87, samples=31 iops : min= 9, max= 36, avg=22.68, stdev= 6.80, samples=31 lat (usec) : 500=0.65%, 750=1.43%, 1000=0.39% lat (msec) : 2=0.39%, 4=0.39%, 10=13.93%, 20=37.24%, 50=43.36% lat (msec) : 100=2.21% cpu : usr=0.09%, sys=29.31%, ctx=2958, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189359: Tue Oct 7 03:36:36 2025 read: IOPS=23, BW=375KiB/s (384kB/s)(6032KiB/16080msec) clat (usec): min=196, max=78097, avg=21939.60, stdev=12411.79 lat (usec): min=196, max=78098, avg=21940.21, stdev=12411.83 clat percentiles (usec): | 1.00th=[ 371], 5.00th=[ 709], 10.00th=[ 9372], 20.00th=[13042], | 30.00th=[15533], 40.00th=[17695], 50.00th=[20055], 60.00th=[22676], | 70.00th=[25560], 80.00th=[30278], 90.00th=[36439], 95.00th=[42206], | 99.00th=[63177], 99.50th=[71828], 99.90th=[78119], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 191, max= 672, per=25.20%, avg=373.97, stdev=120.88, samples=32 iops : min= 11, max= 42, avg=22.91, stdev= 7.51, samples=32 write: IOPS=24, BW=389KiB/s (398kB/s)(6256KiB/16080msec); 0 zone resets clat (usec): min=1234, max=73118, avg=19939.39, stdev=13010.29 lat (usec): min=1235, max=73120, avg=19940.64, stdev=13010.37 clat percentiles (usec): | 1.00th=[ 3884], 5.00th=[ 4817], 10.00th=[ 5669], 20.00th=[ 8586], | 30.00th=[11863], 40.00th=[14615], 50.00th=[17433], 60.00th=[19792], | 70.00th=[23725], 80.00th=[29230], 90.00th=[38011], 95.00th=[44303], | 99.00th=[67634], 99.50th=[70779], 99.90th=[72877], 99.95th=[72877], | 99.99th=[72877] bw ( KiB/s): min= 222, max= 672, per=24.91%, avg=389.03, stdev=110.52, samples=32 iops : min= 13, max= 42, avg=23.81, stdev= 7.03, samples=32 lat (usec) : 250=0.13%, 500=1.30%, 750=1.30%, 1000=0.13% lat (msec) : 2=0.65%, 4=0.39%, 10=13.93%, 20=37.50%, 50=41.28% lat (msec) : 100=3.39% cpu : usr=0.10%, sys=29.14%, ctx=3018, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1485KiB/s (1521kB/s), 356KiB/s-393KiB/s (365kB/s-402kB/s), io=23.4MiB (24.5MB), run=15463-16130msec WRITE: bw=1562KiB/s (1600kB/s), 379KiB/s-412KiB/s (388kB/s-422kB/s), io=24.6MiB (25.8MB), run=15463-16130msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189351: Tue Oct 7 03:36:38 2025 read: IOPS=20, BW=328KiB/s (336kB/s)(5920KiB/18029msec) clat (usec): min=3248, max=91200, avg=20762.16, stdev=15458.77 lat (usec): min=3248, max=91201, avg=20762.98, stdev=15458.91 clat percentiles (usec): | 1.00th=[ 4178], 5.00th=[ 5669], 10.00th=[ 6849], 20.00th=[ 9110], | 30.00th=[10683], 40.00th=[12911], 50.00th=[15139], 60.00th=[18482], | 70.00th=[23987], 80.00th=[31065], 90.00th=[43254], 95.00th=[55313], | 99.00th=[72877], 99.50th=[77071], 99.90th=[90702], 99.95th=[90702], | 99.99th=[90702] bw ( KiB/s): min= 128, max= 830, per=23.85%, avg=315.34, stdev=153.41, samples=35 iops : min= 8, max= 51, avg=19.14, stdev= 9.73, samples=35 write: IOPS=22, BW=353KiB/s (362kB/s)(6368KiB/18029msec); 0 zone resets clat (msec): min=5, max=109, avg=25.89, stdev=17.41 lat (msec): min=5, max=109, avg=25.89, stdev=17.41 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 26], | 70.00th=[ 31], 80.00th=[ 39], 90.00th=[ 52], 95.00th=[ 60], | 99.00th=[ 93], 99.50th=[ 109], 99.90th=[ 110], 99.95th=[ 110], | 99.99th=[ 110] bw ( KiB/s): min= 160, max= 1021, per=24.76%, avg=344.34, stdev=170.68, samples=35 iops : min= 10, max= 63, avg=20.97, stdev=10.73, samples=35 lat (msec) : 4=0.26%, 10=18.62%, 20=36.20%, 50=36.33%, 100=8.20% lat (msec) : 250=0.39% cpu : usr=0.11%, sys=13.05%, ctx=1652, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189352: Tue Oct 7 03:36:38 2025 read: IOPS=19, BW=317KiB/s (325kB/s)(5744KiB/18111msec) clat (usec): min=2806, max=96025, avg=22438.01, stdev=17091.66 lat (usec): min=2807, max=96025, avg=22438.71, stdev=17091.70 clat percentiles (usec): | 1.00th=[ 3032], 5.00th=[ 5473], 10.00th=[ 6980], 20.00th=[ 9110], | 30.00th=[11600], 40.00th=[14091], 50.00th=[16712], 60.00th=[20841], | 70.00th=[26608], 80.00th=[33162], 90.00th=[45876], 95.00th=[56361], | 99.00th=[82314], 99.50th=[94897], 99.90th=[95945], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 96, max= 702, per=22.94%, avg=303.26, stdev=132.31, samples=35 iops : min= 6, max= 43, avg=18.40, stdev= 8.35, samples=35 write: IOPS=22, BW=361KiB/s (370kB/s)(6544KiB/18111msec); 0 zone resets clat (msec): min=4, max=115, avg=24.53, stdev=16.56 lat (msec): min=4, max=115, avg=24.53, stdev=16.56 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 8], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 21], 60.00th=[ 26], | 70.00th=[ 30], 80.00th=[ 38], 90.00th=[ 47], 95.00th=[ 56], | 99.00th=[ 80], 99.50th=[ 90], 99.90th=[ 115], 99.95th=[ 115], | 99.99th=[ 115] bw ( KiB/s): min= 96, max= 1053, per=24.34%, avg=338.60, stdev=182.25, samples=35 iops : min= 6, max= 65, avg=20.60, stdev=11.44, samples=35 lat (msec) : 4=1.04%, 10=19.27%, 20=33.20%, 50=38.54%, 100=7.81% lat (msec) : 250=0.13% cpu : usr=0.05%, sys=12.95%, ctx=1670, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189353: Tue Oct 7 03:36:38 2025 read: IOPS=21, BW=349KiB/s (357kB/s)(6256KiB/17930msec) clat (msec): min=3, max=111, avg=20.68, stdev=15.90 lat (msec): min=3, max=111, avg=20.68, stdev=15.90 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 30], 90.00th=[ 42], 95.00th=[ 57], | 99.00th=[ 72], 99.50th=[ 96], 99.90th=[ 112], 99.95th=[ 112], | 99.99th=[ 112] bw ( KiB/s): min= 94, max= 1213, per=26.22%, avg=346.69, stdev=200.54, samples=35 iops : min= 5, max= 75, avg=21.11, stdev=12.59, samples=35 write: IOPS=21, BW=336KiB/s (344kB/s)(6032KiB/17930msec); 0 zone resets clat (msec): min=5, max=117, avg=26.01, stdev=16.79 lat (msec): min=5, max=117, avg=26.01, stdev=16.79 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 27], | 70.00th=[ 32], 80.00th=[ 40], 90.00th=[ 51], 95.00th=[ 59], | 99.00th=[ 79], 99.50th=[ 89], 99.90th=[ 117], 99.95th=[ 117], | 99.99th=[ 117] bw ( KiB/s): min= 95, max= 672, per=23.60%, avg=328.26, stdev=143.09, samples=35 iops : min= 5, max= 42, avg=19.97, stdev= 9.06, samples=35 lat (msec) : 4=0.13%, 10=19.01%, 20=37.11%, 50=35.55%, 100=7.94% lat (msec) : 250=0.26% cpu : usr=0.07%, sys=13.36%, ctx=1651, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189354: Tue Oct 7 03:36:38 2025 read: IOPS=21, BW=337KiB/s (345kB/s)(6032KiB/17890msec) clat (msec): min=4, max=130, avg=20.85, stdev=15.61 lat (msec): min=4, max=130, avg=20.85, stdev=15.61 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 12], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 23], 80.00th=[ 31], 90.00th=[ 42], 95.00th=[ 50], | 99.00th=[ 77], 99.50th=[ 81], 99.90th=[ 131], 99.95th=[ 131], | 99.99th=[ 131] bw ( KiB/s): min= 95, max= 896, per=24.99%, avg=330.40, stdev=168.04, samples=35 iops : min= 5, max= 56, avg=20.09, stdev=10.68, samples=35 write: IOPS=21, BW=350KiB/s (358kB/s)(6256KiB/17890msec); 0 zone resets clat (msec): min=5, max=117, avg=25.53, stdev=16.39 lat (msec): min=5, max=117, avg=25.53, stdev=16.39 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 27], | 70.00th=[ 30], 80.00th=[ 37], 90.00th=[ 46], 95.00th=[ 58], | 99.00th=[ 85], 99.50th=[ 116], 99.90th=[ 118], 99.95th=[ 118], | 99.99th=[ 118] bw ( KiB/s): min= 159, max= 926, per=24.75%, avg=344.29, stdev=147.54, samples=35 iops : min= 9, max= 57, avg=20.97, stdev= 9.25, samples=35 lat (msec) : 10=16.02%, 20=39.58%, 50=38.15%, 100=5.86%, 250=0.39% cpu : usr=0.08%, sys=13.59%, ctx=1653, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1323KiB/s (1354kB/s), 317KiB/s-349KiB/s (325kB/s-357kB/s), io=23.4MiB (24.5MB), run=17890-18111msec WRITE: bw=1391KiB/s (1425kB/s), 336KiB/s-361KiB/s (344kB/s-370kB/s), io=24.6MiB (25.8MB), run=17890-18111msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189368: Tue Oct 7 03:36:48 2025 read: IOPS=2, BW=2456KiB/s (2515kB/s)(7168KiB/2918msec) clat (usec): min=874, max=154040, avg=117067.31, stdev=52724.44 lat (usec): min=875, max=154041, avg=117068.57, stdev=52724.68 clat percentiles (usec): | 1.00th=[ 873], 5.00th=[ 873], 10.00th=[ 873], 20.00th=[120062], | 30.00th=[122160], 40.00th=[122160], 50.00th=[135267], 60.00th=[139461], | 70.00th=[139461], 80.00th=[147850], 90.00th=[154141], 95.00th=[154141], | 99.00th=[154141], 99.50th=[154141], 99.90th=[154141], 99.95th=[154141], | 99.99th=[154141] bw ( KiB/s): min= 2043, max= 5605, per=61.01%, avg=3914.67, stdev=1787.91, samples=3 iops : min= 1, max= 5, avg= 3.33, stdev= 2.08, samples=3 write: IOPS=1, BW=1755KiB/s (1797kB/s)(5120KiB/2918msec); 0 zone resets clat (msec): min=169, max=916, avg=417.19, stdev=296.55 lat (msec): min=169, max=916, avg=417.23, stdev=296.56 clat percentiles (msec): | 1.00th=[ 169], 5.00th=[ 169], 10.00th=[ 169], 20.00th=[ 169], | 30.00th=[ 271], 40.00th=[ 271], 50.00th=[ 284], 60.00th=[ 284], | 70.00th=[ 447], 80.00th=[ 447], 90.00th=[ 919], 95.00th=[ 919], | 99.00th=[ 919], 99.50th=[ 919], 99.90th=[ 919], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 2043, max= 2048, per=22.77%, avg=2045.50, stdev= 2.89, samples=4 iops : min= 1, max= 2, avg= 1.50, stdev= 0.58, samples=4 lat (usec) : 1000=8.33% lat (msec) : 250=58.33%, 500=25.00%, 1000=8.33% cpu : usr=0.00%, sys=42.92%, ctx=977, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189369: Tue Oct 7 03:36:48 2025 read: IOPS=0, BW=642KiB/s (657kB/s)(2048KiB/3192msec) clat (msec): min=114, max=132, avg=123.53, stdev=13.18 lat (msec): min=114, max=132, avg=123.53, stdev=13.18 clat percentiles (msec): | 1.00th=[ 114], 5.00th=[ 114], 10.00th=[ 114], 20.00th=[ 114], | 30.00th=[ 114], 40.00th=[ 114], 50.00th=[ 114], 60.00th=[ 133], | 70.00th=[ 133], 80.00th=[ 133], 90.00th=[ 133], 95.00th=[ 133], | 99.00th=[ 133], 99.50th=[ 133], 99.90th=[ 133], 99.95th=[ 133], | 99.99th=[ 133] bw ( KiB/s): min= 1865, max= 2048, per=30.49%, avg=1956.50, stdev=129.40, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3208KiB/s (3285kB/s)(10.0MiB/3192msec); 0 zone resets clat (msec): min=172, max=608, avg=290.77, stdev=134.53 lat (msec): min=172, max=608, avg=290.84, stdev=134.52 clat percentiles (msec): | 1.00th=[ 174], 5.00th=[ 174], 10.00th=[ 174], 20.00th=[ 180], | 30.00th=[ 201], 40.00th=[ 215], 50.00th=[ 253], 60.00th=[ 271], | 70.00th=[ 284], 80.00th=[ 292], 90.00th=[ 430], 95.00th=[ 609], | 99.00th=[ 609], 99.50th=[ 609], 99.90th=[ 609], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 2048, max= 4096, per=40.91%, avg=3674.60, stdev=909.43, samples=5 iops : min= 2, max= 4, avg= 3.00, stdev= 0.71, samples=5 lat (msec) : 250=50.00%, 500=41.67%, 750=8.33% cpu : usr=0.03%, sys=49.04%, ctx=588, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189370: Tue Oct 7 03:36:48 2025 read: IOPS=2, BW=2944KiB/s (3015kB/s)(6144KiB/2087msec) clat (usec): min=807, max=232201, avg=96808.90, stdev=108574.50 lat (usec): min=808, max=232201, avg=96809.88, stdev=108574.10 clat percentiles (usec): | 1.00th=[ 807], 5.00th=[ 807], 10.00th=[ 807], 20.00th=[ 832], | 30.00th=[ 832], 40.00th=[ 857], 50.00th=[ 857], 60.00th=[147850], | 70.00th=[198181], 80.00th=[198181], 90.00th=[231736], 95.00th=[231736], | 99.00th=[231736], 99.50th=[231736], 99.90th=[231736], 99.95th=[231736], | 99.99th=[231736] bw ( KiB/s): min= 2048, max= 6131, per=53.13%, avg=3409.00, stdev=2357.32, samples=3 iops : min= 2, max= 5, avg= 3.00, stdev= 1.73, samples=3 write: IOPS=2, BW=2944KiB/s (3015kB/s)(6144KiB/2087msec); 0 zone resets clat (msec): min=185, max=326, avg=244.27, stdev=54.55 lat (msec): min=185, max=326, avg=244.32, stdev=54.56 clat percentiles (msec): | 1.00th=[ 186], 5.00th=[ 186], 10.00th=[ 186], 20.00th=[ 203], | 30.00th=[ 203], 40.00th=[ 228], 50.00th=[ 228], 60.00th=[ 230], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 326], 95.00th=[ 326], | 99.00th=[ 326], 99.50th=[ 326], 99.90th=[ 326], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 2043, max= 3764, per=27.56%, avg=2475.75, stdev=858.84, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (usec) : 1000=25.00% lat (msec) : 250=58.33%, 500=16.67% cpu : usr=0.00%, sys=55.70%, ctx=413, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189371: Tue Oct 7 03:36:48 2025 read: IOPS=1, BW=1979KiB/s (2027kB/s)(5120KiB/2587msec) clat (usec): min=604, max=414987, avg=121890.68, stdev=183672.29 lat (usec): min=605, max=414988, avg=121891.55, stdev=183672.14 clat percentiles (usec): | 1.00th=[ 603], 5.00th=[ 603], 10.00th=[ 603], 20.00th=[ 603], | 30.00th=[ 734], 40.00th=[ 734], 50.00th=[ 742], 60.00th=[ 742], | 70.00th=[191890], 80.00th=[191890], 90.00th=[413139], 95.00th=[413139], | 99.00th=[413139], 99.50th=[413139], 99.90th=[413139], 99.95th=[413139], | 99.99th=[413139] bw ( KiB/s): min= 2048, max= 6131, per=63.74%, avg=4089.50, stdev=2887.12, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 write: IOPS=2, BW=2771KiB/s (2837kB/s)(7168KiB/2587msec); 0 zone resets clat (msec): min=117, max=507, avg=278.50, stdev=145.78 lat (msec): min=117, max=507, avg=278.57, stdev=145.77 clat percentiles (msec): | 1.00th=[ 118], 5.00th=[ 118], 10.00th=[ 118], 20.00th=[ 144], | 30.00th=[ 153], 40.00th=[ 153], 50.00th=[ 300], 60.00th=[ 359], | 70.00th=[ 359], 80.00th=[ 372], 90.00th=[ 506], 95.00th=[ 506], | 99.00th=[ 506], 99.50th=[ 506], 99.90th=[ 506], 99.95th=[ 506], | 99.99th=[ 506] bw ( KiB/s): min= 1946, max= 4087, per=33.87%, avg=3042.00, stdev=1207.38, samples=4 iops : min= 1, max= 3, avg= 2.25, stdev= 0.96, samples=4 lat (usec) : 750=25.00% lat (msec) : 250=33.33%, 500=33.33%, 750=8.33% cpu : usr=0.00%, sys=43.89%, ctx=282, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6416KiB/s (6570kB/s), 642KiB/s-2944KiB/s (657kB/s-3015kB/s), io=20.0MiB (20.0MB), run=2087-3192msec WRITE: bw=8982KiB/s (9198kB/s), 1755KiB/s-3208KiB/s (1797kB/s-3285kB/s), io=28.0MiB (29.4MB), run=2087-3192msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189372: Tue Oct 7 03:36:48 2025 read: IOPS=2, BW=2264KiB/s (2318kB/s)(7168KiB/3166msec) clat (msec): min=55, max=305, avg=145.65, stdev=88.48 lat (msec): min=55, max=305, avg=145.65, stdev=88.48 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 56], 10.00th=[ 56], 20.00th=[ 75], | 30.00th=[ 78], 40.00th=[ 78], 50.00th=[ 138], 60.00th=[ 165], | 70.00th=[ 165], 80.00th=[ 205], 90.00th=[ 305], 95.00th=[ 305], | 99.00th=[ 305], 99.50th=[ 305], 99.90th=[ 305], 99.95th=[ 305], | 99.99th=[ 305] bw ( KiB/s): min= 2048, max= 5688, per=64.06%, avg=3944.00, stdev=1824.75, samples=3 iops : min= 2, max= 5, avg= 3.67, stdev= 1.53, samples=3 write: IOPS=1, BW=1617KiB/s (1656kB/s)(5120KiB/3166msec); 0 zone resets clat (msec): min=159, max=578, avg=425.98, stdev=176.67 lat (msec): min=159, max=578, avg=426.05, stdev=176.68 clat percentiles (msec): | 1.00th=[ 159], 5.00th=[ 159], 10.00th=[ 159], 20.00th=[ 159], | 30.00th=[ 334], 40.00th=[ 334], 50.00th=[ 510], 60.00th=[ 510], | 70.00th=[ 550], 80.00th=[ 550], 90.00th=[ 584], 95.00th=[ 584], | 99.00th=[ 584], 99.50th=[ 584], 99.90th=[ 584], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 1875, max= 2048, per=23.22%, avg=2001.50, stdev=84.56, samples=4 iops : min= 1, max= 2, avg= 1.50, stdev= 0.58, samples=4 lat (msec) : 100=25.00%, 250=33.33%, 500=16.67%, 750=25.00% cpu : usr=0.00%, sys=19.97%, ctx=115, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189373: Tue Oct 7 03:36:48 2025 read: IOPS=0, BW=626KiB/s (641kB/s)(2048KiB/3272msec) clat (msec): min=43, max=118, avg=81.04, stdev=53.02 lat (msec): min=43, max=118, avg=81.04, stdev=53.03 clat percentiles (msec): | 1.00th=[ 44], 5.00th=[ 44], 10.00th=[ 44], 20.00th=[ 44], | 30.00th=[ 44], 40.00th=[ 44], 50.00th=[ 44], 60.00th=[ 120], | 70.00th=[ 120], 80.00th=[ 120], 90.00th=[ 120], 95.00th=[ 120], | 99.00th=[ 120], 99.50th=[ 120], 99.90th=[ 120], 99.95th=[ 120], | 99.99th=[ 120] bw ( KiB/s): min= 1946, max= 2035, per=32.33%, avg=1990.50, stdev=62.93, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3130KiB/s (3205kB/s)(10.0MiB/3272msec); 0 zone resets clat (msec): min=73, max=898, avg=303.36, stdev=264.62 lat (msec): min=73, max=899, avg=303.44, stdev=264.60 clat percentiles (msec): | 1.00th=[ 73], 5.00th=[ 73], 10.00th=[ 73], 20.00th=[ 101], | 30.00th=[ 130], 40.00th=[ 157], 50.00th=[ 201], 60.00th=[ 266], | 70.00th=[ 279], 80.00th=[ 284], 90.00th=[ 642], 95.00th=[ 902], | 99.00th=[ 902], 99.50th=[ 902], 99.90th=[ 902], 99.95th=[ 902], | 99.99th=[ 902] bw ( KiB/s): min= 2048, max= 6144, per=46.44%, avg=4003.25, stdev=1680.73, samples=4 iops : min= 2, max= 6, avg= 3.50, stdev= 1.73, samples=4 lat (msec) : 50=8.33%, 100=8.33%, 250=41.67%, 500=25.00%, 750=8.33% lat (msec) : 1000=8.33% cpu : usr=0.00%, sys=27.76%, ctx=235, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189374: Tue Oct 7 03:36:48 2025 read: IOPS=1, BW=1988KiB/s (2035kB/s)(6144KiB/3091msec) clat (msec): min=89, max=370, avg=230.06, stdev=118.78 lat (msec): min=89, max=370, avg=230.06, stdev=118.78 clat percentiles (msec): | 1.00th=[ 90], 5.00th=[ 90], 10.00th=[ 90], 20.00th=[ 140], | 30.00th=[ 140], 40.00th=[ 144], 50.00th=[ 144], 60.00th=[ 313], | 70.00th=[ 326], 80.00th=[ 326], 90.00th=[ 372], 95.00th=[ 372], | 99.00th=[ 372], 99.50th=[ 372], 99.90th=[ 372], 99.95th=[ 372], | 99.99th=[ 372] bw ( KiB/s): min= 2048, max= 4071, per=53.43%, avg=3289.67, stdev=1087.23, samples=3 iops : min= 2, max= 3, avg= 2.67, stdev= 0.58, samples=3 write: IOPS=1, BW=1988KiB/s (2035kB/s)(6144KiB/3091msec); 0 zone resets clat (msec): min=90, max=523, avg=275.89, stdev=163.89 lat (msec): min=90, max=523, avg=275.97, stdev=163.91 clat percentiles (msec): | 1.00th=[ 91], 5.00th=[ 91], 10.00th=[ 91], 20.00th=[ 194], | 30.00th=[ 194], 40.00th=[ 199], 50.00th=[ 199], 60.00th=[ 222], | 70.00th=[ 430], 80.00th=[ 430], 90.00th=[ 523], 95.00th=[ 523], | 99.00th=[ 523], 99.50th=[ 523], 99.90th=[ 523], 99.95th=[ 523], | 99.99th=[ 523] bw ( KiB/s): min= 1875, max= 4096, per=35.14%, avg=3028.75, stdev=1234.38, samples=4 iops : min= 1, max= 4, avg= 2.75, stdev= 1.50, samples=4 lat (msec) : 100=16.67%, 250=41.67%, 500=33.33%, 750=8.33% cpu : usr=0.03%, sys=16.63%, ctx=145, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189375: Tue Oct 7 03:36:48 2025 read: IOPS=1, BW=1539KiB/s (1576kB/s)(5120KiB/3326msec) clat (msec): min=20, max=449, avg=238.05, stdev=178.40 lat (msec): min=20, max=449, avg=238.05, stdev=178.40 clat percentiles (msec): | 1.00th=[ 21], 5.00th=[ 21], 10.00th=[ 21], 20.00th=[ 21], | 30.00th=[ 94], 40.00th=[ 94], 50.00th=[ 279], 60.00th=[ 279], | 70.00th=[ 351], 80.00th=[ 351], 90.00th=[ 451], 95.00th=[ 451], | 99.00th=[ 451], 99.50th=[ 451], 99.90th=[ 451], 99.95th=[ 451], | 99.99th=[ 451] bw ( KiB/s): min= 2035, max= 6144, per=66.42%, avg=4089.50, stdev=2905.50, samples=2 iops : min= 1, max= 6, avg= 3.50, stdev= 3.54, samples=2 write: IOPS=2, BW=2155KiB/s (2207kB/s)(7168KiB/3326msec); 0 zone resets clat (msec): min=38, max=890, avg=292.69, stdev=289.41 lat (msec): min=38, max=890, avg=292.76, stdev=289.42 clat percentiles (msec): | 1.00th=[ 39], 5.00th=[ 39], 10.00th=[ 39], 20.00th=[ 46], | 30.00th=[ 188], 40.00th=[ 188], 50.00th=[ 249], 60.00th=[ 271], | 70.00th=[ 271], 80.00th=[ 368], 90.00th=[ 894], 95.00th=[ 894], | 99.00th=[ 894], 99.50th=[ 894], 99.90th=[ 894], 99.95th=[ 894], | 99.99th=[ 894] bw ( KiB/s): min= 1875, max= 4096, per=31.01%, avg=2673.00, stdev=1235.39, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 lat (msec) : 50=25.00%, 100=8.33%, 250=16.67%, 500=41.67%, 1000=8.33% cpu : usr=0.03%, sys=14.11%, ctx=88, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6158KiB/s (6305kB/s), 626KiB/s-2264KiB/s (641kB/s-2318kB/s), io=20.0MiB (20.0MB), run=3091-3326msec WRITE: bw=8621KiB/s (8827kB/s), 1617KiB/s-3130KiB/s (1656kB/s-3205kB/s), io=28.0MiB (29.4MB), run=3091-3326msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189381: Tue Oct 7 03:36:58 2025 read: IOPS=1, BW=5105KiB/s (5228kB/s)(12.0MiB/2407msec) clat (msec): min=699, max=942, avg=787.35, stdev=134.44 lat (msec): min=699, max=942, avg=787.35, stdev=134.44 clat percentiles (msec): | 1.00th=[ 701], 5.00th=[ 701], 10.00th=[ 701], 20.00th=[ 701], | 30.00th=[ 701], 40.00th=[ 718], 50.00th=[ 718], 60.00th=[ 718], | 70.00th=[ 944], 80.00th=[ 944], 90.00th=[ 944], 95.00th=[ 944], | 99.00th=[ 944], 99.50th=[ 944], 99.90th=[ 944], 99.95th=[ 944], | 99.99th=[ 944] bw ( KiB/s): min= 8110, max= 8192, per=100.00%, avg=8151.00, stdev=57.98, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 1000=33.33% cpu : usr=0.00%, sys=46.63%, ctx=3122, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189382: Tue Oct 7 03:36:58 2025 read: IOPS=0, BW=1492KiB/s (1528kB/s)(4096KiB/2745msec) clat (nsec): min=941846k, max=941846k, avg=941845786.00, stdev= 0.00 lat (nsec): min=941847k, max=941847k, avg=941847063.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 944], 5.00th=[ 944], 10.00th=[ 944], 20.00th=[ 944], | 30.00th=[ 944], 40.00th=[ 944], 50.00th=[ 944], 60.00th=[ 944], | 70.00th=[ 944], 80.00th=[ 944], 90.00th=[ 944], 95.00th=[ 944], | 99.00th=[ 944], 99.50th=[ 944], 99.90th=[ 944], 99.95th=[ 944], | 99.99th=[ 944] bw ( KiB/s): min= 8110, max= 8110, per=100.00%, avg=8110.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2984KiB/s (3056kB/s)(8192KiB/2745msec); 0 zone resets clat (msec): min=518, max=1241, avg=880.02, stdev=511.15 lat (msec): min=518, max=1241, avg=880.36, stdev=511.05 clat percentiles (msec): | 1.00th=[ 518], 5.00th=[ 518], 10.00th=[ 518], 20.00th=[ 518], | 30.00th=[ 518], 40.00th=[ 518], 50.00th=[ 518], 60.00th=[ 1234], | 70.00th=[ 1234], 80.00th=[ 1234], 90.00th=[ 1234], 95.00th=[ 1234], | 99.00th=[ 1234], 99.50th=[ 1234], 99.90th=[ 1234], 99.95th=[ 1234], | 99.99th=[ 1234] bw ( KiB/s): min= 8192, max= 8192, per=88.41%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=15.67%, ctx=1195, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189383: Tue Oct 7 03:36:58 2025 write: IOPS=0, BW=3475KiB/s (3559kB/s)(12.0MiB/3536msec); 0 zone resets clat (msec): min=74, max=2759, avg=1165.11, stdev=1411.44 lat (msec): min=75, max=2759, avg=1165.41, stdev=1411.46 clat percentiles (msec): | 1.00th=[ 75], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 75], | 30.00th=[ 75], 40.00th=[ 659], 50.00th=[ 659], 60.00th=[ 659], | 70.00th=[ 2769], 80.00th=[ 2769], 90.00th=[ 2769], 95.00th=[ 2769], | 99.00th=[ 2769], 99.50th=[ 2769], 99.90th=[ 2769], 99.95th=[ 2769], | 99.99th=[ 2769] bw ( KiB/s): min= 8062, max= 8062, per=87.01%, avg=8062.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 100=33.33%, 750=33.33%, >=2000=33.33% cpu : usr=0.03%, sys=9.42%, ctx=66, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189384: Tue Oct 7 03:36:58 2025 write: IOPS=0, BW=3983KiB/s (4079kB/s)(12.0MiB/3085msec); 0 zone resets clat (msec): min=615, max=1595, avg=1023.16, stdev=510.24 lat (msec): min=615, max=1595, avg=1023.47, stdev=510.22 clat percentiles (msec): | 1.00th=[ 617], 5.00th=[ 617], 10.00th=[ 617], 20.00th=[ 617], | 30.00th=[ 617], 40.00th=[ 860], 50.00th=[ 860], 60.00th=[ 860], | 70.00th=[ 1603], 80.00th=[ 1603], 90.00th=[ 1603], 95.00th=[ 1603], | 99.00th=[ 1603], 99.50th=[ 1603], 99.90th=[ 1603], 99.95th=[ 1603], | 99.99th=[ 1603] bw ( KiB/s): min= 8110, max= 8175, per=87.88%, avg=8142.50, stdev=45.96, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=13.94%, ctx=114, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5969KiB/s (6112kB/s), 1492KiB/s-5105KiB/s (1528kB/s-5228kB/s), io=16.0MiB (16.8MB), run=2407-2745msec WRITE: bw=9267KiB/s (9489kB/s), 2984KiB/s-3983KiB/s (3056kB/s-4079kB/s), io=32.0MiB (33.6MB), run=2745-3536msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189386: Tue Oct 7 03:36:58 2025 read: IOPS=1, BW=4130KiB/s (4230kB/s)(12.0MiB/2975msec) clat (msec): min=482, max=1666, avg=987.63, stdev=611.25 lat (msec): min=482, max=1666, avg=987.63, stdev=611.25 clat percentiles (msec): | 1.00th=[ 481], 5.00th=[ 481], 10.00th=[ 481], 20.00th=[ 481], | 30.00th=[ 481], 40.00th=[ 818], 50.00th=[ 818], 60.00th=[ 818], | 70.00th=[ 1670], 80.00th=[ 1670], 90.00th=[ 1670], 95.00th=[ 1670], | 99.00th=[ 1670], 99.50th=[ 1670], 99.90th=[ 1670], 99.95th=[ 1670], | 99.99th=[ 1670] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=17.32%, ctx=87, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189387: Tue Oct 7 03:36:58 2025 read: IOPS=0, BW=1253KiB/s (1283kB/s)(4096KiB/3270msec) clat (nsec): min=1004.3M, max=1004.3M, avg=1004298760.00, stdev= 0.00 lat (nsec): min=1004.3M, max=1004.3M, avg=1004301490.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1003], 5.00th=[ 1003], 10.00th=[ 1003], 20.00th=[ 1003], | 30.00th=[ 1003], 40.00th=[ 1003], 50.00th=[ 1003], 60.00th=[ 1003], | 70.00th=[ 1003], 80.00th=[ 1003], 90.00th=[ 1003], 95.00th=[ 1003], | 99.00th=[ 1003], 99.50th=[ 1003], 99.90th=[ 1003], 99.95th=[ 1003], | 99.99th=[ 1003] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2505KiB/s (2565kB/s)(8192KiB/3270msec); 0 zone resets clat (msec): min=694, max=1531, avg=1113.13, stdev=592.33 lat (msec): min=694, max=1532, avg=1113.39, stdev=592.22 clat percentiles (msec): | 1.00th=[ 693], 5.00th=[ 693], 10.00th=[ 693], 20.00th=[ 693], | 30.00th=[ 693], 40.00th=[ 693], 50.00th=[ 693], 60.00th=[ 1536], | 70.00th=[ 1536], 80.00th=[ 1536], 90.00th=[ 1536], 95.00th=[ 1536], | 99.00th=[ 1536], 99.50th=[ 1536], 99.90th=[ 1536], 99.95th=[ 1536], | 99.99th=[ 1536] bw ( KiB/s): min= 8175, max= 8175, per=82.53%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=14.90%, ctx=122, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189388: Tue Oct 7 03:36:58 2025 write: IOPS=0, BW=3746KiB/s (3836kB/s)(12.0MiB/3280msec); 0 zone resets clat (msec): min=365, max=2259, avg=1088.79, stdev=1022.95 lat (msec): min=365, max=2259, avg=1089.06, stdev=1022.96 clat percentiles (msec): | 1.00th=[ 368], 5.00th=[ 368], 10.00th=[ 368], 20.00th=[ 368], | 30.00th=[ 368], 40.00th=[ 642], 50.00th=[ 642], 60.00th=[ 642], | 70.00th=[ 2265], 80.00th=[ 2265], 90.00th=[ 2265], 95.00th=[ 2265], | 99.00th=[ 2265], 99.50th=[ 2265], 99.90th=[ 2265], 99.95th=[ 2265], | 99.99th=[ 2265] bw ( KiB/s): min= 8175, max= 8192, per=82.62%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 750=33.33%, >=2000=33.33% cpu : usr=0.03%, sys=16.32%, ctx=84, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189389: Tue Oct 7 03:36:58 2025 write: IOPS=0, BW=3715KiB/s (3804kB/s)(12.0MiB/3308msec); 0 zone resets clat (msec): min=676, max=1404, avg=1094.17, stdev=375.43 lat (msec): min=677, max=1404, avg=1094.47, stdev=375.43 clat percentiles (msec): | 1.00th=[ 676], 5.00th=[ 676], 10.00th=[ 676], 20.00th=[ 676], | 30.00th=[ 676], 40.00th=[ 1200], 50.00th=[ 1200], 60.00th=[ 1200], | 70.00th=[ 1401], 80.00th=[ 1401], 90.00th=[ 1401], 95.00th=[ 1401], | 99.00th=[ 1401], 99.50th=[ 1401], 99.90th=[ 1401], 99.95th=[ 1401], | 99.99th=[ 1401] bw ( KiB/s): min= 7984, max= 8192, per=81.66%, avg=8088.00, stdev=147.08, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=17.24%, ctx=107, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5010KiB/s (5131kB/s), 1253KiB/s-4130KiB/s (1283kB/s-4230kB/s), io=16.0MiB (16.8MB), run=2975-3270msec WRITE: bw=9906KiB/s (10.1MB/s), 2505KiB/s-3746KiB/s (2565kB/s-3836kB/s), io=32.0MiB (33.6MB), run=3270-3308msec