== sanity test 398b: DIO and buffer IO race ============== 16:17:46 (1760127466) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 4.4567 s, 11.3 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=153196: Fri Oct 10 16:19:41 2025 read: IOPS=15, BW=61.1KiB/s (62.5kB/s)(5992KiB/98117msec) clat (usec): min=89, max=897572, avg=30930.67, stdev=32796.37 lat (usec): min=90, max=897574, avg=30932.27, stdev=32796.00 clat percentiles (usec): | 1.00th=[ 116], 5.00th=[ 155], 10.00th=[ 182], 20.00th=[ 11994], | 30.00th=[ 17957], 40.00th=[ 22152], 50.00th=[ 26608], 60.00th=[ 32375], | 70.00th=[ 39060], 80.00th=[ 48497], 90.00th=[ 60031], 95.00th=[ 72877], | 99.00th=[102237], 99.50th=[117965], 99.90th=[283116], 99.95th=[893387], | 99.99th=[893387] bw ( KiB/s): min= 7, max= 120, per=25.66%, avg=61.33, stdev=23.50, samples=192 iops : min= 1, max= 30, avg=14.71, stdev= 5.93, samples=192 write: IOPS=16, BW=64.2KiB/s (65.7kB/s)(6296KiB/98117msec); 0 zone resets clat (usec): min=906, max=497177, avg=32651.41, stdev=35472.68 lat (usec): min=908, max=497180, avg=32653.69, stdev=35473.01 clat percentiles (usec): | 1.00th=[ 1012], 5.00th=[ 1123], 10.00th=[ 1270], 20.00th=[ 2835], | 30.00th=[ 6915], 40.00th=[ 14877], 50.00th=[ 24511], 60.00th=[ 33424], | 70.00th=[ 44303], 80.00th=[ 56361], 90.00th=[ 73925], 95.00th=[ 94897], | 99.00th=[143655], 99.50th=[160433], 99.90th=[408945], 99.95th=[497026], | 99.99th=[497026] bw ( KiB/s): min= 8, max= 133, per=25.73%, avg=64.33, stdev=23.12, samples=192 iops : min= 2, max= 33, avg=15.47, stdev= 5.83, samples=192 lat (usec) : 100=0.07%, 250=6.77%, 500=1.11%, 750=0.26%, 1000=0.52% lat (msec) : 2=8.43%, 4=3.58%, 10=6.12%, 20=13.44%, 50=37.70% lat (msec) : 100=19.27%, 250=2.57%, 500=0.13%, 1000=0.03% cpu : usr=0.11%, sys=7.60%, ctx=6087, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153199: Fri Oct 10 16:19:41 2025 read: IOPS=14, BW=59.8KiB/s (61.2kB/s)(5940KiB/99327msec) clat (usec): min=103, max=914478, avg=31483.77, stdev=34422.64 lat (usec): min=104, max=914479, avg=31488.71, stdev=34422.32 clat percentiles (usec): | 1.00th=[ 121], 5.00th=[ 153], 10.00th=[ 182], 20.00th=[ 9503], | 30.00th=[ 17695], 40.00th=[ 22152], 50.00th=[ 26346], 60.00th=[ 32375], | 70.00th=[ 39584], 80.00th=[ 49546], 90.00th=[ 62129], 95.00th=[ 76022], | 99.00th=[109577], 99.50th=[130548], 99.90th=[316670], 99.95th=[918553], | 99.99th=[918553] bw ( KiB/s): min= 7, max= 142, per=25.03%, avg=59.83, stdev=23.45, samples=194 iops : min= 1, max= 35, avg=14.34, stdev= 5.89, samples=194 write: IOPS=15, BW=63.9KiB/s (65.4kB/s)(6348KiB/99327msec); 0 zone resets clat (usec): min=897, max=383334, avg=32851.25, stdev=35233.25 lat (usec): min=898, max=383336, avg=32853.67, stdev=35233.66 clat percentiles (usec): | 1.00th=[ 996], 5.00th=[ 1156], 10.00th=[ 1254], 20.00th=[ 2704], | 30.00th=[ 6587], 40.00th=[ 13173], 50.00th=[ 23725], 60.00th=[ 33817], | 70.00th=[ 44303], 80.00th=[ 56886], 90.00th=[ 79168], 95.00th=[ 96994], | 99.00th=[141558], 99.50th=[166724], 99.90th=[278922], 99.95th=[383779], | 99.99th=[383779] bw ( KiB/s): min= 8, max= 167, per=25.39%, avg=63.48, stdev=25.15, samples=195 iops : min= 2, max= 41, avg=15.26, stdev= 6.29, samples=195 lat (usec) : 250=7.03%, 500=1.50%, 750=0.20%, 1000=0.72% lat (msec) : 2=8.76%, 4=3.16%, 10=6.32%, 20=13.22%, 50=36.69% lat (msec) : 100=19.56%, 250=2.67%, 500=0.16%, 1000=0.03% cpu : usr=0.10%, sys=7.66%, ctx=6132, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153200: Fri Oct 10 16:19:41 2025 read: IOPS=15, BW=60.1KiB/s (61.5kB/s)(6028KiB/100373msec) clat (usec): min=95, max=905365, avg=29291.94, stdev=34201.43 lat (usec): min=96, max=905366, avg=29293.79, stdev=34201.35 clat percentiles (usec): | 1.00th=[ 114], 5.00th=[ 145], 10.00th=[ 178], 20.00th=[ 1811], | 30.00th=[ 16712], 40.00th=[ 21103], 50.00th=[ 24773], 60.00th=[ 28705], | 70.00th=[ 35390], 80.00th=[ 46924], 90.00th=[ 59507], 95.00th=[ 72877], | 99.00th=[ 98042], 99.50th=[135267], 99.90th=[337642], 99.95th=[901776], | 99.99th=[901776] bw ( KiB/s): min= 7, max= 152, per=24.95%, avg=59.63, stdev=23.73, samples=197 iops : min= 1, max= 38, avg=14.28, stdev= 5.97, samples=197 write: IOPS=15, BW=62.4KiB/s (63.9kB/s)(6260KiB/100373msec); 0 zone resets clat (usec): min=885, max=408262, avg=35674.08, stdev=37243.06 lat (usec): min=887, max=408264, avg=35678.13, stdev=37243.22 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 1156], 10.00th=[ 1237], 20.00th=[ 2573], | 30.00th=[ 7439], 40.00th=[ 17433], 50.00th=[ 26870], 60.00th=[ 36439], | 70.00th=[ 47449], 80.00th=[ 62129], 90.00th=[ 83362], 95.00th=[100140], | 99.00th=[141558], 99.50th=[175113], 99.90th=[350225], 99.95th=[408945], | 99.99th=[408945] bw ( KiB/s): min= 8, max= 134, per=24.74%, avg=61.85, stdev=24.11, samples=197 iops : min= 2, max= 33, avg=14.85, stdev= 6.04, samples=197 lat (usec) : 100=0.07%, 250=7.29%, 500=1.95%, 750=0.03%, 1000=0.55% lat (msec) : 2=8.85%, 4=3.29%, 10=5.34%, 20=12.21%, 50=37.43% lat (msec) : 100=19.86%, 250=2.93%, 500=0.16%, 1000=0.03% cpu : usr=0.09%, sys=7.50%, ctx=6147, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153203: Fri Oct 10 16:19:41 2025 read: IOPS=15, BW=60.7KiB/s (62.2kB/s)(6052KiB/99695msec) clat (usec): min=100, max=350327, avg=29537.21, stdev=25090.15 lat (usec): min=100, max=350329, avg=29539.77, stdev=25089.55 clat percentiles (usec): | 1.00th=[ 115], 5.00th=[ 147], 10.00th=[ 174], 20.00th=[ 4047], | 30.00th=[ 16909], 40.00th=[ 21365], 50.00th=[ 25560], 60.00th=[ 30540], | 70.00th=[ 36963], 80.00th=[ 49021], 90.00th=[ 61604], 95.00th=[ 72877], | 99.00th=[104334], 99.50th=[114820], 99.90th=[154141], 99.95th=[350225], | 99.99th=[350225] bw ( KiB/s): min= 7, max= 127, per=25.81%, avg=61.69, stdev=23.00, samples=192 iops : min= 1, max= 31, avg=14.81, stdev= 5.78, samples=192 write: IOPS=15, BW=62.5KiB/s (64.1kB/s)(6236KiB/99695msec); 0 zone resets clat (usec): min=909, max=1096.3k, avg=34991.04, stdev=44842.27 lat (usec): min=910, max=1096.3k, avg=34994.75, stdev=44842.08 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1156], 10.00th=[ 1270], | 20.00th=[ 2606], 30.00th=[ 6587], 40.00th=[ 15401], | 50.00th=[ 26608], 60.00th=[ 35914], 70.00th=[ 46400], | 80.00th=[ 59507], 90.00th=[ 80217], 95.00th=[ 100140], | 99.00th=[ 141558], 99.50th=[ 185598], 99.90th=[ 400557], | 99.95th=[1098908], 99.99th=[1098908] bw ( KiB/s): min= 7, max= 119, per=25.01%, avg=62.52, stdev=22.18, samples=195 iops : min= 1, max= 29, avg=15.03, stdev= 5.57, samples=195 lat (usec) : 250=7.65%, 500=1.40%, 750=0.13%, 1000=0.36% lat (msec) : 2=8.76%, 4=3.84%, 10=6.32%, 20=11.65%, 50=36.88% lat (msec) : 100=20.02%, 250=2.86%, 500=0.10%, 2000=0.03% cpu : usr=0.09%, sys=7.52%, ctx=6077, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=239KiB/s (245kB/s), 59.8KiB/s-61.1KiB/s (61.2kB/s-62.5kB/s), io=23.4MiB (24.6MB), run=98117-100373msec WRITE: bw=250KiB/s (256kB/s), 62.4KiB/s-64.2KiB/s (63.9kB/s-65.7kB/s), io=24.6MiB (25.7MB), run=98117-100373msec rand-rw: (groupid=0, jobs=1): err= 0: pid=153197: Fri Oct 10 16:22:30 2025 read: IOPS=5, BW=22.2KiB/s (22.7kB/s)(5992KiB/270239msec) clat (msec): min=6, max=906, avg=30.02, stdev=34.72 lat (msec): min=6, max=906, avg=30.02, stdev=34.72 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 19], 50.00th=[ 22], 60.00th=[ 25], | 70.00th=[ 29], 80.00th=[ 38], 90.00th=[ 60], 95.00th=[ 80], | 99.00th=[ 136], 99.50th=[ 167], 99.90th=[ 368], 99.95th=[ 911], | 99.99th=[ 911] bw ( KiB/s): min= 7, max= 103, per=27.89%, avg=24.54, stdev=16.15, samples=471 iops : min= 1, max= 25, avg= 5.48, stdev= 4.07, samples=471 write: IOPS=5, BW=23.3KiB/s (23.9kB/s)(6296KiB/270239msec); 0 zone resets clat (msec): min=49, max=579, avg=142.87, stdev=53.23 lat (msec): min=49, max=579, avg=142.87, stdev=53.23 clat percentiles (msec): | 1.00th=[ 63], 5.00th=[ 79], 10.00th=[ 88], 20.00th=[ 101], | 30.00th=[ 111], 40.00th=[ 122], 50.00th=[ 133], 60.00th=[ 144], | 70.00th=[ 161], 80.00th=[ 180], 90.00th=[ 211], 95.00th=[ 241], | 99.00th=[ 305], 99.50th=[ 351], 99.90th=[ 502], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 7, max= 40, per=24.71%, avg=22.73, stdev= 8.02, samples=532 iops : min= 1, max= 10, avg= 5.03, stdev= 2.06, samples=532 lat (msec) : 10=2.31%, 20=20.67%, 50=19.27%, 100=15.79%, 250=39.75% lat (msec) : 500=2.12%, 750=0.07%, 1000=0.03% cpu : usr=0.04%, sys=1.22%, ctx=5226, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153198: Fri Oct 10 16:22:30 2025 read: IOPS=5, BW=21.0KiB/s (22.5kB/s)(5940KiB/270382msec) clat (msec): min=5, max=980, avg=29.33, stdev=36.40 lat (msec): min=5, max=980, avg=29.33, stdev=36.40 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 23], | 70.00th=[ 29], 80.00th=[ 37], 90.00th=[ 58], 95.00th=[ 75], | 99.00th=[ 126], 99.50th=[ 144], 99.90th=[ 558], 99.95th=[ 978], | 99.99th=[ 978] bw ( KiB/s): min= 7, max= 96, per=28.73%, avg=25.28, stdev=16.20, samples=453 iops : min= 1, max= 24, avg= 5.67, stdev= 4.09, samples=453 write: IOPS=5, BW=23.5KiB/s (24.0kB/s)(6348KiB/270382msec); 0 zone resets clat (msec): min=44, max=608, avg=142.68, stdev=53.64 lat (msec): min=44, max=608, avg=142.68, stdev=53.64 clat percentiles (msec): | 1.00th=[ 62], 5.00th=[ 75], 10.00th=[ 85], 20.00th=[ 100], | 30.00th=[ 111], 40.00th=[ 123], 50.00th=[ 134], 60.00th=[ 146], | 70.00th=[ 161], 80.00th=[ 182], 90.00th=[ 213], 95.00th=[ 245], | 99.00th=[ 313], 99.50th=[ 351], 99.90th=[ 435], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 7, max= 47, per=24.85%, avg=22.86, stdev= 8.17, samples=533 iops : min= 1, max= 11, avg= 5.06, stdev= 2.09, samples=533 lat (msec) : 10=2.25%, 20=21.71%, 50=18.10%, 100=15.95%, 250=39.78% lat (msec) : 500=2.12%, 750=0.07%, 1000=0.03% cpu : usr=0.05%, sys=1.23%, ctx=5277, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153201: Fri Oct 10 16:22:30 2025 read: IOPS=5, BW=22.3KiB/s (22.8kB/s)(6028KiB/270319msec) clat (msec): min=5, max=275, avg=28.97, stdev=24.18 lat (msec): min=5, max=275, avg=28.97, stdev=24.18 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 19], 50.00th=[ 22], 60.00th=[ 24], | 70.00th=[ 29], 80.00th=[ 38], 90.00th=[ 62], 95.00th=[ 79], | 99.00th=[ 117], 99.50th=[ 148], 99.90th=[ 222], 99.95th=[ 275], | 99.99th=[ 275] bw ( KiB/s): min= 7, max= 102, per=27.19%, avg=23.93, stdev=16.53, samples=485 iops : min= 1, max= 25, avg= 5.33, stdev= 4.15, samples=485 write: IOPS=5, BW=23.2KiB/s (23.7kB/s)(6260KiB/270319msec); 0 zone resets clat (msec): min=50, max=1071, avg=144.58, stdev=58.83 lat (msec): min=50, max=1071, avg=144.59, stdev=58.83 clat percentiles (msec): | 1.00th=[ 63], 5.00th=[ 75], 10.00th=[ 87], 20.00th=[ 102], | 30.00th=[ 113], 40.00th=[ 125], 50.00th=[ 136], 60.00th=[ 148], | 70.00th=[ 163], 80.00th=[ 180], 90.00th=[ 211], 95.00th=[ 243], | 99.00th=[ 326], 99.50th=[ 372], 99.90th=[ 609], 99.95th=[ 1070], | 99.99th=[ 1070] bw ( KiB/s): min= 7, max= 47, per=24.69%, avg=22.72, stdev= 8.24, samples=529 iops : min= 1, max= 11, avg= 5.03, stdev= 2.11, samples=529 lat (msec) : 10=2.54%, 20=20.41%, 50=19.37%, 100=15.59%, 250=40.07% lat (msec) : 500=1.92%, 750=0.07%, 2000=0.03% cpu : usr=0.05%, sys=1.21%, ctx=5105, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153202: Fri Oct 10 16:22:30 2025 read: IOPS=5, BW=22.5KiB/s (23.0kB/s)(6052KiB/269412msec) clat (msec): min=5, max=217, avg=28.94, stdev=23.97 lat (msec): min=5, max=217, avg=28.94, stdev=23.97 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 19], 50.00th=[ 21], 60.00th=[ 24], | 70.00th=[ 29], 80.00th=[ 39], 90.00th=[ 58], 95.00th=[ 79], | 99.00th=[ 126], 99.50th=[ 146], 99.90th=[ 201], 99.95th=[ 218], | 99.99th=[ 218] bw ( KiB/s): min= 7, max= 87, per=28.28%, avg=24.88, stdev=16.33, samples=469 iops : min= 1, max= 21, avg= 5.57, stdev= 4.09, samples=469 write: IOPS=5, BW=23.1KiB/s (23.7kB/s)(6236KiB/269412msec); 0 zone resets clat (msec): min=52, max=1081, avg=144.48, stdev=57.89 lat (msec): min=52, max=1081, avg=144.48, stdev=57.89 clat percentiles (msec): | 1.00th=[ 65], 5.00th=[ 80], 10.00th=[ 89], 20.00th=[ 101], | 30.00th=[ 112], 40.00th=[ 124], 50.00th=[ 136], 60.00th=[ 148], | 70.00th=[ 163], 80.00th=[ 182], 90.00th=[ 213], 95.00th=[ 243], | 99.00th=[ 309], 99.50th=[ 321], 99.90th=[ 592], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 7, max= 48, per=24.55%, avg=22.58, stdev= 8.01, samples=530 iops : min= 1, max= 12, avg= 4.99, stdev= 2.05, samples=530 lat (msec) : 10=2.47%, 20=21.39%, 50=18.59%, 100=16.02%, 250=39.49% lat (msec) : 500=1.95%, 750=0.07%, 2000=0.03% cpu : usr=0.05%, sys=1.23%, ctx=5199, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=88.8KiB/s (90.9kB/s), 21.0KiB/s-22.5KiB/s (22.5kB/s-23.0kB/s), io=23.4MiB (24.6MB), run=269412-270382msec WRITE: bw=92.0KiB/s (95.2kB/s), 23.1KiB/s-23.5KiB/s (23.7kB/s-24.0kB/s), io=24.6MiB (25.7MB), run=269412-270382msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=153257: Fri Oct 10 16:23:00 2025 read: IOPS=19, BW=306KiB/s (313kB/s)(5920KiB/19341msec) clat (usec): min=101, max=92672, avg=24798.08, stdev=21803.32 lat (usec): min=101, max=92673, avg=24799.45, stdev=21803.26 clat percentiles (usec): | 1.00th=[ 113], 5.00th=[ 143], 10.00th=[ 159], 20.00th=[ 194], | 30.00th=[ 2040], 40.00th=[18482], 50.00th=[23725], 60.00th=[28967], | 70.00th=[35914], 80.00th=[42730], 90.00th=[55313], 95.00th=[62129], | 99.00th=[85459], 99.50th=[90702], 99.90th=[92799], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 159, max= 542, per=25.92%, avg=307.73, stdev=89.86, samples=37 iops : min= 9, max= 33, avg=18.59, stdev= 5.70, samples=37 write: IOPS=20, BW=329KiB/s (337kB/s)(6368KiB/19341msec); 0 zone resets clat (usec): min=1269, max=164759, avg=25159.44, stdev=26278.96 lat (usec): min=1270, max=164761, avg=25165.98, stdev=26276.11 clat percentiles (usec): | 1.00th=[ 1385], 5.00th=[ 1565], 10.00th=[ 1729], 20.00th=[ 3163], | 30.00th=[ 6128], 40.00th=[ 9896], 50.00th=[ 17433], 60.00th=[ 22414], | 70.00th=[ 31851], 80.00th=[ 44827], 90.00th=[ 63177], 95.00th=[ 79168], | 99.00th=[104334], 99.50th=[154141], 99.90th=[164627], 99.95th=[164627], | 99.99th=[164627] bw ( KiB/s): min= 127, max= 541, per=26.63%, avg=332.59, stdev=104.54, samples=37 iops : min= 7, max= 33, avg=20.16, stdev= 6.60, samples=37 lat (usec) : 250=11.98%, 500=1.69%, 1000=0.13% lat (msec) : 2=8.20%, 4=4.56%, 10=10.03%, 20=11.59%, 50=36.72% lat (msec) : 100=14.19%, 250=0.91% cpu : usr=0.11%, sys=9.49%, ctx=1277, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153258: Fri Oct 10 16:23:00 2025 read: IOPS=17, BW=285KiB/s (292kB/s)(5744KiB/20168msec) clat (usec): min=94, max=127071, avg=26988.99, stdev=24210.06 lat (usec): min=94, max=127072, avg=26990.44, stdev=24209.99 clat percentiles (usec): | 1.00th=[ 112], 5.00th=[ 139], 10.00th=[ 153], 20.00th=[ 186], | 30.00th=[ 766], 40.00th=[ 20579], 50.00th=[ 25560], 60.00th=[ 29754], | 70.00th=[ 36439], 80.00th=[ 46924], 90.00th=[ 61080], 95.00th=[ 68682], | 99.00th=[ 96994], 99.50th=[126354], 99.90th=[127402], 99.95th=[127402], | 99.99th=[127402] bw ( KiB/s): min= 59, max= 510, per=23.90%, avg=283.67, stdev=88.94, samples=39 iops : min= 3, max= 31, avg=17.08, stdev= 5.55, samples=39 write: IOPS=20, BW=324KiB/s (332kB/s)(6544KiB/20168msec); 0 zone resets clat (usec): min=1254, max=156939, avg=25091.94, stdev=26473.03 lat (usec): min=1255, max=156940, avg=25094.20, stdev=26473.01 clat percentiles (usec): | 1.00th=[ 1385], 5.00th=[ 1532], 10.00th=[ 1614], 20.00th=[ 2212], | 30.00th=[ 5080], 40.00th=[ 8979], 50.00th=[ 15139], 60.00th=[ 22676], | 70.00th=[ 32375], 80.00th=[ 46924], 90.00th=[ 63701], 95.00th=[ 73925], | 99.00th=[103285], 99.50th=[111674], 99.90th=[156238], 99.95th=[156238], | 99.99th=[156238] bw ( KiB/s): min= 95, max= 633, per=25.76%, avg=321.72, stdev=120.85, samples=39 iops : min= 5, max= 39, avg=19.49, stdev= 7.68, samples=39 lat (usec) : 100=0.13%, 250=11.85%, 500=1.30%, 750=0.65%, 1000=0.13% lat (msec) : 2=10.16%, 4=4.43%, 10=8.20%, 20=11.20%, 50=34.38% lat (msec) : 100=16.54%, 250=1.04% cpu : usr=0.16%, sys=9.24%, ctx=1324, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153259: Fri Oct 10 16:23:00 2025 read: IOPS=19, BW=313KiB/s (320kB/s)(6256KiB/20018msec) clat (usec): min=101, max=175382, avg=26263.32, stdev=27153.11 lat (usec): min=101, max=175383, avg=26278.77, stdev=27162.15 clat percentiles (usec): | 1.00th=[ 106], 5.00th=[ 141], 10.00th=[ 155], 20.00th=[ 192], | 30.00th=[ 343], 40.00th=[ 16188], 50.00th=[ 22938], 60.00th=[ 29754], | 70.00th=[ 36439], 80.00th=[ 46400], 90.00th=[ 59507], 95.00th=[ 68682], | 99.00th=[143655], 99.50th=[162530], 99.90th=[175113], 99.95th=[175113], | 99.99th=[175113] bw ( KiB/s): min= 92, max= 564, per=26.44%, avg=313.79, stdev=109.62, samples=39 iops : min= 5, max= 35, avg=18.95, stdev= 6.87, samples=39 write: IOPS=18, BW=301KiB/s (309kB/s)(6032KiB/20018msec); 0 zone resets clat (usec): min=1180, max=163830, avg=25664.25, stdev=28204.23 lat (usec): min=1182, max=163833, avg=25668.57, stdev=28205.24 clat percentiles (usec): | 1.00th=[ 1483], 5.00th=[ 1631], 10.00th=[ 1729], 20.00th=[ 2933], | 30.00th=[ 6128], 40.00th=[ 10159], 50.00th=[ 15008], 60.00th=[ 19530], | 70.00th=[ 30278], 80.00th=[ 46924], 90.00th=[ 67634], 95.00th=[ 81265], | 99.00th=[123208], 99.50th=[132645], 99.90th=[164627], 99.95th=[164627], | 99.99th=[164627] bw ( KiB/s): min= 63, max= 566, per=23.99%, avg=299.67, stdev=117.98, samples=39 iops : min= 3, max= 35, avg=18.05, stdev= 7.44, samples=39 lat (usec) : 250=13.41%, 500=2.86%, 750=0.13%, 1000=0.39% lat (msec) : 2=7.94%, 4=4.30%, 10=8.46%, 20=15.49%, 50=29.17% lat (msec) : 100=15.49%, 250=2.34% cpu : usr=0.14%, sys=8.65%, ctx=1328, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153260: Fri Oct 10 16:23:00 2025 read: IOPS=20, BW=323KiB/s (331kB/s)(6032KiB/18658msec) clat (usec): min=99, max=109508, avg=25916.20, stdev=23624.49 lat (usec): min=99, max=109510, avg=25917.51, stdev=23624.47 clat percentiles (usec): | 1.00th=[ 115], 5.00th=[ 141], 10.00th=[ 151], 20.00th=[ 190], | 30.00th=[ 388], 40.00th=[ 18482], 50.00th=[ 24773], 60.00th=[ 31065], | 70.00th=[ 36963], 80.00th=[ 45876], 90.00th=[ 56886], 95.00th=[ 68682], | 99.00th=[ 91751], 99.50th=[109577], 99.90th=[109577], 99.95th=[109577], | 99.99th=[109577] bw ( KiB/s): min= 147, max= 541, per=27.28%, avg=323.86, stdev=104.69, samples=36 iops : min= 9, max= 33, avg=19.64, stdev= 6.63, samples=36 write: IOPS=20, BW=335KiB/s (343kB/s)(6256KiB/18658msec); 0 zone resets clat (usec): min=1310, max=187067, avg=22427.77, stdev=26908.43 lat (usec): min=1312, max=187069, avg=22430.21, stdev=26908.32 clat percentiles (usec): | 1.00th=[ 1385], 5.00th=[ 1532], 10.00th=[ 1631], 20.00th=[ 1893], | 30.00th=[ 3752], 40.00th=[ 6849], 50.00th=[ 10421], 60.00th=[ 16909], | 70.00th=[ 27657], 80.00th=[ 41681], 90.00th=[ 58983], 95.00th=[ 81265], | 99.00th=[102237], 99.50th=[127402], 99.90th=[187696], 99.95th=[187696], | 99.99th=[187696] bw ( KiB/s): min= 156, max= 692, per=27.04%, avg=337.69, stdev=118.36, samples=36 iops : min= 9, max= 43, avg=20.50, stdev= 7.48, samples=36 lat (usec) : 100=0.13%, 250=11.72%, 500=3.26%, 750=0.13%, 1000=0.13% lat (msec) : 2=11.72%, 4=4.95%, 10=10.03%, 20=11.20%, 50=30.99% lat (msec) : 100=14.84%, 250=0.91% cpu : usr=0.12%, sys=9.33%, ctx=1289, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1188KiB/s (1216kB/s), 285KiB/s-323KiB/s (292kB/s-331kB/s), io=23.4MiB (24.5MB), run=18658-20168msec WRITE: bw=1250KiB/s (1279kB/s), 301KiB/s-335KiB/s (309kB/s-343kB/s), io=24.6MiB (25.8MB), run=18658-20168msec rand-rw: (groupid=0, jobs=1): err= 0: pid=153262: Fri Oct 10 16:23:39 2025 read: IOPS=6, BW=99.2KiB/s (102kB/s)(5920KiB/59648msec) clat (msec): min=5, max=165, avg=28.22, stdev=27.07 lat (msec): min=5, max=165, avg=28.22, stdev=27.07 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 19], 60.00th=[ 22], | 70.00th=[ 26], 80.00th=[ 37], 90.00th=[ 59], 95.00th=[ 75], | 99.00th=[ 155], 99.50th=[ 161], 99.90th=[ 167], 99.95th=[ 167], | 99.99th=[ 167] bw ( KiB/s): min= 31, max= 349, per=26.43%, avg=105.97, stdev=64.32, samples=110 iops : min= 1, max= 21, avg= 5.94, stdev= 4.05, samples=110 write: IOPS=6, BW=107KiB/s (109kB/s)(6368KiB/59648msec); 0 zone resets clat (msec): min=43, max=389, avg=123.37, stdev=45.57 lat (msec): min=43, max=389, avg=123.37, stdev=45.57 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 66], 10.00th=[ 77], 20.00th=[ 89], | 30.00th=[ 97], 40.00th=[ 105], 50.00th=[ 115], 60.00th=[ 127], | 70.00th=[ 140], 80.00th=[ 153], 90.00th=[ 178], 95.00th=[ 211], | 99.00th=[ 271], 99.50th=[ 305], 99.90th=[ 388], 99.95th=[ 388], | 99.99th=[ 388] bw ( KiB/s): min= 25, max= 223, per=25.25%, avg=106.54, stdev=40.29, samples=118 iops : min= 1, max= 13, avg= 5.95, stdev= 2.57, samples=118 lat (msec) : 10=4.04%, 20=21.74%, 50=15.76%, 100=22.53%, 250=35.03% lat (msec) : 500=0.91% cpu : usr=0.04%, sys=1.59%, ctx=1396, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153263: Fri Oct 10 16:23:39 2025 read: IOPS=6, BW=96.5KiB/s (98.8kB/s)(5744KiB/59550msec) clat (msec): min=5, max=134, avg=27.25, stdev=22.48 lat (msec): min=5, max=134, avg=27.25, stdev=22.48 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 19], 60.00th=[ 22], | 70.00th=[ 28], 80.00th=[ 41], 90.00th=[ 57], 95.00th=[ 79], | 99.00th=[ 115], 99.50th=[ 131], 99.90th=[ 136], 99.95th=[ 136], | 99.99th=[ 136] bw ( KiB/s): min= 25, max= 319, per=26.55%, avg=106.47, stdev=63.63, samples=105 iops : min= 1, max= 19, avg= 5.96, stdev= 4.02, samples=105 write: IOPS=6, BW=110KiB/s (113kB/s)(6544KiB/59550msec); 0 zone resets clat (msec): min=41, max=321, avg=121.40, stdev=45.96 lat (msec): min=41, max=321, avg=121.40, stdev=45.96 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 64], 10.00th=[ 69], 20.00th=[ 83], | 30.00th=[ 95], 40.00th=[ 105], 50.00th=[ 115], 60.00th=[ 126], | 70.00th=[ 138], 80.00th=[ 153], 90.00th=[ 182], 95.00th=[ 211], | 99.00th=[ 262], 99.50th=[ 288], 99.90th=[ 321], 99.95th=[ 321], | 99.99th=[ 321] bw ( KiB/s): min= 25, max= 222, per=25.84%, avg=109.04, stdev=38.91, samples=117 iops : min= 1, max= 13, avg= 6.11, stdev= 2.47, samples=117 lat (msec) : 10=4.56%, 20=22.14%, 50=13.41%, 100=24.74%, 250=34.24% lat (msec) : 500=0.91% cpu : usr=0.04%, sys=1.47%, ctx=1374, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153264: Fri Oct 10 16:23:39 2025 read: IOPS=6, BW=110KiB/s (112kB/s)(6256KiB/57080msec) clat (msec): min=6, max=152, avg=26.20, stdev=22.86 lat (msec): min=6, max=152, avg=26.21, stdev=22.87 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 10], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 19], 60.00th=[ 22], | 70.00th=[ 28], 80.00th=[ 35], 90.00th=[ 50], 95.00th=[ 63], | 99.00th=[ 144], 99.50th=[ 148], 99.90th=[ 153], 99.95th=[ 153], | 99.99th=[ 153] bw ( KiB/s): min= 31, max= 416, per=29.69%, avg=119.04, stdev=77.96, samples=104 iops : min= 1, max= 26, avg= 6.74, stdev= 4.97, samples=104 write: IOPS=6, BW=106KiB/s (108kB/s)(6032KiB/57080msec); 0 zone resets clat (msec): min=49, max=346, avg=124.02, stdev=50.12 lat (msec): min=49, max=346, avg=124.02, stdev=50.12 clat percentiles (msec): | 1.00th=[ 58], 5.00th=[ 67], 10.00th=[ 73], 20.00th=[ 84], | 30.00th=[ 94], 40.00th=[ 105], 50.00th=[ 113], 60.00th=[ 122], | 70.00th=[ 136], 80.00th=[ 157], 90.00th=[ 192], 95.00th=[ 228], | 99.00th=[ 292], 99.50th=[ 313], 99.90th=[ 347], 99.95th=[ 347], | 99.99th=[ 347] bw ( KiB/s): min= 31, max= 190, per=25.18%, avg=106.25, stdev=35.47, samples=112 iops : min= 1, max= 11, avg= 5.95, stdev= 2.27, samples=112 lat (msec) : 10=5.60%, 20=22.66%, 50=17.84%, 100=21.22%, 250=31.12% lat (msec) : 500=1.56% cpu : usr=0.08%, sys=1.61%, ctx=1424, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153265: Fri Oct 10 16:23:39 2025 read: IOPS=6, BW=103KiB/s (106kB/s)(6032KiB/58370msec) clat (msec): min=6, max=167, avg=27.12, stdev=22.63 lat (msec): min=6, max=167, avg=27.13, stdev=22.63 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 27], 80.00th=[ 37], 90.00th=[ 56], 95.00th=[ 77], | 99.00th=[ 115], 99.50th=[ 136], 99.90th=[ 169], 99.95th=[ 169], | 99.99th=[ 169] bw ( KiB/s): min= 31, max= 318, per=27.90%, avg=111.90, stdev=66.86, samples=105 iops : min= 1, max= 19, avg= 6.29, stdev= 4.26, samples=105 write: IOPS=6, BW=107KiB/s (110kB/s)(6256KiB/58370msec); 0 zone resets clat (msec): min=39, max=321, avg=122.90, stdev=46.12 lat (msec): min=39, max=321, avg=122.91, stdev=46.12 clat percentiles (msec): | 1.00th=[ 57], 5.00th=[ 69], 10.00th=[ 74], 20.00th=[ 85], | 30.00th=[ 94], 40.00th=[ 104], 50.00th=[ 114], 60.00th=[ 124], | 70.00th=[ 140], 80.00th=[ 157], 90.00th=[ 182], 95.00th=[ 213], | 99.00th=[ 268], 99.50th=[ 309], 99.90th=[ 321], 99.95th=[ 321], | 99.99th=[ 321] bw ( KiB/s): min= 31, max= 191, per=25.30%, avg=106.77, stdev=37.44, samples=115 iops : min= 1, max= 11, avg= 5.97, stdev= 2.41, samples=115 lat (msec) : 10=3.26%, 20=22.27%, 50=17.84%, 100=24.22%, 250=31.51% lat (msec) : 500=0.91% cpu : usr=0.04%, sys=1.63%, ctx=1397, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=402KiB/s (411kB/s), 96.5KiB/s-110KiB/s (98.8kB/s-112kB/s), io=23.4MiB (24.5MB), run=57080-59648msec WRITE: bw=422KiB/s (433kB/s), 106KiB/s-110KiB/s (108kB/s-113kB/s), io=24.6MiB (25.8MB), run=57080-59648msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=153279: Fri Oct 10 16:23:50 2025 read: IOPS=2, BW=2472KiB/s (2531kB/s)(7168KiB/2900msec) clat (msec): min=97, max=309, avg=203.23, stdev=93.83 lat (msec): min=97, max=309, avg=203.23, stdev=93.83 clat percentiles (msec): | 1.00th=[ 97], 5.00th=[ 97], 10.00th=[ 97], 20.00th=[ 104], | 30.00th=[ 118], 40.00th=[ 118], 50.00th=[ 234], 60.00th=[ 279], | 70.00th=[ 279], 80.00th=[ 284], 90.00th=[ 309], 95.00th=[ 309], | 99.00th=[ 309], 99.50th=[ 309], 99.90th=[ 309], 99.95th=[ 309], | 99.99th=[ 309] bw ( KiB/s): min= 2003, max= 4079, per=62.97%, avg=3007.50, stdev=1136.42, samples=4 iops : min= 1, max= 3, avg= 2.25, stdev= 0.96, samples=4 write: IOPS=1, BW=1766KiB/s (1808kB/s)(5120KiB/2900msec); 0 zone resets clat (msec): min=214, max=332, avg=286.41, stdev=50.76 lat (msec): min=215, max=332, avg=286.53, stdev=50.73 clat percentiles (msec): | 1.00th=[ 215], 5.00th=[ 215], 10.00th=[ 215], 20.00th=[ 215], | 30.00th=[ 253], 40.00th=[ 253], 50.00th=[ 309], 60.00th=[ 309], | 70.00th=[ 326], 80.00th=[ 326], 90.00th=[ 334], 95.00th=[ 334], | 99.00th=[ 334], 99.50th=[ 334], 99.90th=[ 334], 99.95th=[ 334], | 99.99th=[ 334] bw ( KiB/s): min= 2003, max= 4023, per=40.25%, avg=2691.33, stdev=1153.48, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 lat (msec) : 100=8.33%, 250=33.33%, 500=58.33% cpu : usr=0.03%, sys=8.90%, ctx=202, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153280: Fri Oct 10 16:23:50 2025 read: IOPS=0, BW=478KiB/s (489kB/s)(2048KiB/4288msec) clat (msec): min=109, max=321, avg=215.99, stdev=149.89 lat (msec): min=109, max=321, avg=215.99, stdev=149.89 clat percentiles (msec): | 1.00th=[ 110], 5.00th=[ 110], 10.00th=[ 110], 20.00th=[ 110], | 30.00th=[ 110], 40.00th=[ 110], 50.00th=[ 110], 60.00th=[ 321], | 70.00th=[ 321], 80.00th=[ 321], 90.00th=[ 321], 95.00th=[ 321], | 99.00th=[ 321], 99.50th=[ 321], 99.90th=[ 321], 99.95th=[ 321], | 99.99th=[ 321] bw ( KiB/s): min= 1932, max= 2003, per=41.20%, avg=1967.50, stdev=50.20, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2388KiB/s (2445kB/s)(10.0MiB/4288msec); 0 zone resets clat (msec): min=104, max=796, avg=369.74, stdev=221.56 lat (msec): min=104, max=796, avg=369.96, stdev=221.53 clat percentiles (msec): | 1.00th=[ 105], 5.00th=[ 105], 10.00th=[ 105], 20.00th=[ 148], | 30.00th=[ 192], 40.00th=[ 201], 50.00th=[ 380], 60.00th=[ 384], | 70.00th=[ 422], 80.00th=[ 426], 90.00th=[ 642], 95.00th=[ 793], | 99.00th=[ 793], 99.50th=[ 793], 99.90th=[ 793], 99.95th=[ 793], | 99.99th=[ 793] bw ( KiB/s): min= 2003, max= 6144, per=54.94%, avg=3673.00, stdev=2249.84, samples=5 iops : min= 1, max= 6, avg= 3.00, stdev= 2.35, samples=5 lat (msec) : 250=41.67%, 500=41.67%, 750=8.33%, 1000=8.33% cpu : usr=0.05%, sys=6.34%, ctx=96, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153282: Fri Oct 10 16:23:50 2025 read: IOPS=1, BW=1585KiB/s (1623kB/s)(6144KiB/3877msec) clat (msec): min=83, max=319, avg=169.86, stdev=86.54 lat (msec): min=83, max=319, avg=169.86, stdev=86.54 clat percentiles (msec): | 1.00th=[ 84], 5.00th=[ 84], 10.00th=[ 84], 20.00th=[ 97], | 30.00th=[ 97], 40.00th=[ 140], 50.00th=[ 140], 60.00th=[ 178], | 70.00th=[ 205], 80.00th=[ 205], 90.00th=[ 321], 95.00th=[ 321], | 99.00th=[ 321], 99.50th=[ 321], 99.90th=[ 321], 99.95th=[ 321], | 99.99th=[ 321] bw ( KiB/s): min= 2048, max= 4087, per=70.90%, avg=3386.00, stdev=1159.18, samples=3 iops : min= 2, max= 3, avg= 2.67, stdev= 0.58, samples=3 write: IOPS=1, BW=1585KiB/s (1623kB/s)(6144KiB/3877msec); 0 zone resets clat (msec): min=124, max=1043, avg=448.74, stdev=346.48 lat (msec): min=126, max=1043, avg=449.10, stdev=346.17 clat percentiles (msec): | 1.00th=[ 125], 5.00th=[ 125], 10.00th=[ 125], 20.00th=[ 174], | 30.00th=[ 174], 40.00th=[ 249], 50.00th=[ 249], 60.00th=[ 510], | 70.00th=[ 592], 80.00th=[ 592], 90.00th=[ 1045], 95.00th=[ 1045], | 99.00th=[ 1045], 99.50th=[ 1045], 99.90th=[ 1045], 99.95th=[ 1045], | 99.99th=[ 1045] bw ( KiB/s): min= 2039, max= 4087, per=50.21%, avg=3357.33, stdev=1143.88, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 lat (msec) : 100=16.67%, 250=50.00%, 500=8.33%, 750=16.67%, 2000=8.33% cpu : usr=0.00%, sys=5.68%, ctx=70, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153284: Fri Oct 10 16:23:50 2025 read: IOPS=1, BW=1253KiB/s (1283kB/s)(5120KiB/4087msec) clat (msec): min=54, max=304, avg=188.35, stdev=115.70 lat (msec): min=54, max=304, avg=188.35, stdev=115.70 clat percentiles (msec): | 1.00th=[ 55], 5.00th=[ 55], 10.00th=[ 55], 20.00th=[ 55], | 30.00th=[ 85], 40.00th=[ 85], 50.00th=[ 205], 60.00th=[ 205], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 305], 95.00th=[ 305], | 99.00th=[ 305], 99.50th=[ 305], 99.90th=[ 305], 99.95th=[ 305], | 99.99th=[ 305] bw ( KiB/s): min= 4087, max= 4096, per=85.67%, avg=4091.50, stdev= 6.36, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1754KiB/s (1796kB/s)(7168KiB/4087msec); 0 zone resets clat (msec): min=125, max=1019, avg=424.46, stdev=313.60 lat (msec): min=125, max=1019, avg=424.57, stdev=313.60 clat percentiles (msec): | 1.00th=[ 126], 5.00th=[ 126], 10.00th=[ 126], 20.00th=[ 144], | 30.00th=[ 209], 40.00th=[ 209], 50.00th=[ 422], 60.00th=[ 523], | 70.00th=[ 523], 80.00th=[ 531], 90.00th=[ 1020], 95.00th=[ 1020], | 99.00th=[ 1020], 99.50th=[ 1020], 99.90th=[ 1020], 99.95th=[ 1020], | 99.99th=[ 1020] bw ( KiB/s): min= 1372, max= 2043, per=28.32%, avg=1893.80, stdev=292.32, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (msec) : 100=16.67%, 250=33.33%, 500=25.00%, 750=16.67%, 2000=8.33% cpu : usr=0.05%, sys=6.39%, ctx=87, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4776KiB/s (4891kB/s), 478KiB/s-2472KiB/s (489kB/s-2531kB/s), io=20.0MiB (20.0MB), run=2900-4288msec WRITE: bw=6687KiB/s (6847kB/s), 1585KiB/s-2388KiB/s (1623kB/s-2445kB/s), io=28.0MiB (29.4MB), run=2900-4288msec rand-rw: (groupid=0, jobs=1): err= 0: pid=153281: Fri Oct 10 16:23:50 2025 read: IOPS=2, BW=2364KiB/s (2421kB/s)(7168KiB/3032msec) clat (usec): min=634, max=646200, avg=199485.72, stdev=218410.32 lat (usec): min=635, max=646201, avg=199487.18, stdev=218410.39 clat percentiles (usec): | 1.00th=[ 635], 5.00th=[ 635], 10.00th=[ 635], 20.00th=[ 709], | 30.00th=[120062], 40.00th=[120062], 50.00th=[183501], 60.00th=[217056], | 70.00th=[217056], 80.00th=[227541], 90.00th=[650118], 95.00th=[650118], | 99.00th=[650118], 99.50th=[650118], 99.90th=[650118], 99.95th=[650118], | 99.99th=[650118] bw ( KiB/s): min= 1387, max= 3886, per=49.52%, avg=2341.00, stdev=1075.76, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 write: IOPS=1, BW=1689KiB/s (1729kB/s)(5120KiB/3032msec); 0 zone resets clat (msec): min=198, max=412, avg=284.32, stdev=98.28 lat (msec): min=198, max=412, avg=284.41, stdev=98.28 clat percentiles (msec): | 1.00th=[ 199], 5.00th=[ 199], 10.00th=[ 199], 20.00th=[ 199], | 30.00th=[ 207], 40.00th=[ 207], 50.00th=[ 241], 60.00th=[ 241], | 70.00th=[ 368], 80.00th=[ 368], 90.00th=[ 414], 95.00th=[ 414], | 99.00th=[ 414], 99.50th=[ 414], 99.90th=[ 414], 99.95th=[ 414], | 99.99th=[ 414] bw ( KiB/s): min= 1943, max= 3976, per=40.13%, avg=2655.67, stdev=1144.65, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 lat (usec) : 750=16.67% lat (msec) : 250=58.33%, 500=16.67%, 750=8.33% cpu : usr=0.07%, sys=17.22%, ctx=248, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153283: Fri Oct 10 16:23:50 2025 read: IOPS=0, BW=473KiB/s (484kB/s)(2048KiB/4332msec) clat (msec): min=191, max=266, avg=229.20, stdev=53.08 lat (msec): min=191, max=266, avg=229.20, stdev=53.08 clat percentiles (msec): | 1.00th=[ 192], 5.00th=[ 192], 10.00th=[ 192], 20.00th=[ 192], | 30.00th=[ 192], 40.00th=[ 192], 50.00th=[ 192], 60.00th=[ 268], | 70.00th=[ 268], 80.00th=[ 268], 90.00th=[ 268], 95.00th=[ 268], | 99.00th=[ 268], 99.50th=[ 268], 99.90th=[ 268], 99.95th=[ 268], | 99.99th=[ 268] bw ( KiB/s): min= 1416, max= 1988, per=36.01%, avg=1702.00, stdev=404.47, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2364KiB/s (2421kB/s)(10.0MiB/4332msec); 0 zone resets clat (msec): min=68, max=975, avg=350.61, stdev=300.27 lat (msec): min=68, max=975, avg=350.72, stdev=300.25 clat percentiles (msec): | 1.00th=[ 69], 5.00th=[ 69], 10.00th=[ 69], 20.00th=[ 136], | 30.00th=[ 140], 40.00th=[ 161], 50.00th=[ 180], 60.00th=[ 203], | 70.00th=[ 388], 80.00th=[ 575], 90.00th=[ 684], 95.00th=[ 978], | 99.00th=[ 978], 99.50th=[ 978], 99.90th=[ 978], 99.95th=[ 978], | 99.99th=[ 978] bw ( KiB/s): min= 1943, max= 6131, per=54.98%, avg=3638.60, stdev=2253.70, samples=5 iops : min= 1, max= 5, avg= 2.80, stdev= 2.05, samples=5 lat (msec) : 100=8.33%, 250=50.00%, 500=16.67%, 750=16.67%, 1000=8.33% cpu : usr=0.00%, sys=17.29%, ctx=165, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153285: Fri Oct 10 16:23:50 2025 read: IOPS=1, BW=1560KiB/s (1597kB/s)(6144KiB/3939msec) clat (msec): min=44, max=307, avg=146.43, stdev=95.65 lat (msec): min=44, max=307, avg=146.43, stdev=95.65 clat percentiles (msec): | 1.00th=[ 45], 5.00th=[ 45], 10.00th=[ 45], 20.00th=[ 57], | 30.00th=[ 57], 40.00th=[ 131], 50.00th=[ 131], 60.00th=[ 171], | 70.00th=[ 171], 80.00th=[ 171], 90.00th=[ 309], 95.00th=[ 309], | 99.00th=[ 309], 99.50th=[ 309], 99.90th=[ 309], 99.95th=[ 309], | 99.99th=[ 309] bw ( KiB/s): min= 2048, max= 4096, per=71.26%, avg=3368.33, stdev=1145.43, samples=3 iops : min= 2, max= 4, avg= 3.00, stdev= 1.00, samples=3 write: IOPS=1, BW=1560KiB/s (1597kB/s)(6144KiB/3939msec); 0 zone resets clat (msec): min=122, max=1127, avg=467.61, stdev=387.37 lat (msec): min=122, max=1127, avg=467.70, stdev=387.35 clat percentiles (msec): | 1.00th=[ 123], 5.00th=[ 123], 10.00th=[ 123], 20.00th=[ 174], | 30.00th=[ 174], 40.00th=[ 209], 50.00th=[ 209], 60.00th=[ 506], | 70.00th=[ 667], 80.00th=[ 667], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 2048, max= 3961, per=44.55%, avg=2948.33, stdev=961.43, samples=3 iops : min= 2, max= 3, avg= 2.33, stdev= 0.58, samples=3 lat (msec) : 50=8.33%, 100=8.33%, 250=50.00%, 500=8.33%, 750=16.67% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=15.24%, ctx=263, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153286: Fri Oct 10 16:23:51 2025 read: IOPS=1, BW=1813KiB/s (1857kB/s)(5120KiB/2824msec) clat (usec): min=834, max=244441, avg=163456.52, stdev=96472.88 lat (usec): min=836, max=244442, avg=163457.78, stdev=96472.84 clat percentiles (usec): | 1.00th=[ 832], 5.00th=[ 832], 10.00th=[ 832], 20.00th=[ 832], | 30.00th=[168821], 40.00th=[168821], 50.00th=[177210], 60.00th=[177210], | 70.00th=[227541], 80.00th=[227541], 90.00th=[244319], 95.00th=[244319], | 99.00th=[244319], 99.50th=[244319], 99.90th=[244319], 99.95th=[244319], | 99.99th=[244319] bw ( KiB/s): min= 2048, max= 5829, per=83.32%, avg=3938.50, stdev=2673.57, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 write: IOPS=2, BW=2538KiB/s (2599kB/s)(7168KiB/2824msec); 0 zone resets clat (msec): min=175, max=435, avg=273.04, stdev=95.63 lat (msec): min=175, max=435, avg=274.07, stdev=96.55 clat percentiles (msec): | 1.00th=[ 176], 5.00th=[ 176], 10.00th=[ 176], 20.00th=[ 190], | 30.00th=[ 201], 40.00th=[ 201], 50.00th=[ 262], 60.00th=[ 300], | 70.00th=[ 300], 80.00th=[ 351], 90.00th=[ 435], 95.00th=[ 435], | 99.00th=[ 435], 99.50th=[ 435], 99.90th=[ 435], 99.95th=[ 435], | 99.99th=[ 435] bw ( KiB/s): min= 1943, max= 3976, per=38.64%, avg=2557.00, stdev=860.43, samples=5 iops : min= 1, max= 3, avg= 1.80, stdev= 0.84, samples=5 lat (usec) : 1000=8.33% lat (msec) : 250=58.33%, 500=33.33% cpu : usr=0.00%, sys=27.45%, ctx=345, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4728KiB/s (4841kB/s), 473KiB/s-2364KiB/s (484kB/s-2421kB/s), io=20.0MiB (20.0MB), run=2824-4332msec WRITE: bw=6619KiB/s (6777kB/s), 1560KiB/s-2538KiB/s (1597kB/s-2599kB/s), io=28.0MiB (29.4MB), run=2824-4332msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=153294: Fri Oct 10 16:24:02 2025 read: IOPS=0, BW=3733KiB/s (3822kB/s)(12.0MiB/3292msec) clat (msec): min=182, max=1647, avg=1050.92, stdev=769.12 lat (msec): min=182, max=1647, avg=1050.92, stdev=769.12 clat percentiles (msec): | 1.00th=[ 184], 5.00th=[ 184], 10.00th=[ 184], 20.00th=[ 184], | 30.00th=[ 184], 40.00th=[ 1318], 50.00th=[ 1318], 60.00th=[ 1318], | 70.00th=[ 1653], 80.00th=[ 1653], 90.00th=[ 1653], 95.00th=[ 1653], | 99.00th=[ 1653], 99.50th=[ 1653], 99.90th=[ 1653], 99.95th=[ 1653], | 99.99th=[ 1653] bw ( KiB/s): min= 8143, max= 8143, per=100.00%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=66.67% cpu : usr=0.00%, sys=3.22%, ctx=43, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153296: Fri Oct 10 16:24:02 2025 read: IOPS=0, BW=1968KiB/s (2016kB/s)(4096KiB/2081msec) clat (nsec): min=383298k, max=383298k, avg=383297538.00, stdev= 0.00 lat (nsec): min=383301k, max=383301k, avg=383300513.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 384], 5.00th=[ 384], 10.00th=[ 384], 20.00th=[ 384], | 30.00th=[ 384], 40.00th=[ 384], 50.00th=[ 384], 60.00th=[ 384], | 70.00th=[ 384], 80.00th=[ 384], 90.00th=[ 384], 95.00th=[ 384], | 99.00th=[ 384], 99.50th=[ 384], 99.90th=[ 384], 99.95th=[ 384], | 99.99th=[ 384] bw ( KiB/s): min= 6501, max= 6501, per=100.00%, avg=6501.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3937KiB/s (4031kB/s)(8192KiB/2081msec); 0 zone resets clat (msec): min=429, max=1096, avg=762.60, stdev=471.61 lat (msec): min=429, max=1096, avg=763.14, stdev=471.20 clat percentiles (msec): | 1.00th=[ 430], 5.00th=[ 430], 10.00th=[ 430], 20.00th=[ 430], | 30.00th=[ 430], 40.00th=[ 430], 50.00th=[ 430], 60.00th=[ 1099], | 70.00th=[ 1099], 80.00th=[ 1099], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 7876, max= 7876, per=74.87%, avg=7876.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=9.62%, ctx=81, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153298: Fri Oct 10 16:24:02 2025 write: IOPS=0, BW=3945KiB/s (4039kB/s)(12.0MiB/3115msec); 0 zone resets clat (msec): min=479, max=1245, avg=979.40, stdev=433.11 lat (msec): min=480, max=1245, avg=979.88, stdev=433.10 clat percentiles (msec): | 1.00th=[ 481], 5.00th=[ 481], 10.00th=[ 481], 20.00th=[ 481], | 30.00th=[ 481], 40.00th=[ 1217], 50.00th=[ 1217], 60.00th=[ 1217], | 70.00th=[ 1250], 80.00th=[ 1250], 90.00th=[ 1250], 95.00th=[ 1250], | 99.00th=[ 1250], 99.50th=[ 1250], 99.90th=[ 1250], 99.95th=[ 1250], | 99.99th=[ 1250] bw ( KiB/s): min= 8143, max= 8143, per=77.41%, avg=8143.00, stdev= 0.00, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.00%, sys=5.84%, ctx=46, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153300: Fri Oct 10 16:24:02 2025 write: IOPS=1, BW=4732KiB/s (4845kB/s)(12.0MiB/2597msec); 0 zone resets clat (msec): min=237, max=1594, avg=792.67, stdev=711.52 lat (msec): min=237, max=1595, avg=793.13, stdev=711.47 clat percentiles (msec): | 1.00th=[ 239], 5.00th=[ 239], 10.00th=[ 239], 20.00th=[ 239], | 30.00th=[ 239], 40.00th=[ 550], 50.00th=[ 550], 60.00th=[ 550], | 70.00th=[ 1603], 80.00th=[ 1603], 90.00th=[ 1603], 95.00th=[ 1603], | 99.00th=[ 1603], 99.50th=[ 1603], 99.90th=[ 1603], 99.95th=[ 1603], | 99.99th=[ 1603] bw ( KiB/s): min= 7876, max= 7876, per=74.87%, avg=7876.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.04%, sys=6.28%, ctx=49, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4977KiB/s (5096kB/s), 1968KiB/s-3733KiB/s (2016kB/s-3822kB/s), io=16.0MiB (16.8MB), run=2081-3292msec WRITE: bw=10.3MiB/s (10.8MB/s), 3937KiB/s-4732KiB/s (4031kB/s-4845kB/s), io=32.0MiB (33.6MB), run=2081-3115msec rand-rw: (groupid=0, jobs=1): err= 0: pid=153295: Fri Oct 10 16:24:03 2025 read: IOPS=1, BW=6086KiB/s (6232kB/s)(12.0MiB/2019msec) clat (msec): min=483, max=787, avg=631.89, stdev=151.84 lat (msec): min=483, max=787, avg=631.89, stdev=151.84 clat percentiles (msec): | 1.00th=[ 485], 5.00th=[ 485], 10.00th=[ 485], 20.00th=[ 485], | 30.00th=[ 485], 40.00th=[ 625], 50.00th=[ 625], 60.00th=[ 625], | 70.00th=[ 785], 80.00th=[ 785], 90.00th=[ 785], 95.00th=[ 785], | 99.00th=[ 785], 99.50th=[ 785], 99.90th=[ 785], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 8126, max= 8143, per=100.00%, avg=8134.50, stdev=12.02, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=25.62%, ctx=330, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153297: Fri Oct 10 16:24:03 2025 read: IOPS=0, BW=1426KiB/s (1460kB/s)(4096KiB/2872msec) clat (nsec): min=733085k, max=733085k, avg=733084865.00, stdev= 0.00 lat (nsec): min=733088k, max=733088k, avg=733087502.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 735], 5.00th=[ 735], 10.00th=[ 735], 20.00th=[ 735], | 30.00th=[ 735], 40.00th=[ 735], 50.00th=[ 735], 60.00th=[ 735], | 70.00th=[ 735], 80.00th=[ 735], 90.00th=[ 735], 95.00th=[ 735], | 99.00th=[ 735], 99.50th=[ 735], 99.90th=[ 735], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 8126, max= 8126, per=100.00%, avg=8126.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2852KiB/s (2921kB/s)(8192KiB/2872msec); 0 zone resets clat (msec): min=472, max=1432, avg=952.04, stdev=678.85 lat (msec): min=472, max=1433, avg=952.90, stdev=679.57 clat percentiles (msec): | 1.00th=[ 472], 5.00th=[ 472], 10.00th=[ 472], 20.00th=[ 472], | 30.00th=[ 472], 40.00th=[ 472], 50.00th=[ 472], 60.00th=[ 1435], | 70.00th=[ 1435], 80.00th=[ 1435], 90.00th=[ 1435], 95.00th=[ 1435], | 99.00th=[ 1435], 99.50th=[ 1435], 99.90th=[ 1435], 99.95th=[ 1435], | 99.99th=[ 1435] bw ( KiB/s): min= 8143, max= 8143, per=89.46%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=5.02%, ctx=124, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153299: Fri Oct 10 16:24:03 2025 write: IOPS=0, BW=3413KiB/s (3495kB/s)(12.0MiB/3600msec); 0 zone resets clat (msec): min=214, max=1731, avg=1155.81, stdev=822.03 lat (msec): min=214, max=1732, avg=1156.27, stdev=822.22 clat percentiles (msec): | 1.00th=[ 215], 5.00th=[ 215], 10.00th=[ 215], 20.00th=[ 215], | 30.00th=[ 215], 40.00th=[ 1519], 50.00th=[ 1519], 60.00th=[ 1519], | 70.00th=[ 1737], 80.00th=[ 1737], 90.00th=[ 1737], 95.00th=[ 1737], | 99.00th=[ 1737], 99.50th=[ 1737], 99.90th=[ 1737], 99.95th=[ 1737], | 99.99th=[ 1737] bw ( KiB/s): min= 8143, max= 8175, per=89.64%, avg=8159.00, stdev=22.63, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 2000=66.67% cpu : usr=0.00%, sys=5.36%, ctx=67, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153301: Fri Oct 10 16:24:03 2025 write: IOPS=1, BW=5951KiB/s (6093kB/s)(12.0MiB/2065msec); 0 zone resets clat (msec): min=336, max=885, avg=621.69, stdev=275.10 lat (msec): min=337, max=885, avg=622.00, stdev=275.04 clat percentiles (msec): | 1.00th=[ 338], 5.00th=[ 338], 10.00th=[ 338], 20.00th=[ 338], | 30.00th=[ 338], 40.00th=[ 642], 50.00th=[ 642], 60.00th=[ 642], | 70.00th=[ 885], 80.00th=[ 885], 90.00th=[ 885], 95.00th=[ 885], | 99.00th=[ 885], 99.50th=[ 885], 99.90th=[ 885], 99.95th=[ 885], | 99.99th=[ 885] bw ( KiB/s): min= 6340, max= 8143, per=79.56%, avg=7241.50, stdev=1274.91, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=10.76%, ctx=78, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5705KiB/s (5842kB/s), 1426KiB/s-6086KiB/s (1460kB/s-6232kB/s), io=16.0MiB (16.8MB), run=2019-2872msec WRITE: bw=9102KiB/s (9321kB/s), 2852KiB/s-5951KiB/s (2921kB/s-6093kB/s), io=32.0MiB (33.6MB), run=2065-3600msec