== sanity test 398b: DIO and buffer IO race ============== 17:31:17 (1764109877) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.99769 s, 16.8 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=190712: Tue Nov 25 17:32:20 2025 read: IOPS=28, BW=115KiB/s (118kB/s)(5992KiB/52063msec) clat (usec): min=82, max=723058, avg=19127.65, stdev=21062.99 lat (usec): min=84, max=723059, avg=19128.78, stdev=21063.05 clat percentiles (usec): | 1.00th=[ 281], 5.00th=[ 7308], 10.00th=[ 8586], 20.00th=[ 10421], | 30.00th=[ 11994], 40.00th=[ 14091], 50.00th=[ 16581], 60.00th=[ 19530], | 70.00th=[ 22152], 80.00th=[ 25560], 90.00th=[ 31327], 95.00th=[ 38011], | 99.00th=[ 57410], 99.50th=[ 63177], 99.90th=[ 89654], 99.95th=[725615], | 99.99th=[725615] bw ( KiB/s): min= 7, max= 232, per=24.96%, avg=114.57, stdev=28.54, samples=103 iops : min= 1, max= 58, avg=28.25, stdev= 7.15, samples=103 write: IOPS=30, BW=121KiB/s (124kB/s)(6296KiB/52063msec); 0 zone resets clat (usec): min=702, max=73406, avg=14727.47, stdev=11924.55 lat (usec): min=702, max=73407, avg=14728.69, stdev=11924.57 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 1876], 10.00th=[ 2474], 20.00th=[ 3195], | 30.00th=[ 4948], 40.00th=[ 9765], 50.00th=[12387], 60.00th=[15401], | 70.00th=[19006], 80.00th=[24511], 90.00th=[31851], 95.00th=[36963], | 99.00th=[50070], 99.50th=[58459], 99.90th=[68682], 99.95th=[73925], | 99.99th=[73925] bw ( KiB/s): min= 48, max= 240, per=25.33%, avg=121.58, stdev=33.31, samples=102 iops : min= 12, max= 60, avg=30.01, stdev= 8.33, samples=102 lat (usec) : 100=0.03%, 250=0.26%, 500=0.81%, 750=0.16%, 1000=0.29% lat (msec) : 2=2.70%, 4=10.45%, 10=14.75%, 20=37.76%, 50=31.32% lat (msec) : 100=1.43%, 750=0.03% cpu : usr=0.12%, sys=20.08%, ctx=7956, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190714: Tue Nov 25 17:32:20 2025 read: IOPS=28, BW=114KiB/s (116kB/s)(5940KiB/52287msec) clat (usec): min=106, max=696886, avg=19017.19, stdev=20156.84 lat (usec): min=107, max=696887, avg=19018.41, stdev=20156.91 clat percentiles (usec): | 1.00th=[ 326], 5.00th=[ 7504], 10.00th=[ 8848], 20.00th=[ 10683], | 30.00th=[ 12649], 40.00th=[ 14746], 50.00th=[ 16909], 60.00th=[ 19530], | 70.00th=[ 21890], 80.00th=[ 25297], 90.00th=[ 30278], 95.00th=[ 35914], | 99.00th=[ 53216], 99.50th=[ 56886], 99.90th=[ 89654], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 16, max= 199, per=24.95%, avg=114.50, stdev=29.61, samples=103 iops : min= 4, max= 49, avg=28.24, stdev= 7.40, samples=103 write: IOPS=30, BW=121KiB/s (124kB/s)(6348KiB/52287msec); 0 zone resets clat (usec): min=742, max=79865, avg=14999.63, stdev=12834.33 lat (usec): min=743, max=79866, avg=15000.87, stdev=12834.34 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 1614], 10.00th=[ 2311], 20.00th=[ 2999], | 30.00th=[ 4883], 40.00th=[ 9372], 50.00th=[12256], 60.00th=[15139], | 70.00th=[19268], 80.00th=[24773], 90.00th=[32637], 95.00th=[39584], | 99.00th=[53740], 99.50th=[68682], 99.90th=[77071], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 40, max= 248, per=25.42%, avg=122.03, stdev=35.58, samples=103 iops : min= 10, max= 62, avg=30.13, stdev= 8.92, samples=103 lat (usec) : 250=0.16%, 500=0.75%, 750=0.23%, 1000=0.26% lat (msec) : 2=3.35%, 4=10.22%, 10=14.49%, 20=37.79%, 50=31.32% lat (msec) : 100=1.40%, 750=0.03% cpu : usr=0.13%, sys=20.05%, ctx=8007, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190715: Tue Nov 25 17:32:20 2025 read: IOPS=30, BW=121KiB/s (124kB/s)(6028KiB/49860msec) clat (usec): min=124, max=86442, avg=17831.59, stdev=9969.57 lat (usec): min=125, max=86444, avg=17834.16, stdev=9968.90 clat percentiles (usec): | 1.00th=[ 269], 5.00th=[ 6783], 10.00th=[ 8586], 20.00th=[10421], | 30.00th=[12125], 40.00th=[13698], 50.00th=[16057], 60.00th=[18220], | 70.00th=[20841], 80.00th=[24773], 90.00th=[30016], 95.00th=[35914], | 99.00th=[49021], 99.50th=[59507], 99.90th=[76022], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 56, max= 191, per=26.55%, avg=121.86, stdev=28.01, samples=98 iops : min= 14, max= 47, avg=30.08, stdev= 6.98, samples=98 write: IOPS=31, BW=126KiB/s (129kB/s)(6260KiB/49860msec); 0 zone resets clat (usec): min=725, max=716034, avg=14532.27, stdev=21424.35 lat (usec): min=727, max=716036, avg=14534.05, stdev=21424.09 clat percentiles (usec): | 1.00th=[ 1106], 5.00th=[ 1745], 10.00th=[ 2343], 20.00th=[ 3032], | 30.00th=[ 4686], 40.00th=[ 8979], 50.00th=[ 11469], 60.00th=[ 14091], | 70.00th=[ 17695], 80.00th=[ 23725], 90.00th=[ 30540], 95.00th=[ 36963], | 99.00th=[ 50594], 99.50th=[ 66323], 99.90th=[100140], 99.95th=[717226], | 99.99th=[717226] bw ( KiB/s): min= 14, max= 215, per=26.01%, avg=124.86, stdev=34.45, samples=99 iops : min= 3, max= 53, avg=30.84, stdev= 8.56, samples=99 lat (usec) : 250=0.36%, 500=1.43%, 750=0.20%, 1000=0.10% lat (msec) : 2=3.39%, 4=10.22%, 10=15.49%, 20=39.71%, 50=28.09% lat (msec) : 100=0.94%, 250=0.03%, 750=0.03% cpu : usr=0.12%, sys=20.29%, ctx=7767, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190717: Tue Nov 25 17:32:20 2025 read: IOPS=29, BW=118KiB/s (120kB/s)(6052KiB/51471msec) clat (usec): min=129, max=81335, avg=18275.92, stdev=10197.33 lat (usec): min=129, max=81337, avg=18277.79, stdev=10198.74 clat percentiles (usec): | 1.00th=[ 239], 5.00th=[ 7177], 10.00th=[ 8717], 20.00th=[10552], | 30.00th=[11994], 40.00th=[14091], 50.00th=[16450], 60.00th=[18744], | 70.00th=[21365], 80.00th=[25035], 90.00th=[30802], 95.00th=[36963], | 99.00th=[53216], 99.50th=[62653], 99.90th=[70779], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 56, max= 175, per=25.79%, avg=118.36, stdev=25.73, samples=101 iops : min= 14, max= 43, avg=29.20, stdev= 6.44, samples=101 write: IOPS=30, BW=121KiB/s (124kB/s)(6236KiB/51471msec); 0 zone resets clat (usec): min=827, max=693434, avg=15132.21, stdev=21485.06 lat (usec): min=829, max=693436, avg=15133.52, stdev=21485.11 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 1745], 10.00th=[ 2245], 20.00th=[ 2999], | 30.00th=[ 4752], 40.00th=[ 9241], 50.00th=[ 11863], 60.00th=[ 14484], | 70.00th=[ 18482], 80.00th=[ 24511], 90.00th=[ 32375], 95.00th=[ 39584], | 99.00th=[ 56361], 99.50th=[ 70779], 99.90th=[ 83362], 99.95th=[692061], | 99.99th=[692061] bw ( KiB/s): min= 21, max= 255, per=25.16%, avg=120.76, stdev=33.34, samples=102 iops : min= 5, max= 63, avg=29.80, stdev= 8.32, samples=102 lat (usec) : 250=0.55%, 500=0.81%, 750=0.13%, 1000=0.26% lat (msec) : 2=3.39%, 4=10.29%, 10=14.49%, 20=39.16%, 50=29.20% lat (msec) : 100=1.69%, 750=0.03% cpu : usr=0.10%, sys=19.98%, ctx=7879, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=459KiB/s (470kB/s), 114KiB/s-121KiB/s (116kB/s-124kB/s), io=23.4MiB (24.6MB), run=49860-52287msec WRITE: bw=481KiB/s (492kB/s), 121KiB/s-126KiB/s (124kB/s-129kB/s), io=24.6MiB (25.7MB), run=49860-52287msec rand-rw: (groupid=0, jobs=1): err= 0: pid=190710: Tue Nov 25 17:32:32 2025 read: IOPS=22, BW=91.0KiB/s (94.2kB/s)(5992KiB/65150msec) clat (msec): min=3, max=721, avg=17.76, stdev=22.97 lat (msec): min=3, max=721, avg=17.76, stdev=22.97 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 15], | 70.00th=[ 18], 80.00th=[ 25], 90.00th=[ 38], 95.00th=[ 47], | 99.00th=[ 71], 99.50th=[ 82], 99.90th=[ 96], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 24, max= 288, per=25.00%, avg=92.01, stdev=53.30, samples=128 iops : min= 6, max= 72, avg=22.68, stdev=13.36, samples=128 write: IOPS=24, BW=96.6KiB/s (98.0kB/s)(6296KiB/65150msec); 0 zone resets clat (msec): min=5, max=107, avg=24.35, stdev=15.78 lat (msec): min=5, max=107, avg=24.35, stdev=15.78 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 29], 80.00th=[ 36], 90.00th=[ 47], 95.00th=[ 56], | 99.00th=[ 79], 99.50th=[ 90], 99.90th=[ 107], 99.95th=[ 108], | 99.99th=[ 108] bw ( KiB/s): min= 16, max= 248, per=24.99%, avg=96.20, stdev=51.92, samples=128 iops : min= 4, max= 62, avg=23.73, stdev=13.03, samples=128 lat (msec) : 4=0.29%, 10=23.05%, 20=39.39%, 50=31.48%, 100=5.70% lat (msec) : 250=0.07%, 750=0.03% cpu : usr=0.13%, sys=8.62%, ctx=5124, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190711: Tue Nov 25 17:32:32 2025 read: IOPS=22, BW=91.5KiB/s (93.7kB/s)(5940KiB/64897msec) clat (msec): min=3, max=704, avg=17.87, stdev=22.73 lat (msec): min=3, max=704, avg=17.87, stdev=22.73 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 15], | 70.00th=[ 18], 80.00th=[ 25], 90.00th=[ 38], 95.00th=[ 48], | 99.00th=[ 68], 99.50th=[ 79], 99.90th=[ 112], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 7, max= 280, per=25.02%, avg=92.07, stdev=53.86, samples=128 iops : min= 1, max= 70, avg=22.70, stdev=13.49, samples=128 write: IOPS=24, BW=97.8KiB/s (100kB/s)(6348KiB/64897msec); 0 zone resets clat (msec): min=5, max=105, avg=24.02, stdev=15.20 lat (msec): min=5, max=105, avg=24.03, stdev=15.20 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 30], 80.00th=[ 36], 90.00th=[ 46], 95.00th=[ 54], | 99.00th=[ 75], 99.50th=[ 85], 99.90th=[ 102], 99.95th=[ 106], | 99.99th=[ 106] bw ( KiB/s): min= 23, max= 280, per=25.47%, avg=98.07, stdev=52.48, samples=128 iops : min= 5, max= 70, avg=24.20, stdev=13.19, samples=128 lat (msec) : 4=0.29%, 10=22.62%, 20=39.78%, 50=31.93%, 100=5.21% lat (msec) : 250=0.13%, 750=0.03% cpu : usr=0.13%, sys=8.45%, ctx=5217, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190713: Tue Nov 25 17:32:32 2025 read: IOPS=23, BW=93.9KiB/s (96.1kB/s)(6028KiB/64218msec) clat (msec): min=3, max=107, avg=17.00, stdev=13.17 lat (msec): min=3, max=107, avg=17.00, stdev=13.17 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 15], | 70.00th=[ 19], 80.00th=[ 24], 90.00th=[ 35], 95.00th=[ 45], | 99.00th=[ 64], 99.50th=[ 79], 99.90th=[ 91], 99.95th=[ 108], | 99.99th=[ 108] bw ( KiB/s): min= 16, max= 304, per=25.68%, avg=94.50, stdev=52.87, samples=127 iops : min= 4, max= 76, avg=23.30, stdev=13.27, samples=127 write: IOPS=24, BW=97.5KiB/s (99.8kB/s)(6260KiB/64218msec); 0 zone resets clat (msec): min=5, max=710, avg=24.52, stdev=22.99 lat (msec): min=5, max=710, avg=24.52, stdev=22.99 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 29], 80.00th=[ 36], 90.00th=[ 45], 95.00th=[ 53], | 99.00th=[ 79], 99.50th=[ 91], 99.90th=[ 114], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 32, max= 264, per=25.48%, avg=98.09, stdev=51.04, samples=127 iops : min= 8, max= 66, avg=24.20, stdev=12.80, samples=127 lat (msec) : 4=0.29%, 10=22.43%, 20=40.14%, 50=32.03%, 100=4.95% lat (msec) : 250=0.13%, 750=0.03% cpu : usr=0.14%, sys=8.68%, ctx=5127, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190716: Tue Nov 25 17:32:32 2025 read: IOPS=23, BW=93.3KiB/s (95.5kB/s)(6052KiB/64895msec) clat (msec): min=3, max=111, avg=17.57, stdev=14.04 lat (msec): min=3, max=111, avg=17.57, stdev=14.04 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 13], 60.00th=[ 16], | 70.00th=[ 19], 80.00th=[ 25], 90.00th=[ 37], 95.00th=[ 47], | 99.00th=[ 68], 99.50th=[ 81], 99.90th=[ 96], 99.95th=[ 112], | 99.99th=[ 112] bw ( KiB/s): min= 15, max= 328, per=25.46%, avg=93.70, stdev=53.03, samples=128 iops : min= 3, max= 82, avg=23.10, stdev=13.29, samples=128 write: IOPS=24, BW=96.1KiB/s (98.4kB/s)(6236KiB/64895msec); 0 zone resets clat (msec): min=5, max=700, avg=24.42, stdev=22.74 lat (msec): min=5, max=700, avg=24.42, stdev=22.74 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 29], 80.00th=[ 35], 90.00th=[ 45], 95.00th=[ 53], | 99.00th=[ 75], 99.50th=[ 91], 99.90th=[ 116], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 39, max= 264, per=25.03%, avg=96.38, stdev=50.28, samples=128 iops : min= 9, max= 66, avg=23.77, stdev=12.62, samples=128 lat (msec) : 4=0.20%, 10=22.20%, 20=39.88%, 50=32.36%, 100=5.21% lat (msec) : 250=0.13%, 750=0.03% cpu : usr=0.09%, sys=8.60%, ctx=5160, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=369KiB/s (377kB/s), 91.5KiB/s-93.9KiB/s (93.7kB/s-96.1kB/s), io=23.4MiB (24.6MB), run=64218-65150msec WRITE: bw=386KiB/s (395kB/s), 96.1KiB/s-97.8KiB/s (98.4kB/s-100kB/s), io=24.6MiB (25.7MB), run=64218-65150msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=190745: Tue Nov 25 17:32:53 2025 read: IOPS=25, BW=410KiB/s (420kB/s)(5920KiB/14450msec) clat (usec): min=150, max=63073, avg=20341.19, stdev=10236.00 lat (usec): min=152, max=63074, avg=20342.95, stdev=10235.54 clat percentiles (usec): | 1.00th=[ 269], 5.00th=[ 1074], 10.00th=[10028], 20.00th=[12518], | 30.00th=[14353], 40.00th=[16712], 50.00th=[19268], 60.00th=[21365], | 70.00th=[23987], 80.00th=[28443], 90.00th=[34341], 95.00th=[38536], | 99.00th=[50594], 99.50th=[54264], 99.90th=[63177], 99.95th=[63177], | 99.99th=[63177] bw ( KiB/s): min= 256, max= 638, per=24.44%, avg=404.07, stdev=88.07, samples=28 iops : min= 16, max= 39, avg=24.71, stdev= 5.39, samples=28 write: IOPS=27, BW=441KiB/s (451kB/s)(6368KiB/14450msec); 0 zone resets clat (usec): min=1635, max=78430, avg=17217.04, stdev=12514.25 lat (usec): min=1636, max=78431, avg=17219.00, stdev=12514.17 clat percentiles (usec): | 1.00th=[ 1942], 5.00th=[ 3556], 10.00th=[ 4228], 20.00th=[ 5997], | 30.00th=[ 8291], 40.00th=[12387], 50.00th=[14484], 60.00th=[17695], | 70.00th=[21890], 80.00th=[26346], 90.00th=[32900], 95.00th=[41681], | 99.00th=[60556], 99.50th=[73925], 99.90th=[78119], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 223, max= 640, per=25.25%, avg=439.18, stdev=98.28, samples=28 iops : min= 13, max= 40, avg=26.93, stdev= 6.26, samples=28 lat (usec) : 250=0.26%, 500=1.95%, 750=0.13% lat (msec) : 2=0.65%, 4=3.65%, 10=15.76%, 20=38.15%, 50=37.89% lat (msec) : 100=1.56% cpu : usr=0.13%, sys=24.97%, ctx=2560, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190746: Tue Nov 25 17:32:53 2025 read: IOPS=24, BW=396KiB/s (406kB/s)(5744KiB/14488msec) clat (usec): min=163, max=78928, avg=20771.32, stdev=10811.22 lat (usec): min=164, max=78929, avg=20772.28, stdev=10811.26 clat percentiles (usec): | 1.00th=[ 297], 5.00th=[ 6849], 10.00th=[ 9372], 20.00th=[12911], | 30.00th=[14877], 40.00th=[16909], 50.00th=[19530], 60.00th=[21627], | 70.00th=[24249], 80.00th=[28181], 90.00th=[33162], 95.00th=[41157], | 99.00th=[51643], 99.50th=[76022], 99.90th=[79168], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 159, max= 544, per=24.09%, avg=398.21, stdev=104.44, samples=28 iops : min= 9, max= 34, avg=24.36, stdev= 6.65, samples=28 write: IOPS=28, BW=452KiB/s (463kB/s)(6544KiB/14488msec); 0 zone resets clat (usec): min=1389, max=73020, avg=16932.23, stdev=12826.94 lat (usec): min=1391, max=73021, avg=16939.75, stdev=12839.74 clat percentiles (usec): | 1.00th=[ 1729], 5.00th=[ 3654], 10.00th=[ 4293], 20.00th=[ 5276], | 30.00th=[ 8848], 40.00th=[11207], 50.00th=[13698], 60.00th=[16450], | 70.00th=[20841], 80.00th=[26608], 90.00th=[34866], 95.00th=[41681], | 99.00th=[56886], 99.50th=[67634], 99.90th=[72877], 99.95th=[72877], | 99.99th=[72877] bw ( KiB/s): min= 255, max= 670, per=25.86%, avg=449.75, stdev=89.46, samples=28 iops : min= 15, max= 41, avg=27.57, stdev= 5.66, samples=28 lat (usec) : 250=0.26%, 500=1.56%, 750=0.26% lat (msec) : 2=1.30%, 4=2.73%, 10=17.97%, 20=36.46%, 50=37.63% lat (msec) : 100=1.82% cpu : usr=0.11%, sys=25.17%, ctx=2537, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190747: Tue Nov 25 17:32:53 2025 read: IOPS=27, BW=435KiB/s (445kB/s)(6256KiB/14392msec) clat (usec): min=183, max=70637, avg=19653.09, stdev=11134.60 lat (usec): min=184, max=70638, avg=19654.13, stdev=11134.61 clat percentiles (usec): | 1.00th=[ 269], 5.00th=[ 2638], 10.00th=[ 9634], 20.00th=[12125], | 30.00th=[13698], 40.00th=[15401], 50.00th=[17957], 60.00th=[19792], | 70.00th=[22414], 80.00th=[25560], 90.00th=[31589], 95.00th=[41157], | 99.00th=[61080], 99.50th=[66847], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 224, max= 672, per=26.41%, avg=436.57, stdev=110.81, samples=28 iops : min= 14, max= 42, avg=26.75, stdev= 7.02, samples=28 write: IOPS=26, BW=419KiB/s (429kB/s)(6032KiB/14392msec); 0 zone resets clat (usec): min=1471, max=76910, avg=17706.96, stdev=13653.48 lat (usec): min=1473, max=76912, avg=17708.75, stdev=13653.49 clat percentiles (usec): | 1.00th=[ 1942], 5.00th=[ 3425], 10.00th=[ 4293], 20.00th=[ 5669], | 30.00th=[ 9110], 40.00th=[11600], 50.00th=[14222], 60.00th=[16909], | 70.00th=[21365], 80.00th=[26870], 90.00th=[37487], 95.00th=[46400], | 99.00th=[61080], 99.50th=[76022], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 189, max= 666, per=23.83%, avg=414.32, stdev=125.10, samples=28 iops : min= 11, max= 41, avg=25.36, stdev= 7.94, samples=28 lat (usec) : 250=0.39%, 500=1.69%, 750=0.26% lat (msec) : 2=0.65%, 4=3.65%, 10=15.49%, 20=41.54%, 50=33.33% lat (msec) : 100=2.99% cpu : usr=0.10%, sys=24.22%, ctx=2554, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190748: Tue Nov 25 17:32:53 2025 read: IOPS=26, BW=419KiB/s (429kB/s)(6032KiB/14406msec) clat (usec): min=156, max=65245, avg=20978.67, stdev=11292.95 lat (usec): min=158, max=65246, avg=20979.75, stdev=11293.03 clat percentiles (usec): | 1.00th=[ 176], 5.00th=[ 553], 10.00th=[ 9896], 20.00th=[12649], | 30.00th=[14615], 40.00th=[16909], 50.00th=[19792], 60.00th=[22152], | 70.00th=[25297], 80.00th=[28181], 90.00th=[33162], 95.00th=[42206], | 99.00th=[57934], 99.50th=[60556], 99.90th=[65274], 99.95th=[65274], | 99.99th=[65274] bw ( KiB/s): min= 254, max= 544, per=25.17%, avg=416.07, stdev=86.90, samples=28 iops : min= 15, max= 34, avg=25.46, stdev= 5.54, samples=28 write: IOPS=27, BW=434KiB/s (445kB/s)(6256KiB/14406msec); 0 zone resets clat (usec): min=1778, max=55889, avg=16441.53, stdev=11957.30 lat (usec): min=1779, max=55890, avg=16449.75, stdev=11963.20 clat percentiles (usec): | 1.00th=[ 2114], 5.00th=[ 3425], 10.00th=[ 4228], 20.00th=[ 5276], | 30.00th=[ 7308], 40.00th=[10814], 50.00th=[13304], 60.00th=[16909], | 70.00th=[21103], 80.00th=[26346], 90.00th=[34341], 95.00th=[40109], | 99.00th=[51643], 99.50th=[55313], 99.90th=[55837], 99.95th=[55837], | 99.99th=[55837] bw ( KiB/s): min= 256, max= 702, per=24.90%, avg=432.93, stdev=111.25, samples=28 iops : min= 16, max= 43, avg=26.54, stdev= 6.95, samples=28 lat (usec) : 250=0.65%, 500=1.56%, 750=0.39%, 1000=0.13% lat (msec) : 2=0.39%, 4=3.91%, 10=16.93%, 20=35.42%, 50=38.15% lat (msec) : 100=2.47% cpu : usr=0.13%, sys=25.09%, ctx=2528, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1653KiB/s (1693kB/s), 396KiB/s-435KiB/s (406kB/s-445kB/s), io=23.4MiB (24.5MB), run=14392-14488msec WRITE: bw=1739KiB/s (1781kB/s), 419KiB/s-452KiB/s (429kB/s-463kB/s), io=24.6MiB (25.8MB), run=14392-14488msec rand-rw: (groupid=0, jobs=1): err= 0: pid=190749: Tue Nov 25 17:32:56 2025 read: IOPS=21, BW=347KiB/s (355kB/s)(5920KiB/17070msec) clat (usec): min=4119, max=81141, avg=18321.56, stdev=14551.11 lat (usec): min=4120, max=81143, avg=18322.58, stdev=14551.11 clat percentiles (usec): | 1.00th=[ 4817], 5.00th=[ 5669], 10.00th=[ 6259], 20.00th=[ 8094], | 30.00th=[ 9372], 40.00th=[11076], 50.00th=[12387], 60.00th=[14746], | 70.00th=[19530], 80.00th=[26608], 90.00th=[39060], 95.00th=[49546], | 99.00th=[73925], 99.50th=[79168], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 159, max= 896, per=24.09%, avg=335.85, stdev=189.85, samples=33 iops : min= 9, max= 56, avg=20.48, stdev=11.98, samples=33 write: IOPS=23, BW=373KiB/s (382kB/s)(6368KiB/17070msec); 0 zone resets clat (usec): min=6731, max=89125, avg=25614.57, stdev=15221.07 lat (usec): min=6733, max=89126, avg=25616.75, stdev=15221.06 clat percentiles (usec): | 1.00th=[ 7635], 5.00th=[ 9241], 10.00th=[ 9896], 20.00th=[12256], | 30.00th=[14222], 40.00th=[17695], 50.00th=[20841], 60.00th=[26084], | 70.00th=[31065], 80.00th=[39060], 90.00th=[47449], 95.00th=[55837], | 99.00th=[69731], 99.50th=[81265], 99.90th=[88605], 99.95th=[88605], | 99.99th=[88605] bw ( KiB/s): min= 192, max= 960, per=25.18%, avg=369.42, stdev=191.18, samples=33 iops : min= 12, max= 60, avg=22.58, stdev=12.04, samples=33 lat (msec) : 10=21.88%, 20=36.98%, 50=35.16%, 100=5.99% cpu : usr=0.14%, sys=10.16%, ctx=1422, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190750: Tue Nov 25 17:32:56 2025 read: IOPS=21, BW=339KiB/s (348kB/s)(5744KiB/16926msec) clat (usec): min=4435, max=87218, avg=18553.73, stdev=13644.29 lat (usec): min=4437, max=87219, avg=18555.21, stdev=13644.32 clat percentiles (usec): | 1.00th=[ 4621], 5.00th=[ 5866], 10.00th=[ 6587], 20.00th=[ 8455], | 30.00th=[10159], 40.00th=[11731], 50.00th=[13304], 60.00th=[15795], | 70.00th=[20317], 80.00th=[25822], 90.00th=[41157], 95.00th=[46924], | 99.00th=[63177], 99.50th=[71828], 99.90th=[87557], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 128, max= 800, per=24.22%, avg=337.61, stdev=154.93, samples=33 iops : min= 8, max= 50, avg=20.61, stdev= 9.68, samples=33 write: IOPS=24, BW=387KiB/s (396kB/s)(6544KiB/16926msec); 0 zone resets clat (usec): min=7309, max=95974, avg=25017.86, stdev=15359.14 lat (usec): min=7311, max=95975, avg=25019.81, stdev=15359.14 clat percentiles (usec): | 1.00th=[ 8455], 5.00th=[ 9634], 10.00th=[10814], 20.00th=[12256], | 30.00th=[14353], 40.00th=[17433], 50.00th=[20317], 60.00th=[23725], | 70.00th=[28705], 80.00th=[35390], 90.00th=[47449], 95.00th=[55837], | 99.00th=[80217], 99.50th=[87557], 99.90th=[95945], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 190, max= 992, per=26.06%, avg=382.30, stdev=192.99, samples=33 iops : min= 11, max= 62, avg=23.39, stdev=12.17, samples=33 lat (msec) : 10=17.19%, 20=40.89%, 50=35.94%, 100=5.99% cpu : usr=0.09%, sys=10.30%, ctx=1381, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190751: Tue Nov 25 17:32:56 2025 read: IOPS=22, BW=366KiB/s (375kB/s)(6256KiB/17073msec) clat (usec): min=4465, max=86169, avg=19727.52, stdev=15266.55 lat (usec): min=4466, max=86170, avg=19728.83, stdev=15266.46 clat percentiles (usec): | 1.00th=[ 4817], 5.00th=[ 5932], 10.00th=[ 6456], 20.00th=[ 8160], | 30.00th=[ 9634], 40.00th=[11338], 50.00th=[13698], 60.00th=[17171], | 70.00th=[22676], 80.00th=[30802], 90.00th=[43254], 95.00th=[51119], | 99.00th=[73925], 99.50th=[74974], 99.90th=[86508], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 96, max= 1021, per=26.11%, avg=363.97, stdev=219.39, samples=33 iops : min= 6, max= 63, avg=22.24, stdev=13.79, samples=33 write: IOPS=22, BW=353KiB/s (362kB/s)(6032KiB/17073msec); 0 zone resets clat (msec): min=6, max=103, avg=24.58, stdev=15.99 lat (msec): min=6, max=103, avg=24.58, stdev=15.99 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 29], 80.00th=[ 36], 90.00th=[ 46], 95.00th=[ 55], | 99.00th=[ 87], 99.50th=[ 97], 99.90th=[ 104], 99.95th=[ 104], | 99.99th=[ 104] bw ( KiB/s): min= 127, max= 928, per=23.30%, avg=341.79, stdev=172.59, samples=33 iops : min= 7, max= 58, avg=20.85, stdev=10.91, samples=33 lat (msec) : 10=20.83%, 20=38.80%, 50=34.24%, 100=5.99%, 250=0.13% cpu : usr=0.12%, sys=10.09%, ctx=1372, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190752: Tue Nov 25 17:32:56 2025 read: IOPS=21, BW=351KiB/s (360kB/s)(6032KiB/17173msec) clat (usec): min=4270, max=75596, avg=18647.12, stdev=14411.86 lat (usec): min=4271, max=75597, avg=18648.13, stdev=14411.83 clat percentiles (usec): | 1.00th=[ 4817], 5.00th=[ 5669], 10.00th=[ 6390], 20.00th=[ 7439], | 30.00th=[ 9503], 40.00th=[11338], 50.00th=[13042], 60.00th=[15401], | 70.00th=[20841], 80.00th=[28181], 90.00th=[41681], 95.00th=[53216], | 99.00th=[61604], 99.50th=[65274], 99.90th=[76022], 99.95th=[76022], | 99.99th=[76022] bw ( KiB/s): min= 159, max= 896, per=25.02%, avg=348.82, stdev=195.47, samples=34 iops : min= 9, max= 56, avg=21.32, stdev=12.32, samples=34 write: IOPS=22, BW=364KiB/s (373kB/s)(6256KiB/17173msec); 0 zone resets clat (usec): min=6864, max=96689, avg=25795.65, stdev=17625.88 lat (usec): min=6866, max=96690, avg=25797.76, stdev=17626.07 clat percentiles (usec): | 1.00th=[ 7242], 5.00th=[ 9110], 10.00th=[10159], 20.00th=[11600], | 30.00th=[13435], 40.00th=[16319], 50.00th=[18744], 60.00th=[23200], | 70.00th=[30278], 80.00th=[41157], 90.00th=[51643], 95.00th=[62653], | 99.00th=[81265], 99.50th=[92799], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 191, max= 1056, per=24.86%, avg=364.71, stdev=199.75, samples=34 iops : min= 11, max= 66, avg=22.29, stdev=12.60, samples=34 lat (msec) : 10=20.31%, 20=39.32%, 50=31.64%, 100=8.72% cpu : usr=0.08%, sys=10.05%, ctx=1423, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1395KiB/s (1428kB/s), 339KiB/s-366KiB/s (348kB/s-375kB/s), io=23.4MiB (24.5MB), run=16926-17173msec WRITE: bw=1467KiB/s (1503kB/s), 353KiB/s-387KiB/s (362kB/s-396kB/s), io=24.6MiB (25.8MB), run=16926-17173msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=190761: Tue Nov 25 17:33:05 2025 read: IOPS=3, BW=3490KiB/s (3574kB/s)(7168KiB/2054msec) clat (msec): min=48, max=268, avg=116.32, stdev=82.75 lat (msec): min=48, max=268, avg=116.32, stdev=82.75 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 50], 20.00th=[ 62], | 30.00th=[ 64], 40.00th=[ 64], 50.00th=[ 69], 60.00th=[ 114], | 70.00th=[ 114], 80.00th=[ 190], 90.00th=[ 268], 95.00th=[ 268], | 99.00th=[ 268], 99.50th=[ 268], 99.90th=[ 268], 99.95th=[ 268], | 99.99th=[ 268] bw ( KiB/s): min= 2019, max= 7861, per=71.81%, avg=4940.00, stdev=4130.92, samples=2 iops : min= 1, max= 7, avg= 4.00, stdev= 4.24, samples=2 write: IOPS=2, BW=2493KiB/s (2553kB/s)(5120KiB/2054msec); 0 zone resets clat (msec): min=98, max=553, avg=244.21, stdev=185.69 lat (msec): min=98, max=553, avg=244.32, stdev=185.72 clat percentiles (msec): | 1.00th=[ 99], 5.00th=[ 99], 10.00th=[ 99], 20.00th=[ 99], | 30.00th=[ 122], 40.00th=[ 122], 50.00th=[ 176], 60.00th=[ 176], | 70.00th=[ 275], 80.00th=[ 275], 90.00th=[ 558], 95.00th=[ 558], | 99.00th=[ 558], 99.50th=[ 558], 99.90th=[ 558], 99.95th=[ 558], | 99.99th=[ 558] bw ( KiB/s): min= 1965, max= 4096, per=27.97%, avg=2693.33, stdev=1215.04, samples=3 iops : min= 1, max= 4, avg= 2.00, stdev= 1.73, samples=3 lat (msec) : 50=8.33%, 100=33.33%, 250=33.33%, 500=16.67%, 750=8.33% cpu : usr=0.00%, sys=13.40%, ctx=59, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190763: Tue Nov 25 17:33:05 2025 read: IOPS=0, BW=697KiB/s (714kB/s)(2048KiB/2937msec) clat (msec): min=291, max=299, avg=295.45, stdev= 5.11 lat (msec): min=291, max=299, avg=295.45, stdev= 5.11 clat percentiles (msec): | 1.00th=[ 292], 5.00th=[ 292], 10.00th=[ 292], 20.00th=[ 292], | 30.00th=[ 292], 40.00th=[ 292], 50.00th=[ 292], 60.00th=[ 300], | 70.00th=[ 300], 80.00th=[ 300], 90.00th=[ 300], 95.00th=[ 300], | 99.00th=[ 300], 99.50th=[ 300], 99.90th=[ 300], 99.95th=[ 300], | 99.99th=[ 300] bw ( KiB/s): min= 1917, max= 1957, per=28.16%, avg=1937.00, stdev=28.28, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3487KiB/s (3570kB/s)(10.0MiB/2937msec); 0 zone resets clat (msec): min=47, max=375, avg=229.51, stdev=114.78 lat (msec): min=47, max=375, avg=229.58, stdev=114.78 clat percentiles (msec): | 1.00th=[ 47], 5.00th=[ 47], 10.00th=[ 47], 20.00th=[ 97], | 30.00th=[ 136], 40.00th=[ 190], 50.00th=[ 209], 60.00th=[ 251], | 70.00th=[ 275], 80.00th=[ 355], 90.00th=[ 363], 95.00th=[ 376], | 99.00th=[ 376], 99.50th=[ 376], 99.90th=[ 376], 99.95th=[ 376], | 99.99th=[ 376] bw ( KiB/s): min= 1917, max= 4096, per=36.73%, avg=3537.00, stdev=1080.33, samples=4 iops : min= 1, max= 4, avg= 3.00, stdev= 1.41, samples=4 lat (msec) : 50=8.33%, 100=8.33%, 250=33.33%, 500=50.00% cpu : usr=0.00%, sys=16.08%, ctx=248, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190764: Tue Nov 25 17:33:05 2025 read: IOPS=2, BW=2064KiB/s (2113kB/s)(6144KiB/2977msec) clat (msec): min=45, max=651, avg=187.01, stdev=234.70 lat (msec): min=45, max=651, avg=187.01, stdev=234.70 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 46], 10.00th=[ 46], 20.00th=[ 57], | 30.00th=[ 57], 40.00th=[ 61], 50.00th=[ 61], 60.00th=[ 107], | 70.00th=[ 203], 80.00th=[ 203], 90.00th=[ 651], 95.00th=[ 651], | 99.00th=[ 651], 99.50th=[ 651], 99.90th=[ 651], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 3835, max= 6144, per=72.53%, avg=4989.50, stdev=1632.71, samples=2 iops : min= 3, max= 6, avg= 4.50, stdev= 2.12, samples=2 write: IOPS=2, BW=2064KiB/s (2113kB/s)(6144KiB/2977msec); 0 zone resets clat (msec): min=60, max=669, avg=296.63, stdev=201.25 lat (msec): min=60, max=669, avg=296.68, stdev=201.26 clat percentiles (msec): | 1.00th=[ 61], 5.00th=[ 61], 10.00th=[ 61], 20.00th=[ 245], | 30.00th=[ 245], 40.00th=[ 247], 50.00th=[ 247], 60.00th=[ 251], | 70.00th=[ 309], 80.00th=[ 309], 90.00th=[ 667], 95.00th=[ 667], | 99.00th=[ 667], 99.50th=[ 667], 99.90th=[ 667], 99.95th=[ 667], | 99.99th=[ 667] bw ( KiB/s): min= 1917, max= 2727, per=22.63%, avg=2179.75, stdev=369.33, samples=4 iops : min= 1, max= 2, avg= 1.50, stdev= 0.58, samples=4 lat (msec) : 50=8.33%, 100=25.00%, 250=33.33%, 500=16.67%, 750=16.67% cpu : usr=0.00%, sys=21.10%, ctx=160, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190766: Tue Nov 25 17:33:05 2025 read: IOPS=1, BW=1849KiB/s (1893kB/s)(5120KiB/2769msec) clat (msec): min=52, max=594, avg=176.40, stdev=234.51 lat (msec): min=52, max=594, avg=176.40, stdev=234.51 clat percentiles (msec): | 1.00th=[ 53], 5.00th=[ 53], 10.00th=[ 53], 20.00th=[ 53], | 30.00th=[ 65], 40.00th=[ 65], 50.00th=[ 73], 60.00th=[ 73], | 70.00th=[ 97], 80.00th=[ 97], 90.00th=[ 592], 95.00th=[ 592], | 99.00th=[ 592], 99.50th=[ 592], 99.90th=[ 592], 99.95th=[ 592], | 99.99th=[ 592] bw ( KiB/s): min= 3835, max= 4096, per=57.65%, avg=3965.50, stdev=184.55, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2589KiB/s (2651kB/s)(7168KiB/2769msec); 0 zone resets clat (msec): min=105, max=413, avg=260.86, stdev=108.92 lat (msec): min=105, max=413, avg=260.93, stdev=108.92 clat percentiles (msec): | 1.00th=[ 106], 5.00th=[ 106], 10.00th=[ 106], 20.00th=[ 134], | 30.00th=[ 257], 40.00th=[ 257], 50.00th=[ 275], 60.00th=[ 313], | 70.00th=[ 313], 80.00th=[ 330], 90.00th=[ 414], 95.00th=[ 414], | 99.00th=[ 414], 99.50th=[ 414], 99.90th=[ 414], 99.95th=[ 414], | 99.99th=[ 414] bw ( KiB/s): min= 1957, max= 4039, per=26.20%, avg=2523.00, stdev=1011.58, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (msec) : 100=33.33%, 250=16.67%, 500=41.67%, 750=8.33% cpu : usr=0.00%, sys=19.00%, ctx=293, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6879KiB/s (7045kB/s), 697KiB/s-3490KiB/s (714kB/s-3574kB/s), io=20.0MiB (20.0MB), run=2054-2977msec WRITE: bw=9631KiB/s (9862kB/s), 2064KiB/s-3487KiB/s (2113kB/s-3570kB/s), io=28.0MiB (29.4MB), run=2054-2977msec rand-rw: (groupid=0, jobs=1): err= 0: pid=190762: Tue Nov 25 17:33:05 2025 read: IOPS=2, BW=2932KiB/s (3002kB/s)(7168KiB/2445msec) clat (msec): min=92, max=181, avg=130.30, stdev=30.49 lat (msec): min=92, max=181, avg=130.30, stdev=30.49 clat percentiles (msec): | 1.00th=[ 93], 5.00th=[ 93], 10.00th=[ 93], 20.00th=[ 108], | 30.00th=[ 111], 40.00th=[ 111], 50.00th=[ 129], 60.00th=[ 138], | 70.00th=[ 138], 80.00th=[ 155], 90.00th=[ 182], 95.00th=[ 182], | 99.00th=[ 182], 99.50th=[ 182], 99.90th=[ 182], 99.95th=[ 182], | 99.99th=[ 182] bw ( KiB/s): min= 2031, max= 4106, per=50.12%, avg=3397.33, stdev=1183.55, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=2, BW=2094KiB/s (2144kB/s)(5120KiB/2445msec); 0 zone resets clat (msec): min=98, max=567, avg=280.90, stdev=186.60 lat (msec): min=98, max=567, avg=280.96, stdev=186.60 clat percentiles (msec): | 1.00th=[ 99], 5.00th=[ 99], 10.00th=[ 99], 20.00th=[ 99], | 30.00th=[ 169], 40.00th=[ 169], 50.00th=[ 211], 60.00th=[ 211], | 70.00th=[ 359], 80.00th=[ 359], 90.00th=[ 567], 95.00th=[ 567], | 99.00th=[ 567], 99.50th=[ 567], 99.90th=[ 567], 99.95th=[ 567], | 99.99th=[ 567] bw ( KiB/s): min= 1368, max= 2031, per=19.34%, avg=1835.75, stdev=316.28, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 lat (msec) : 100=16.67%, 250=66.67%, 500=8.33%, 750=8.33% cpu : usr=0.08%, sys=39.75%, ctx=855, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190765: Tue Nov 25 17:33:05 2025 read: IOPS=0, BW=678KiB/s (694kB/s)(2048KiB/3021msec) clat (msec): min=184, max=343, avg=264.04, stdev=112.09 lat (msec): min=184, max=343, avg=264.04, stdev=112.09 clat percentiles (msec): | 1.00th=[ 186], 5.00th=[ 186], 10.00th=[ 186], 20.00th=[ 186], | 30.00th=[ 186], 40.00th=[ 186], 50.00th=[ 186], 60.00th=[ 342], | 70.00th=[ 342], 80.00th=[ 342], 90.00th=[ 342], 95.00th=[ 342], | 99.00th=[ 342], 99.50th=[ 342], 99.90th=[ 342], 99.95th=[ 342], | 99.99th=[ 342] bw ( KiB/s): min= 2031, max= 2048, per=30.09%, avg=2039.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3390KiB/s (3471kB/s)(10.0MiB/3021msec); 0 zone resets clat (msec): min=61, max=629, avg=240.91, stdev=179.01 lat (msec): min=61, max=630, avg=240.98, stdev=179.01 clat percentiles (msec): | 1.00th=[ 62], 5.00th=[ 62], 10.00th=[ 62], 20.00th=[ 116], | 30.00th=[ 116], 40.00th=[ 134], 50.00th=[ 138], 60.00th=[ 176], | 70.00th=[ 255], 80.00th=[ 384], 90.00th=[ 401], 95.00th=[ 634], | 99.00th=[ 634], 99.50th=[ 634], 99.90th=[ 634], 99.95th=[ 634], | 99.99th=[ 634] bw ( KiB/s): min= 2023, max= 3835, per=26.18%, avg=2484.25, stdev=900.56, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 100=8.33%, 250=50.00%, 500=33.33%, 750=8.33% cpu : usr=0.00%, sys=40.66%, ctx=237, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190767: Tue Nov 25 17:33:05 2025 read: IOPS=2, BW=2541KiB/s (2602kB/s)(6144KiB/2418msec) clat (usec): min=715, max=168612, avg=111119.65, stdev=69201.16 lat (usec): min=716, max=168613, avg=111120.92, stdev=69201.17 clat percentiles (usec): | 1.00th=[ 717], 5.00th=[ 717], 10.00th=[ 717], 20.00th=[ 49021], | 30.00th=[ 49021], 40.00th=[139461], 50.00th=[139461], 60.00th=[154141], | 70.00th=[154141], 80.00th=[154141], 90.00th=[168821], 95.00th=[168821], | 99.00th=[168821], 99.50th=[168821], 99.90th=[168821], 99.95th=[168821], | 99.99th=[168821] bw ( KiB/s): min= 1917, max= 6144, per=49.58%, avg=3361.33, stdev=2410.44, samples=3 iops : min= 1, max= 6, avg= 2.67, stdev= 2.89, samples=3 write: IOPS=2, BW=2541KiB/s (2602kB/s)(6144KiB/2418msec); 0 zone resets clat (msec): min=115, max=448, avg=288.16, stdev=105.36 lat (msec): min=115, max=448, avg=288.23, stdev=105.36 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 284], | 30.00th=[ 284], 40.00th=[ 292], 50.00th=[ 292], 60.00th=[ 296], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 447], 95.00th=[ 447], | 99.00th=[ 447], 99.50th=[ 447], 99.90th=[ 447], 99.95th=[ 447], | 99.99th=[ 447] bw ( KiB/s): min= 1917, max= 4039, per=26.41%, avg=2506.75, stdev=1023.08, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (usec) : 750=8.33% lat (msec) : 50=8.33%, 250=41.67%, 500=41.67% cpu : usr=0.08%, sys=44.77%, ctx=787, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190768: Tue Nov 25 17:33:05 2025 read: IOPS=2, BW=2101KiB/s (2151kB/s)(5120KiB/2437msec) clat (usec): min=780, max=259838, avg=115541.23, stdev=112656.39 lat (usec): min=780, max=259839, avg=115542.06, stdev=112656.55 clat percentiles (usec): | 1.00th=[ 783], 5.00th=[ 783], 10.00th=[ 783], 20.00th=[ 783], | 30.00th=[ 873], 40.00th=[ 873], 50.00th=[158335], 60.00th=[158335], | 70.00th=[158335], 80.00th=[158335], 90.00th=[258999], 95.00th=[258999], | 99.00th=[258999], 99.50th=[258999], 99.90th=[258999], 99.95th=[258999], | 99.99th=[258999] bw ( KiB/s): min= 3835, max= 4063, per=58.25%, avg=3949.00, stdev=161.22, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2941KiB/s (3012kB/s)(7168KiB/2437msec); 0 zone resets clat (msec): min=114, max=527, avg=248.01, stdev=156.30 lat (msec): min=114, max=527, avg=248.07, stdev=156.32 clat percentiles (msec): | 1.00th=[ 115], 5.00th=[ 115], 10.00th=[ 115], 20.00th=[ 133], | 30.00th=[ 142], 40.00th=[ 142], 50.00th=[ 146], 60.00th=[ 330], | 70.00th=[ 330], 80.00th=[ 342], 90.00th=[ 527], 95.00th=[ 527], | 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527], | 99.99th=[ 527] bw ( KiB/s): min= 1374, max= 6095, per=35.41%, avg=3360.25, stdev=2159.45, samples=4 iops : min= 1, max= 5, avg= 2.50, stdev= 1.91, samples=4 lat (usec) : 1000=16.67% lat (msec) : 250=50.00%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=42.16%, ctx=251, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6779KiB/s (6942kB/s), 678KiB/s-2932KiB/s (694kB/s-3002kB/s), io=20.0MiB (20.0MB), run=2418-3021msec WRITE: bw=9491KiB/s (9719kB/s), 2094KiB/s-3390KiB/s (2144kB/s-3471kB/s), io=28.0MiB (29.4MB), run=2418-3021msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=190775: Tue Nov 25 17:33:13 2025 read: IOPS=2, BW=8821KiB/s (9033kB/s)(12.0MiB/1393msec) clat (msec): min=371, max=493, avg=431.29, stdev=60.84 lat (msec): min=371, max=493, avg=431.29, stdev=60.84 clat percentiles (msec): | 1.00th=[ 372], 5.00th=[ 372], 10.00th=[ 372], 20.00th=[ 372], | 30.00th=[ 372], 40.00th=[ 430], 50.00th=[ 430], 60.00th=[ 430], | 70.00th=[ 493], 80.00th=[ 493], 90.00th=[ 493], 95.00th=[ 493], | 99.00th=[ 493], 99.50th=[ 493], 99.90th=[ 493], 99.95th=[ 493], | 99.99th=[ 493] bw ( KiB/s): min= 6660, max= 8192, per=67.49%, avg=7426.00, stdev=1083.29, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=100.00% cpu : usr=0.00%, sys=53.02%, ctx=2445, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190777: Tue Nov 25 17:33:13 2025 read: IOPS=0, BW=2751KiB/s (2817kB/s)(4096KiB/1489msec) clat (nsec): min=490371k, max=490371k, avg=490370850.00, stdev= 0.00 lat (nsec): min=490372k, max=490372k, avg=490372000.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 489], 5.00th=[ 489], 10.00th=[ 489], 20.00th=[ 489], | 30.00th=[ 489], 40.00th=[ 489], 50.00th=[ 489], 60.00th=[ 489], | 70.00th=[ 489], 80.00th=[ 489], 90.00th=[ 489], 95.00th=[ 489], | 99.00th=[ 489], 99.50th=[ 489], 99.90th=[ 489], 99.95th=[ 489], | 99.99th=[ 489] bw ( KiB/s): min= 6954, max= 6954, per=63.20%, avg=6954.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=5502KiB/s (5634kB/s)(8192KiB/1489msec); 0 zone resets clat (msec): min=338, max=584, avg=461.42, stdev=173.66 lat (msec): min=338, max=584, avg=461.72, stdev=173.68 clat percentiles (msec): | 1.00th=[ 338], 5.00th=[ 338], 10.00th=[ 338], 20.00th=[ 338], | 30.00th=[ 338], 40.00th=[ 338], 50.00th=[ 338], 60.00th=[ 584], | 70.00th=[ 584], 80.00th=[ 584], 90.00th=[ 584], 95.00th=[ 584], | 99.00th=[ 584], 99.50th=[ 584], 99.90th=[ 584], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 8192, max= 8192, per=50.30%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.00%, sys=19.42%, ctx=589, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190779: Tue Nov 25 17:33:13 2025 write: IOPS=1, BW=6107KiB/s (6254kB/s)(12.0MiB/2012msec); 0 zone resets clat (msec): min=228, max=1035, avg=628.80, stdev=403.57 lat (msec): min=228, max=1037, avg=629.55, stdev=404.40 clat percentiles (msec): | 1.00th=[ 228], 5.00th=[ 228], 10.00th=[ 228], 20.00th=[ 228], | 30.00th=[ 228], 40.00th=[ 625], 50.00th=[ 625], 60.00th=[ 625], | 70.00th=[ 1036], 80.00th=[ 1036], 90.00th=[ 1036], 95.00th=[ 1036], | 99.00th=[ 1036], 99.50th=[ 1036], 99.90th=[ 1036], 99.95th=[ 1036], | 99.99th=[ 1036] bw ( KiB/s): min= 8143, max= 8143, per=50.00%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=15.02%, ctx=99, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190781: Tue Nov 25 17:33:13 2025 write: IOPS=1, BW=6225KiB/s (6374kB/s)(12.0MiB/1974msec); 0 zone resets clat (msec): min=396, max=1035, avg=625.93, stdev=355.76 lat (msec): min=396, max=1036, avg=626.24, stdev=355.82 clat percentiles (msec): | 1.00th=[ 397], 5.00th=[ 397], 10.00th=[ 397], 20.00th=[ 397], | 30.00th=[ 397], 40.00th=[ 447], 50.00th=[ 447], 60.00th=[ 447], | 70.00th=[ 1036], 80.00th=[ 1036], 90.00th=[ 1036], 95.00th=[ 1036], | 99.00th=[ 1036], 99.50th=[ 1036], 99.90th=[ 1036], 99.95th=[ 1036], | 99.99th=[ 1036] bw ( KiB/s): min= 6770, max= 8192, per=45.94%, avg=7481.00, stdev=1005.51, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.05%, sys=15.15%, ctx=79, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=10.7MiB/s (11.3MB/s), 2751KiB/s-8821KiB/s (2817kB/s-9033kB/s), io=16.0MiB (16.8MB), run=1393-1489msec WRITE: bw=15.9MiB/s (16.7MB/s), 5502KiB/s-6225KiB/s (5634kB/s-6374kB/s), io=32.0MiB (33.6MB), run=1489-2012msec rand-rw: (groupid=0, jobs=1): err= 0: pid=190774: Tue Nov 25 17:33:13 2025 read: IOPS=1, BW=5621KiB/s (5756kB/s)(12.0MiB/2186msec) clat (msec): min=170, max=1353, avg=674.08, stdev=610.68 lat (msec): min=170, max=1353, avg=674.08, stdev=610.68 clat percentiles (msec): | 1.00th=[ 171], 5.00th=[ 171], 10.00th=[ 171], 20.00th=[ 171], | 30.00th=[ 171], 40.00th=[ 498], 50.00th=[ 498], 60.00th=[ 498], | 70.00th=[ 1351], 80.00th=[ 1351], 90.00th=[ 1351], 95.00th=[ 1351], | 99.00th=[ 1351], 99.50th=[ 1351], 99.90th=[ 1351], 99.95th=[ 1351], | 99.99th=[ 1351] bw ( KiB/s): min= 8062, max= 8192, per=100.00%, avg=8127.00, stdev=91.92, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=14.42%, ctx=94, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190776: Tue Nov 25 17:33:13 2025 read: IOPS=0, BW=1888KiB/s (1934kB/s)(4096KiB/2169msec) clat (nsec): min=1478.3M, max=1478.3M, avg=1478289119.00, stdev= 0.00 lat (nsec): min=1478.3M, max=1478.3M, avg=1478290649.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1485], 5.00th=[ 1485], 10.00th=[ 1485], 20.00th=[ 1485], | 30.00th=[ 1485], 40.00th=[ 1485], 50.00th=[ 1485], 60.00th=[ 1485], | 70.00th=[ 1485], 80.00th=[ 1485], 90.00th=[ 1485], 95.00th=[ 1485], | 99.00th=[ 1485], 99.50th=[ 1485], 99.90th=[ 1485], 99.95th=[ 1485], | 99.99th=[ 1485] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3777KiB/s (3868kB/s)(8192KiB/2169msec); 0 zone resets clat (msec): min=189, max=342, avg=265.75, stdev=107.84 lat (msec): min=189, max=342, avg=266.08, stdev=107.80 clat percentiles (msec): | 1.00th=[ 190], 5.00th=[ 190], 10.00th=[ 190], 20.00th=[ 190], | 30.00th=[ 190], 40.00th=[ 190], 50.00th=[ 190], 60.00th=[ 342], | 70.00th=[ 342], 80.00th=[ 342], 90.00th=[ 342], 95.00th=[ 342], | 99.00th=[ 342], 99.50th=[ 342], 99.90th=[ 342], 99.95th=[ 342], | 99.99th=[ 342] lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=14.53%, ctx=71, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190778: Tue Nov 25 17:33:13 2025 write: IOPS=1, BW=7637KiB/s (7820kB/s)(12.0MiB/1609msec); 0 zone resets clat (msec): min=302, max=617, avg=505.43, stdev=176.29 lat (msec): min=302, max=617, avg=505.76, stdev=176.28 clat percentiles (msec): | 1.00th=[ 305], 5.00th=[ 305], 10.00th=[ 305], 20.00th=[ 305], | 30.00th=[ 305], 40.00th=[ 600], 50.00th=[ 600], 60.00th=[ 600], | 70.00th=[ 617], 80.00th=[ 617], 90.00th=[ 617], 95.00th=[ 617], | 99.00th=[ 617], 99.50th=[ 617], 99.90th=[ 617], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 8192, max= 8192, per=54.23%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.00%, sys=26.99%, ctx=79, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190780: Tue Nov 25 17:33:13 2025 write: IOPS=1, BW=7070KiB/s (7240kB/s)(12.0MiB/1738msec); 0 zone resets clat (msec): min=138, max=1235, avg=547.41, stdev=599.06 lat (msec): min=139, max=1235, avg=547.85, stdev=599.00 clat percentiles (msec): | 1.00th=[ 140], 5.00th=[ 140], 10.00th=[ 140], 20.00th=[ 140], | 30.00th=[ 140], 40.00th=[ 268], 50.00th=[ 268], 60.00th=[ 268], | 70.00th=[ 1234], 80.00th=[ 1234], 90.00th=[ 1234], 95.00th=[ 1234], | 99.00th=[ 1234], 99.50th=[ 1234], 99.90th=[ 1234], 99.95th=[ 1234], | 99.99th=[ 1234] bw ( KiB/s): min=13607, max=13607, per=90.07%, avg=13607.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=17.27%, ctx=67, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7495KiB/s (7675kB/s), 1888KiB/s-5621KiB/s (1934kB/s-5756kB/s), io=16.0MiB (16.8MB), run=2169-2186msec WRITE: bw=14.8MiB/s (15.5MB/s), 3777KiB/s-7637KiB/s (3868kB/s-7820kB/s), io=32.0MiB (33.6MB), run=1609-2169msec