== sanity test 398b: DIO and buffer IO race ============== 23:21:53 (1761103313) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.24284 s, 15.5 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159331: Tue Oct 21 23:23:04 2025 read: IOPS=24, BW=98.9KiB/s (101kB/s)(5992KiB/60577msec) clat (usec): min=83, max=662445, avg=20949.16, stdev=21257.95 lat (usec): min=83, max=662446, avg=20949.76, stdev=21257.95 clat percentiles (usec): | 1.00th=[ 269], 5.00th=[ 725], 10.00th=[ 7504], 20.00th=[ 10683], | 30.00th=[ 13173], 40.00th=[ 15270], 50.00th=[ 17957], 60.00th=[ 20841], | 70.00th=[ 25297], 80.00th=[ 30016], 90.00th=[ 35914], 95.00th=[ 44303], | 99.00th=[ 62129], 99.50th=[ 73925], 99.90th=[156238], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 6, max= 240, per=25.61%, avg=98.61, stdev=32.82, samples=120 iops : min= 1, max= 60, avg=24.23, stdev= 8.25, samples=120 write: IOPS=25, BW=104KiB/s (106kB/s)(6296KiB/60577msec); 0 zone resets clat (usec): min=643, max=209547, avg=18443.07, stdev=16385.93 lat (usec): min=643, max=209549, avg=18443.76, stdev=16385.93 clat percentiles (usec): | 1.00th=[ 750], 5.00th=[ 2147], 10.00th=[ 3130], 20.00th=[ 4948], | 30.00th=[ 6456], 40.00th=[ 9896], 50.00th=[ 14877], 60.00th=[ 19006], | 70.00th=[ 23200], 80.00th=[ 30016], 90.00th=[ 40633], 95.00th=[ 49021], | 99.00th=[ 66323], 99.50th=[ 77071], 99.90th=[120062], 99.95th=[208667], | 99.99th=[208667] bw ( KiB/s): min= 32, max= 208, per=25.98%, avg=104.70, stdev=35.88, samples=119 iops : min= 8, max= 52, avg=25.76, stdev= 9.00, samples=119 lat (usec) : 100=0.10%, 250=0.29%, 500=0.91%, 750=1.69%, 1000=1.01% lat (msec) : 2=1.14%, 4=4.82%, 10=18.85%, 20=31.12%, 50=36.30% lat (msec) : 100=3.58%, 250=0.16%, 750=0.03% cpu : usr=0.08%, sys=27.87%, ctx=8842, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159332: Tue Oct 21 23:23:04 2025 read: IOPS=24, BW=96.4KiB/s (98.7kB/s)(5940KiB/61601msec) clat (usec): min=71, max=167655, avg=21084.66, stdev=13594.91 lat (usec): min=71, max=167655, avg=21085.77, stdev=13594.52 clat percentiles (usec): | 1.00th=[ 297], 5.00th=[ 1029], 10.00th=[ 7898], 20.00th=[ 10945], | 30.00th=[ 13698], 40.00th=[ 15926], 50.00th=[ 18482], 60.00th=[ 21627], | 70.00th=[ 25822], 80.00th=[ 30016], 90.00th=[ 36439], 95.00th=[ 43779], | 99.00th=[ 64750], 99.50th=[ 78119], 99.90th=[121111], 99.95th=[166724], | 99.99th=[166724] bw ( KiB/s): min= 6, max= 192, per=24.87%, avg=95.74, stdev=33.16, samples=122 iops : min= 1, max= 48, avg=23.52, stdev= 8.32, samples=122 write: IOPS=25, BW=103KiB/s (106kB/s)(6348KiB/61601msec); 0 zone resets clat (usec): min=639, max=659903, avg=18975.76, stdev=23443.06 lat (usec): min=640, max=659904, avg=18976.55, stdev=23443.14 clat percentiles (usec): | 1.00th=[ 709], 5.00th=[ 1778], 10.00th=[ 2868], 20.00th=[ 4817], | 30.00th=[ 6849], 40.00th=[ 10028], 50.00th=[ 13960], 60.00th=[ 18220], | 70.00th=[ 23462], 80.00th=[ 30540], 90.00th=[ 40109], 95.00th=[ 50594], | 99.00th=[ 73925], 99.50th=[ 84411], 99.90th=[193987], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 32, max= 232, per=25.52%, avg=102.83, stdev=37.38, samples=121 iops : min= 8, max= 58, avg=25.29, stdev= 9.36, samples=121 lat (usec) : 100=0.16%, 250=0.16%, 500=0.55%, 750=1.79%, 1000=1.27% lat (msec) : 2=1.66%, 4=4.82%, 10=18.33%, 20=31.32%, 50=35.61% lat (msec) : 100=4.10%, 250=0.20%, 750=0.03% cpu : usr=0.06%, sys=27.75%, ctx=9015, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159333: Tue Oct 21 23:23:04 2025 read: IOPS=24, BW=96.9KiB/s (99.2kB/s)(6028KiB/62238msec) clat (usec): min=80, max=166801, avg=20159.55, stdev=13818.01 lat (usec): min=80, max=166802, avg=20160.22, stdev=13818.02 clat percentiles (usec): | 1.00th=[ 306], 5.00th=[ 766], 10.00th=[ 7242], 20.00th=[ 10552], | 30.00th=[ 12911], 40.00th=[ 15008], 50.00th=[ 17433], 60.00th=[ 20317], | 70.00th=[ 23725], 80.00th=[ 28967], 90.00th=[ 35914], 95.00th=[ 43779], | 99.00th=[ 64226], 99.50th=[ 79168], 99.90th=[119014], 99.95th=[166724], | 99.99th=[166724] bw ( KiB/s): min= 31, max= 200, per=25.13%, avg=96.75, stdev=34.71, samples=122 iops : min= 7, max= 50, avg=23.77, stdev= 8.69, samples=122 write: IOPS=25, BW=101KiB/s (103kB/s)(6260KiB/62238msec); 0 zone resets clat (usec): min=671, max=727849, avg=20246.96, stdev=24683.00 lat (usec): min=672, max=727851, avg=20247.69, stdev=24683.01 clat percentiles (usec): | 1.00th=[ 701], 5.00th=[ 2147], 10.00th=[ 3261], 20.00th=[ 5080], | 30.00th=[ 7635], 40.00th=[ 11600], 50.00th=[ 15664], 60.00th=[ 20055], | 70.00th=[ 25560], 80.00th=[ 33424], 90.00th=[ 42730], 95.00th=[ 51119], | 99.00th=[ 68682], 99.50th=[ 85459], 99.90th=[204473], 99.95th=[725615], | 99.99th=[725615] bw ( KiB/s): min= 15, max= 216, per=24.87%, avg=100.21, stdev=33.70, samples=122 iops : min= 3, max= 54, avg=24.64, stdev= 8.51, samples=122 lat (usec) : 100=0.16%, 250=0.23%, 500=0.75%, 750=1.99%, 1000=0.98% lat (msec) : 2=1.20%, 4=5.01%, 10=17.06%, 20=32.26%, 50=36.20% lat (msec) : 100=3.84%, 250=0.29%, 750=0.03% cpu : usr=0.08%, sys=27.25%, ctx=8932, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159334: Tue Oct 21 23:23:04 2025 read: IOPS=24, BW=98.6KiB/s (101kB/s)(6052KiB/61407msec) clat (usec): min=68, max=91467, avg=19859.72, stdev=12414.76 lat (usec): min=69, max=91468, avg=19860.49, stdev=12414.76 clat percentiles (usec): | 1.00th=[ 241], 5.00th=[ 693], 10.00th=[ 7373], 20.00th=[10421], | 30.00th=[13042], 40.00th=[15008], 50.00th=[17433], 60.00th=[20317], | 70.00th=[24249], 80.00th=[28967], 90.00th=[35390], 95.00th=[41157], | 99.00th=[62129], 99.50th=[72877], 99.90th=[88605], 99.95th=[91751], | 99.99th=[91751] bw ( KiB/s): min= 47, max= 207, per=25.77%, avg=99.21, stdev=34.56, samples=121 iops : min= 11, max= 51, avg=24.38, stdev= 8.67, samples=121 write: IOPS=25, BW=102KiB/s (104kB/s)(6236KiB/61407msec); 0 zone resets clat (usec): min=635, max=692787, avg=20010.20, stdev=24849.81 lat (usec): min=635, max=692788, avg=20010.98, stdev=24849.79 clat percentiles (usec): | 1.00th=[ 725], 5.00th=[ 1958], 10.00th=[ 3032], 20.00th=[ 5014], | 30.00th=[ 7570], 40.00th=[ 11207], 50.00th=[ 15270], 60.00th=[ 19530], | 70.00th=[ 24249], 80.00th=[ 32113], 90.00th=[ 41157], 95.00th=[ 52167], | 99.00th=[ 80217], 99.50th=[107480], 99.90th=[196084], 99.95th=[692061], | 99.99th=[692061] bw ( KiB/s): min= 12, max= 224, per=25.13%, avg=101.29, stdev=36.02, samples=122 iops : min= 3, max= 56, avg=24.91, stdev= 9.07, samples=122 lat (usec) : 100=0.16%, 250=0.36%, 500=0.88%, 750=2.02%, 1000=0.98% lat (msec) : 2=1.30%, 4=4.79%, 10=16.93%, 20=32.71%, 50=35.81% lat (msec) : 100=3.78%, 250=0.26%, 750=0.03% cpu : usr=0.07%, sys=27.17%, ctx=8976, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=386KiB/s (395kB/s), 96.4KiB/s-98.9KiB/s (98.7kB/s-101kB/s), io=23.4MiB (24.6MB), run=60577-62238msec WRITE: bw=404KiB/s (414kB/s), 101KiB/s-104KiB/s (103kB/s-106kB/s), io=24.6MiB (25.7MB), run=60577-62238msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159326: Tue Oct 21 23:23:30 2025 read: IOPS=17, BW=68.4KiB/s (70.1kB/s)(5992KiB/87543msec) clat (msec): min=2, max=671, avg=17.34, stdev=25.38 lat (msec): min=2, max=671, avg=17.34, stdev=25.38 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 11], 60.00th=[ 15], | 70.00th=[ 20], 80.00th=[ 27], 90.00th=[ 39], 95.00th=[ 53], | 99.00th=[ 99], 99.50th=[ 123], 99.90th=[ 232], 99.95th=[ 667], | 99.99th=[ 667] bw ( KiB/s): min= 7, max= 200, per=25.37%, avg=68.75, stdev=40.92, samples=173 iops : min= 1, max= 50, avg=16.97, stdev=10.36, samples=173 write: IOPS=17, BW=71.9KiB/s (73.6kB/s)(6296KiB/87543msec); 0 zone resets clat (msec): min=12, max=219, avg=39.00, stdev=20.36 lat (msec): min=12, max=219, avg=39.00, stdev=20.36 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 24], | 30.00th=[ 27], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 43], 80.00th=[ 53], 90.00th=[ 66], 95.00th=[ 77], | 99.00th=[ 108], 99.50th=[ 144], 99.90th=[ 169], 99.95th=[ 220], | 99.99th=[ 220] bw ( KiB/s): min= 16, max= 152, per=25.39%, avg=72.12, stdev=36.88, samples=173 iops : min= 4, max= 38, avg=17.81, stdev= 9.36, samples=173 lat (msec) : 4=8.14%, 10=16.18%, 20=13.28%, 50=48.21%, 100=12.99% lat (msec) : 250=1.17%, 750=0.03% cpu : usr=0.05%, sys=7.92%, ctx=5978, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159327: Tue Oct 21 23:23:30 2025 read: IOPS=16, BW=67.2KiB/s (68.8kB/s)(5940KiB/88352msec) clat (msec): min=2, max=700, avg=17.54, stdev=25.66 lat (msec): min=2, max=700, avg=17.54, stdev=25.66 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 11], 60.00th=[ 15], | 70.00th=[ 20], 80.00th=[ 27], 90.00th=[ 40], 95.00th=[ 56], | 99.00th=[ 94], 99.50th=[ 101], 99.90th=[ 138], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 7, max= 200, per=24.93%, avg=67.55, stdev=42.23, samples=175 iops : min= 1, max= 50, avg=16.67, stdev=10.67, samples=175 write: IOPS=17, BW=71.8KiB/s (73.6kB/s)(6348KiB/88352msec); 0 zone resets clat (msec): min=12, max=236, avg=39.14, stdev=21.21 lat (msec): min=12, max=236, avg=39.14, stdev=21.21 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 20], 10.00th=[ 21], 20.00th=[ 24], | 30.00th=[ 27], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 44], 80.00th=[ 54], 90.00th=[ 66], 95.00th=[ 80], | 99.00th=[ 110], 99.50th=[ 134], 99.90th=[ 228], 99.95th=[ 236], | 99.99th=[ 236] bw ( KiB/s): min= 15, max= 176, per=25.41%, avg=72.17, stdev=37.84, samples=175 iops : min= 3, max= 44, avg=17.82, stdev= 9.61, samples=175 lat (msec) : 4=7.68%, 10=16.41%, 20=13.93%, 50=46.91%, 100=13.83% lat (msec) : 250=1.20%, 750=0.03% cpu : usr=0.05%, sys=7.97%, ctx=6065, majf=1, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159328: Tue Oct 21 23:23:30 2025 read: IOPS=17, BW=68.4KiB/s (70.1kB/s)(6028KiB/88077msec) clat (msec): min=2, max=183, avg=17.33, stdev=18.99 lat (msec): min=2, max=183, avg=17.33, stdev=18.99 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 11], 60.00th=[ 15], | 70.00th=[ 20], 80.00th=[ 28], 90.00th=[ 41], 95.00th=[ 54], | 99.00th=[ 94], 99.50th=[ 116], 99.90th=[ 142], 99.95th=[ 184], | 99.99th=[ 184] bw ( KiB/s): min= 7, max= 192, per=25.32%, avg=68.63, stdev=40.12, samples=174 iops : min= 1, max= 48, avg=16.94, stdev=10.17, samples=174 write: IOPS=17, BW=71.1KiB/s (72.8kB/s)(6260KiB/88077msec); 0 zone resets clat (msec): min=13, max=723, avg=39.47, stdev=26.66 lat (msec): min=13, max=723, avg=39.48, stdev=26.66 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 24], | 30.00th=[ 27], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 44], 80.00th=[ 53], 90.00th=[ 65], 95.00th=[ 77], | 99.00th=[ 117], 99.50th=[ 142], 99.90th=[ 226], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 16, max= 152, per=25.02%, avg=71.06, stdev=36.87, samples=174 iops : min= 4, max= 38, avg=17.55, stdev= 9.36, samples=174 lat (msec) : 4=7.16%, 10=16.83%, 20=14.03%, 50=47.62%, 100=13.22% lat (msec) : 250=1.11%, 750=0.03% cpu : usr=0.05%, sys=7.99%, ctx=6100, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159330: Tue Oct 21 23:23:30 2025 read: IOPS=17, BW=68.9KiB/s (70.6kB/s)(6052KiB/87838msec) clat (msec): min=2, max=207, avg=16.95, stdev=18.29 lat (msec): min=2, max=207, avg=16.95, stdev=18.29 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 11], 60.00th=[ 15], | 70.00th=[ 19], 80.00th=[ 28], 90.00th=[ 40], 95.00th=[ 53], | 99.00th=[ 80], 99.50th=[ 105], 99.90th=[ 148], 99.95th=[ 207], | 99.99th=[ 207] bw ( KiB/s): min= 7, max= 208, per=25.54%, avg=69.22, stdev=41.71, samples=174 iops : min= 1, max= 52, avg=17.09, stdev=10.55, samples=174 write: IOPS=17, BW=70.0KiB/s (72.7kB/s)(6236KiB/87838msec); 0 zone resets clat (msec): min=14, max=707, avg=39.78, stdev=27.10 lat (msec): min=14, max=707, avg=39.78, stdev=27.10 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 25], | 30.00th=[ 27], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 43], 80.00th=[ 53], 90.00th=[ 66], 95.00th=[ 79], | 99.00th=[ 123], 99.50th=[ 153], 99.90th=[ 239], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 16, max= 152, per=25.12%, avg=71.34, stdev=35.87, samples=174 iops : min= 4, max= 38, avg=17.61, stdev= 9.12, samples=174 lat (msec) : 4=7.19%, 10=17.02%, 20=14.39%, 50=47.17%, 100=12.96% lat (msec) : 250=1.24%, 750=0.03% cpu : usr=0.04%, sys=8.16%, ctx=6048, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=272KiB/s (278kB/s), 67.2KiB/s-68.9KiB/s (68.8kB/s-70.6kB/s), io=23.4MiB (24.6MB), run=87543-88352msec WRITE: bw=285KiB/s (291kB/s), 70.0KiB/s-71.9KiB/s (72.7kB/s-73.6kB/s), io=24.6MiB (25.7MB), run=87543-88352msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159360: Tue Oct 21 23:23:48 2025 read: IOPS=30, BW=489KiB/s (501kB/s)(5920KiB/12097msec) clat (usec): min=86, max=65841, avg=16601.60, stdev=11346.76 lat (usec): min=86, max=65842, avg=16602.18, stdev=11346.78 clat percentiles (usec): | 1.00th=[ 109], 5.00th=[ 375], 10.00th=[ 437], 20.00th=[ 8717], | 30.00th=[10945], 40.00th=[12911], 50.00th=[15139], 60.00th=[17695], | 70.00th=[20579], 80.00th=[24511], 90.00th=[31065], 95.00th=[38011], | 99.00th=[52691], 99.50th=[64750], 99.90th=[65799], 99.95th=[65799], | 99.99th=[65799] bw ( KiB/s): min= 256, max= 736, per=25.30%, avg=480.78, stdev=132.36, samples=23 iops : min= 16, max= 46, avg=29.52, stdev= 8.22, samples=23 write: IOPS=32, BW=526KiB/s (539kB/s)(6368KiB/12097msec); 0 zone resets clat (usec): min=915, max=70199, avg=14773.72, stdev=11951.34 lat (usec): min=915, max=70202, avg=14774.60, stdev=11951.41 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 2040], 10.00th=[ 3589], 20.00th=[ 4621], | 30.00th=[ 5407], 40.00th=[ 8455], 50.00th=[12125], 60.00th=[15008], | 70.00th=[19268], 80.00th=[23725], 90.00th=[31065], 95.00th=[36963], | 99.00th=[55837], 99.50th=[58983], 99.90th=[69731], 99.95th=[69731], | 99.99th=[69731] bw ( KiB/s): min= 254, max= 736, per=26.30%, avg=525.78, stdev=165.43, samples=23 iops : min= 15, max= 46, avg=32.35, stdev=10.41, samples=23 lat (usec) : 100=0.39%, 250=0.39%, 500=4.82%, 750=0.91%, 1000=0.78% lat (msec) : 2=2.34%, 4=4.56%, 10=20.18%, 20=35.29%, 50=28.78% lat (msec) : 100=1.56% cpu : usr=0.07%, sys=32.09%, ctx=2257, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159361: Tue Oct 21 23:23:48 2025 read: IOPS=28, BW=457KiB/s (468kB/s)(5744KiB/12579msec) clat (usec): min=84, max=87400, avg=17868.75, stdev=12378.33 lat (usec): min=84, max=87401, avg=17873.72, stdev=12385.73 clat percentiles (usec): | 1.00th=[ 247], 5.00th=[ 388], 10.00th=[ 465], 20.00th=[ 9765], | 30.00th=[11338], 40.00th=[13566], 50.00th=[16188], 60.00th=[19006], | 70.00th=[21627], 80.00th=[25560], 90.00th=[32113], 95.00th=[40633], | 99.00th=[65799], 99.50th=[76022], 99.90th=[87557], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 191, max= 704, per=23.54%, avg=447.25, stdev=146.00, samples=24 iops : min= 11, max= 44, avg=27.46, stdev= 9.13, samples=24 write: IOPS=32, BW=520KiB/s (533kB/s)(6544KiB/12579msec); 0 zone resets clat (usec): min=810, max=84684, avg=14877.99, stdev=12355.28 lat (usec): min=811, max=84685, avg=14879.37, stdev=12354.87 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 2900], 10.00th=[ 3916], 20.00th=[ 4752], | 30.00th=[ 6128], 40.00th=[ 9241], 50.00th=[12125], 60.00th=[14877], | 70.00th=[18744], 80.00th=[22152], 90.00th=[28967], 95.00th=[36439], | 99.00th=[58459], 99.50th=[65799], 99.90th=[84411], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 224, max= 896, per=25.11%, avg=501.92, stdev=186.83, samples=24 iops : min= 14, max= 56, avg=30.88, stdev=11.74, samples=24 lat (usec) : 100=0.13%, 250=0.39%, 500=4.30%, 750=0.52%, 1000=0.39% lat (msec) : 2=1.30%, 4=3.91%, 10=21.48%, 20=37.63%, 50=27.73% lat (msec) : 100=2.21% cpu : usr=0.07%, sys=32.00%, ctx=2397, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159362: Tue Oct 21 23:23:48 2025 read: IOPS=31, BW=496KiB/s (508kB/s)(6256KiB/12604msec) clat (usec): min=101, max=63227, avg=17359.77, stdev=10980.98 lat (usec): min=101, max=63227, avg=17360.36, stdev=10981.02 clat percentiles (usec): | 1.00th=[ 231], 5.00th=[ 355], 10.00th=[ 441], 20.00th=[ 9896], | 30.00th=[11731], 40.00th=[13042], 50.00th=[16057], 60.00th=[19006], | 70.00th=[21627], 80.00th=[25560], 90.00th=[32375], 95.00th=[36963], | 99.00th=[50594], 99.50th=[59507], 99.90th=[63177], 99.95th=[63177], | 99.99th=[63177] bw ( KiB/s): min= 224, max= 958, per=25.69%, avg=488.08, stdev=190.86, samples=24 iops : min= 14, max= 59, avg=30.04, stdev=11.90, samples=24 write: IOPS=29, BW=479KiB/s (490kB/s)(6032KiB/12604msec); 0 zone resets clat (usec): min=927, max=122513, avg=15366.45, stdev=13554.33 lat (usec): min=928, max=122514, avg=15367.46, stdev=13554.28 clat percentiles (usec): | 1.00th=[ 1012], 5.00th=[ 3195], 10.00th=[ 3982], 20.00th=[ 4752], | 30.00th=[ 5407], 40.00th=[ 8094], 50.00th=[ 11863], 60.00th=[ 14353], | 70.00th=[ 19268], 80.00th=[ 23987], 90.00th=[ 32113], 95.00th=[ 42206], | 99.00th=[ 58459], 99.50th=[ 63701], 99.90th=[122160], 99.95th=[122160], | 99.99th=[122160] bw ( KiB/s): min= 127, max= 800, per=22.75%, avg=454.71, stdev=166.99, samples=24 iops : min= 7, max= 50, avg=27.92, stdev=10.50, samples=24 lat (usec) : 250=1.04%, 500=4.82%, 750=0.13%, 1000=0.52% lat (msec) : 2=1.04%, 4=3.52%, 10=21.48%, 20=34.77%, 50=31.12% lat (msec) : 100=1.43%, 250=0.13% cpu : usr=0.06%, sys=31.45%, ctx=2430, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159363: Tue Oct 21 23:23:48 2025 read: IOPS=30, BW=487KiB/s (498kB/s)(6032KiB/12397msec) clat (usec): min=121, max=60984, avg=16083.39, stdev=9673.19 lat (usec): min=121, max=60984, avg=16084.02, stdev=9673.20 clat percentiles (usec): | 1.00th=[ 293], 5.00th=[ 392], 10.00th=[ 482], 20.00th=[ 9765], | 30.00th=[11600], 40.00th=[13173], 50.00th=[15401], 60.00th=[17695], | 70.00th=[20055], 80.00th=[22414], 90.00th=[27657], 95.00th=[33424], | 99.00th=[46400], 99.50th=[53740], 99.90th=[61080], 99.95th=[61080], | 99.99th=[61080] bw ( KiB/s): min= 192, max= 800, per=25.28%, avg=480.38, stdev=175.92, samples=24 iops : min= 12, max= 50, avg=29.54, stdev=11.06, samples=24 write: IOPS=31, BW=505KiB/s (517kB/s)(6256KiB/12397msec); 0 zone resets clat (usec): min=881, max=119937, avg=16072.48, stdev=14578.63 lat (usec): min=881, max=119938, avg=16073.34, stdev=14578.65 clat percentiles (usec): | 1.00th=[ 1303], 5.00th=[ 3490], 10.00th=[ 4178], 20.00th=[ 4883], | 30.00th=[ 6194], 40.00th=[ 8586], 50.00th=[ 11731], 60.00th=[ 14746], | 70.00th=[ 19792], 80.00th=[ 24773], 90.00th=[ 33817], 95.00th=[ 44303], | 99.00th=[ 70779], 99.50th=[ 95945], 99.90th=[120062], 99.95th=[120062], | 99.99th=[120062] bw ( KiB/s): min= 223, max= 733, per=25.01%, avg=499.96, stdev=160.89, samples=24 iops : min= 13, max= 45, avg=30.75, stdev=10.02, samples=24 lat (usec) : 250=0.39%, 500=4.56%, 750=0.52%, 1000=0.39% lat (msec) : 2=1.17%, 4=3.12%, 10=23.05%, 20=36.72%, 50=27.99% lat (msec) : 100=1.95%, 250=0.13% cpu : usr=0.05%, sys=31.74%, ctx=2428, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1900KiB/s (1946kB/s), 457KiB/s-496KiB/s (468kB/s-508kB/s), io=23.4MiB (24.5MB), run=12097-12604msec WRITE: bw=1999KiB/s (2047kB/s), 479KiB/s-526KiB/s (490kB/s-539kB/s), io=24.6MiB (25.8MB), run=12097-12604msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159365: Tue Oct 21 23:23:55 2025 read: IOPS=18, BW=298KiB/s (305kB/s)(5920KiB/19852msec) clat (msec): min=2, max=115, avg=14.56, stdev=15.78 lat (msec): min=2, max=115, avg=14.56, stdev=15.78 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 5], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 12], | 70.00th=[ 16], 80.00th=[ 22], 90.00th=[ 33], 95.00th=[ 47], | 99.00th=[ 75], 99.50th=[ 100], 99.90th=[ 116], 99.95th=[ 116], | 99.99th=[ 116] bw ( KiB/s): min= 94, max= 640, per=24.25%, avg=292.51, stdev=150.00, samples=39 iops : min= 5, max= 40, avg=17.97, stdev= 9.64, samples=39 write: IOPS=20, BW=321KiB/s (328kB/s)(6368KiB/19852msec); 0 zone resets clat (msec): min=15, max=124, avg=36.08, stdev=17.30 lat (msec): min=15, max=124, avg=36.08, stdev=17.30 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 19], 10.00th=[ 21], 20.00th=[ 24], | 30.00th=[ 26], 40.00th=[ 29], 50.00th=[ 31], 60.00th=[ 35], | 70.00th=[ 39], 80.00th=[ 48], 90.00th=[ 59], 95.00th=[ 75], | 99.00th=[ 94], 99.50th=[ 109], 99.90th=[ 125], 99.95th=[ 125], | 99.99th=[ 125] bw ( KiB/s): min= 154, max= 576, per=25.10%, avg=318.54, stdev=147.41, samples=39 iops : min= 9, max= 36, avg=19.59, stdev= 9.44, samples=39 lat (msec) : 4=7.42%, 10=19.53%, 20=14.84%, 50=47.66%, 100=10.16% lat (msec) : 250=0.39% cpu : usr=0.05%, sys=8.62%, ctx=1533, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159366: Tue Oct 21 23:23:55 2025 read: IOPS=18, BW=291KiB/s (297kB/s)(5744KiB/19772msec) clat (msec): min=2, max=126, avg=14.23, stdev=16.65 lat (msec): min=2, max=126, avg=14.23, stdev=16.65 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 5], 40.00th=[ 6], 50.00th=[ 8], 60.00th=[ 11], | 70.00th=[ 14], 80.00th=[ 20], 90.00th=[ 36], 95.00th=[ 52], | 99.00th=[ 71], 99.50th=[ 115], 99.90th=[ 127], 99.95th=[ 127], | 99.99th=[ 127] bw ( KiB/s): min= 51, max= 704, per=24.09%, avg=290.49, stdev=168.89, samples=39 iops : min= 3, max= 44, avg=17.85, stdev=10.76, samples=39 write: IOPS=20, BW=331KiB/s (339kB/s)(6544KiB/19772msec); 0 zone resets clat (msec): min=14, max=151, avg=35.55, stdev=16.34 lat (msec): min=14, max=151, avg=35.55, stdev=16.34 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 24], | 30.00th=[ 26], 40.00th=[ 28], 50.00th=[ 31], 60.00th=[ 35], | 70.00th=[ 40], 80.00th=[ 47], 90.00th=[ 59], 95.00th=[ 68], | 99.00th=[ 85], 99.50th=[ 95], 99.90th=[ 153], 99.95th=[ 153], | 99.99th=[ 153] bw ( KiB/s): min= 128, max= 608, per=25.99%, avg=329.87, stdev=144.04, samples=39 iops : min= 8, max= 38, avg=20.31, stdev= 9.24, samples=39 lat (msec) : 4=7.94%, 10=19.53%, 20=13.41%, 50=48.57%, 100=10.16% lat (msec) : 250=0.39% cpu : usr=0.08%, sys=8.31%, ctx=1460, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159367: Tue Oct 21 23:23:55 2025 read: IOPS=20, BW=320KiB/s (328kB/s)(6256KiB/19535msec) clat (usec): min=2829, max=88793, avg=14019.13, stdev=14665.15 lat (usec): min=2829, max=88793, avg=14019.92, stdev=14665.08 clat percentiles (usec): | 1.00th=[ 2868], 5.00th=[ 3326], 10.00th=[ 3654], 20.00th=[ 4555], | 30.00th=[ 5211], 40.00th=[ 6325], 50.00th=[ 7767], 60.00th=[10945], | 70.00th=[14091], 80.00th=[20579], 90.00th=[32637], 95.00th=[45876], | 99.00th=[74974], 99.50th=[77071], 99.90th=[88605], 99.95th=[88605], | 99.99th=[88605] bw ( KiB/s): min= 95, max= 928, per=26.69%, avg=321.89, stdev=199.72, samples=38 iops : min= 5, max= 58, avg=19.79, stdev=12.71, samples=38 write: IOPS=19, BW=309KiB/s (316kB/s)(6032KiB/19535msec); 0 zone resets clat (msec): min=15, max=151, avg=37.06, stdev=18.06 lat (msec): min=15, max=151, avg=37.06, stdev=18.06 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 24], | 30.00th=[ 27], 40.00th=[ 30], 50.00th=[ 32], 60.00th=[ 35], | 70.00th=[ 40], 80.00th=[ 46], 90.00th=[ 62], 95.00th=[ 73], | 99.00th=[ 102], 99.50th=[ 131], 99.90th=[ 153], 99.95th=[ 153], | 99.99th=[ 153] bw ( KiB/s): min= 126, max= 544, per=24.08%, avg=305.55, stdev=137.17, samples=38 iops : min= 7, max= 34, avg=18.76, stdev= 8.80, samples=38 lat (msec) : 4=6.64%, 10=22.14%, 20=14.97%, 50=45.83%, 100=9.77% lat (msec) : 250=0.65% cpu : usr=0.04%, sys=8.99%, ctx=1481, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159368: Tue Oct 21 23:23:55 2025 read: IOPS=19, BW=306KiB/s (313kB/s)(6032KiB/19729msec) clat (msec): min=2, max=125, avg=15.11, stdev=16.80 lat (msec): min=2, max=125, avg=15.11, stdev=16.80 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 6], 50.00th=[ 8], 60.00th=[ 12], | 70.00th=[ 17], 80.00th=[ 23], 90.00th=[ 36], 95.00th=[ 53], | 99.00th=[ 83], 99.50th=[ 97], 99.90th=[ 127], 99.95th=[ 127], | 99.99th=[ 127] bw ( KiB/s): min= 95, max= 672, per=25.22%, avg=304.10, stdev=163.36, samples=39 iops : min= 5, max= 42, avg=18.69, stdev=10.43, samples=39 write: IOPS=19, BW=317KiB/s (325kB/s)(6256KiB/19729msec); 0 zone resets clat (msec): min=14, max=131, avg=35.74, stdev=18.32 lat (msec): min=14, max=131, avg=35.74, stdev=18.32 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 24], | 30.00th=[ 25], 40.00th=[ 28], 50.00th=[ 30], 60.00th=[ 33], | 70.00th=[ 37], 80.00th=[ 44], 90.00th=[ 62], 95.00th=[ 77], | 99.00th=[ 107], 99.50th=[ 120], 99.90th=[ 132], 99.95th=[ 132], | 99.99th=[ 132] bw ( KiB/s): min= 63, max= 608, per=25.03%, avg=317.59, stdev=162.40, samples=39 iops : min= 3, max= 38, avg=19.54, stdev=10.37, samples=39 lat (msec) : 4=6.77%, 10=20.70%, 20=12.89%, 50=49.22%, 100=9.64% lat (msec) : 250=0.78% cpu : usr=0.04%, sys=8.88%, ctx=1544, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1207KiB/s (1235kB/s), 291KiB/s-320KiB/s (297kB/s-328kB/s), io=23.4MiB (24.5MB), run=19535-19852msec WRITE: bw=1269KiB/s (1300kB/s), 309KiB/s-331KiB/s (316kB/s-339kB/s), io=24.6MiB (25.8MB), run=19535-19852msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159376: Tue Oct 21 23:24:04 2025 read: IOPS=2, BW=2187KiB/s (2239kB/s)(7168KiB/3278msec) clat (usec): min=677, max=1937.4k, avg=359019.95, stdev=699911.55 lat (usec): min=678, max=1937.4k, avg=359020.84, stdev=699911.73 clat percentiles (usec): | 1.00th=[ 676], 5.00th=[ 676], 10.00th=[ 676], | 20.00th=[ 758], 30.00th=[ 106431], 40.00th=[ 106431], | 50.00th=[ 113771], 60.00th=[ 152044], 70.00th=[ 152044], | 80.00th=[ 200279], 90.00th=[1937769], 95.00th=[1937769], | 99.00th=[1937769], 99.50th=[1937769], 99.90th=[1937769], | 99.95th=[1937769], 99.99th=[1937769] bw ( KiB/s): min= 2015, max= 6144, per=65.86%, avg=3848.00, stdev=2103.08, samples=3 iops : min= 1, max= 6, avg= 3.33, stdev= 2.52, samples=3 write: IOPS=1, BW=1562KiB/s (1599kB/s)(5120KiB/3278msec); 0 zone resets clat (msec): min=89, max=197, avg=138.75, stdev=40.43 lat (msec): min=89, max=197, avg=138.80, stdev=40.42 clat percentiles (msec): | 1.00th=[ 89], 5.00th=[ 89], 10.00th=[ 89], 20.00th=[ 89], | 30.00th=[ 124], 40.00th=[ 124], 50.00th=[ 129], 60.00th=[ 129], | 70.00th=[ 157], 80.00th=[ 157], 90.00th=[ 199], 95.00th=[ 199], | 99.00th=[ 199], 99.50th=[ 199], 99.90th=[ 199], 99.95th=[ 199], | 99.99th=[ 199] bw ( KiB/s): min= 2048, max= 6047, per=49.48%, avg=4047.50, stdev=2827.72, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 lat (usec) : 750=8.33%, 1000=8.33% lat (msec) : 100=8.33%, 250=66.67%, 2000=8.33% cpu : usr=0.03%, sys=34.38%, ctx=1326, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159377: Tue Oct 21 23:24:04 2025 read: IOPS=0, BW=584KiB/s (598kB/s)(2048KiB/3505msec) clat (msec): min=136, max=175, avg=156.20, stdev=27.95 lat (msec): min=136, max=175, avg=156.20, stdev=27.95 clat percentiles (msec): | 1.00th=[ 138], 5.00th=[ 138], 10.00th=[ 138], 20.00th=[ 138], | 30.00th=[ 138], 40.00th=[ 138], 50.00th=[ 138], 60.00th=[ 176], | 70.00th=[ 176], 80.00th=[ 176], 90.00th=[ 176], 95.00th=[ 176], | 99.00th=[ 176], 99.50th=[ 176], 99.90th=[ 176], 99.95th=[ 176], | 99.99th=[ 176] bw ( KiB/s): min= 1689, max= 2048, per=31.98%, avg=1868.50, stdev=253.85, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2922KiB/s (2992kB/s)(10.0MiB/3505msec); 0 zone resets clat (msec): min=72, max=1227, avg=311.96, stdev=340.19 lat (msec): min=72, max=1227, avg=312.02, stdev=340.20 clat percentiles (msec): | 1.00th=[ 73], 5.00th=[ 73], 10.00th=[ 73], 20.00th=[ 104], | 30.00th=[ 110], 40.00th=[ 132], 50.00th=[ 192], 60.00th=[ 228], | 70.00th=[ 326], 80.00th=[ 334], 90.00th=[ 397], 95.00th=[ 1234], | 99.00th=[ 1234], 99.50th=[ 1234], 99.90th=[ 1234], 99.95th=[ 1234], | 99.99th=[ 1234] bw ( KiB/s): min= 4031, max= 6144, per=65.81%, avg=5383.67, stdev=1174.45, samples=3 iops : min= 3, max= 6, avg= 4.67, stdev= 1.53, samples=3 lat (msec) : 100=8.33%, 250=58.33%, 500=25.00%, 2000=8.33% cpu : usr=0.00%, sys=37.27%, ctx=548, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159378: Tue Oct 21 23:24:04 2025 read: IOPS=2, BW=2198KiB/s (2251kB/s)(6144KiB/2795msec) clat (usec): min=674, max=264695, avg=109373.74, stdev=99122.68 lat (usec): min=675, max=264695, avg=109374.55, stdev=99122.47 clat percentiles (usec): | 1.00th=[ 676], 5.00th=[ 676], 10.00th=[ 676], 20.00th=[ 783], | 30.00th=[ 783], 40.00th=[121111], 50.00th=[121111], 60.00th=[133694], | 70.00th=[135267], 80.00th=[135267], 90.00th=[265290], 95.00th=[265290], | 99.00th=[265290], 99.50th=[265290], 99.90th=[265290], 99.95th=[265290], | 99.99th=[265290] bw ( KiB/s): min= 1992, max= 6047, per=68.79%, avg=4019.50, stdev=2867.32, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=2, BW=2198KiB/s (2251kB/s)(6144KiB/2795msec); 0 zone resets clat (msec): min=61, max=1223, avg=354.87, stdev=439.25 lat (msec): min=61, max=1223, avg=354.92, stdev=439.25 clat percentiles (msec): | 1.00th=[ 62], 5.00th=[ 62], 10.00th=[ 62], 20.00th=[ 129], | 30.00th=[ 129], 40.00th=[ 163], 50.00th=[ 163], 60.00th=[ 167], | 70.00th=[ 388], 80.00th=[ 388], 90.00th=[ 1217], 95.00th=[ 1217], | 99.00th=[ 1217], 99.50th=[ 1217], 99.90th=[ 1217], 99.95th=[ 1217], | 99.99th=[ 1217] bw ( KiB/s): min= 2048, max= 4031, per=38.59%, avg=3156.33, stdev=1011.94, samples=3 iops : min= 2, max= 3, avg= 2.67, stdev= 0.58, samples=3 lat (usec) : 750=8.33%, 1000=8.33% lat (msec) : 100=8.33%, 250=50.00%, 500=16.67%, 2000=8.33% cpu : usr=0.00%, sys=35.08%, ctx=560, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159379: Tue Oct 21 23:24:04 2025 read: IOPS=1, BW=1665KiB/s (1705kB/s)(5120KiB/3075msec) clat (msec): min=14, max=210, avg=113.43, stdev=83.38 lat (msec): min=14, max=210, avg=113.43, stdev=83.38 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 15], 10.00th=[ 15], 20.00th=[ 15], | 30.00th=[ 39], 40.00th=[ 39], 50.00th=[ 146], 60.00th=[ 146], | 70.00th=[ 159], 80.00th=[ 159], 90.00th=[ 211], 95.00th=[ 211], | 99.00th=[ 211], 99.50th=[ 211], 99.90th=[ 211], 99.95th=[ 211], | 99.99th=[ 211] bw ( KiB/s): min= 2048, max= 5976, per=68.66%, avg=4012.00, stdev=2777.52, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 write: IOPS=2, BW=2331KiB/s (2387kB/s)(7168KiB/3075msec); 0 zone resets clat (msec): min=140, max=990, avg=347.34, stdev=313.52 lat (msec): min=140, max=990, avg=347.42, stdev=313.51 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 163], | 30.00th=[ 165], 40.00th=[ 165], 50.00th=[ 201], 60.00th=[ 241], | 70.00th=[ 241], 80.00th=[ 531], 90.00th=[ 995], 95.00th=[ 995], | 99.00th=[ 995], 99.50th=[ 995], 99.90th=[ 995], 99.95th=[ 995], | 99.99th=[ 995] bw ( KiB/s): min= 1695, max= 6047, per=47.32%, avg=3871.00, stdev=3077.33, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 lat (msec) : 20=8.33%, 50=8.33%, 250=66.67%, 750=8.33%, 1000=8.33% cpu : usr=0.00%, sys=34.87%, ctx=882, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5843KiB/s (5983kB/s), 584KiB/s-2198KiB/s (598kB/s-2251kB/s), io=20.0MiB (20.0MB), run=2795-3505msec WRITE: bw=8180KiB/s (8377kB/s), 1562KiB/s-2922KiB/s (1599kB/s-2992kB/s), io=28.0MiB (29.4MB), run=2795-3505msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159381: Tue Oct 21 23:24:05 2025 read: IOPS=1, BW=1993KiB/s (2041kB/s)(7168KiB/3596msec) clat (msec): min=30, max=1430, avg=392.11, stdev=507.98 lat (msec): min=30, max=1430, avg=392.11, stdev=507.98 clat percentiles (msec): | 1.00th=[ 31], 5.00th=[ 31], 10.00th=[ 31], 20.00th=[ 41], | 30.00th=[ 54], 40.00th=[ 54], 50.00th=[ 159], 60.00th=[ 426], | 70.00th=[ 426], 80.00th=[ 609], 90.00th=[ 1435], 95.00th=[ 1435], | 99.00th=[ 1435], 99.50th=[ 1435], 99.90th=[ 1435], 99.95th=[ 1435], | 99.99th=[ 1435] bw ( KiB/s): min= 1686, max= 4096, per=43.36%, avg=2469.50, stdev=1097.68, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.26, samples=4 write: IOPS=1, BW=1424KiB/s (1458kB/s)(5120KiB/3596msec); 0 zone resets clat (msec): min=73, max=193, avg=143.22, stdev=59.60 lat (msec): min=73, max=193, avg=143.27, stdev=59.59 clat percentiles (msec): | 1.00th=[ 74], 5.00th=[ 74], 10.00th=[ 74], 20.00th=[ 74], | 30.00th=[ 84], 40.00th=[ 84], 50.00th=[ 176], 60.00th=[ 176], | 70.00th=[ 190], 80.00th=[ 190], 90.00th=[ 194], 95.00th=[ 194], | 99.00th=[ 194], 99.50th=[ 194], 99.90th=[ 194], 99.95th=[ 194], | 99.99th=[ 194] bw ( KiB/s): min= 2048, max= 2048, per=25.69%, avg=2048.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 50=16.67%, 100=25.00%, 250=33.33%, 500=8.33%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=10.60%, ctx=102, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159382: Tue Oct 21 23:24:05 2025 read: IOPS=0, BW=572KiB/s (585kB/s)(2048KiB/3582msec) clat (msec): min=44, max=352, avg=198.62, stdev=217.53 lat (msec): min=44, max=352, avg=198.62, stdev=217.53 clat percentiles (msec): | 1.00th=[ 45], 5.00th=[ 45], 10.00th=[ 45], 20.00th=[ 45], | 30.00th=[ 45], 40.00th=[ 45], 50.00th=[ 45], 60.00th=[ 355], | 70.00th=[ 355], 80.00th=[ 355], 90.00th=[ 355], 95.00th=[ 355], | 99.00th=[ 355], 99.50th=[ 355], 99.90th=[ 355], 99.95th=[ 355], | 99.99th=[ 355] bw ( KiB/s): min= 1684, max= 2048, per=32.77%, avg=1866.00, stdev=257.39, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2859KiB/s (2927kB/s)(10.0MiB/3582msec); 0 zone resets clat (msec): min=47, max=1285, avg=310.32, stdev=373.66 lat (msec): min=48, max=1285, avg=310.37, stdev=373.66 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 48], 10.00th=[ 48], 20.00th=[ 63], | 30.00th=[ 122], 40.00th=[ 148], 50.00th=[ 157], 60.00th=[ 180], | 70.00th=[ 192], 80.00th=[ 368], 90.00th=[ 550], 95.00th=[ 1284], | 99.00th=[ 1284], 99.50th=[ 1284], 99.90th=[ 1284], 99.95th=[ 1284], | 99.99th=[ 1284] bw ( KiB/s): min= 2048, max= 8192, per=51.35%, avg=4093.75, stdev=2896.31, samples=4 iops : min= 2, max= 8, avg= 3.75, stdev= 2.87, samples=4 lat (msec) : 50=16.67%, 100=8.33%, 250=41.67%, 500=16.67%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=15.33%, ctx=622, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159383: Tue Oct 21 23:24:05 2025 read: IOPS=1, BW=1795KiB/s (1838kB/s)(6144KiB/3423msec) clat (msec): min=58, max=236, avg=120.58, stdev=65.72 lat (msec): min=58, max=236, avg=120.58, stdev=65.72 clat percentiles (msec): | 1.00th=[ 59], 5.00th=[ 59], 10.00th=[ 59], 20.00th=[ 83], | 30.00th=[ 83], 40.00th=[ 85], 50.00th=[ 85], 60.00th=[ 106], | 70.00th=[ 157], 80.00th=[ 157], 90.00th=[ 236], 95.00th=[ 236], | 99.00th=[ 236], 99.50th=[ 236], 99.90th=[ 236], 99.95th=[ 236], | 99.99th=[ 236] bw ( KiB/s): min= 4087, max= 4096, per=71.84%, avg=4091.50, stdev= 6.36, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1795KiB/s (1838kB/s)(6144KiB/3423msec); 0 zone resets clat (msec): min=70, max=1203, avg=429.58, stdev=421.69 lat (msec): min=70, max=1204, avg=429.64, stdev=421.69 clat percentiles (msec): | 1.00th=[ 71], 5.00th=[ 71], 10.00th=[ 71], 20.00th=[ 148], | 30.00th=[ 148], 40.00th=[ 236], 50.00th=[ 236], 60.00th=[ 317], | 70.00th=[ 609], 80.00th=[ 609], 90.00th=[ 1200], 95.00th=[ 1200], | 99.00th=[ 1200], 99.50th=[ 1200], 99.90th=[ 1200], 99.95th=[ 1200], | 99.99th=[ 1200] bw ( KiB/s): min= 2048, max= 4087, per=39.94%, avg=3184.67, stdev=1039.50, samples=3 iops : min= 2, max= 3, avg= 2.67, stdev= 0.58, samples=3 lat (msec) : 100=33.33%, 250=41.67%, 500=8.33%, 750=8.33%, 2000=8.33% cpu : usr=0.03%, sys=11.31%, ctx=132, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159384: Tue Oct 21 23:24:05 2025 read: IOPS=1, BW=1485KiB/s (1521kB/s)(5120KiB/3448msec) clat (msec): min=51, max=113, avg=90.61, stdev=24.07 lat (msec): min=51, max=113, avg=90.61, stdev=24.07 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 52], 10.00th=[ 52], 20.00th=[ 52], | 30.00th=[ 85], 40.00th=[ 85], 50.00th=[ 99], 60.00th=[ 99], | 70.00th=[ 106], 80.00th=[ 106], 90.00th=[ 113], 95.00th=[ 113], | 99.00th=[ 113], 99.50th=[ 113], 99.90th=[ 113], 99.95th=[ 113], | 99.99th=[ 113] bw ( KiB/s): min= 4087, max= 4096, per=71.84%, avg=4091.50, stdev= 6.36, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2079KiB/s (2129kB/s)(7168KiB/3448msec); 0 zone resets clat (msec): min=149, max=1292, avg=407.46, stdev=410.74 lat (msec): min=149, max=1292, avg=407.51, stdev=410.74 clat percentiles (msec): | 1.00th=[ 150], 5.00th=[ 150], 10.00th=[ 150], 20.00th=[ 186], | 30.00th=[ 188], 40.00th=[ 188], 50.00th=[ 213], 60.00th=[ 296], | 70.00th=[ 296], 80.00th=[ 531], 90.00th=[ 1301], 95.00th=[ 1301], | 99.00th=[ 1301], 99.50th=[ 1301], 99.90th=[ 1301], 99.95th=[ 1301], | 99.99th=[ 1301] bw ( KiB/s): min= 1709, max= 4087, per=31.02%, avg=2473.00, stdev=1087.80, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (msec) : 100=25.00%, 250=50.00%, 500=8.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=14.01%, ctx=305, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5695KiB/s (5832kB/s), 572KiB/s-1993KiB/s (585kB/s-2041kB/s), io=20.0MiB (20.0MB), run=3423-3596msec WRITE: bw=7973KiB/s (8165kB/s), 1424KiB/s-2859KiB/s (1458kB/s-2927kB/s), io=28.0MiB (29.4MB), run=3423-3596msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159395: Tue Oct 21 23:24:13 2025 read: IOPS=2, BW=9069KiB/s (9286kB/s)(12.0MiB/1355msec) clat (msec): min=394, max=487, avg=440.73, stdev=46.44 lat (msec): min=394, max=487, avg=440.73, stdev=46.44 clat percentiles (msec): | 1.00th=[ 393], 5.00th=[ 393], 10.00th=[ 393], 20.00th=[ 393], | 30.00th=[ 393], 40.00th=[ 443], 50.00th=[ 443], 60.00th=[ 443], | 70.00th=[ 489], 80.00th=[ 489], 90.00th=[ 489], 95.00th=[ 489], | 99.00th=[ 489], 99.50th=[ 489], 99.90th=[ 489], 99.95th=[ 489], | 99.99th=[ 489] bw ( KiB/s): min= 5557, max= 8192, per=56.86%, avg=6874.50, stdev=1863.23, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=100.00% cpu : usr=0.00%, sys=47.19%, ctx=2441, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159396: Tue Oct 21 23:24:13 2025 read: IOPS=0, BW=3101KiB/s (3175kB/s)(4096KiB/1321msec) clat (nsec): min=531271k, max=531271k, avg=531271128.00, stdev= 0.00 lat (nsec): min=531273k, max=531273k, avg=531273142.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 531], 5.00th=[ 531], 10.00th=[ 531], 20.00th=[ 531], | 30.00th=[ 531], 40.00th=[ 531], 50.00th=[ 531], 60.00th=[ 531], | 70.00th=[ 531], 80.00th=[ 531], 90.00th=[ 531], 95.00th=[ 531], | 99.00th=[ 531], 99.50th=[ 531], 99.90th=[ 531], 99.95th=[ 531], | 99.99th=[ 531] bw ( KiB/s): min= 8143, max= 8143, per=67.35%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=6201KiB/s (6350kB/s)(8192KiB/1321msec); 0 zone resets clat (msec): min=377, max=390, avg=384.04, stdev= 9.23 lat (msec): min=377, max=390, avg=384.29, stdev= 9.26 clat percentiles (msec): | 1.00th=[ 380], 5.00th=[ 380], 10.00th=[ 380], 20.00th=[ 380], | 30.00th=[ 380], 40.00th=[ 380], 50.00th=[ 380], 60.00th=[ 393], | 70.00th=[ 393], 80.00th=[ 393], 90.00th=[ 393], 95.00th=[ 393], | 99.00th=[ 393], 99.50th=[ 393], 99.90th=[ 393], 99.95th=[ 393], | 99.99th=[ 393] bw ( KiB/s): min= 8143, max= 8143, per=45.30%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.00%, sys=36.97%, ctx=817, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159397: Tue Oct 21 23:24:13 2025 write: IOPS=1, BW=7288KiB/s (7463kB/s)(12.0MiB/1686msec); 0 zone resets clat (msec): min=208, max=1039, avg=554.35, stdev=432.92 lat (msec): min=208, max=1040, avg=554.62, stdev=432.94 clat percentiles (msec): | 1.00th=[ 209], 5.00th=[ 209], 10.00th=[ 209], 20.00th=[ 209], | 30.00th=[ 209], 40.00th=[ 414], 50.00th=[ 414], 60.00th=[ 414], | 70.00th=[ 1036], 80.00th=[ 1036], 90.00th=[ 1036], 95.00th=[ 1036], | 99.00th=[ 1036], 99.50th=[ 1036], 99.90th=[ 1036], 99.95th=[ 1036], | 99.99th=[ 1036] bw ( KiB/s): min= 8192, max= 8192, per=45.58%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=16.85%, ctx=173, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159398: Tue Oct 21 23:24:13 2025 write: IOPS=1, BW=6741KiB/s (6902kB/s)(12.0MiB/1823msec); 0 zone resets clat (msec): min=303, max=1062, avg=598.82, stdev=406.43 lat (msec): min=303, max=1062, avg=599.09, stdev=406.42 clat percentiles (msec): | 1.00th=[ 305], 5.00th=[ 305], 10.00th=[ 305], 20.00th=[ 305], | 30.00th=[ 305], 40.00th=[ 430], 50.00th=[ 430], 60.00th=[ 430], | 70.00th=[ 1062], 80.00th=[ 1062], 90.00th=[ 1062], 95.00th=[ 1062], | 99.00th=[ 1062], 99.50th=[ 1062], 99.90th=[ 1062], 99.95th=[ 1062], | 99.99th=[ 1062] bw ( KiB/s): min= 5475, max= 8192, per=38.02%, avg=6833.50, stdev=1921.21, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=15.97%, ctx=223, majf=0, minf=8 rand-rw: (groupid=0, jobs=1): err= 0: pid=159391: Tue Oct 21 23:24:13 2025 read: IOPS=4, BW=16.0MiB/s (17.8MB/s)(12.0MiB/707msec) clat (msec): min=143, max=400, avg=232.94, stdev=145.05 lat (msec): min=143, max=400, avg=232.94, stdev=145.05 clat percentiles (msec): | 1.00th=[ 144], 5.00th=[ 144], 10.00th=[ 144], 20.00th=[ 144], | 30.00th=[ 144], 40.00th=[ 155], 50.00th=[ 155], 60.00th=[ 155], | 70.00th=[ 401], 80.00th=[ 401], 90.00th=[ 401], 95.00th=[ 401], | 99.00th=[ 401], 99.50th=[ 401], 99.90th=[ 401], 99.95th=[ 401], | 99.99th=[ 401] bw ( KiB/s): min=14397, max=14397, per=100.00%, avg=14397.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, 500=33.33% cpu : usr=0.00%, sys=33.43%, ctx=92, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159392: Tue Oct 21 23:24:13 2025 read: IOPS=0, BW=2365KiB/s (2422kB/s)(4096KiB/1732msec) clat (nsec): min=656123k, max=656123k, avg=656122948.00, stdev= 0.00 lat (nsec): min=656125k, max=656125k, avg=656125498.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 659], 5.00th=[ 659], 10.00th=[ 659], 20.00th=[ 659], | 30.00th=[ 659], 40.00th=[ 659], 50.00th=[ 659], 60.00th=[ 659], | 70.00th=[ 659], 80.00th=[ 659], 90.00th=[ 659], 95.00th=[ 659], | 99.00th=[ 659], 99.50th=[ 659], 99.90th=[ 659], 99.95th=[ 659], | 99.99th=[ 659] bw ( KiB/s): min= 8047, max= 8047, per=85.07%, avg=8047.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4730KiB/s (4843kB/s)(8192KiB/1732msec); 0 zone resets clat (msec): min=353, max=676, avg=514.73, stdev=228.70 lat (msec): min=353, max=676, avg=515.00, stdev=228.74 clat percentiles (msec): | 1.00th=[ 355], 5.00th=[ 355], 10.00th=[ 355], 20.00th=[ 355], | 30.00th=[ 355], 40.00th=[ 355], 50.00th=[ 355], 60.00th=[ 676], | 70.00th=[ 676], 80.00th=[ 676], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 8192, max= 8192, per=47.75%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.06%, sys=15.25%, ctx=76, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159393: Tue Oct 21 23:24:13 2025 write: IOPS=1, BW=6434KiB/s (6588kB/s)(12.0MiB/1910msec); 0 zone resets clat (msec): min=90, max=1648, avg=630.26, stdev=882.62 lat (msec): min=91, max=1649, avg=630.56, stdev=882.62 clat percentiles (msec): | 1.00th=[ 91], 5.00th=[ 91], 10.00th=[ 91], 20.00th=[ 91], | 30.00th=[ 91], 40.00th=[ 153], 50.00th=[ 153], 60.00th=[ 153], | 70.00th=[ 1653], 80.00th=[ 1653], 90.00th=[ 1653], 95.00th=[ 1653], | 99.00th=[ 1653], 99.50th=[ 1653], 99.90th=[ 1653], 99.95th=[ 1653], | 99.99th=[ 1653] bw ( KiB/s): min= 8192, max= 8192, per=47.75%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 100=33.33%, 250=33.33%, 2000=33.33% cpu : usr=0.05%, sys=14.20%, ctx=62, majf=0, m IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% inf=9 submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% latency : target=0, window=0, percentile=100.00%, depth=16 issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): rand-rw: (groupid=0, jobs=1): err= 0: pid=159394: Tue Oct 21 23:24:13 2025 READ: bw=11.8MiB/s (12.4MB/s), 3101KiB/s-9069KiB/s (3175kB/s-9286kB/s), io=16.0MiB (16.8MB), run=1321-1355msec write: IOPS=2, BW=8365KiB/s (8566kB/s)(12.0MiB/1469msec); 0 zone resets clat (msec): min=201, max=920, avg=473.59, stdev=390.26 WRITE: bw=17.6MiB/s (18.4MB/s), 6201KiB/s-7288KiB/s (6350kB/s-7463kB/s), io=32.0MiB (33.6MB), run=1321-1823msec lat (msec): min=201, max=920, avg=473.84, stdev=390.17 clat percentiles (msec): | 1.00th=[ 201], 5.00th=[ 201], 10.00th=[ 201], 20.00th=[ 201], | 30.00th=[ 201], 40.00th=[ 300], 50.00th=[ 300], 60.00th=[ 300], | 70.00th=[ 919], 80.00th=[ 919], 90.00th=[ 919], 95.00th=[ 919], | 99.00th=[ 919], 99.50th=[ 919], 99.90th=[ 919], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min=14397, max=14397, per=83.92%, avg=14397.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 1000=33.33% cpu : usr=0.00%, sys=19.21%, ctx=86, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=9460KiB/s (9687kB/s), 2365KiB/s-16.0MiB/s (2422kB/s-17.8MB/s), io=16.0MiB (16.8MB), run=707-1732msec WRITE: bw=16.8MiB/s (17.6MB/s), 4730KiB/s-8365KiB/s (4843kB/s-8566kB/s), io=32.0MiB (33.6MB), run=1469-1910msec