== sanity test 398b: DIO and buffer IO race ============== 13:40:39 (1758735639) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.73661 s, 18.4 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=150934: Wed Sep 24 13:41:36 2025 read: IOPS=31, BW=127KiB/s (130kB/s)(5992KiB/47314msec) clat (usec): min=71, max=679713, avg=16616.40, stdev=20087.56 lat (usec): min=71, max=679714, avg=16618.36, stdev=20088.91 clat percentiles (usec): | 1.00th=[ 289], 5.00th=[ 4817], 10.00th=[ 6783], 20.00th=[ 8455], | 30.00th=[ 10028], 40.00th=[ 11863], 50.00th=[ 13829], 60.00th=[ 16319], | 70.00th=[ 19006], 80.00th=[ 22938], 90.00th=[ 28705], 95.00th=[ 35914], | 99.00th=[ 50070], 99.50th=[ 62653], 99.90th=[103285], 99.95th=[683672], | 99.99th=[683672] bw ( KiB/s): min= 14, max= 224, per=25.20%, avg=126.51, stdev=38.75, samples=94 iops : min= 3, max= 56, avg=31.30, stdev= 9.77, samples=94 write: IOPS=33, BW=133KiB/s (136kB/s)(6296KiB/47314msec); 0 zone resets clat (usec): min=599, max=206236, avg=14131.67, stdev=12974.27 lat (usec): min=599, max=206237, avg=14132.60, stdev=12974.30 clat percentiles (usec): | 1.00th=[ 873], 5.00th=[ 1876], 10.00th=[ 2474], 20.00th=[ 3359], | 30.00th=[ 5211], 40.00th=[ 8291], 50.00th=[ 11076], 60.00th=[ 14222], | 70.00th=[ 17695], 80.00th=[ 22152], 90.00th=[ 30278], 95.00th=[ 36963], | 99.00th=[ 55313], 99.50th=[ 64750], 99.90th=[ 93848], 99.95th=[206570], | 99.99th=[206570] bw ( KiB/s): min= 40, max= 232, per=25.57%, avg=134.26, stdev=44.03, samples=93 iops : min= 10, max= 58, avg=33.24, stdev=11.08, samples=93 lat (usec) : 100=0.07%, 250=0.29%, 500=1.07%, 750=0.78%, 1000=0.78% lat (msec) : 2=2.02%, 4=9.47%, 10=23.67%, 20=36.07%, 50=24.58% lat (msec) : 100=1.07%, 250=0.10%, 750=0.03% cpu : usr=0.10%, sys=26.89%, ctx=8249, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150936: Wed Sep 24 13:41:36 2025 read: IOPS=31, BW=124KiB/s (127kB/s)(5940KiB/47796msec) clat (usec): min=76, max=144991, avg=16204.82, stdev=10338.84 lat (usec): min=76, max=144992, avg=16205.55, stdev=10338.91 clat percentiles (usec): | 1.00th=[ 289], 5.00th=[ 4359], 10.00th=[ 6915], 20.00th=[ 8717], | 30.00th=[ 10290], 40.00th=[ 12125], 50.00th=[ 14222], 60.00th=[ 16319], | 70.00th=[ 19268], 80.00th=[ 22414], 90.00th=[ 28967], 95.00th=[ 35390], | 99.00th=[ 49021], 99.50th=[ 55837], 99.90th=[ 81265], 99.95th=[145753], | 99.99th=[145753] bw ( KiB/s): min= 7, max= 208, per=24.62%, avg=123.59, stdev=38.20, samples=95 iops : min= 1, max= 52, avg=30.57, stdev= 9.69, samples=95 write: IOPS=33, BW=133KiB/s (136kB/s)(6348KiB/47796msec); 0 zone resets clat (usec): min=591, max=678962, avg=14839.94, stdev=21101.74 lat (usec): min=591, max=678964, avg=14840.82, stdev=21101.77 clat percentiles (usec): | 1.00th=[ 816], 5.00th=[ 1811], 10.00th=[ 2409], 20.00th=[ 3425], | 30.00th=[ 5473], 40.00th=[ 8848], 50.00th=[ 11469], 60.00th=[ 14746], | 70.00th=[ 17957], 80.00th=[ 22676], 90.00th=[ 30540], 95.00th=[ 38011], | 99.00th=[ 60556], 99.50th=[ 78119], 99.90th=[121111], 99.95th=[675283], | 99.99th=[675283] bw ( KiB/s): min= 55, max= 296, per=25.36%, avg=133.16, stdev=42.87, samples=94 iops : min= 13, max= 74, avg=32.97, stdev=10.80, samples=94 lat (usec) : 100=0.23%, 250=0.20%, 500=1.20%, 750=0.75%, 1000=0.59% lat (msec) : 2=2.31%, 4=9.24%, 10=21.68%, 20=37.89%, 50=24.61% lat (msec) : 100=1.14%, 250=0.13%, 750=0.03% cpu : usr=0.10%, sys=26.63%, ctx=8510, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150938: Wed Sep 24 13:41:36 2025 read: IOPS=31, BW=127KiB/s (130kB/s)(6028KiB/47530msec) clat (usec): min=85, max=204863, avg=16111.62, stdev=11241.57 lat (usec): min=86, max=204864, avg=16112.45, stdev=11241.68 clat percentiles (usec): | 1.00th=[ 318], 5.00th=[ 5276], 10.00th=[ 6783], 20.00th=[ 8586], | 30.00th=[ 10159], 40.00th=[ 12125], 50.00th=[ 14091], 60.00th=[ 16319], | 70.00th=[ 18482], 80.00th=[ 22152], 90.00th=[ 28443], 95.00th=[ 33162], | 99.00th=[ 54264], 99.50th=[ 64750], 99.90th=[103285], 99.95th=[204473], | 99.99th=[204473] bw ( KiB/s): min= 39, max= 232, per=25.43%, avg=127.65, stdev=40.38, samples=93 iops : min= 9, max= 58, avg=31.58, stdev=10.25, samples=93 write: IOPS=32, BW=132KiB/s (135kB/s)(6260KiB/47530msec); 0 zone resets clat (usec): min=558, max=704942, avg=14739.02, stdev=21488.93 lat (usec): min=558, max=704944, avg=14740.01, stdev=21488.96 clat percentiles (usec): | 1.00th=[ 807], 5.00th=[ 1729], 10.00th=[ 2376], 20.00th=[ 3294], | 30.00th=[ 5276], 40.00th=[ 8455], 50.00th=[ 11469], 60.00th=[ 14746], | 70.00th=[ 18220], 80.00th=[ 23462], 90.00th=[ 30802], 95.00th=[ 35914], | 99.00th=[ 53216], 99.50th=[ 79168], 99.90th=[103285], 99.95th=[708838], | 99.99th=[708838] bw ( KiB/s): min= 39, max= 224, per=25.16%, avg=132.09, stdev=39.52, samples=93 iops : min= 9, max= 56, avg=32.70, stdev= 9.95, samples=93 lat (usec) : 100=0.07%, 250=0.16%, 500=1.27%, 750=0.75%, 1000=0.88% lat (msec) : 2=2.12%, 4=9.05%, 10=23.27%, 20=36.33%, 50=24.80% lat (msec) : 100=1.17%, 250=0.10%, 750=0.03% cpu : usr=0.13%, sys=26.52%, ctx=8288, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150940: Wed Sep 24 13:41:36 2025 read: IOPS=31, BW=127KiB/s (130kB/s)(6052KiB/47696msec) clat (usec): min=78, max=126405, avg=16029.84, stdev=10906.08 lat (usec): min=78, max=126406, avg=16030.63, stdev=10906.28 clat percentiles (usec): | 1.00th=[ 262], 5.00th=[ 824], 10.00th=[ 6718], 20.00th=[ 8455], | 30.00th=[ 10028], 40.00th=[ 11731], 50.00th=[ 13566], 60.00th=[ 15664], | 70.00th=[ 18482], 80.00th=[ 22414], 90.00th=[ 29230], 95.00th=[ 35914], | 99.00th=[ 52691], 99.50th=[ 68682], 99.90th=[ 94897], 99.95th=[126354], | 99.99th=[126354] bw ( KiB/s): min= 31, max= 248, per=25.55%, avg=128.24, stdev=42.13, samples=94 iops : min= 7, max= 62, avg=31.73, stdev=10.66, samples=94 write: IOPS=32, BW=131KiB/s (134kB/s)(6236KiB/47696msec); 0 zone resets clat (usec): min=580, max=701809, avg=14920.03, stdev=21818.87 lat (usec): min=580, max=701811, avg=14920.96, stdev=21818.89 clat percentiles (usec): | 1.00th=[ 750], 5.00th=[ 1663], 10.00th=[ 2376], 20.00th=[ 3294], | 30.00th=[ 5669], 40.00th=[ 8586], 50.00th=[ 11469], 60.00th=[ 14615], | 70.00th=[ 18482], 80.00th=[ 23200], 90.00th=[ 30802], 95.00th=[ 38011], | 99.00th=[ 53740], 99.50th=[ 60556], 99.90th=[166724], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 47, max= 240, per=25.16%, avg=132.06, stdev=42.17, samples=94 iops : min= 11, max= 60, avg=32.69, stdev=10.72, samples=94 lat (usec) : 100=0.10%, 250=0.36%, 500=1.40%, 750=1.04%, 1000=0.88% lat (msec) : 2=2.41%, 4=8.82%, 10=22.88%, 20=36.49%, 50=24.25% lat (msec) : 100=1.17%, 250=0.16%, 750=0.03% cpu : usr=0.13%, sys=26.41%, ctx=8250, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=502KiB/s (514kB/s), 124KiB/s-127KiB/s (127kB/s-130kB/s), io=23.4MiB (24.6MB), run=47314-47796msec WRITE: bw=526KiB/s (539kB/s), 131KiB/s-133KiB/s (134kB/s-136kB/s), io=24.6MiB (25.7MB), run=47314-47796msec rand-rw: (groupid=0, jobs=1): err= 0: pid=150935: Wed Sep 24 13:42:06 2025 read: IOPS=19, BW=76.7KiB/s (78.6kB/s)(5992KiB/78106msec) clat (msec): min=2, max=686, avg=15.54, stdev=22.13 lat (msec): min=2, max=686, avg=15.55, stdev=22.13 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 21], 90.00th=[ 31], 95.00th=[ 43], | 99.00th=[ 77], 99.50th=[ 84], 99.90th=[ 108], 99.95th=[ 684], | 99.99th=[ 684] bw ( KiB/s): min= 7, max= 240, per=25.03%, avg=76.84, stdev=40.49, samples=154 iops : min= 1, max= 60, avg=18.98, stdev=10.23, samples=154 write: IOPS=20, BW=80.6KiB/s (82.5kB/s)(6296KiB/78106msec); 0 zone resets clat (msec): min=12, max=182, avg=34.71, stdev=17.40 lat (msec): min=12, max=182, avg=34.71, stdev=17.40 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 18], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 26], 40.00th=[ 27], 50.00th=[ 30], 60.00th=[ 34], | 70.00th=[ 38], 80.00th=[ 45], 90.00th=[ 56], 95.00th=[ 66], | 99.00th=[ 93], 99.50th=[ 130], 99.90th=[ 180], 99.95th=[ 182], | 99.99th=[ 182] bw ( KiB/s): min= 16, max= 168, per=25.15%, avg=80.73, stdev=34.19, samples=154 iops : min= 4, max= 42, avg=19.95, stdev= 8.68, samples=154 lat (msec) : 4=4.39%, 10=17.32%, 20=22.66%, 50=46.48%, 100=8.72% lat (msec) : 250=0.39%, 750=0.03% cpu : usr=0.10%, sys=8.54%, ctx=5959, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150937: Wed Sep 24 13:42:06 2025 read: IOPS=19, BW=76.1KiB/s (77.9kB/s)(5940KiB/78045msec) clat (msec): min=2, max=695, avg=14.88, stdev=21.81 lat (msec): min=2, max=695, avg=14.88, stdev=21.81 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 9], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 16], 80.00th=[ 20], 90.00th=[ 29], 95.00th=[ 39], | 99.00th=[ 73], 99.50th=[ 77], 99.90th=[ 157], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 8, max= 224, per=24.89%, avg=76.42, stdev=42.29, samples=154 iops : min= 2, max= 56, avg=18.88, stdev=10.67, samples=154 write: IOPS=20, BW=81.3KiB/s (83.3kB/s)(6348KiB/78045msec); 0 zone resets clat (msec): min=13, max=220, avg=35.14, stdev=17.57 lat (msec): min=13, max=220, avg=35.14, stdev=17.57 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 19], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 25], 40.00th=[ 28], 50.00th=[ 31], 60.00th=[ 34], | 70.00th=[ 40], 80.00th=[ 46], 90.00th=[ 57], 95.00th=[ 67], | 99.00th=[ 97], 99.50th=[ 116], 99.90th=[ 192], 99.95th=[ 222], | 99.99th=[ 222] bw ( KiB/s): min= 23, max= 168, per=25.39%, avg=81.51, stdev=33.56, samples=154 iops : min= 5, max= 42, avg=20.15, stdev= 8.52, samples=154 lat (msec) : 4=3.55%, 10=18.23%, 20=22.69%, 50=46.48%, 100=8.50% lat (msec) : 250=0.52%, 750=0.03% cpu : usr=0.12%, sys=8.52%, ctx=5841, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150939: Wed Sep 24 13:42:06 2025 read: IOPS=19, BW=77.1KiB/s (78.0kB/s)(6028KiB/78146msec) clat (msec): min=2, max=111, avg=15.32, stdev=13.12 lat (msec): min=2, max=111, avg=15.32, stdev=13.12 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 8], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 15], | 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 32], 95.00th=[ 41], | 99.00th=[ 68], 99.50th=[ 79], 99.90th=[ 99], 99.95th=[ 111], | 99.99th=[ 111] bw ( KiB/s): min= 16, max= 216, per=25.18%, avg=77.31, stdev=39.99, samples=154 iops : min= 4, max= 54, avg=19.10, stdev=10.11, samples=154 write: IOPS=20, BW=80.1KiB/s (82.0kB/s)(6260KiB/78146msec); 0 zone resets clat (msec): min=12, max=700, avg=35.07, stdev=24.29 lat (msec): min=12, max=700, avg=35.07, stdev=24.29 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 18], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 25], 40.00th=[ 28], 50.00th=[ 30], 60.00th=[ 34], | 70.00th=[ 38], 80.00th=[ 45], 90.00th=[ 56], 95.00th=[ 67], | 99.00th=[ 95], 99.50th=[ 131], 99.90th=[ 218], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 23, max= 168, per=24.97%, avg=80.16, stdev=33.46, samples=154 iops : min= 5, max= 42, avg=19.81, stdev= 8.50, samples=154 lat (msec) : 4=4.10%, 10=16.54%, 20=23.83%, 50=46.35%, 100=8.76% lat (msec) : 250=0.39%, 750=0.03% cpu : usr=0.10%, sys=8.62%, ctx=5947, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150941: Wed Sep 24 13:42:06 2025 read: IOPS=19, BW=77.8KiB/s (79.6kB/s)(6052KiB/77809msec) clat (msec): min=2, max=161, avg=15.38, stdev=13.85 lat (msec): min=2, max=161, avg=15.38, stdev=13.85 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 8], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 15], | 70.00th=[ 18], 80.00th=[ 22], 90.00th=[ 33], 95.00th=[ 42], | 99.00th=[ 67], 99.50th=[ 91], 99.90th=[ 113], 99.95th=[ 161], | 99.99th=[ 161] bw ( KiB/s): min= 15, max= 248, per=25.49%, avg=78.24, stdev=40.89, samples=154 iops : min= 3, max= 62, avg=19.33, stdev=10.34, samples=154 write: IOPS=20, BW=80.1KiB/s (82.1kB/s)(6236KiB/77809msec); 0 zone resets clat (msec): min=13, max=699, avg=34.87, stdev=23.42 lat (msec): min=13, max=699, avg=34.87, stdev=23.42 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 18], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 26], 40.00th=[ 28], 50.00th=[ 30], 60.00th=[ 33], | 70.00th=[ 38], 80.00th=[ 45], 90.00th=[ 56], 95.00th=[ 66], | 99.00th=[ 93], 99.50th=[ 124], 99.90th=[ 142], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 23, max= 160, per=25.09%, avg=80.53, stdev=34.26, samples=154 iops : min= 5, max= 40, avg=19.90, stdev= 8.70, samples=154 lat (msec) : 4=4.33%, 10=16.63%, 20=23.54%, 50=46.91%, 100=8.04% lat (msec) : 250=0.52%, 750=0.03% cpu : usr=0.08%, sys=8.60%, ctx=5875, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=307KiB/s (315kB/s), 76.1KiB/s-77.8KiB/s (77.9kB/s-79.6kB/s), io=23.4MiB (24.6MB), run=77809-78146msec WRITE: bw=322KiB/s (329kB/s), 80.1KiB/s-81.3KiB/s (82.0kB/s-83.3kB/s), io=24.6MiB (25.7MB), run=77809-78146msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=150970: Wed Sep 24 13:42:23 2025 read: IOPS=32, BW=522KiB/s (534kB/s)(5920KiB/11350msec) clat (usec): min=112, max=62160, avg=15262.56, stdev=9465.46 lat (usec): min=113, max=62162, avg=15263.41, stdev=9465.56 clat percentiles (usec): | 1.00th=[ 258], 5.00th=[ 359], 10.00th=[ 474], 20.00th=[ 8848], | 30.00th=[10552], 40.00th=[12518], 50.00th=[14353], 60.00th=[16188], | 70.00th=[19006], 80.00th=[21627], 90.00th=[26870], 95.00th=[31589], | 99.00th=[46924], 99.50th=[55313], 99.90th=[62129], 99.95th=[62129], | 99.99th=[62129] bw ( KiB/s): min= 347, max= 796, per=24.21%, avg=507.23, stdev=106.50, samples=22 iops : min= 21, max= 49, avg=31.18, stdev= 6.76, samples=22 write: IOPS=35, BW=561KiB/s (575kB/s)(6368KiB/11350msec); 0 zone resets clat (usec): min=1013, max=74381, avg=14186.95, stdev=11748.23 lat (usec): min=1015, max=74383, avg=14188.39, stdev=11748.38 clat percentiles (usec): | 1.00th=[ 1139], 5.00th=[ 2638], 10.00th=[ 3589], 20.00th=[ 4293], | 30.00th=[ 5276], 40.00th=[ 7242], 50.00th=[11207], 60.00th=[13566], | 70.00th=[17171], 80.00th=[23462], 90.00th=[31589], 95.00th=[38011], | 99.00th=[49021], 99.50th=[59507], 99.90th=[73925], 99.95th=[73925], | 99.99th=[73925] bw ( KiB/s): min= 288, max= 828, per=25.24%, avg=556.50, stdev=171.64, samples=22 iops : min= 18, max= 51, avg=34.27, stdev=10.58, samples=22 lat (usec) : 250=0.26%, 500=4.69%, 750=0.52%, 1000=0.13% lat (msec) : 2=2.08%, 4=6.25%, 10=23.05%, 20=37.50%, 50=24.87% lat (msec) : 100=0.65% cpu : usr=0.11%, sys=30.80%, ctx=2310, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150971: Wed Sep 24 13:42:23 2025 read: IOPS=31, BW=510KiB/s (522kB/s)(5744KiB/11266msec) clat (usec): min=133, max=62348, avg=15015.08, stdev=9325.24 lat (usec): min=134, max=62349, avg=15015.94, stdev=9325.31 clat percentiles (usec): | 1.00th=[ 233], 5.00th=[ 322], 10.00th=[ 519], 20.00th=[ 8979], | 30.00th=[10683], 40.00th=[12256], 50.00th=[13829], 60.00th=[15926], | 70.00th=[18482], 80.00th=[20841], 90.00th=[25560], 95.00th=[31065], | 99.00th=[47449], 99.50th=[51119], 99.90th=[62129], 99.95th=[62129], | 99.99th=[62129] bw ( KiB/s): min= 224, max= 766, per=24.19%, avg=506.77, stdev=140.25, samples=22 iops : min= 14, max= 47, avg=31.18, stdev= 8.77, samples=22 write: IOPS=36, BW=581KiB/s (595kB/s)(6544KiB/11266msec); 0 zone resets clat (usec): min=825, max=92645, avg=14310.54, stdev=12394.04 lat (usec): min=826, max=92646, avg=14312.27, stdev=12394.08 clat percentiles (usec): | 1.00th=[ 1172], 5.00th=[ 2802], 10.00th=[ 3523], 20.00th=[ 4621], | 30.00th=[ 5932], 40.00th=[ 8356], 50.00th=[10814], 60.00th=[13829], | 70.00th=[17433], 80.00th=[22414], 90.00th=[30278], 95.00th=[36963], | 99.00th=[51643], 99.50th=[81265], 99.90th=[92799], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 319, max= 1147, per=26.17%, avg=576.95, stdev=194.98, samples=22 iops : min= 19, max= 71, avg=35.55, stdev=12.15, samples=22 lat (usec) : 250=0.65%, 500=3.78%, 750=0.52%, 1000=0.39% lat (msec) : 2=1.69%, 4=5.34%, 10=25.00%, 20=38.67%, 50=22.66% lat (msec) : 100=1.30% cpu : usr=0.12%, sys=30.91%, ctx=2199, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150972: Wed Sep 24 13:42:23 2025 read: IOPS=34, BW=547KiB/s (561kB/s)(6256KiB/11428msec) clat (usec): min=81, max=71811, avg=15745.14, stdev=10497.22 lat (usec): min=82, max=71813, avg=15746.07, stdev=10497.25 clat percentiles (usec): | 1.00th=[ 202], 5.00th=[ 343], 10.00th=[ 461], 20.00th=[ 8848], | 30.00th=[10945], 40.00th=[12256], 50.00th=[14746], 60.00th=[16581], | 70.00th=[19530], 80.00th=[22676], 90.00th=[27395], 95.00th=[33424], | 99.00th=[52691], 99.50th=[70779], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min= 349, max= 832, per=26.05%, avg=545.82, stdev=133.24, samples=22 iops : min= 21, max= 52, avg=33.59, stdev= 8.35, samples=22 write: IOPS=32, BW=528KiB/s (540kB/s)(6032KiB/11428msec); 0 zone resets clat (usec): min=812, max=86073, avg=13871.73, stdev=11725.69 lat (usec): min=814, max=86075, avg=13873.33, stdev=11725.63 clat percentiles (usec): | 1.00th=[ 1336], 5.00th=[ 3195], 10.00th=[ 3720], 20.00th=[ 4359], | 30.00th=[ 5407], 40.00th=[ 8029], 50.00th=[10683], 60.00th=[12780], | 70.00th=[16319], 80.00th=[22152], 90.00th=[29492], 95.00th=[37487], | 99.00th=[51643], 99.50th=[66323], 99.90th=[86508], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 192, max= 733, per=23.17%, avg=510.91, stdev=127.63, samples=22 iops : min= 12, max= 45, avg=31.41, stdev= 7.86, samples=22 lat (usec) : 100=0.13%, 250=0.52%, 500=5.34%, 750=0.39%, 1000=0.26% lat (msec) : 2=0.65%, 4=5.73%, 10=23.83%, 20=37.37%, 50=24.74% lat (msec) : 100=1.04% cpu : usr=0.15%, sys=30.38%, ctx=2334, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150973: Wed Sep 24 13:42:23 2025 read: IOPS=33, BW=539KiB/s (552kB/s)(6032KiB/11184msec) clat (usec): min=62, max=76666, avg=15255.26, stdev=11339.07 lat (usec): min=62, max=76667, avg=15256.41, stdev=11338.96 clat percentiles (usec): | 1.00th=[ 87], 5.00th=[ 293], 10.00th=[ 379], 20.00th=[ 8455], | 30.00th=[ 9765], 40.00th=[11338], 50.00th=[13566], 60.00th=[16057], | 70.00th=[18744], 80.00th=[22414], 90.00th=[28443], 95.00th=[32900], | 99.00th=[66847], 99.50th=[68682], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 288, max= 828, per=25.63%, avg=536.86, stdev=135.30, samples=22 iops : min= 18, max= 51, avg=33.05, stdev= 8.40, samples=22 write: IOPS=34, BW=559KiB/s (573kB/s)(6256KiB/11184msec); 0 zone resets clat (usec): min=777, max=107024, avg=13786.89, stdev=12112.31 lat (usec): min=778, max=107026, avg=13788.32, stdev=12112.43 clat percentiles (usec): | 1.00th=[ 988], 5.00th=[ 2671], 10.00th=[ 3720], 20.00th=[ 4424], | 30.00th=[ 5276], 40.00th=[ 6783], 50.00th=[ 10421], 60.00th=[ 13042], | 70.00th=[ 16188], 80.00th=[ 22414], 90.00th=[ 30278], 95.00th=[ 37487], | 99.00th=[ 51643], 99.50th=[ 64226], 99.90th=[107480], 99.95th=[107480], | 99.99th=[107480] bw ( KiB/s): min= 320, max= 830, per=25.44%, avg=560.86, stdev=147.63, samples=22 iops : min= 20, max= 51, avg=34.55, stdev= 9.27, samples=22 lat (usec) : 100=0.65%, 250=1.30%, 500=4.69%, 750=0.65%, 1000=0.65% lat (msec) : 2=1.43%, 4=5.86%, 10=25.13%, 20=34.64%, 50=23.57% lat (msec) : 100=1.30%, 250=0.13% cpu : usr=0.08%, sys=30.30%, ctx=2197, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2096KiB/s (2146kB/s), 510KiB/s-547KiB/s (522kB/s-561kB/s), io=23.4MiB (24.5MB), run=11184-11428msec WRITE: bw=2205KiB/s (2258kB/s), 528KiB/s-581KiB/s (540kB/s-595kB/s), io=24.6MiB (25.8MB), run=11184-11428msec rand-rw: (groupid=0, jobs=1): err= 0: pid=150975: Wed Sep 24 13:42:31 2025 read: IOPS=19, BW=313KiB/s (321kB/s)(5920KiB/18885msec) clat (usec): min=2500, max=96700, avg=13956.74, stdev=15533.57 lat (usec): min=2500, max=96701, avg=13957.56, stdev=15533.55 clat percentiles (usec): | 1.00th=[ 2868], 5.00th=[ 3294], 10.00th=[ 3720], 20.00th=[ 4555], | 30.00th=[ 5276], 40.00th=[ 6456], 50.00th=[ 7701], 60.00th=[ 9765], | 70.00th=[12649], 80.00th=[19792], 90.00th=[32375], 95.00th=[49021], | 99.00th=[83362], 99.50th=[84411], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 95, max= 704, per=24.29%, avg=307.22, stdev=155.52, samples=37 iops : min= 5, max= 44, avg=18.95, stdev= 9.86, samples=37 write: IOPS=21, BW=337KiB/s (345kB/s)(6368KiB/18885msec); 0 zone resets clat (msec): min=13, max=123, avg=34.31, stdev=18.19 lat (msec): min=13, max=123, avg=34.31, stdev=18.19 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 17], 10.00th=[ 19], 20.00th=[ 21], | 30.00th=[ 23], 40.00th=[ 26], 50.00th=[ 28], 60.00th=[ 32], | 70.00th=[ 36], 80.00th=[ 47], 90.00th=[ 61], 95.00th=[ 74], | 99.00th=[ 104], 99.50th=[ 108], 99.90th=[ 124], 99.95th=[ 124], | 99.99th=[ 124] bw ( KiB/s): min= 159, max= 672, per=25.06%, avg=333.49, stdev=172.28, samples=37 iops : min= 9, max= 42, avg=20.57, stdev=10.97, samples=37 lat (msec) : 4=5.73%, 10=23.96%, 20=16.80%, 50=42.32%, 100=10.55% lat (msec) : 250=0.65% cpu : usr=0.05%, sys=8.13%, ctx=1441, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150976: Wed Sep 24 13:42:31 2025 read: IOPS=18, BW=304KiB/s (311kB/s)(5744KiB/18922msec) clat (usec): min=2235, max=84871, avg=13232.56, stdev=15107.44 lat (usec): min=2236, max=84872, avg=13233.40, stdev=15107.44 clat percentiles (usec): | 1.00th=[ 2835], 5.00th=[ 3425], 10.00th=[ 3851], 20.00th=[ 4490], | 30.00th=[ 5145], 40.00th=[ 5735], 50.00th=[ 6783], 60.00th=[ 8717], | 70.00th=[11600], 80.00th=[17433], 90.00th=[32900], 95.00th=[46924], | 99.00th=[76022], 99.50th=[82314], 99.90th=[84411], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 53, max= 672, per=23.88%, avg=302.08, stdev=178.15, samples=37 iops : min= 3, max= 42, avg=18.62, stdev=11.28, samples=37 write: IOPS=21, BW=346KiB/s (354kB/s)(6544KiB/18922msec); 0 zone resets clat (msec): min=12, max=114, avg=34.44, stdev=17.47 lat (msec): min=12, max=114, avg=34.45, stdev=17.47 clat percentiles (msec): | 1.00th=[ 14], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 21], | 30.00th=[ 24], 40.00th=[ 27], 50.00th=[ 30], 60.00th=[ 33], | 70.00th=[ 39], 80.00th=[ 47], 90.00th=[ 61], 95.00th=[ 72], | 99.00th=[ 94], 99.50th=[ 100], 99.90th=[ 115], 99.95th=[ 115], | 99.99th=[ 115] bw ( KiB/s): min= 128, max= 672, per=25.49%, avg=339.24, stdev=159.96, samples=37 iops : min= 8, max= 42, avg=20.95, stdev=10.15, samples=37 lat (msec) : 4=5.86%, 10=24.22%, 20=17.84%, 50=41.54%, 100=10.29% lat (msec) : 250=0.26% cpu : usr=0.07%, sys=7.91%, ctx=1419, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150977: Wed Sep 24 13:42:31 2025 read: IOPS=21, BW=341KiB/s (349kB/s)(6256KiB/18370msec) clat (usec): min=2605, max=95904, avg=13421.87, stdev=14674.13 lat (usec): min=2605, max=95905, avg=13422.84, stdev=14674.13 clat percentiles (usec): | 1.00th=[ 2769], 5.00th=[ 3359], 10.00th=[ 3785], 20.00th=[ 4555], | 30.00th=[ 5211], 40.00th=[ 5932], 50.00th=[ 7570], 60.00th=[10159], | 70.00th=[13435], 80.00th=[18220], 90.00th=[30802], 95.00th=[45351], | 99.00th=[76022], 99.50th=[93848], 99.90th=[95945], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 95, max= 960, per=27.22%, avg=344.31, stdev=210.59, samples=36 iops : min= 5, max= 60, avg=21.25, stdev=13.31, samples=36 write: IOPS=20, BW=328KiB/s (336kB/s)(6032KiB/18370msec); 0 zone resets clat (msec): min=12, max=111, avg=34.55, stdev=16.47 lat (msec): min=12, max=111, avg=34.55, stdev=16.47 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 18], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 25], 40.00th=[ 28], 50.00th=[ 30], 60.00th=[ 33], | 70.00th=[ 39], 80.00th=[ 45], 90.00th=[ 56], 95.00th=[ 68], | 99.00th=[ 92], 99.50th=[ 107], 99.90th=[ 112], 99.95th=[ 112], | 99.99th=[ 112] bw ( KiB/s): min= 96, max= 608, per=24.46%, avg=325.56, stdev=153.52, samples=36 iops : min= 6, max= 38, avg=20.08, stdev= 9.74, samples=36 lat (msec) : 4=6.64%, 10=22.53%, 20=18.10%, 50=43.36%, 100=8.98% lat (msec) : 250=0.39% cpu : usr=0.10%, sys=8.31%, ctx=1443, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150978: Wed Sep 24 13:42:31 2025 read: IOPS=20, BW=324KiB/s (331kB/s)(6032KiB/18634msec) clat (msec): min=2, max=129, avg=13.85, stdev=16.18 lat (msec): min=2, max=129, avg=13.85, stdev=16.18 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 10], | 70.00th=[ 13], 80.00th=[ 20], 90.00th=[ 33], 95.00th=[ 44], | 99.00th=[ 86], 99.50th=[ 110], 99.90th=[ 130], 99.95th=[ 130], | 99.99th=[ 130] bw ( KiB/s): min= 95, max= 800, per=24.91%, avg=315.11, stdev=167.75, samples=36 iops : min= 5, max= 50, avg=19.42, stdev=10.67, samples=36 write: IOPS=20, BW=336KiB/s (344kB/s)(6256KiB/18634msec); 0 zone resets clat (msec): min=14, max=106, avg=34.07, stdev=16.84 lat (msec): min=14, max=106, avg=34.07, stdev=16.84 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 18], 10.00th=[ 20], 20.00th=[ 22], | 30.00th=[ 24], 40.00th=[ 26], 50.00th=[ 29], 60.00th=[ 32], | 70.00th=[ 39], 80.00th=[ 46], 90.00th=[ 57], 95.00th=[ 71], | 99.00th=[ 92], 99.50th=[ 103], 99.90th=[ 107], 99.95th=[ 107], | 99.99th=[ 107] bw ( KiB/s): min= 158, max= 672, per=24.81%, avg=330.17, stdev=162.28, samples=36 iops : min= 9, max= 42, avg=20.36, stdev=10.33, samples=36 lat (msec) : 4=4.82%, 10=26.30%, 20=15.36%, 50=44.01%, 100=8.85% lat (msec) : 250=0.65% cpu : usr=0.09%, sys=8.22%, ctx=1477, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1266KiB/s (1296kB/s), 304KiB/s-341KiB/s (311kB/s-349kB/s), io=23.4MiB (24.5MB), run=18370-18922msec WRITE: bw=1332KiB/s (1364kB/s), 328KiB/s-346KiB/s (336kB/s-354kB/s), io=24.6MiB (25.8MB), run=18370-18922msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=150986: Wed Sep 24 13:42:40 2025 read: IOPS=2, BW=2699KiB/s (2764kB/s)(7168KiB/2656msec) clat (msec): min=68, max=336, avg=160.69, stdev=92.06 lat (msec): min=68, max=336, avg=160.69, stdev=92.06 clat percentiles (msec): | 1.00th=[ 69], 5.00th=[ 69], 10.00th=[ 69], 20.00th=[ 73], | 30.00th=[ 122], 40.00th=[ 122], 50.00th=[ 146], 60.00th=[ 186], | 70.00th=[ 186], 80.00th=[ 197], 90.00th=[ 338], 95.00th=[ 338], | 99.00th=[ 338], 99.50th=[ 338], 99.90th=[ 338], 99.95th=[ 338], | 99.99th=[ 338] bw ( KiB/s): min= 2048, max= 5807, per=63.04%, avg=3967.33, stdev=1880.77, samples=3 iops : min= 2, max= 5, avg= 3.33, stdev= 1.53, samples=3 write: IOPS=1, BW=1928KiB/s (1974kB/s)(5120KiB/2656msec); 0 zone resets clat (msec): min=74, max=539, avg=298.98, stdev=181.96 lat (msec): min=75, max=539, avg=299.06, stdev=181.97 clat percentiles (msec): | 1.00th=[ 75], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 75], | 30.00th=[ 171], 40.00th=[ 171], 50.00th=[ 326], 60.00th=[ 326], | 70.00th=[ 384], 80.00th=[ 384], 90.00th=[ 542], 95.00th=[ 542], | 99.00th=[ 542], 99.50th=[ 542], 99.90th=[ 542], 99.95th=[ 542], | 99.99th=[ 542] bw ( KiB/s): min= 2031, max= 4096, per=30.88%, avg=2720.67, stdev=1191.08, samples=3 iops : min= 1, max= 4, avg= 2.00, stdev= 1.73, samples=3 lat (msec) : 100=25.00%, 250=41.67%, 500=25.00%, 750=8.33% cpu : usr=0.08%, sys=19.81%, ctx=136, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150988: Wed Sep 24 13:42:40 2025 read: IOPS=0, BW=629KiB/s (644kB/s)(2048KiB/3254msec) clat (msec): min=84, max=433, avg=259.07, stdev=246.69 lat (msec): min=84, max=433, avg=259.07, stdev=246.69 clat percentiles (msec): | 1.00th=[ 85], 5.00th=[ 85], 10.00th=[ 85], 20.00th=[ 85], | 30.00th=[ 85], 40.00th=[ 85], 50.00th=[ 85], 60.00th=[ 435], | 70.00th=[ 435], 80.00th=[ 435], 90.00th=[ 435], 95.00th=[ 435], | 99.00th=[ 435], 99.50th=[ 435], 99.90th=[ 435], 99.95th=[ 435], | 99.99th=[ 435] bw ( KiB/s): min= 1980, max= 2048, per=32.00%, avg=2014.00, stdev=48.08, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3147KiB/s (3222kB/s)(10.0MiB/3254msec); 0 zone resets clat (msec): min=39, max=619, avg=270.28, stdev=167.96 lat (msec): min=39, max=619, avg=270.35, stdev=167.97 clat percentiles (msec): | 1.00th=[ 40], 5.00th=[ 40], 10.00th=[ 40], 20.00th=[ 95], | 30.00th=[ 159], 40.00th=[ 203], 50.00th=[ 253], 60.00th=[ 279], | 70.00th=[ 284], 80.00th=[ 351], 90.00th=[ 426], 95.00th=[ 617], | 99.00th=[ 617], 99.50th=[ 617], 99.90th=[ 617], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 1980, max= 4096, per=32.23%, avg=2840.20, stdev=1135.17, samples=5 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=5 lat (msec) : 50=8.33%, 100=16.67%, 250=16.67%, 500=50.00%, 750=8.33% cpu : usr=0.03%, sys=16.54%, ctx=360, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150990: Wed Sep 24 13:42:40 2025 read: IOPS=2, BW=2056KiB/s (2105kB/s)(6144KiB/2989msec) clat (msec): min=67, max=649, avg=240.84, stdev=218.21 lat (msec): min=67, max=649, avg=240.84, stdev=218.21 clat percentiles (msec): | 1.00th=[ 68], 5.00th=[ 68], 10.00th=[ 68], 20.00th=[ 99], | 30.00th=[ 99], 40.00th=[ 117], 50.00th=[ 117], 60.00th=[ 209], | 70.00th=[ 305], 80.00th=[ 305], 90.00th=[ 651], 95.00th=[ 651], | 99.00th=[ 651], 99.50th=[ 651], 99.90th=[ 651], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 2023, max= 4071, per=43.13%, avg=2714.00, stdev=1175.26, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 write: IOPS=2, BW=2056KiB/s (2105kB/s)(6144KiB/2989msec); 0 zone resets clat (msec): min=80, max=443, avg=253.90, stdev=155.13 lat (msec): min=80, max=443, avg=253.98, stdev=155.13 clat percentiles (msec): | 1.00th=[ 81], 5.00th=[ 81], 10.00th=[ 81], 20.00th=[ 138], | 30.00th=[ 138], 40.00th=[ 163], 50.00th=[ 163], 60.00th=[ 266], | 70.00th=[ 435], 80.00th=[ 435], 90.00th=[ 443], 95.00th=[ 443], | 99.00th=[ 443], 99.50th=[ 443], 99.90th=[ 443], 99.95th=[ 443], | 99.99th=[ 443] bw ( KiB/s): min= 2023, max= 3828, per=28.19%, avg=2483.50, stdev=896.39, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 100=25.00%, 250=33.33%, 500=33.33%, 750=8.33% cpu : usr=0.03%, sys=14.42%, ctx=114, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150992: Wed Sep 24 13:42:40 2025 read: IOPS=1, BW=1831KiB/s (1875kB/s)(5120KiB/2796msec) clat (msec): min=54, max=261, avg=176.20, stdev=87.12 lat (msec): min=54, max=261, avg=176.20, stdev=87.12 clat percentiles (msec): | 1.00th=[ 55], 5.00th=[ 55], 10.00th=[ 55], 20.00th=[ 55], | 30.00th=[ 142], 40.00th=[ 142], 50.00th=[ 163], 60.00th=[ 163], | 70.00th=[ 259], 80.00th=[ 259], 90.00th=[ 262], 95.00th=[ 262], | 99.00th=[ 262], 99.50th=[ 262], 99.90th=[ 262], 99.95th=[ 262], | 99.99th=[ 262] bw ( KiB/s): min= 4047, max= 4063, per=64.44%, avg=4055.00, stdev=11.31, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2564KiB/s (2625kB/s)(7168KiB/2796msec); 0 zone resets clat (msec): min=112, max=501, avg=266.50, stdev=153.14 lat (msec): min=112, max=501, avg=266.59, stdev=153.12 clat percentiles (msec): | 1.00th=[ 112], 5.00th=[ 112], 10.00th=[ 112], 20.00th=[ 125], | 30.00th=[ 213], 40.00th=[ 213], 50.00th=[ 220], 60.00th=[ 241], | 70.00th=[ 241], 80.00th=[ 456], 90.00th=[ 502], 95.00th=[ 502], | 99.00th=[ 502], 99.50th=[ 502], 99.90th=[ 502], 99.95th=[ 502], | 99.99th=[ 502] bw ( KiB/s): min= 1976, max= 4096, per=34.52%, avg=3041.50, stdev=1203.39, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.50, samples=4 lat (msec) : 100=8.33%, 250=58.33%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=15.03%, ctx=109, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6294KiB/s (6445kB/s), 629KiB/s-2699KiB/s (644kB/s-2764kB/s), io=20.0MiB (20.0MB), run=2656-3254msec WRITE: bw=8811KiB/s (9023kB/s), 1928KiB/s-3147KiB/s (1974kB/s-3222kB/s), io=28.0MiB (29.4MB), run=2656-3254msec rand-rw: (groupid=0, jobs=1): err= 0: pid=150987: Wed Sep 24 13:42:40 2025 read: IOPS=2, BW=2949KiB/s (3019kB/s)(7168KiB/2431msec) clat (msec): min=4, max=257, avg=145.92, stdev=77.45 lat (msec): min=4, max=257, avg=145.93, stdev=77.45 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 118], | 30.00th=[ 124], 40.00th=[ 124], 50.00th=[ 161], 60.00th=[ 180], | 70.00th=[ 180], 80.00th=[ 180], 90.00th=[ 257], 95.00th=[ 257], | 99.00th=[ 257], 99.50th=[ 257], 99.90th=[ 257], 99.95th=[ 257], | 99.99th=[ 257] bw ( KiB/s): min= 1984, max= 5885, per=46.48%, avg=2987.00, stdev=1932.19, samples=4 iops : min= 1, max= 5, avg= 2.25, stdev= 1.89, samples=4 write: IOPS=2, BW=2106KiB/s (2157kB/s)(5120KiB/2431msec); 0 zone resets clat (msec): min=189, max=377, avg=280.57, stdev=70.79 lat (msec): min=189, max=377, avg=280.62, stdev=70.78 clat percentiles (msec): | 1.00th=[ 190], 5.00th=[ 190], 10.00th=[ 190], 20.00th=[ 190], | 30.00th=[ 247], 40.00th=[ 247], 50.00th=[ 279], 60.00th=[ 279], | 70.00th=[ 313], 80.00th=[ 313], 90.00th=[ 380], 95.00th=[ 380], | 99.00th=[ 380], 99.50th=[ 380], 99.90th=[ 380], 99.95th=[ 380], | 99.99th=[ 380] bw ( KiB/s): min= 1984, max= 4063, per=29.99%, avg=2698.33, stdev=1182.27, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 lat (msec) : 10=8.33%, 250=58.33%, 500=33.33% cpu : usr=0.00%, sys=51.65%, ctx=735, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150989: Wed Sep 24 13:42:40 2025 read: IOPS=0, BW=646KiB/s (662kB/s)(2048KiB/3168msec) clat (usec): min=859, max=113101, avg=56980.24, stdev=79366.89 lat (usec): min=860, max=113103, avg=56982.02, stdev=79367.43 clat percentiles (usec): | 1.00th=[ 857], 5.00th=[ 857], 10.00th=[ 857], 20.00th=[ 857], | 30.00th=[ 857], 40.00th=[ 857], 50.00th=[ 857], 60.00th=[112722], | 70.00th=[112722], 80.00th=[112722], 90.00th=[112722], 95.00th=[112722], | 99.00th=[112722], 99.50th=[112722], 99.90th=[112722], 99.95th=[112722], | 99.99th=[112722] bw ( KiB/s): min= 2007, max= 2031, per=31.42%, avg=2019.00, stdev=16.97, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3232KiB/s (3310kB/s)(10.0MiB/3168msec); 0 zone resets clat (msec): min=147, max=518, avg=298.48, stdev=124.97 lat (msec): min=147, max=518, avg=298.56, stdev=124.98 clat percentiles (msec): | 1.00th=[ 148], 5.00th=[ 148], 10.00th=[ 148], 20.00th=[ 153], | 30.00th=[ 199], 40.00th=[ 228], 50.00th=[ 271], 60.00th=[ 296], | 70.00th=[ 342], 80.00th=[ 376], 90.00th=[ 456], 95.00th=[ 518], | 99.00th=[ 518], 99.50th=[ 518], 99.90th=[ 518], 99.95th=[ 518], | 99.99th=[ 518] bw ( KiB/s): min= 1984, max= 6144, per=40.76%, avg=3667.00, stdev=1726.99, samples=5 iops : min= 1, max= 6, avg= 3.20, stdev= 1.92, samples=5 lat (usec) : 1000=8.33% lat (msec) : 250=41.67%, 500=41.67%, 750=8.33% cpu : usr=0.06%, sys=44.87%, ctx=555, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150991: Wed Sep 24 13:42:40 2025 read: IOPS=2, BW=2427KiB/s (2485kB/s)(6144KiB/2532msec) clat (usec): min=978, max=246112, avg=112204.87, stdev=97633.06 lat (usec): min=979, max=246113, avg=112206.03, stdev=97632.84 clat percentiles (usec): | 1.00th=[ 979], 5.00th=[ 979], 10.00th=[ 979], 20.00th=[ 1205], | 30.00th=[ 1205], 40.00th=[121111], 50.00th=[121111], 60.00th=[122160], | 70.00th=[181404], 80.00th=[181404], 90.00th=[246416], 95.00th=[246416], | 99.00th=[246416], 99.50th=[246416], 99.90th=[246416], 99.95th=[246416], | 99.99th=[246416] bw ( KiB/s): min= 2048, max= 4096, per=52.30%, avg=3360.67, stdev=1139.54, samples=3 iops : min= 2, max= 4, avg= 3.00, stdev= 1.00, samples=3 write: IOPS=2, BW=2427KiB/s (2485kB/s)(6144KiB/2532msec); 0 zone resets clat (msec): min=158, max=474, avg=304.95, stdev=119.99 lat (msec): min=158, max=474, avg=305.01, stdev=120.00 clat percentiles (msec): | 1.00th=[ 159], 5.00th=[ 159], 10.00th=[ 159], 20.00th=[ 215], | 30.00th=[ 215], 40.00th=[ 259], 50.00th=[ 259], 60.00th=[ 309], | 70.00th=[ 414], 80.00th=[ 414], 90.00th=[ 477], 95.00th=[ 477], | 99.00th=[ 477], 99.50th=[ 477], 99.90th=[ 477], 99.95th=[ 477], | 99.99th=[ 477] bw ( KiB/s): min= 1969, max= 2737, per=24.46%, avg=2200.50, stdev=359.60, samples=4 iops : min= 1, max= 2, avg= 1.75, stdev= 0.50, samples=4 lat (usec) : 1000=8.33% lat (msec) : 2=8.33%, 250=50.00%, 500=33.33% cpu : usr=0.00%, sys=39.55%, ctx=467, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=150993: Wed Sep 24 13:42:40 2025 read: IOPS=1, BW=1607KiB/s (1645kB/s)(5120KiB/3187msec) clat (usec): min=1017, max=580684, avg=211077.14, stdev=234157.23 lat (usec): min=1019, max=580685, avg=211078.55, stdev=234157.05 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1020], 10.00th=[ 1020], 20.00th=[ 1020], | 30.00th=[ 52167], 40.00th=[ 52167], 50.00th=[129500], 60.00th=[129500], | 70.00th=[291505], 80.00th=[291505], 90.00th=[583009], 95.00th=[583009], | 99.00th=[583009], 99.50th=[583009], 99.90th=[583009], 99.95th=[583009], | 99.99th=[583009] bw ( KiB/s): min= 4063, max= 4096, per=63.48%, avg=4079.50, stdev=23.33, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2249KiB/s (2303kB/s)(7168KiB/3187msec); 0 zone resets clat (msec): min=139, max=525, avg=296.25, stdev=135.19 lat (msec): min=139, max=525, avg=296.31, stdev=135.20 clat percentiles (msec): | 1.00th=[ 140], 5.00th=[ 140], 10.00th=[ 140], 20.00th=[ 197], | 30.00th=[ 211], 40.00th=[ 211], 50.00th=[ 249], 60.00th=[ 376], | 70.00th=[ 376], 80.00th=[ 380], 90.00th=[ 527], 95.00th=[ 527], | 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527], | 99.99th=[ 527] bw ( KiB/s): min= 1984, max= 4096, per=27.04%, avg=2432.40, stdev=930.31, samples=5 iops : min= 1, max= 4, avg= 1.80, stdev= 1.30, samples=5 lat (msec) : 2=8.33%, 100=8.33%, 250=41.67%, 500=25.00%, 750=16.67% cpu : usr=0.00%, sys=37.66%, ctx=593, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6426KiB/s (6580kB/s), 646KiB/s-2949KiB/s (662kB/s-3019kB/s), io=20.0MiB (20.0MB), run=2431-3187msec WRITE: bw=8997KiB/s (9212kB/s), 2106KiB/s-3232KiB/s (2157kB/s-3310kB/s), io=28.0MiB (29.4MB), run=2431-3187msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=151004: Wed Sep 24 13:42:51 2025 read: IOPS=1, BW=6596KiB/s (6754kB/s)(12.0MiB/1863msec) clat (msec): min=523, max=742, avg=615.77, stdev=113.34 lat (msec): min=523, max=742, avg=615.77, stdev=113.34 clat percentiles (msec): | 1.00th=[ 523], 5.00th=[ 523], 10.00th=[ 523], 20.00th=[ 523], | 30.00th=[ 523], 40.00th=[ 584], 50.00th=[ 584], 60.00th=[ 584], | 70.00th=[ 743], 80.00th=[ 743], 90.00th=[ 743], 95.00th=[ 743], | 99.00th=[ 743], 99.50th=[ 743], 99.90th=[ 743], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 5603, max= 7816, per=100.00%, avg=6709.50, stdev=1564.83, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=100.00% cpu : usr=0.00%, sys=39.04%, ctx=1750, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151005: Wed Sep 24 13:42:51 2025 read: IOPS=0, BW=1050KiB/s (1075kB/s)(4096KiB/3902msec) clat (nsec): min=589756k, max=589756k, avg=589755886.00, stdev= 0.00 lat (nsec): min=589757k, max=589757k, avg=589757279.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 592], 5.00th=[ 592], 10.00th=[ 592], 20.00th=[ 592], | 30.00th=[ 592], 40.00th=[ 592], 50.00th=[ 592], 60.00th=[ 592], | 70.00th=[ 592], 80.00th=[ 592], 90.00th=[ 592], 95.00th=[ 592], | 99.00th=[ 592], 99.50th=[ 592], 99.90th=[ 592], 99.95th=[ 592], | 99.99th=[ 592] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2099KiB/s (2150kB/s)(8192KiB/3902msec); 0 zone resets clat (msec): min=1476, max=1790, avg=1633.14, stdev=221.85 lat (msec): min=1476, max=1790, avg=1633.51, stdev=221.99 clat percentiles (msec): | 1.00th=[ 1469], 5.00th=[ 1469], 10.00th=[ 1469], 20.00th=[ 1469], | 30.00th=[ 1469], 40.00th=[ 1469], 50.00th=[ 1469], 60.00th=[ 1787], | 70.00th=[ 1787], 80.00th=[ 1787], 90.00th=[ 1787], 95.00th=[ 1787], | 99.00th=[ 1787], 99.50th=[ 1787], 99.90th=[ 1787], 99.95th=[ 1787], | 99.99th=[ 1787] bw ( KiB/s): min= 8192, max= 8192, per=99.26%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=37.99%, ctx=1135, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151006: Wed Sep 24 13:42:51 2025 write: IOPS=0, BW=3095KiB/s (3169kB/s)(12.0MiB/3970msec); 0 zone resets clat (msec): min=736, max=1769, avg=1314.38, stdev=527.09 lat (msec): min=737, max=1769, avg=1314.64, stdev=527.15 clat percentiles (msec): | 1.00th=[ 735], 5.00th=[ 735], 10.00th=[ 735], 20.00th=[ 735], | 30.00th=[ 735], 40.00th=[ 1435], 50.00th=[ 1435], 60.00th=[ 1435], | 70.00th=[ 1770], 80.00th=[ 1770], 90.00th=[ 1770], 95.00th=[ 1770], | 99.00th=[ 1770], 99.50th=[ 1770], 99.90th=[ 1770], 99.95th=[ 1770], | 99.99th=[ 1770] bw ( KiB/s): min= 7447, max= 8047, per=93.87%, avg=7747.00, stdev=424.26, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=49.66%, ctx=128, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151007: Wed Sep 24 13:42:51 2025 write: IOPS=0, BW=3343KiB/s (3423kB/s)(12.0MiB/3676msec); 0 zone resets clat (msec): min=829, max=1606, avg=1215.58, stdev=388.62 lat (msec): min=829, max=1606, avg=1215.89, stdev=388.56 clat percentiles (msec): | 1.00th=[ 827], 5.00th=[ 827], 10.00th=[ 827], 20.00th=[ 827], | 30.00th=[ 827], 40.00th=[ 1217], 50.00th=[ 1217], 60.00th=[ 1217], | 70.00th=[ 1603], 80.00th=[ 1603], 90.00th=[ 1603], 95.00th=[ 1603], | 99.00th=[ 1603], 99.50th=[ 1603], 99.90th=[ 1603], 99.95th=[ 1603], | 99.99th=[ 1603] bw ( KiB/s): min= 8175, max= 8192, per=99.16%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.03%, sys=54.64%, ctx=291, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4199KiB/s (4300kB/s), 1050KiB/s-6596KiB/s (1075kB/s-6754kB/s), io=16.0MiB (16.8MB), run=1863-3902msec WRITE: bw=8254KiB/s (8452kB/s), 2099KiB/s-3343KiB/s (2150kB/s-3423kB/s), io=32.0MiB (33.6MB), run=3676-3970msec rand-rw: (groupid=0, jobs=1): err= 0: pid=150999: Wed Sep 24 13:42:52 2025 read: IOPS=0, BW=3697KiB/s (3785kB/s)(12.0MiB/3324msec) clat (msec): min=725, max=1796, avg=1092.33, stdev=610.09 lat (msec): min=725, max=1796, avg=1092.33, stdev=610.09 clat percentiles (msec): | 1.00th=[ 726], 5.00th=[ 726], 10.00th=[ 726], 20.00th=[ 726], | 30.00th=[ 726], 40.00th=[ 760], 50.00th=[ 760], 60.00th=[ 760], | 70.00th=[ 1804], 80.00th=[ 1804], 90.00th=[ 1804], 95.00th=[ 1804], | 99.00th=[ 1804], 99.50th=[ 1804], 99.90th=[ 1804], 99.95th=[ 1804], | 99.99th=[ 1804] bw ( KiB/s): min= 8175, max= 8192, per=100.00%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=11.77%, ctx=90, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151000: Wed Sep 24 13:42:52 2025 read: IOPS=0, BW=941KiB/s (964kB/s)(4096KiB/4352msec) clat (nsec): min=149365k, max=149365k, avg=149365112.00, stdev= 0.00 lat (nsec): min=149368k, max=149368k, avg=149367827.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 150], 5.00th=[ 150], 10.00th=[ 150], 20.00th=[ 150], | 30.00th=[ 150], 40.00th=[ 150], 50.00th=[ 150], 60.00th=[ 150], | 70.00th=[ 150], 80.00th=[ 150], 90.00th=[ 150], 95.00th=[ 150], | 99.00th=[ 150], 99.50th=[ 150], 99.90th=[ 150], 99.95th=[ 150], | 99.99th=[ 150] bw ( KiB/s): min= 7529, max= 7529, per=100.00%, avg=7529.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1882KiB/s (1928kB/s)(8192KiB/4352msec); 0 zone resets clat (msec): min=1770, max=2385, avg=2077.86, stdev=435.07 lat (msec): min=1770, max=2385, avg=2078.16, stdev=434.80 clat percentiles (msec): | 1.00th=[ 1770], 5.00th=[ 1770], 10.00th=[ 1770], 20.00th=[ 1770], | 30.00th=[ 1770], 40.00th=[ 1770], 50.00th=[ 1770], 60.00th=[ 2400], | 70.00th=[ 2400], 80.00th=[ 2400], 90.00th=[ 2400], 95.00th=[ 2400], | 99.00th=[ 2400], 99.50th=[ 2400], 99.90th=[ 2400], 99.95th=[ 2400], | 99.99th=[ 2400] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.02%, sys=12.16%, ctx=102, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151001: Wed Sep 24 13:42:52 2025 write: IOPS=0, BW=2773KiB/s (2839kB/s)(12.0MiB/4432msec); 0 zone resets clat (msec): min=240, max=2332, avg=1462.52, stdev=1089.34 lat (msec): min=240, max=2332, avg=1462.90, stdev=1089.37 clat percentiles (msec): | 1.00th=[ 241], 5.00th=[ 241], 10.00th=[ 241], 20.00th=[ 241], | 30.00th=[ 241], 40.00th=[ 1821], 50.00th=[ 1821], 60.00th=[ 1821], | 70.00th=[ 2333], 80.00th=[ 2333], 90.00th=[ 2333], 95.00th=[ 2333], | 99.00th=[ 2333], 99.50th=[ 2333], 99.90th=[ 2333], 99.95th=[ 2333], | 99.99th=[ 2333] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.02%, sys=15.78%, ctx=135, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151002: Wed Sep 24 13:42:52 2025 write: IOPS=1, BW=4585KiB/s (4695kB/s)(12.0MiB/2680msec); 0 zone resets clat (msec): min=260, max=2100, avg=878.94, stdev=1057.71 lat (msec): min=260, max=2100, avg=879.27, stdev=1057.67 clat percentiles (msec): | 1.00th=[ 259], 5.00th=[ 259], 10.00th=[ 259], 20.00th=[ 259], | 30.00th=[ 259], 40.00th=[ 275], 50.00th=[ 275], 60.00th=[ 275], | 70.00th=[ 2106], 80.00th=[ 2106], 90.00th=[ 2106], 95.00th=[ 2106], | 99.00th=[ 2106], 99.50th=[ 2106], 99.90th=[ 2106], 99.95th=[ 2106], | 99.99th=[ 2106] bw ( KiB/s): min= 7585, max= 8175, per=100.00%, avg=7880.00, stdev=417.19, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, >=2000=33.33% cpu : usr=0.04%, sys=12.69%, ctx=112, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3765KiB/s (3855kB/s), 941KiB/s-3697KiB/s (964kB/s-3785kB/s), io=16.0MiB (16.8MB), run=3324-4352msec WRITE: bw=7394KiB/s (7571kB/s), 1882KiB/s-4585KiB/s (1928kB/s-4695kB/s), io=32.0MiB (33.6MB), run=2680-4432msec