== sanity test 398b: DIO and buffer IO race ============== 19:37:39 (1753745859) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.79858 s, 13.3 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147934: Mon Jul 28 19:39:18 2025 read: IOPS=17, BW=69.7KiB/s (71.4kB/s)(5992KiB/85908msec) clat (usec): min=97, max=680359, avg=28056.47, stdev=26924.38 lat (usec): min=97, max=680360, avg=28058.74, stdev=26924.19 clat percentiles (usec): | 1.00th=[ 141], 5.00th=[ 200], 10.00th=[ 465], 20.00th=[ 13960], | 30.00th=[ 16909], 40.00th=[ 20317], 50.00th=[ 23462], 60.00th=[ 27919], | 70.00th=[ 33817], 80.00th=[ 41681], 90.00th=[ 51643], 95.00th=[ 63701], | 99.00th=[ 96994], 99.50th=[132645], 99.90th=[240124], 99.95th=[683672], | 99.99th=[683672] bw ( KiB/s): min= 5, max= 136, per=24.85%, avg=69.34, stdev=24.50, samples=170 iops : min= 1, max= 34, avg=16.75, stdev= 6.16, samples=170 write: IOPS=18, BW=73.3KiB/s (75.0kB/s)(6296KiB/85908msec); 0 zone resets clat (usec): min=868, max=272905, avg=27814.50, stdev=28425.74 lat (usec): min=868, max=272908, avg=27818.67, stdev=28427.10 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 1287], 10.00th=[ 1516], 20.00th=[ 2868], | 30.00th=[ 6063], 40.00th=[ 12649], 50.00th=[ 21365], 60.00th=[ 28443], | 70.00th=[ 36439], 80.00th=[ 47449], 90.00th=[ 65274], 95.00th=[ 83362], | 99.00th=[125305], 99.50th=[141558], 99.90th=[179307], 99.95th=[274727], | 99.99th=[274727] bw ( KiB/s): min= 15, max= 143, per=25.25%, avg=73.74, stdev=26.40, samples=168 iops : min= 3, max= 35, avg=17.84, stdev= 6.63, samples=168 lat (usec) : 100=0.03%, 250=3.35%, 500=1.53%, 750=0.33%, 1000=0.36% lat (msec) : 2=7.32%, 4=5.37%, 10=7.19%, 20=18.62%, 50=41.15% lat (msec) : 100=13.09%, 250=1.60%, 500=0.03%, 750=0.03% cpu : usr=0.12%, sys=9.88%, ctx=6158, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147935: Mon Jul 28 19:39:18 2025 read: IOPS=17, BW=69.1KiB/s (70.7kB/s)(5940KiB/86017msec) clat (usec): min=105, max=749433, avg=28300.12, stdev=28039.67 lat (usec): min=105, max=749434, avg=28301.98, stdev=28039.57 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 184], 10.00th=[ 420], 20.00th=[ 13698], | 30.00th=[ 17171], 40.00th=[ 20841], 50.00th=[ 24249], 60.00th=[ 28705], | 70.00th=[ 34866], 80.00th=[ 41157], 90.00th=[ 52167], 95.00th=[ 61080], | 99.00th=[ 92799], 99.50th=[139461], 99.90th=[267387], 99.95th=[750781], | 99.99th=[750781] bw ( KiB/s): min= 7, max= 143, per=24.67%, avg=68.83, stdev=24.66, samples=169 iops : min= 1, max= 35, avg=16.61, stdev= 6.18, samples=169 write: IOPS=18, BW=73.8KiB/s (75.6kB/s)(6348KiB/86017msec); 0 zone resets clat (usec): min=879, max=187593, avg=27625.12, stdev=27315.79 lat (usec): min=880, max=187595, avg=27627.41, stdev=27315.75 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 1287], 10.00th=[ 1549], 20.00th=[ 2966], | 30.00th=[ 6128], 40.00th=[ 13960], 50.00th=[ 21627], 60.00th=[ 28181], | 70.00th=[ 36439], 80.00th=[ 47449], 90.00th=[ 63701], 95.00th=[ 80217], | 99.00th=[117965], 99.50th=[137364], 99.90th=[173016], 99.95th=[187696], | 99.99th=[187696] bw ( KiB/s): min= 8, max= 165, per=25.13%, avg=73.38, stdev=26.85, samples=169 iops : min= 2, max= 41, avg=17.77, stdev= 6.75, samples=169 lat (usec) : 250=3.87%, 500=1.24%, 750=0.10%, 1000=0.36% lat (msec) : 2=7.71%, 4=4.85%, 10=6.93%, 20=17.81%, 50=41.99% lat (msec) : 100=13.44%, 250=1.63%, 500=0.03%, 750=0.03% cpu : usr=0.15%, sys=9.86%, ctx=6026, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147936: Mon Jul 28 19:39:18 2025 read: IOPS=17, BW=70.2KiB/s (71.9kB/s)(6028KiB/85809msec) clat (usec): min=103, max=156099, avg=26585.15, stdev=19703.85 lat (usec): min=103, max=156101, avg=26587.76, stdev=19702.69 clat percentiles (usec): | 1.00th=[ 135], 5.00th=[ 190], 10.00th=[ 326], 20.00th=[ 12387], | 30.00th=[ 16319], 40.00th=[ 19530], 50.00th=[ 23200], 60.00th=[ 27657], | 70.00th=[ 32900], 80.00th=[ 41157], 90.00th=[ 51119], 95.00th=[ 58983], | 99.00th=[ 92799], 99.50th=[116917], 99.90th=[149947], 99.95th=[156238], | 99.99th=[156238] bw ( KiB/s): min= 15, max= 141, per=25.21%, avg=70.35, stdev=26.08, samples=168 iops : min= 3, max= 35, avg=16.99, stdev= 6.56, samples=168 write: IOPS=18, BW=72.0KiB/s (74.7kB/s)(6260KiB/85809msec); 0 zone resets clat (usec): min=891, max=871900, avg=29154.35, stdev=35604.61 lat (usec): min=891, max=871902, avg=29156.24, stdev=35604.64 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 1319], 10.00th=[ 1631], 20.00th=[ 3326], | 30.00th=[ 6980], 40.00th=[ 14615], 50.00th=[ 21890], 60.00th=[ 28443], | 70.00th=[ 36963], 80.00th=[ 48497], 90.00th=[ 65799], 95.00th=[ 83362], | 99.00th=[128451], 99.50th=[152044], 99.90th=[208667], 99.95th=[868221], | 99.99th=[868221] bw ( KiB/s): min= 23, max= 158, per=24.96%, avg=72.87, stdev=24.17, samples=168 iops : min= 5, max= 39, avg=17.62, stdev= 6.07, samples=168 lat (usec) : 250=3.84%, 500=1.86%, 750=0.33%, 1000=0.26% lat (msec) : 2=7.00%, 4=5.08%, 10=6.41%, 20=19.79%, 50=40.07% lat (msec) : 100=13.44%, 250=1.89%, 1000=0.03% cpu : usr=0.15%, sys=9.92%, ctx=6031, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147937: Mon Jul 28 19:39:18 2025 read: IOPS=17, BW=70.8KiB/s (72.5kB/s)(6052KiB/85479msec) clat (usec): min=111, max=201189, avg=26248.03, stdev=19287.61 lat (usec): min=112, max=201190, avg=26249.81, stdev=19287.52 clat percentiles (usec): | 1.00th=[ 151], 5.00th=[ 194], 10.00th=[ 363], 20.00th=[ 13173], | 30.00th=[ 16909], 40.00th=[ 19530], 50.00th=[ 22938], 60.00th=[ 27395], | 70.00th=[ 32375], 80.00th=[ 39584], 90.00th=[ 48497], 95.00th=[ 58459], | 99.00th=[ 80217], 99.50th=[103285], 99.90th=[179307], 99.95th=[200279], | 99.99th=[200279] bw ( KiB/s): min= 8, max= 143, per=25.57%, avg=71.34, stdev=24.49, samples=167 iops : min= 2, max= 35, avg=17.25, stdev= 6.19, samples=167 write: IOPS=18, BW=72.0KiB/s (74.7kB/s)(6236KiB/85479msec); 0 zone resets clat (usec): min=806, max=893509, avg=29277.56, stdev=37999.50 lat (usec): min=806, max=893512, avg=29279.80, stdev=37999.32 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 1287], 10.00th=[ 1549], 20.00th=[ 3130], | 30.00th=[ 6194], 40.00th=[ 12780], 50.00th=[ 22676], 60.00th=[ 29754], | 70.00th=[ 38536], 80.00th=[ 49546], 90.00th=[ 65799], 95.00th=[ 80217], | 99.00th=[125305], 99.50th=[149947], 99.90th=[480248], 99.95th=[893387], | 99.99th=[893387] bw ( KiB/s): min= 8, max= 175, per=25.02%, avg=73.07, stdev=25.18, samples=168 iops : min= 2, max= 43, avg=17.67, stdev= 6.32, samples=168 lat (usec) : 250=3.78%, 500=1.89%, 750=0.36%, 1000=0.39% lat (msec) : 2=7.10%, 4=4.95%, 10=7.03%, 20=18.52%, 50=41.47% lat (msec) : 100=13.18%, 250=1.24%, 500=0.07%, 1000=0.03% cpu : usr=0.16%, sys=9.93%, ctx=6029, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=279KiB/s (286kB/s), 69.1KiB/s-70.8KiB/s (70.7kB/s-72.5kB/s), io=23.4MiB (24.6MB), run=85479-86017msec WRITE: bw=292KiB/s (299kB/s), 72.0KiB/s-73.8KiB/s (74.7kB/s-75.6kB/s), io=24.6MiB (25.7MB), run=85479-86017msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147928: Mon Jul 28 19:41:13 2025 read: IOPS=7, BW=29.8KiB/s (30.6kB/s)(5992KiB/200830msec) clat (msec): min=5, max=732, avg=29.49, stdev=29.24 lat (msec): min=5, max=732, avg=29.50, stdev=29.24 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 19], 50.00th=[ 24], 60.00th=[ 29], | 70.00th=[ 35], 80.00th=[ 43], 90.00th=[ 54], 95.00th=[ 67], | 99.00th=[ 121], 99.50th=[ 146], 99.90th=[ 342], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 7, max= 104, per=26.16%, avg=31.14, stdev=19.63, samples=375 iops : min= 1, max= 26, avg= 7.17, stdev= 4.93, samples=375 write: IOPS=7, BW=31.3KiB/s (32.1kB/s)(6296KiB/200830msec); 0 zone resets clat (msec): min=36, max=390, avg=99.30, stdev=46.33 lat (msec): min=36, max=390, avg=99.30, stdev=46.32 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 50], 10.00th=[ 55], 20.00th=[ 63], | 30.00th=[ 70], 40.00th=[ 78], 50.00th=[ 86], 60.00th=[ 97], | 70.00th=[ 112], 80.00th=[ 133], 90.00th=[ 167], 95.00th=[ 192], | 99.00th=[ 257], 99.50th=[ 284], 99.90th=[ 380], 99.95th=[ 393], | 99.99th=[ 393] bw ( KiB/s): min= 7, max= 64, per=24.76%, avg=30.94, stdev=12.65, samples=396 iops : min= 1, max= 16, avg= 7.13, stdev= 3.20, samples=396 lat (msec) : 10=5.92%, 20=15.23%, 50=24.35%, 100=34.80%, 250=19.08% lat (msec) : 500=0.59%, 750=0.03% cpu : usr=0.07%, sys=2.35%, ctx=4391, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147929: Mon Jul 28 19:41:13 2025 read: IOPS=7, BW=29.6KiB/s (30.3kB/s)(5940KiB/200573msec) clat (msec): min=5, max=804, avg=28.13, stdev=29.32 lat (msec): min=5, max=804, avg=28.14, stdev=29.32 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 18], 50.00th=[ 23], 60.00th=[ 28], | 70.00th=[ 34], 80.00th=[ 41], 90.00th=[ 50], 95.00th=[ 63], | 99.00th=[ 110], 99.50th=[ 138], 99.90th=[ 266], 99.95th=[ 802], | 99.99th=[ 802] bw ( KiB/s): min= 7, max= 111, per=26.24%, avg=31.23, stdev=19.13, samples=370 iops : min= 1, max= 27, avg= 7.20, stdev= 4.81, samples=370 write: IOPS=7, BW=31.6KiB/s (32.4kB/s)(6348KiB/200573msec); 0 zone resets clat (msec): min=33, max=363, avg=99.84, stdev=45.96 lat (msec): min=33, max=363, avg=99.84, stdev=45.96 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 50], 10.00th=[ 56], 20.00th=[ 65], | 30.00th=[ 72], 40.00th=[ 81], 50.00th=[ 88], 60.00th=[ 97], | 70.00th=[ 111], 80.00th=[ 128], 90.00th=[ 161], 95.00th=[ 197], | 99.00th=[ 264], 99.50th=[ 300], 99.90th=[ 363], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 7, max= 63, per=25.10%, avg=31.37, stdev=11.73, samples=393 iops : min= 1, max= 15, avg= 7.23, stdev= 2.97, samples=393 lat (msec) : 10=6.74%, 20=14.81%, 50=24.87%, 100=33.82%, 250=19.08% lat (msec) : 500=0.65%, 1000=0.03% cpu : usr=0.07%, sys=2.35%, ctx=4335, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147930: Mon Jul 28 19:41:13 2025 read: IOPS=7, BW=30.2KiB/s (30.9kB/s)(6028KiB/199715msec) clat (msec): min=4, max=229, avg=28.64, stdev=21.15 lat (msec): min=4, max=229, avg=28.64, stdev=21.15 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 13], | 30.00th=[ 16], 40.00th=[ 21], 50.00th=[ 25], 60.00th=[ 30], | 70.00th=[ 34], 80.00th=[ 41], 90.00th=[ 52], 95.00th=[ 63], | 99.00th=[ 101], 99.50th=[ 125], 99.90th=[ 211], 99.95th=[ 230], | 99.99th=[ 230] bw ( KiB/s): min= 7, max= 119, per=26.15%, avg=31.12, stdev=19.44, samples=376 iops : min= 1, max= 29, avg= 7.16, stdev= 4.88, samples=376 write: IOPS=7, BW=31.3KiB/s (32.1kB/s)(6260KiB/199715msec); 0 zone resets clat (msec): min=32, max=903, avg=99.79, stdev=50.60 lat (msec): min=32, max=903, avg=99.80, stdev=50.60 clat percentiles (msec): | 1.00th=[ 43], 5.00th=[ 51], 10.00th=[ 56], 20.00th=[ 64], | 30.00th=[ 72], 40.00th=[ 80], 50.00th=[ 87], 60.00th=[ 97], | 70.00th=[ 109], 80.00th=[ 128], 90.00th=[ 163], 95.00th=[ 192], | 99.00th=[ 245], 99.50th=[ 305], 99.90th=[ 531], 99.95th=[ 902], | 99.99th=[ 902] bw ( KiB/s): min= 7, max= 63, per=24.73%, avg=30.92, stdev=12.62, samples=393 iops : min= 1, max= 15, avg= 7.12, stdev= 3.19, samples=393 lat (msec) : 10=5.53%, 20=14.03%, 50=26.66%, 100=34.41%, 250=18.91% lat (msec) : 500=0.39%, 750=0.03%, 1000=0.03% cpu : usr=0.07%, sys=2.38%, ctx=4358, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147931: Mon Jul 28 19:41:13 2025 read: IOPS=7, BW=30.2KiB/s (30.0kB/s)(6052KiB/200197msec) clat (msec): min=5, max=219, avg=28.25, stdev=20.33 lat (msec): min=5, max=219, avg=28.25, stdev=20.33 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 19], 50.00th=[ 24], 60.00th=[ 30], | 70.00th=[ 35], 80.00th=[ 42], 90.00th=[ 52], 95.00th=[ 65], | 99.00th=[ 96], 99.50th=[ 123], 99.90th=[ 159], 99.95th=[ 220], | 99.99th=[ 220] bw ( KiB/s): min= 7, max= 96, per=26.56%, avg=31.61, stdev=18.94, samples=372 iops : min= 1, max= 24, avg= 7.29, stdev= 4.74, samples=372 write: IOPS=7, BW=31.1KiB/s (31.9kB/s)(6236KiB/200197msec); 0 zone resets clat (msec): min=27, max=902, avg=100.78, stdev=52.11 lat (msec): min=27, max=902, avg=100.78, stdev=52.11 clat percentiles (msec): | 1.00th=[ 40], 5.00th=[ 50], 10.00th=[ 56], 20.00th=[ 64], | 30.00th=[ 72], 40.00th=[ 80], 50.00th=[ 87], 60.00th=[ 97], | 70.00th=[ 111], 80.00th=[ 131], 90.00th=[ 165], 95.00th=[ 201], | 99.00th=[ 255], 99.50th=[ 296], 99.90th=[ 527], 99.95th=[ 902], | 99.99th=[ 902] bw ( KiB/s): min= 7, max= 63, per=24.49%, avg=30.62, stdev=12.58, samples=395 iops : min= 1, max= 15, avg= 7.04, stdev= 3.18, samples=395 lat (msec) : 10=6.90%, 20=14.32%, 50=25.42%, 100=34.21%, 250=18.49% lat (msec) : 500=0.59%, 750=0.03%, 1000=0.03% cpu : usr=0.06%, sys=2.37%, ctx=4377, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=120KiB/s (122kB/s), 29.6KiB/s-30.2KiB/s (30.3kB/s-30.0kB/s), io=23.4MiB (24.6MB), run=199715-200830msec WRITE: bw=125KiB/s (128kB/s), 31.1KiB/s-31.6KiB/s (31.9kB/s-32.4kB/s), io=24.6MiB (25.7MB), run=199715-200830msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147984: Mon Jul 28 19:41:35 2025 read: IOPS=24, BW=388KiB/s (398kB/s)(5920KiB/15246msec) clat (usec): min=115, max=138029, avg=19281.33, stdev=17624.20 lat (usec): min=115, max=138030, avg=19282.49, stdev=17624.20 clat percentiles (usec): | 1.00th=[ 130], 5.00th=[ 172], 10.00th=[ 217], 20.00th=[ 461], | 30.00th=[ 11207], 40.00th=[ 14222], 50.00th=[ 17171], 60.00th=[ 20317], | 70.00th=[ 25297], 80.00th=[ 30802], 90.00th=[ 38536], 95.00th=[ 45351], | 99.00th=[ 80217], 99.50th=[122160], 99.90th=[137364], 99.95th=[137364], | 99.99th=[137364] bw ( KiB/s): min= 191, max= 800, per=25.72%, avg=387.37, stdev=150.22, samples=30 iops : min= 11, max= 50, avg=23.63, stdev= 9.46, samples=30 write: IOPS=26, BW=418KiB/s (428kB/s)(6368KiB/15246msec); 0 zone resets clat (usec): min=1471, max=174845, avg=20276.83, stdev=21730.89 lat (usec): min=1473, max=174847, avg=20281.05, stdev=21729.75 clat percentiles (usec): | 1.00th=[ 1582], 5.00th=[ 2376], 10.00th=[ 2737], 20.00th=[ 4228], | 30.00th=[ 6521], 40.00th=[ 9503], 50.00th=[ 13829], 60.00th=[ 19268], | 70.00th=[ 24511], 80.00th=[ 31851], 90.00th=[ 44303], 95.00th=[ 55313], | 99.00th=[112722], 99.50th=[175113], 99.90th=[175113], 99.95th=[175113], | 99.99th=[175113] bw ( KiB/s): min= 188, max= 704, per=26.41%, avg=418.27, stdev=115.47, samples=30 iops : min= 11, max= 44, avg=25.57, stdev= 7.34, samples=30 lat (usec) : 250=5.47%, 500=4.43%, 750=1.43% lat (msec) : 2=2.21%, 4=8.72%, 10=12.24%, 20=26.04%, 50=33.98% lat (msec) : 100=4.69%, 250=0.78% cpu : usr=0.14%, sys=17.38%, ctx=1656, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147985: Mon Jul 28 19:41:35 2025 read: IOPS=22, BW=361KiB/s (370kB/s)(5744KiB/15901msec) clat (usec): min=110, max=160596, avg=20505.75, stdev=18429.28 lat (usec): min=111, max=160597, avg=20506.87, stdev=18429.33 clat percentiles (usec): | 1.00th=[ 147], 5.00th=[ 206], 10.00th=[ 285], 20.00th=[ 537], | 30.00th=[ 11207], 40.00th=[ 15926], 50.00th=[ 19006], 60.00th=[ 22676], | 70.00th=[ 26608], 80.00th=[ 31589], 90.00th=[ 39584], 95.00th=[ 48497], | 99.00th=[103285], 99.50th=[106431], 99.90th=[160433], 99.95th=[160433], | 99.99th=[160433] bw ( KiB/s): min= 31, max= 672, per=23.74%, avg=357.55, stdev=138.20, samples=31 iops : min= 1, max= 42, avg=21.74, stdev= 8.72, samples=31 write: IOPS=25, BW=412KiB/s (421kB/s)(6544KiB/15901msec); 0 zone resets clat (usec): min=1151, max=180030, avg=20818.49, stdev=23955.58 lat (usec): min=1153, max=180033, avg=20820.56, stdev=23955.56 clat percentiles (usec): | 1.00th=[ 1467], 5.00th=[ 1762], 10.00th=[ 2278], 20.00th=[ 3785], | 30.00th=[ 5604], 40.00th=[ 7963], 50.00th=[ 12911], 60.00th=[ 18220], | 70.00th=[ 23987], 80.00th=[ 31589], 90.00th=[ 49546], 95.00th=[ 64226], | 99.00th=[112722], 99.50th=[123208], 99.90th=[179307], 99.95th=[179307], | 99.99th=[179307] bw ( KiB/s): min= 159, max= 638, per=25.35%, avg=401.55, stdev=137.60, samples=31 iops : min= 9, max= 39, avg=24.52, stdev= 8.60, samples=31 lat (usec) : 250=3.65%, 500=5.47%, 750=0.78%, 1000=0.39% lat (msec) : 2=4.17%, 4=7.42%, 10=14.84%, 20=22.40%, 50=33.59% lat (msec) : 100=5.73%, 250=1.56% cpu : usr=0.10%, sys=16.48%, ctx=1598, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147986: Mon Jul 28 19:41:35 2025 read: IOPS=25, BW=409KiB/s (419kB/s)(6256KiB/15284msec) clat (usec): min=108, max=217850, avg=20494.25, stdev=19951.28 lat (usec): min=109, max=217851, avg=20495.60, stdev=19951.33 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 184], 10.00th=[ 281], 20.00th=[ 627], | 30.00th=[ 13042], 40.00th=[ 16450], 50.00th=[ 19006], 60.00th=[ 22676], | 70.00th=[ 26870], 80.00th=[ 30540], 90.00th=[ 37487], 95.00th=[ 43254], | 99.00th=[ 74974], 99.50th=[191890], 99.90th=[217056], 99.95th=[217056], | 99.99th=[217056] bw ( KiB/s): min= 128, max= 830, per=27.42%, avg=412.90, stdev=157.17, samples=30 iops : min= 8, max= 51, avg=25.23, stdev= 9.74, samples=30 write: IOPS=24, BW=395KiB/s (404kB/s)(6032KiB/15284msec); 0 zone resets clat (usec): min=1289, max=173332, avg=19116.88, stdev=21520.76 lat (usec): min=1290, max=173333, avg=19159.27, stdev=21506.62 clat percentiles (usec): | 1.00th=[ 1483], 5.00th=[ 1876], 10.00th=[ 2606], 20.00th=[ 3654], | 30.00th=[ 5145], 40.00th=[ 7308], 50.00th=[ 11600], 60.00th=[ 17171], | 70.00th=[ 22414], 80.00th=[ 32375], 90.00th=[ 44303], 95.00th=[ 57410], | 99.00th=[114820], 99.50th=[143655], 99.90th=[173016], 99.95th=[173016], | 99.99th=[173016] bw ( KiB/s): min= 64, max= 665, per=24.72%, avg=391.50, stdev=114.46, samples=30 iops : min= 4, max= 41, avg=23.90, stdev= 7.16, samples=30 lat (usec) : 250=4.43%, 500=4.95%, 750=1.30%, 1000=0.39% lat (msec) : 2=3.52%, 4=7.68%, 10=12.89%, 20=23.70%, 50=36.07% lat (msec) : 100=4.17%, 250=0.91% cpu : usr=0.17%, sys=17.29%, ctx=1559, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147987: Mon Jul 28 19:41:35 2025 read: IOPS=24, BW=390KiB/s (400kB/s)(6032KiB/15456msec) clat (usec): min=119, max=164038, avg=20735.16, stdev=18274.66 lat (usec): min=120, max=164039, avg=20736.41, stdev=18274.73 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 190], 10.00th=[ 269], 20.00th=[ 979], | 30.00th=[ 12911], 40.00th=[ 16188], 50.00th=[ 19006], 60.00th=[ 21365], | 70.00th=[ 25822], 80.00th=[ 30540], 90.00th=[ 40633], 95.00th=[ 52691], | 99.00th=[101188], 99.50th=[106431], 99.90th=[164627], 99.95th=[164627], | 99.99th=[164627] bw ( KiB/s): min= 192, max= 606, per=25.58%, avg=385.17, stdev=110.70, samples=30 iops : min= 12, max= 37, avg=23.50, stdev= 6.95, samples=30 write: IOPS=25, BW=405KiB/s (414kB/s)(6256KiB/15456msec); 0 zone resets clat (usec): min=1589, max=132360, avg=19461.09, stdev=20440.45 lat (usec): min=1591, max=132363, avg=19463.27, stdev=20440.42 clat percentiles (usec): | 1.00th=[ 1729], 5.00th=[ 2180], 10.00th=[ 2802], 20.00th=[ 4080], | 30.00th=[ 5866], 40.00th=[ 8356], 50.00th=[ 13173], 60.00th=[ 17433], | 70.00th=[ 24249], 80.00th=[ 32900], 90.00th=[ 43779], 95.00th=[ 54264], | 99.00th=[115868], 99.50th=[127402], 99.90th=[132645], 99.95th=[132645], | 99.99th=[132645] bw ( KiB/s): min= 160, max= 608, per=25.53%, avg=404.33, stdev=109.54, samples=30 iops : min= 10, max= 38, avg=24.70, stdev= 6.86, samples=30 lat (usec) : 250=4.04%, 500=4.69%, 750=0.65%, 1000=0.52% lat (msec) : 2=2.08%, 4=8.85%, 10=13.28%, 20=25.52%, 50=34.90% lat (msec) : 100=4.17%, 250=1.30% cpu : usr=0.17%, sys=17.32%, ctx=1579, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1506KiB/s (1542kB/s), 361KiB/s-409KiB/s (370kB/s-419kB/s), io=23.4MiB (24.5MB), run=15246-15901msec WRITE: bw=1585KiB/s (1623kB/s), 395KiB/s-418KiB/s (404kB/s-428kB/s), io=24.6MiB (25.8MB), run=15246-15901msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147979: Mon Jul 28 19:42:05 2025 read: IOPS=7, BW=126KiB/s (129kB/s)(5920KiB/47040msec) clat (msec): min=6, max=229, avg=24.09, stdev=21.17 lat (msec): min=6, max=229, avg=24.09, stdev=21.17 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 13], | 30.00th=[ 14], 40.00th=[ 15], 50.00th=[ 18], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 33], 90.00th=[ 45], 95.00th=[ 68], | 99.00th=[ 109], 99.50th=[ 120], 99.90th=[ 230], 99.95th=[ 230], | 99.99th=[ 230] bw ( KiB/s): min= 31, max= 542, per=24.98%, avg=127.15, stdev=80.04, samples=91 iops : min= 1, max= 33, avg= 7.26, stdev= 5.04, samples=91 write: IOPS=8, BW=135KiB/s (139kB/s)(6368KiB/47040msec); 0 zone resets clat (msec): min=35, max=299, avg=95.55, stdev=37.30 lat (msec): min=35, max=299, avg=95.55, stdev=37.30 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 53], 10.00th=[ 59], 20.00th=[ 68], | 30.00th=[ 74], 40.00th=[ 82], 50.00th=[ 87], 60.00th=[ 94], | 70.00th=[ 106], 80.00th=[ 120], 90.00th=[ 140], 95.00th=[ 167], | 99.00th=[ 232], 99.50th=[ 284], 99.90th=[ 300], 99.95th=[ 300], | 99.99th=[ 300] bw ( KiB/s): min= 32, max= 224, per=25.13%, avg=134.43, stdev=44.13, samples=93 iops : min= 2, max= 14, avg= 7.71, stdev= 2.83, samples=93 lat (msec) : 10=4.82%, 20=25.39%, 50=15.89%, 100=35.42%, 250=18.23% lat (msec) : 500=0.26% cpu : usr=0.05%, sys=2.94%, ctx=1052, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147980: Mon Jul 28 19:42:05 2025 read: IOPS=7, BW=123KiB/s (126kB/s)(5744KiB/46540msec) clat (msec): min=5, max=116, avg=22.32, stdev=17.77 lat (msec): min=5, max=116, avg=22.32, stdev=17.77 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 23], 80.00th=[ 31], 90.00th=[ 42], 95.00th=[ 62], | 99.00th=[ 106], 99.50th=[ 114], 99.90th=[ 116], 99.95th=[ 116], | 99.99th=[ 116] bw ( KiB/s): min= 31, max= 478, per=24.60%, avg=125.21, stdev=80.32, samples=90 iops : min= 1, max= 29, avg= 7.13, stdev= 5.06, samples=90 write: IOPS=8, BW=141KiB/s (144kB/s)(6544KiB/46540msec); 0 zone resets clat (msec): min=40, max=422, avg=93.95, stdev=36.13 lat (msec): min=40, max=422, avg=93.96, stdev=36.14 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 54], 10.00th=[ 59], 20.00th=[ 69], | 30.00th=[ 75], 40.00th=[ 81], 50.00th=[ 88], 60.00th=[ 95], | 70.00th=[ 105], 80.00th=[ 113], 90.00th=[ 133], 95.00th=[ 157], | 99.00th=[ 220], 99.50th=[ 245], 99.90th=[ 422], 99.95th=[ 422], | 99.99th=[ 422] bw ( KiB/s): min= 32, max= 224, per=26.23%, avg=140.34, stdev=38.49, samples=92 iops : min= 2, max= 14, avg= 8.08, stdev= 2.46, samples=92 lat (msec) : 10=5.47%, 20=25.39%, 50=13.93%, 100=36.59%, 250=18.36% lat (msec) : 500=0.26% cpu : usr=0.08%, sys=2.95%, ctx=1024, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147981: Mon Jul 28 19:42:05 2025 read: IOPS=8, BW=141KiB/s (145kB/s)(6256KiB/44264msec) clat (msec): min=5, max=280, avg=22.84, stdev=24.94 lat (msec): min=5, max=280, avg=22.84, stdev=24.94 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 13], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 23], 80.00th=[ 28], 90.00th=[ 42], 95.00th=[ 61], | 99.00th=[ 115], 99.50th=[ 275], 99.90th=[ 279], 99.95th=[ 279], | 99.99th=[ 279] bw ( KiB/s): min= 31, max= 407, per=27.94%, avg=142.21, stdev=95.51, samples=87 iops : min= 1, max= 25, avg= 8.20, stdev= 6.05, samples=87 write: IOPS=8, BW=136KiB/s (140kB/s)(6032KiB/44264msec); 0 zone resets clat (msec): min=39, max=296, avg=93.45, stdev=37.06 lat (msec): min=39, max=296, avg=93.46, stdev=37.06 clat percentiles (msec): | 1.00th=[ 47], 5.00th=[ 52], 10.00th=[ 58], 20.00th=[ 66], | 30.00th=[ 72], 40.00th=[ 80], 50.00th=[ 88], 60.00th=[ 94], | 70.00th=[ 103], 80.00th=[ 114], 90.00th=[ 131], 95.00th=[ 163], | 99.00th=[ 257], 99.50th=[ 279], 99.90th=[ 296], 99.95th=[ 296], | 99.99th=[ 296] bw ( KiB/s): min= 63, max= 255, per=25.66%, avg=137.26, stdev=37.58, samples=86 iops : min= 3, max= 15, avg= 7.88, stdev= 2.39, samples=86 lat (msec) : 10=6.25%, 20=27.60%, 50=15.23%, 100=34.77%, 250=15.36% lat (msec) : 500=0.78% cpu : usr=0.07%, sys=3.02%, ctx=1032, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147982: Mon Jul 28 19:42:05 2025 read: IOPS=8, BW=130KiB/s (133kB/s)(6032KiB/46369msec) clat (msec): min=6, max=230, avg=24.38, stdev=24.44 lat (msec): min=6, max=230, avg=24.38, stdev=24.44 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 25], 80.00th=[ 33], 90.00th=[ 45], 95.00th=[ 63], | 99.00th=[ 132], 99.50th=[ 215], 99.90th=[ 230], 99.95th=[ 230], | 99.99th=[ 230] bw ( KiB/s): min= 31, max= 382, per=25.82%, avg=131.43, stdev=73.18, samples=90 iops : min= 1, max= 23, avg= 7.52, stdev= 4.62, samples=90 write: IOPS=8, BW=135KiB/s (138kB/s)(6256KiB/46369msec); 0 zone resets clat (msec): min=29, max=299, avg=94.95, stdev=35.58 lat (msec): min=29, max=299, avg=94.96, stdev=35.58 clat percentiles (msec): | 1.00th=[ 45], 5.00th=[ 54], 10.00th=[ 61], 20.00th=[ 68], | 30.00th=[ 77], 40.00th=[ 83], 50.00th=[ 88], 60.00th=[ 94], | 70.00th=[ 104], 80.00th=[ 115], 90.00th=[ 142], 95.00th=[ 157], | 99.00th=[ 220], 99.50th=[ 275], 99.90th=[ 300], 99.95th=[ 300], | 99.99th=[ 300] bw ( KiB/s): min= 31, max= 224, per=25.12%, avg=134.41, stdev=41.64, samples=92 iops : min= 1, max= 14, avg= 7.71, stdev= 2.65, samples=92 lat (msec) : 10=7.42%, 20=23.70%, 50=15.89%, 100=34.11%, 250=18.49% lat (msec) : 500=0.39% cpu : usr=0.08%, sys=2.97%, ctx=1042, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=509KiB/s (521kB/s), 123KiB/s-141KiB/s (126kB/s-145kB/s), io=23.4MiB (24.5MB), run=44264-47040msec WRITE: bw=536KiB/s (549kB/s), 135KiB/s-141KiB/s (138kB/s-144kB/s), io=24.6MiB (25.8MB), run=44264-47040msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148003: Mon Jul 28 19:42:16 2025 read: IOPS=2, BW=2531KiB/s (2592kB/s)(7168KiB/2832msec) clat (msec): min=137, max=266, avg=183.92, stdev=43.55 lat (msec): min=137, max=266, avg=183.92, stdev=43.55 clat percentiles (msec): | 1.00th=[ 138], 5.00th=[ 138], 10.00th=[ 138], 20.00th=[ 153], | 30.00th=[ 171], 40.00th=[ 171], 50.00th=[ 174], 60.00th=[ 174], | 70.00th=[ 174], 80.00th=[ 215], 90.00th=[ 268], 95.00th=[ 268], | 99.00th=[ 268], 99.50th=[ 268], 99.90th=[ 268], 99.95th=[ 268], | 99.99th=[ 268] bw ( KiB/s): min= 2035, max= 5427, per=73.55%, avg=3818.00, stdev=1702.68, samples=3 iops : min= 1, max= 5, avg= 3.00, stdev= 2.00, samples=3 write: IOPS=1, BW=1808KiB/s (1851kB/s)(5120KiB/2832msec); 0 zone resets clat (msec): min=125, max=675, avg=302.97, stdev=219.94 lat (msec): min=125, max=675, avg=303.06, stdev=219.93 clat percentiles (msec): | 1.00th=[ 126], 5.00th=[ 126], 10.00th=[ 126], 20.00th=[ 126], | 30.00th=[ 174], 40.00th=[ 174], 50.00th=[ 224], 60.00th=[ 224], | 70.00th=[ 317], 80.00th=[ 317], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 1996, max= 3792, per=33.90%, avg=2463.50, stdev=885.84, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 250=75.00%, 500=16.67%, 750=8.33% cpu : usr=0.04%, sys=28.65%, ctx=340, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148004: Mon Jul 28 19:42:16 2025 read: IOPS=0, BW=519KiB/s (532kB/s)(2048KiB/3945msec) clat (usec): min=1174, max=368016, avg=184595.20, stdev=259396.48 lat (usec): min=1175, max=368018, avg=184597.27, stdev=259397.01 clat percentiles (usec): | 1.00th=[ 1172], 5.00th=[ 1172], 10.00th=[ 1172], 20.00th=[ 1172], | 30.00th=[ 1172], 40.00th=[ 1172], 50.00th=[ 1172], 60.00th=[367002], | 70.00th=[367002], 80.00th=[367002], 90.00th=[367002], 95.00th=[367002], | 99.00th=[367002], 99.50th=[367002], 99.90th=[367002], 99.95th=[367002], | 99.99th=[367002] bw ( KiB/s): min= 1996, max= 2043, per=38.90%, avg=2019.50, stdev=33.23, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2596KiB/s (2658kB/s)(10.0MiB/3945msec); 0 zone resets clat (msec): min=107, max=1199, avg=332.79, stdev=320.39 lat (msec): min=107, max=1199, avg=332.90, stdev=320.40 clat percentiles (msec): | 1.00th=[ 108], 5.00th=[ 108], 10.00th=[ 108], 20.00th=[ 127], | 30.00th=[ 153], 40.00th=[ 169], 50.00th=[ 247], 60.00th=[ 262], | 70.00th=[ 317], 80.00th=[ 330], 90.00th=[ 418], 95.00th=[ 1200], | 99.00th=[ 1200], 99.50th=[ 1200], 99.90th=[ 1200], 99.95th=[ 1200], | 99.99th=[ 1200] bw ( KiB/s): min= 1996, max= 4096, per=44.05%, avg=3201.20, stdev=1089.37, samples=5 iops : min= 1, max= 4, avg= 2.40, stdev= 1.34, samples=5 lat (msec) : 2=8.33%, 250=41.67%, 500=41.67%, 2000=8.33% cpu : usr=0.10%, sys=25.13%, ctx=125, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148005: Mon Jul 28 19:42:16 2025 read: IOPS=2, BW=2149KiB/s (2201kB/s)(6144KiB/2859msec) clat (usec): min=688, max=198125, avg=37087.79, stdev=79139.60 lat (usec): min=689, max=198127, avg=37089.14, stdev=79140.02 clat percentiles (usec): | 1.00th=[ 693], 5.00th=[ 693], 10.00th=[ 693], 20.00th=[ 758], | 30.00th=[ 758], 40.00th=[ 1778], 50.00th=[ 1778], 60.00th=[ 4047], | 70.00th=[ 17171], 80.00th=[ 17171], 90.00th=[198181], 95.00th=[198181], | 99.00th=[198181], 99.50th=[198181], 99.90th=[198181], 99.95th=[198181], | 99.99th=[198181] bw ( KiB/s): min= 3792, max= 6107, per=95.35%, avg=4949.50, stdev=1636.95, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=2, BW=2149KiB/s (2201kB/s)(6144KiB/2859msec); 0 zone resets clat (msec): min=154, max=703, avg=420.87, stdev=239.63 lat (msec): min=154, max=703, avg=420.94, stdev=239.61 clat percentiles (msec): | 1.00th=[ 155], 5.00th=[ 155], 10.00th=[ 155], 20.00th=[ 174], | 30.00th=[ 174], 40.00th=[ 347], 50.00th=[ 347], 60.00th=[ 464], | 70.00th=[ 684], 80.00th=[ 684], 90.00th=[ 701], 95.00th=[ 701], | 99.00th=[ 701], 99.50th=[ 701], 99.90th=[ 701], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 1756, max= 4071, per=33.44%, avg=2429.75, stdev=1098.59, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (usec) : 750=8.33%, 1000=8.33% lat (msec) : 2=8.33%, 10=8.33%, 20=8.33%, 250=25.00%, 500=16.67% lat (msec) : 750=16.67% cpu : usr=0.00%, sys=21.90%, ctx=107, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148006: Mon Jul 28 19:42:16 2025 read: IOPS=1, BW=1589KiB/s (1627kB/s)(5120KiB/3222msec) clat (usec): min=966, max=238811, avg=82816.75, stdev=113677.13 lat (usec): min=968, max=238812, avg=82818.34, stdev=113676.88 clat percentiles (usec): | 1.00th=[ 963], 5.00th=[ 963], 10.00th=[ 963], 20.00th=[ 963], | 30.00th=[ 1549], 40.00th=[ 1549], 50.00th=[ 2638], 60.00th=[ 2638], | 70.00th=[170918], 80.00th=[170918], 90.00th=[238027], 95.00th=[238027], | 99.00th=[238027], 99.50th=[238027], 99.90th=[238027], 99.95th=[238027], | 99.99th=[238027] bw ( KiB/s): min= 2035, max= 6095, per=78.31%, avg=4065.00, stdev=2870.85, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=2, BW=2225KiB/s (2278kB/s)(7168KiB/3222msec); 0 zone resets clat (msec): min=276, max=537, avg=391.82, stdev=101.38 lat (msec): min=276, max=538, avg=391.92, stdev=101.38 clat percentiles (msec): | 1.00th=[ 275], 5.00th=[ 275], 10.00th=[ 275], 20.00th=[ 284], | 30.00th=[ 326], 40.00th=[ 326], 50.00th=[ 401], 60.00th=[ 443], | 70.00th=[ 443], 80.00th=[ 481], 90.00th=[ 542], 95.00th=[ 542], | 99.00th=[ 542], 99.50th=[ 542], 99.90th=[ 542], 99.95th=[ 542], | 99.99th=[ 542] bw ( KiB/s): min= 1996, max= 3792, per=32.74%, avg=2379.40, stdev=789.87, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 lat (usec) : 1000=8.33% lat (msec) : 2=8.33%, 4=8.33%, 250=16.67%, 500=50.00%, 750=8.33% cpu : usr=0.03%, sys=25.92%, ctx=188, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5191KiB/s (5316kB/s), 519KiB/s-2531KiB/s (532kB/s-2592kB/s), io=20.0MiB (20.0MB), run=2832-3945msec WRITE: bw=7268KiB/s (7442kB/s), 1808KiB/s-2596KiB/s (1851kB/s-2658kB/s), io=28.0MiB (29.4MB), run=2832-3945msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148007: Mon Jul 28 19:42:16 2025 read: IOPS=2, BW=2205KiB/s (2258kB/s)(7168KiB/3251msec) clat (msec): min=74, max=324, avg=184.93, stdev=101.06 lat (msec): min=74, max=324, avg=184.93, stdev=101.06 clat percentiles (msec): | 1.00th=[ 74], 5.00th=[ 74], 10.00th=[ 74], 20.00th=[ 99], | 30.00th=[ 123], 40.00th=[ 123], 50.00th=[ 171], 60.00th=[ 186], | 70.00th=[ 186], 80.00th=[ 321], 90.00th=[ 326], 95.00th=[ 326], | 99.00th=[ 326], 99.50th=[ 326], 99.90th=[ 326], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 1765, max= 4151, per=52.55%, avg=2453.50, stdev=1135.53, samples=4 iops : min= 1, max= 4, avg= 1.75, stdev= 1.50, samples=4 write: IOPS=1, BW=1575KiB/s (1613kB/s)(5120KiB/3251msec); 0 zone resets clat (msec): min=207, max=757, avg=352.44, stdev=229.63 lat (msec): min=207, max=757, avg=352.53, stdev=229.60 clat percentiles (msec): | 1.00th=[ 209], 5.00th=[ 209], 10.00th=[ 209], 20.00th=[ 209], | 30.00th=[ 222], 40.00th=[ 222], 50.00th=[ 275], 60.00th=[ 275], | 70.00th=[ 300], 80.00th=[ 300], 90.00th=[ 760], 95.00th=[ 760], | 99.00th=[ 760], 99.50th=[ 760], 99.90th=[ 760], 99.95th=[ 760], | 99.99th=[ 760] bw ( KiB/s): min= 1765, max= 3984, per=39.73%, avg=2597.33, stdev=1208.91, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 100=16.67%, 250=41.67%, 500=33.33%, 1000=8.33% cpu : usr=0.09%, sys=13.60%, ctx=159, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148008: Mon Jul 28 19:42:16 2025 read: IOPS=0, BW=467KiB/s (478kB/s)(2048KiB/4386msec) clat (msec): min=133, max=463, avg=298.48, stdev=232.76 lat (msec): min=133, max=463, avg=298.48, stdev=232.76 clat percentiles (msec): | 1.00th=[ 134], 5.00th=[ 134], 10.00th=[ 134], 20.00th=[ 134], | 30.00th=[ 134], 40.00th=[ 134], 50.00th=[ 134], 60.00th=[ 464], | 70.00th=[ 464], 80.00th=[ 464], 90.00th=[ 464], 95.00th=[ 464], | 99.00th=[ 464], 99.50th=[ 464], 99.90th=[ 464], 99.95th=[ 464], | 99.99th=[ 464] bw ( KiB/s): min= 2000, max= 2027, per=43.12%, avg=2013.50, stdev=19.09, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2335KiB/s (2391kB/s)(10.0MiB/4386msec); 0 zone resets clat (msec): min=107, max=799, avg=357.85, stdev=232.66 lat (msec): min=107, max=799, avg=357.95, stdev=232.67 clat percentiles (msec): | 1.00th=[ 108], 5.00th=[ 108], 10.00th=[ 108], 20.00th=[ 157], | 30.00th=[ 218], 40.00th=[ 243], 50.00th=[ 253], 60.00th=[ 266], | 70.00th=[ 372], 80.00th=[ 460], 90.00th=[ 709], 95.00th=[ 802], | 99.00th=[ 802], 99.50th=[ 802], 99.90th=[ 802], 99.95th=[ 802], | 99.99th=[ 802] bw ( KiB/s): min= 1774, max= 6131, per=48.76%, avg=3187.40, stdev=1871.18, samples=5 iops : min= 1, max= 5, avg= 2.40, stdev= 1.67, samples=5 lat (msec) : 250=41.67%, 500=41.67%, 750=8.33%, 1000=8.33% cpu : usr=0.05%, sys=10.08%, ctx=124, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148009: Mon Jul 28 19:42:16 2025 read: IOPS=1, BW=1420KiB/s (1454kB/s)(6144KiB/4326msec) clat (msec): min=50, max=597, avg=199.32, stdev=223.69 lat (msec): min=50, max=597, avg=199.32, stdev=223.69 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 51], 10.00th=[ 51], 20.00th=[ 57], | 30.00th=[ 57], 40.00th=[ 66], 50.00th=[ 66], 60.00th=[ 89], | 70.00th=[ 338], 80.00th=[ 338], 90.00th=[ 600], 95.00th=[ 600], | 99.00th=[ 600], 99.50th=[ 600], 99.90th=[ 600], 99.95th=[ 600], | 99.99th=[ 600] bw ( KiB/s): min= 2000, max= 6131, per=72.67%, avg=3393.00, stdev=2371.30, samples=3 iops : min= 1, max= 5, avg= 2.67, stdev= 2.08, samples=3 write: IOPS=1, BW=1420KiB/s (1454kB/s)(6144KiB/4326msec); 0 zone resets clat (msec): min=103, max=1631, avg=488.66, stdev=577.76 lat (msec): min=103, max=1631, avg=488.76, stdev=577.76 clat percentiles (msec): | 1.00th=[ 104], 5.00th=[ 104], 10.00th=[ 104], 20.00th=[ 113], | 30.00th=[ 113], 40.00th=[ 275], 50.00th=[ 275], 60.00th=[ 326], | 70.00th=[ 489], 80.00th=[ 489], 90.00th=[ 1636], 95.00th=[ 1636], | 99.00th=[ 1636], 99.50th=[ 1636], 99.90th=[ 1636], 99.95th=[ 1636], | 99.99th=[ 1636] bw ( KiB/s): min= 2048, max= 4087, per=51.96%, avg=3396.67, stdev=1168.09, samples=3 iops : min= 2, max= 3, avg= 2.67, stdev= 0.58, samples=3 lat (msec) : 100=33.33%, 250=16.67%, 500=33.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=8.02%, ctx=180, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148010: Mon Jul 28 19:42:16 2025 read: IOPS=1, BW=1225KiB/s (1255kB/s)(5120KiB/4179msec) clat (msec): min=69, max=187, avg=128.22, stdev=49.01 lat (msec): min=69, max=187, avg=128.22, stdev=49.01 clat percentiles (msec): | 1.00th=[ 70], 5.00th=[ 70], 10.00th=[ 70], 20.00th=[ 70], | 30.00th=[ 99], 40.00th=[ 99], 50.00th=[ 117], 60.00th=[ 117], | 70.00th=[ 169], 80.00th=[ 169], 90.00th=[ 188], 95.00th=[ 188], | 99.00th=[ 188], 99.50th=[ 188], 99.90th=[ 188], 99.95th=[ 188], | 99.99th=[ 188] bw ( KiB/s): min= 4007, max= 4087, per=86.68%, avg=4047.00, stdev=56.57, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1715KiB/s (1756kB/s)(7168KiB/4179msec); 0 zone resets clat (msec): min=113, max=1421, avg=498.32, stdev=473.60 lat (msec): min=113, max=1422, avg=498.52, stdev=473.69 clat percentiles (msec): | 1.00th=[ 114], 5.00th=[ 114], 10.00th=[ 114], 20.00th=[ 161], | 30.00th=[ 257], 40.00th=[ 257], 50.00th=[ 326], 60.00th=[ 363], | 70.00th=[ 363], 80.00th=[ 852], 90.00th=[ 1418], 95.00th=[ 1418], | 99.00th=[ 1418], 99.50th=[ 1418], 99.90th=[ 1418], 99.95th=[ 1418], | 99.99th=[ 1418] bw ( KiB/s): min= 1765, max= 2043, per=30.12%, avg=1969.20, stdev=116.46, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (msec) : 100=16.67%, 250=41.67%, 500=25.00%, 1000=8.33%, 2000=8.33% cpu : usr=0.02%, sys=11.51%, ctx=233, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4669KiB/s (4781kB/s), 467KiB/s-2205KiB/s (478kB/s-2258kB/s), io=20.0MiB (20.0MB), run=3251-4386msec WRITE: bw=6537KiB/s (6694kB/s), 1420KiB/s-2335KiB/s (1454kB/s-2391kB/s), io=28.0MiB (29.4MB), run=3251-4386msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148016: Mon Jul 28 19:42:26 2025 read: IOPS=0, BW=3339KiB/s (3419kB/s)(12.0MiB/3680msec) clat (msec): min=678, max=1862, avg=1186.75, stdev=609.31 lat (msec): min=678, max=1862, avg=1186.75, stdev=609.31 clat percentiles (msec): | 1.00th=[ 676], 5.00th=[ 676], 10.00th=[ 676], 20.00th=[ 676], | 30.00th=[ 676], 40.00th=[ 1020], 50.00th=[ 1020], 60.00th=[ 1020], | 70.00th=[ 1871], 80.00th=[ 1871], 90.00th=[ 1871], 95.00th=[ 1871], | 99.00th=[ 1871], 99.50th=[ 1871], 99.90th=[ 1871], 99.95th=[ 1871], | 99.99th=[ 1871] bw ( KiB/s): min= 7907, max= 7953, per=100.00%, avg=7930.00, stdev=32.53, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=6.80%, ctx=67, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148017: Mon Jul 28 19:42:26 2025 read: IOPS=0, BW=2189KiB/s (2242kB/s)(4096KiB/1871msec) clat (nsec): min=179126k, max=179126k, avg=179126005.00, stdev= 0.00 lat (nsec): min=179128k, max=179128k, avg=179128232.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 180], 5.00th=[ 180], 10.00th=[ 180], 20.00th=[ 180], | 30.00th=[ 180], 40.00th=[ 180], 50.00th=[ 180], 60.00th=[ 180], | 70.00th=[ 180], 80.00th=[ 180], 90.00th=[ 180], 95.00th=[ 180], | 99.00th=[ 180], 99.50th=[ 180], 99.90th=[ 180], 99.95th=[ 180], | 99.99th=[ 180] bw ( KiB/s): min= 7907, max= 7907, per=100.00%, avg=7907.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4378KiB/s (4483kB/s)(8192KiB/1871msec); 0 zone resets clat (msec): min=494, max=1112, avg=803.41, stdev=436.90 lat (msec): min=494, max=1112, avg=803.79, stdev=436.99 clat percentiles (msec): | 1.00th=[ 493], 5.00th=[ 493], 10.00th=[ 493], 20.00th=[ 493], | 30.00th=[ 493], 40.00th=[ 493], 50.00th=[ 493], 60.00th=[ 1116], | 70.00th=[ 1116], 80.00th=[ 1116], 90.00th=[ 1116], 95.00th=[ 1116], | 99.00th=[ 1116], 99.50th=[ 1116], 99.90th=[ 1116], 99.95th=[ 1116], | 99.99th=[ 1116] bw ( KiB/s): min= 7953, max= 7953, per=82.91%, avg=7953.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=9.95%, ctx=44, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148018: Mon Jul 28 19:42:26 2025 write: IOPS=0, BW=3597KiB/s (3684kB/s)(12.0MiB/3416msec); 0 zone resets clat (msec): min=359, max=2636, avg=1121.11, stdev=1312.53 lat (msec): min=359, max=2637, avg=1121.47, stdev=1312.67 clat percentiles (msec): | 1.00th=[ 359], 5.00th=[ 359], 10.00th=[ 359], 20.00th=[ 359], | 30.00th=[ 359], 40.00th=[ 368], 50.00th=[ 368], 60.00th=[ 368], | 70.00th=[ 2635], 80.00th=[ 2635], 90.00th=[ 2635], 95.00th=[ 2635], | 99.00th=[ 2635], 99.50th=[ 2635], 99.90th=[ 2635], 99.95th=[ 2635], | 99.99th=[ 2635] bw ( KiB/s): min= 5618, max= 7984, per=70.90%, avg=6801.00, stdev=1673.01, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, >=2000=33.33% cpu : usr=0.03%, sys=9.69%, ctx=88, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148019: Mon Jul 28 19:42:26 2025 write: IOPS=1, BW=4546KiB/s (4655kB/s)(12.0MiB/2703msec); 0 zone resets clat (msec): min=341, max=1369, avg=883.12, stdev=515.92 lat (msec): min=342, max=1369, avg=883.58, stdev=516.02 clat percentiles (msec): | 1.00th=[ 342], 5.00th=[ 342], 10.00th=[ 342], 20.00th=[ 342], | 30.00th=[ 342], 40.00th=[ 936], 50.00th=[ 936], 60.00th=[ 936], | 70.00th=[ 1368], 80.00th=[ 1368], 90.00th=[ 1368], 95.00th=[ 1368], | 99.00th=[ 1368], 99.50th=[ 1368], 99.90th=[ 1368], 99.95th=[ 1368], | 99.99th=[ 1368] bw ( KiB/s): min= 5834, max= 7953, per=71.87%, avg=6893.50, stdev=1498.36, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.07%, sys=14.84%, ctx=88, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4452KiB/s (4559kB/s), 2189KiB/s-3339KiB/s (2242kB/s-3419kB/s), io=16.0MiB (16.8MB), run=1871-3680msec WRITE: bw=9593KiB/s (9823kB/s), 3597KiB/s-4546KiB/s (3684kB/s-4655kB/s), io=32.0MiB (33.6MB), run=1871-3416msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148021: Mon Jul 28 19:42:27 2025 read: IOPS=1, BW=5220KiB/s (5345kB/s)(12.0MiB/2354msec) clat (msec): min=491, max=977, avg=784.24, stdev=258.21 lat (msec): min=491, max=977, avg=784.24, stdev=258.21 clat percentiles (msec): | 1.00th=[ 493], 5.00th=[ 493], 10.00th=[ 493], 20.00th=[ 493], | 30.00th=[ 493], 40.00th=[ 885], 50.00th=[ 885], 60.00th=[ 885], | 70.00th=[ 978], 80.00th=[ 978], 90.00th=[ 978], 95.00th=[ 978], | 99.00th=[ 978], 99.50th=[ 978], 99.90th=[ 978], 99.95th=[ 978], | 99.99th=[ 978] bw ( KiB/s): min= 7641, max= 8192, per=100.00%, avg=7916.50, stdev=389.62, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 1000=66.67% cpu : usr=0.00%, sys=21.16%, ctx=456, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148022: Mon Jul 28 19:42:27 2025 read: IOPS=0, BW=1046KiB/s (1071kB/s)(4096KiB/3917msec) clat (nsec): min=1009.2M, max=1009.2M, avg=1009234935.00, stdev= 0.00 lat (nsec): min=1009.2M, max=1009.2M, avg=1009236892.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1011], 5.00th=[ 1011], 10.00th=[ 1011], 20.00th=[ 1011], | 30.00th=[ 1011], 40.00th=[ 1011], 50.00th=[ 1011], 60.00th=[ 1011], | 70.00th=[ 1011], 80.00th=[ 1011], 90.00th=[ 1011], 95.00th=[ 1011], | 99.00th=[ 1011], 99.50th=[ 1011], 99.90th=[ 1011], 99.95th=[ 1011], | 99.99th=[ 1011] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2091KiB/s (2142kB/s)(8192KiB/3917msec); 0 zone resets clat (msec): min=1384, max=1520, avg=1452.52, stdev=95.86 lat (msec): min=1385, max=1520, avg=1452.93, stdev=96.00 clat percentiles (msec): | 1.00th=[ 1385], 5.00th=[ 1385], 10.00th=[ 1385], 20.00th=[ 1385], | 30.00th=[ 1385], 40.00th=[ 1385], 50.00th=[ 1385], 60.00th=[ 1519], | 70.00th=[ 1519], 80.00th=[ 1519], 90.00th=[ 1519], 95.00th=[ 1519], | 99.00th=[ 1519], 99.50th=[ 1519], 99.90th=[ 1519], 99.95th=[ 1519], | 99.99th=[ 1519] bw ( KiB/s): min= 7599, max= 7599, per=92.12%, avg=7599.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 2000=100.00% cpu : usr=0.00%, sys=23.29%, ctx=421, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148023: Mon Jul 28 19:42:27 2025 write: IOPS=0, BW=3192KiB/s (3268kB/s)(12.0MiB/3850msec); 0 zone resets clat (msec): min=1160, max=1470, avg=1282.07, stdev=165.50 lat (msec): min=1160, max=1470, avg=1282.48, stdev=165.55 clat percentiles (msec): | 1.00th=[ 1167], 5.00th=[ 1167], 10.00th=[ 1167], 20.00th=[ 1167], | 30.00th=[ 1167], 40.00th=[ 1217], 50.00th=[ 1217], 60.00th=[ 1217], | 70.00th=[ 1469], 80.00th=[ 1469], 90.00th=[ 1469], 95.00th=[ 1469], | 99.00th=[ 1469], 99.50th=[ 1469], 99.90th=[ 1469], 99.95th=[ 1469], | 99.99th=[ 1469] bw ( KiB/s): min= 8126, max= 8192, per=98.91%, avg=8159.00, stdev=46.67, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 2000=100.00% cpu : usr=0.00%, sys=32.45%, ctx=114, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148024: Mon Jul 28 19:42:27 2025 write: IOPS=0, BW=3094KiB/s (3168kB/s)(12.0MiB/3972msec); 0 zone resets clat (msec): min=863, max=1900, avg=1322.78, stdev=528.64 lat (msec): min=863, max=1901, avg=1323.15, stdev=528.84 clat percentiles (msec): | 1.00th=[ 860], 5.00th=[ 860], 10.00th=[ 860], 20.00th=[ 860], | 30.00th=[ 860], 40.00th=[ 1200], 50.00th=[ 1200], 60.00th=[ 1200], | 70.00th=[ 1905], 80.00th=[ 1905], 90.00th=[ 1905], 95.00th=[ 1905], | 99.00th=[ 1905], 99.50th=[ 1905], 99.90th=[ 1905], 99.95th=[ 1905], | 99.99th=[ 1905] bw ( KiB/s): min= 8175, max= 8192, per=99.21%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.08%, sys=29.77%, ctx=109, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4183KiB/s (4283kB/s), 1046KiB/s-5220KiB/s (1071kB/s-5345kB/s), io=16.0MiB (16.8MB), run=2354-3917msec WRITE: bw=8250KiB/s (8448kB/s), 2091KiB/s-3192KiB/s (2142kB/s-3268kB/s), io=32.0MiB (33.6MB), run=3850-3972msec