== sanity test 398b: DIO and buffer IO race ============== 03:19:25 (1759821565) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.88975 s, 17.4 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=153782: Tue Oct 7 03:20:25 2025 read: IOPS=29, BW=118KiB/s (121kB/s)(5992KiB/50628msec) clat (usec): min=82, max=681829, avg=17835.14, stdev=19943.77 lat (usec): min=82, max=681830, avg=17837.71, stdev=19946.02 clat percentiles (usec): | 1.00th=[ 338], 5.00th=[ 5342], 10.00th=[ 7767], 20.00th=[ 9765], | 30.00th=[ 11469], 40.00th=[ 13304], 50.00th=[ 15533], 60.00th=[ 17957], | 70.00th=[ 20579], 80.00th=[ 24249], 90.00th=[ 30278], 95.00th=[ 35390], | 99.00th=[ 47973], 99.50th=[ 55313], 99.90th=[113771], 99.95th=[683672], | 99.99th=[683672] bw ( KiB/s): min= 15, max= 222, per=24.92%, avg=117.35, stdev=37.02, samples=100 iops : min= 3, max= 55, avg=28.99, stdev= 9.35, samples=100 write: IOPS=31, BW=124KiB/s (127kB/s)(6296KiB/50628msec); 0 zone resets clat (usec): min=646, max=88063, avg=15065.72, stdev=11794.58 lat (usec): min=647, max=88063, avg=15066.66, stdev=11794.86 clat percentiles (usec): | 1.00th=[ 848], 5.00th=[ 2147], 10.00th=[ 2966], 20.00th=[ 4359], | 30.00th=[ 6652], 40.00th=[ 9634], 50.00th=[12649], 60.00th=[15926], | 70.00th=[19268], 80.00th=[23987], 90.00th=[30016], 95.00th=[36439], | 99.00th=[54264], 99.50th=[63701], 99.90th=[71828], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 48, max= 208, per=25.18%, avg=124.12, stdev=33.19, samples=99 iops : min= 12, max= 52, avg=30.69, stdev= 8.33, samples=99 lat (usec) : 100=0.07%, 250=0.10%, 500=1.11%, 750=0.98%, 1000=0.49% lat (msec) : 2=1.43%, 4=7.42%, 10=19.99%, 20=38.35%, 50=28.97% lat (msec) : 100=1.04%, 250=0.03%, 750=0.03% cpu : usr=0.08%, sys=29.50%, ctx=9210, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153783: Tue Oct 7 03:20:25 2025 read: IOPS=29, BW=118KiB/s (121kB/s)(5940KiB/50214msec) clat (usec): min=88, max=87298, avg=17127.28, stdev=9848.70 lat (usec): min=89, max=87299, avg=17128.41, stdev=9848.33 clat percentiles (usec): | 1.00th=[ 322], 5.00th=[ 4752], 10.00th=[ 7635], 20.00th=[ 9634], | 30.00th=[11207], 40.00th=[13173], 50.00th=[15664], 60.00th=[17957], | 70.00th=[20317], 80.00th=[23462], 90.00th=[28967], 95.00th=[34866], | 99.00th=[47449], 99.50th=[63701], 99.90th=[76022], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 7, max= 200, per=25.07%, avg=118.06, stdev=33.38, samples=100 iops : min= 1, max= 50, avg=29.17, stdev= 8.40, samples=100 write: IOPS=31, BW=126KiB/s (129kB/s)(6348KiB/50214msec); 0 zone resets clat (usec): min=605, max=630709, avg=15493.05, stdev=19727.71 lat (usec): min=606, max=630709, avg=15493.98, stdev=19727.76 clat percentiles (usec): | 1.00th=[ 799], 5.00th=[ 2343], 10.00th=[ 2999], 20.00th=[ 4490], | 30.00th=[ 6849], 40.00th=[ 9896], 50.00th=[ 12387], 60.00th=[ 15401], | 70.00th=[ 19268], 80.00th=[ 23462], 90.00th=[ 30016], 95.00th=[ 35914], | 99.00th=[ 61604], 99.50th=[ 76022], 99.90th=[ 96994], 99.95th=[633340], | 99.99th=[633340] bw ( KiB/s): min= 7, max= 247, per=25.59%, avg=126.17, stdev=39.04, samples=100 iops : min= 1, max= 61, avg=31.19, stdev= 9.83, samples=100 lat (usec) : 100=0.03%, 250=0.23%, 500=1.33%, 750=0.68%, 1000=0.59% lat (msec) : 2=1.20%, 4=7.23%, 10=20.02%, 20=38.44%, 50=28.91% lat (msec) : 100=1.30%, 750=0.03% cpu : usr=0.13%, sys=29.30%, ctx=9167, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153785: Tue Oct 7 03:20:25 2025 read: IOPS=30, BW=120KiB/s (123kB/s)(6028KiB/50049msec) clat (usec): min=79, max=85334, avg=16955.79, stdev=9757.01 lat (usec): min=79, max=85335, avg=16968.12, stdev=9752.75 clat percentiles (usec): | 1.00th=[ 318], 5.00th=[ 1287], 10.00th=[ 7439], 20.00th=[ 9634], | 30.00th=[11338], 40.00th=[13173], 50.00th=[15533], 60.00th=[17695], | 70.00th=[20317], 80.00th=[23725], 90.00th=[28705], 95.00th=[34341], | 99.00th=[48497], 99.50th=[59507], 99.90th=[69731], 99.95th=[85459], | 99.99th=[85459] bw ( KiB/s): min= 32, max= 223, per=25.79%, avg=121.47, stdev=33.89, samples=98 iops : min= 8, max= 55, avg=30.02, stdev= 8.50, samples=98 write: IOPS=31, BW=125KiB/s (128kB/s)(6260KiB/50049msec); 0 zone resets clat (usec): min=667, max=654814, avg=15505.42, stdev=20231.46 lat (usec): min=668, max=654816, avg=15506.35, stdev=20231.48 clat percentiles (usec): | 1.00th=[ 979], 5.00th=[ 2507], 10.00th=[ 3130], 20.00th=[ 4424], | 30.00th=[ 6325], 40.00th=[ 9503], 50.00th=[ 12387], 60.00th=[ 15533], | 70.00th=[ 19530], 80.00th=[ 24511], 90.00th=[ 30802], 95.00th=[ 36439], | 99.00th=[ 52691], 99.50th=[ 70779], 99.90th=[ 99091], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 40, max= 239, per=25.52%, avg=125.81, stdev=37.63, samples=98 iops : min= 10, max= 59, avg=31.10, stdev= 9.47, samples=98 lat (usec) : 100=0.03%, 250=0.20%, 500=1.43%, 750=0.65%, 1000=0.78% lat (msec) : 2=1.20%, 4=7.16%, 10=20.64%, 20=37.99%, 50=28.78% lat (msec) : 100=1.11%, 750=0.03% cpu : usr=0.08%, sys=29.65%, ctx=9055, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153787: Tue Oct 7 03:20:25 2025 read: IOPS=29, BW=119KiB/s (122kB/s)(6052KiB/50957msec) clat (usec): min=80, max=83436, avg=17497.29, stdev=10277.86 lat (usec): min=80, max=83437, avg=17498.11, stdev=10277.91 clat percentiles (usec): | 1.00th=[ 338], 5.00th=[ 930], 10.00th=[ 7504], 20.00th=[ 9634], | 30.00th=[11469], 40.00th=[13566], 50.00th=[16057], 60.00th=[18220], | 70.00th=[21103], 80.00th=[25035], 90.00th=[29230], 95.00th=[34341], | 99.00th=[51119], 99.50th=[60556], 99.90th=[82314], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 32, max= 232, per=25.17%, avg=118.56, stdev=31.24, samples=100 iops : min= 8, max= 58, avg=29.30, stdev= 7.87, samples=100 write: IOPS=30, BW=122KiB/s (125kB/s)(6236KiB/50957msec); 0 zone resets clat (usec): min=672, max=658285, avg=15574.24, stdev=20132.35 lat (usec): min=673, max=658286, avg=15575.06, stdev=20132.37 clat percentiles (usec): | 1.00th=[ 988], 5.00th=[ 2573], 10.00th=[ 3163], 20.00th=[ 4490], | 30.00th=[ 6783], 40.00th=[ 9896], 50.00th=[ 12649], 60.00th=[ 15401], | 70.00th=[ 19006], 80.00th=[ 24249], 90.00th=[ 30540], 95.00th=[ 36963], | 99.00th=[ 56886], 99.50th=[ 63701], 99.90th=[ 86508], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 32, max= 272, per=24.84%, avg=122.48, stdev=36.65, samples=100 iops : min= 8, max= 68, avg=30.28, stdev= 9.24, samples=100 lat (usec) : 100=0.07%, 250=0.07%, 500=1.50%, 750=0.78%, 1000=0.62% lat (msec) : 2=1.14%, 4=7.06%, 10=20.15%, 20=37.79%, 50=29.46% lat (msec) : 100=1.33%, 750=0.03% cpu : usr=0.11%, sys=29.43%, ctx=9168, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=471KiB/s (483kB/s), 118KiB/s-120KiB/s (121kB/s-123kB/s), io=23.4MiB (24.6MB), run=50049-50957msec WRITE: bw=493KiB/s (505kB/s), 122KiB/s-126KiB/s (125kB/s-129kB/s), io=24.6MiB (25.7MB), run=50049-50957msec rand-rw: (groupid=0, jobs=1): err= 0: pid=153781: Tue Oct 7 03:20:48 2025 read: IOPS=20, BW=81.7KiB/s (83.6kB/s)(5992KiB/73361msec) clat (msec): min=2, max=693, avg=14.78, stdev=21.73 lat (msec): min=2, max=693, avg=14.78, stdev=21.73 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 10], 60.00th=[ 13], | 70.00th=[ 16], 80.00th=[ 22], 90.00th=[ 32], 95.00th=[ 44], | 99.00th=[ 62], 99.50th=[ 73], 99.90th=[ 86], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 15, max= 224, per=25.07%, avg=81.97, stdev=44.60, samples=145 iops : min= 3, max= 56, avg=20.23, stdev=11.29, samples=145 write: IOPS=21, BW=85.8KiB/s (87.9kB/s)(6296KiB/73361msec); 0 zone resets clat (msec): min=11, max=147, avg=32.42, stdev=16.01 lat (msec): min=11, max=147, avg=32.42, stdev=16.01 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 21], | 30.00th=[ 23], 40.00th=[ 25], 50.00th=[ 28], 60.00th=[ 31], | 70.00th=[ 36], 80.00th=[ 43], 90.00th=[ 53], 95.00th=[ 64], | 99.00th=[ 88], 99.50th=[ 101], 99.90th=[ 131], 99.95th=[ 148], | 99.99th=[ 148] bw ( KiB/s): min= 15, max= 176, per=25.16%, avg=86.06, stdev=40.45, samples=145 iops : min= 3, max= 44, avg=21.26, stdev=10.29, samples=145 lat (msec) : 4=5.24%, 10=19.21%, 20=22.85%, 50=45.18%, 100=7.26% lat (msec) : 250=0.23%, 750=0.03% cpu : usr=0.11%, sys=9.71%, ctx=5827, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153784: Tue Oct 7 03:20:48 2025 read: IOPS=20, BW=80.0KiB/s (82.9kB/s)(5940KiB/73348msec) clat (msec): min=2, max=639, avg=15.04, stdev=21.28 lat (msec): min=2, max=639, avg=15.04, stdev=21.28 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 32], 95.00th=[ 42], | 99.00th=[ 70], 99.50th=[ 85], 99.90th=[ 122], 99.95th=[ 642], | 99.99th=[ 642] bw ( KiB/s): min= 15, max= 256, per=24.87%, avg=81.31, stdev=45.96, samples=145 iops : min= 3, max= 64, avg=20.07, stdev=11.64, samples=145 write: IOPS=21, BW=86.5KiB/s (88.6kB/s)(6348KiB/73348msec); 0 zone resets clat (msec): min=11, max=162, avg=32.02, stdev=15.71 lat (msec): min=11, max=162, avg=32.02, stdev=15.71 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 21], | 30.00th=[ 23], 40.00th=[ 25], 50.00th=[ 28], 60.00th=[ 31], | 70.00th=[ 36], 80.00th=[ 43], 90.00th=[ 52], 95.00th=[ 61], | 99.00th=[ 90], 99.50th=[ 102], 99.90th=[ 148], 99.95th=[ 163], | 99.99th=[ 163] bw ( KiB/s): min= 32, max= 168, per=25.36%, avg=86.72, stdev=40.19, samples=145 iops : min= 8, max= 42, avg=21.42, stdev=10.21, samples=145 lat (msec) : 4=5.86%, 10=17.90%, 20=23.50%, 50=45.38%, 100=7.03% lat (msec) : 250=0.29%, 750=0.03% cpu : usr=0.10%, sys=9.75%, ctx=5723, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153786: Tue Oct 7 03:20:48 2025 read: IOPS=20, BW=82.4KiB/s (84.3kB/s)(6028KiB/73183msec) clat (msec): min=2, max=105, avg=14.69, stdev=13.24 lat (msec): min=2, max=105, avg=14.69, stdev=13.24 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 11], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 23], 90.00th=[ 32], 95.00th=[ 42], | 99.00th=[ 67], 99.50th=[ 74], 99.90th=[ 101], 99.95th=[ 107], | 99.99th=[ 107] bw ( KiB/s): min= 23, max= 248, per=25.27%, avg=82.65, stdev=43.55, samples=144 iops : min= 5, max= 62, avg=20.40, stdev=11.03, samples=144 write: IOPS=21, BW=85.5KiB/s (87.6kB/s)(6260KiB/73183msec); 0 zone resets clat (msec): min=13, max=651, avg=32.49, stdev=22.22 lat (msec): min=13, max=651, avg=32.49, stdev=22.22 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 21], | 30.00th=[ 23], 40.00th=[ 25], 50.00th=[ 27], 60.00th=[ 31], | 70.00th=[ 36], 80.00th=[ 42], 90.00th=[ 53], 95.00th=[ 63], | 99.00th=[ 92], 99.50th=[ 99], 99.90th=[ 184], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 31, max= 176, per=25.06%, avg=85.71, stdev=40.39, samples=144 iops : min= 7, max= 44, avg=21.17, stdev=10.27, samples=144 lat (msec) : 4=5.34%, 10=18.59%, 20=23.93%, 50=44.63%, 100=7.26% lat (msec) : 250=0.23%, 750=0.03% cpu : usr=0.09%, sys=9.79%, ctx=5777, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153788: Tue Oct 7 03:20:48 2025 read: IOPS=20, BW=82.0KiB/s (84.0kB/s)(6052KiB/72949msec) clat (msec): min=2, max=123, avg=14.32, stdev=13.68 lat (msec): min=2, max=123, avg=14.33, stdev=13.68 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 8], 50.00th=[ 10], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 32], 95.00th=[ 44], | 99.00th=[ 64], 99.50th=[ 73], 99.90th=[ 107], 99.95th=[ 124], | 99.99th=[ 124] bw ( KiB/s): min= 15, max= 248, per=25.51%, avg=83.43, stdev=47.97, samples=144 iops : min= 3, max= 62, avg=20.60, stdev=12.16, samples=144 write: IOPS=21, BW=85.5KiB/s (87.5kB/s)(6236KiB/72949msec); 0 zone resets clat (msec): min=13, max=651, avg=32.77, stdev=22.27 lat (msec): min=13, max=651, avg=32.77, stdev=22.27 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 21], | 30.00th=[ 23], 40.00th=[ 25], 50.00th=[ 28], 60.00th=[ 32], | 70.00th=[ 37], 80.00th=[ 44], 90.00th=[ 54], 95.00th=[ 64], | 99.00th=[ 91], 99.50th=[ 106], 99.90th=[ 131], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 32, max= 176, per=25.08%, avg=85.76, stdev=40.30, samples=144 iops : min= 8, max= 44, avg=21.18, stdev=10.23, samples=144 lat (msec) : 4=6.02%, 10=19.21%, 20=23.18%, 50=43.82%, 100=7.42% lat (msec) : 250=0.33%, 750=0.03% cpu : usr=0.09%, sys=9.77%, ctx=5882, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=327KiB/s (335kB/s), 80.0KiB/s-82.0KiB/s (82.9kB/s-84.0kB/s), io=23.4MiB (24.6MB), run=72949-73361msec WRITE: bw=343KiB/s (351kB/s), 85.5KiB/s-86.5KiB/s (87.5kB/s-88.6kB/s), io=24.6MiB (25.7MB), run=72949-73361msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=153813: Tue Oct 7 03:21:06 2025 read: IOPS=30, BW=484KiB/s (495kB/s)(5920KiB/12237msec) clat (usec): min=257, max=101265, avg=16527.24, stdev=9946.31 lat (usec): min=257, max=101266, avg=16527.89, stdev=9946.48 clat percentiles (usec): | 1.00th=[ 318], 5.00th=[ 453], 10.00th=[ 6521], 20.00th=[ 10552], | 30.00th=[ 11994], 40.00th=[ 13435], 50.00th=[ 15270], 60.00th=[ 17171], | 70.00th=[ 20055], 80.00th=[ 21890], 90.00th=[ 26608], 95.00th=[ 35390], | 99.00th=[ 44827], 99.50th=[ 50594], 99.90th=[101188], 99.95th=[101188], | 99.99th=[101188] bw ( KiB/s): min= 319, max= 736, per=25.00%, avg=482.04, stdev=107.88, samples=24 iops : min= 19, max= 46, avg=29.58, stdev= 6.85, samples=24 write: IOPS=32, BW=520KiB/s (533kB/s)(6368KiB/12237msec); 0 zone resets clat (usec): min=1832, max=65788, avg=15327.87, stdev=10108.57 lat (usec): min=1833, max=65789, avg=15329.04, stdev=10108.46 clat percentiles (usec): | 1.00th=[ 3064], 5.00th=[ 4146], 10.00th=[ 4621], 20.00th=[ 5669], | 30.00th=[ 8356], 40.00th=[10552], 50.00th=[13042], 60.00th=[16188], | 70.00th=[19792], 80.00th=[22938], 90.00th=[29230], 95.00th=[34341], | 99.00th=[47449], 99.50th=[53216], 99.90th=[65799], 99.95th=[65799], | 99.99th=[65799] bw ( KiB/s): min= 255, max= 733, per=25.66%, avg=520.46, stdev=132.82, samples=24 iops : min= 15, max= 45, avg=31.96, stdev= 8.28, samples=24 lat (usec) : 500=3.39%, 750=0.65%, 1000=0.13% lat (msec) : 2=0.13%, 4=2.34%, 10=20.96%, 20=42.58%, 50=29.30% lat (msec) : 100=0.39%, 250=0.13% cpu : usr=0.11%, sys=31.81%, ctx=2676, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153815: Tue Oct 7 03:21:06 2025 read: IOPS=29, BW=467KiB/s (478kB/s)(5744KiB/12312msec) clat (usec): min=117, max=85840, avg=16183.24, stdev=8767.27 lat (usec): min=117, max=85840, avg=16183.81, stdev=8767.29 clat percentiles (usec): | 1.00th=[ 338], 5.00th=[ 457], 10.00th=[ 6783], 20.00th=[10552], | 30.00th=[12125], 40.00th=[13566], 50.00th=[15664], 60.00th=[17433], | 70.00th=[20055], 80.00th=[22414], 90.00th=[25822], 95.00th=[27919], | 99.00th=[42730], 99.50th=[50070], 99.90th=[85459], 99.95th=[85459], | 99.99th=[85459] bw ( KiB/s): min= 119, max= 701, per=24.11%, avg=464.83, stdev=116.63, samples=24 iops : min= 7, max= 43, avg=28.50, stdev= 7.19, samples=24 write: IOPS=33, BW=532KiB/s (544kB/s)(6544KiB/12312msec); 0 zone resets clat (usec): min=1222, max=84752, avg=15683.29, stdev=11733.81 lat (usec): min=1223, max=84754, avg=15684.41, stdev=11733.87 clat percentiles (usec): | 1.00th=[ 2245], 5.00th=[ 4146], 10.00th=[ 4621], 20.00th=[ 5735], | 30.00th=[ 7832], 40.00th=[10814], 50.00th=[13304], 60.00th=[14877], | 70.00th=[17957], 80.00th=[24249], 90.00th=[30802], 95.00th=[36963], | 99.00th=[56361], 99.50th=[74974], 99.90th=[84411], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 287, max= 864, per=25.84%, avg=524.13, stdev=143.13, samples=24 iops : min= 17, max= 54, avg=32.21, stdev= 9.06, samples=24 lat (usec) : 250=0.39%, 500=2.73%, 750=0.52% lat (msec) : 2=0.65%, 4=1.82%, 10=22.27%, 20=43.62%, 50=27.08% lat (msec) : 100=0.91% cpu : usr=0.12%, sys=31.87%, ctx=2677, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153816: Tue Oct 7 03:21:06 2025 read: IOPS=31, BW=504KiB/s (516kB/s)(6256KiB/12421msec) clat (usec): min=90, max=74135, avg=16424.51, stdev=9759.63 lat (usec): min=90, max=74135, avg=16425.16, stdev=9759.64 clat percentiles (usec): | 1.00th=[ 277], 5.00th=[ 404], 10.00th=[ 2835], 20.00th=[10290], | 30.00th=[11731], 40.00th=[13304], 50.00th=[15270], 60.00th=[17957], | 70.00th=[20055], 80.00th=[22676], 90.00th=[27132], 95.00th=[33424], | 99.00th=[51119], 99.50th=[63701], 99.90th=[73925], 99.95th=[73925], | 99.99th=[73925] bw ( KiB/s): min= 191, max= 825, per=25.88%, avg=498.92, stdev=160.37, samples=24 iops : min= 11, max= 51, avg=30.62, stdev=10.16, samples=24 write: IOPS=30, BW=486KiB/s (497kB/s)(6032KiB/12421msec); 0 zone resets clat (usec): min=1079, max=68182, avg=15733.15, stdev=11987.72 lat (usec): min=1080, max=68186, avg=15735.54, stdev=11987.73 clat percentiles (usec): | 1.00th=[ 2180], 5.00th=[ 3818], 10.00th=[ 4490], 20.00th=[ 5669], | 30.00th=[ 7504], 40.00th=[10290], 50.00th=[12649], 60.00th=[15008], | 70.00th=[18220], 80.00th=[23462], 90.00th=[31327], 95.00th=[40633], | 99.00th=[56886], 99.50th=[60031], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 223, max= 670, per=23.09%, avg=468.25, stdev=128.59, samples=24 iops : min= 13, max= 41, avg=28.71, stdev= 8.15, samples=24 lat (usec) : 100=0.13%, 250=0.13%, 500=3.65%, 750=0.65%, 1000=0.26% lat (msec) : 2=0.65%, 4=2.99%, 10=19.79%, 20=43.10%, 50=26.82% lat (msec) : 100=1.82% cpu : usr=0.11%, sys=31.37%, ctx=2704, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153817: Tue Oct 7 03:21:06 2025 read: IOPS=31, BW=509KiB/s (521kB/s)(6032KiB/11847msec) clat (usec): min=219, max=87887, avg=15767.47, stdev=10018.00 lat (usec): min=220, max=87887, avg=15768.17, stdev=10017.83 clat percentiles (usec): | 1.00th=[ 269], 5.00th=[ 400], 10.00th=[ 529], 20.00th=[ 9896], | 30.00th=[11600], 40.00th=[13173], 50.00th=[15008], 60.00th=[17171], | 70.00th=[19006], 80.00th=[22152], 90.00th=[26084], 95.00th=[29492], | 99.00th=[50594], 99.50th=[78119], 99.90th=[87557], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 256, max= 768, per=26.04%, avg=502.00, stdev=125.48, samples=23 iops : min= 16, max= 48, avg=30.83, stdev= 7.81, samples=23 write: IOPS=33, BW=528KiB/s (541kB/s)(6256KiB/11847msec); 0 zone resets clat (usec): min=995, max=70957, avg=14942.72, stdev=10578.21 lat (usec): min=996, max=70958, avg=14944.00, stdev=10578.10 clat percentiles (usec): | 1.00th=[ 2147], 5.00th=[ 4015], 10.00th=[ 4686], 20.00th=[ 5604], | 30.00th=[ 7111], 40.00th=[ 9896], 50.00th=[12518], 60.00th=[15401], | 70.00th=[17957], 80.00th=[22676], 90.00th=[27919], 95.00th=[34866], | 99.00th=[53740], 99.50th=[62653], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 253, max= 698, per=25.96%, avg=526.52, stdev=122.05, samples=23 iops : min= 15, max= 43, avg=32.35, stdev= 7.58, samples=23 lat (usec) : 250=0.13%, 500=4.56%, 750=1.17%, 1000=0.26% lat (msec) : 2=0.26%, 4=1.95%, 10=22.14%, 20=42.58%, 50=25.91% lat (msec) : 100=1.04% cpu : usr=0.12%, sys=32.60%, ctx=2620, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1928KiB/s (1975kB/s), 467KiB/s-509KiB/s (478kB/s-521kB/s), io=23.4MiB (24.5MB), run=11847-12421msec WRITE: bw=2029KiB/s (2078kB/s), 486KiB/s-532KiB/s (497kB/s-544kB/s), io=24.6MiB (25.8MB), run=11847-12421msec rand-rw: (groupid=0, jobs=1): err= 0: pid=153810: Tue Oct 7 03:21:11 2025 read: IOPS=20, BW=331KiB/s (339kB/s)(5920KiB/17873msec) clat (usec): min=3029, max=81158, avg=14101.72, stdev=13634.35 lat (usec): min=3030, max=81159, avg=14102.65, stdev=13634.57 clat percentiles (usec): | 1.00th=[ 3228], 5.00th=[ 3687], 10.00th=[ 4080], 20.00th=[ 4883], | 30.00th=[ 5538], 40.00th=[ 6980], 50.00th=[ 8848], 60.00th=[11207], | 70.00th=[14484], 80.00th=[20317], 90.00th=[33162], 95.00th=[40633], | 99.00th=[66847], 99.50th=[74974], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 127, max= 736, per=24.30%, avg=323.17, stdev=160.38, samples=35 iops : min= 7, max= 46, avg=19.83, stdev=10.22, samples=35 write: IOPS=22, BW=356KiB/s (365kB/s)(6368KiB/17873msec); 0 zone resets clat (msec): min=13, max=128, avg=31.58, stdev=14.96 lat (msec): min=13, max=128, avg=31.58, stdev=14.96 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 18], 10.00th=[ 19], 20.00th=[ 21], | 30.00th=[ 23], 40.00th=[ 25], 50.00th=[ 28], 60.00th=[ 30], | 70.00th=[ 34], 80.00th=[ 42], 90.00th=[ 51], 95.00th=[ 65], | 99.00th=[ 87], 99.50th=[ 103], 99.90th=[ 129], 99.95th=[ 129], | 99.99th=[ 129] bw ( KiB/s): min= 95, max= 640, per=25.14%, avg=351.71, stdev=156.51, samples=35 iops : min= 5, max= 40, avg=21.60, stdev=10.05, samples=35 lat (msec) : 4=4.30%, 10=21.88%, 20=20.18%, 50=46.48%, 100=6.90% lat (msec) : 250=0.26% cpu : usr=0.11%, sys=9.90%, ctx=1461, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153811: Tue Oct 7 03:21:11 2025 read: IOPS=19, BW=319KiB/s (327kB/s)(5744KiB/18009msec) clat (msec): min=2, max=102, avg=13.99, stdev=14.54 lat (msec): min=2, max=102, avg=13.99, stdev=14.54 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 11], | 70.00th=[ 14], 80.00th=[ 20], 90.00th=[ 36], 95.00th=[ 51], | 99.00th=[ 65], 99.50th=[ 69], 99.90th=[ 103], 99.95th=[ 103], | 99.99th=[ 103] bw ( KiB/s): min= 95, max= 736, per=23.66%, avg=314.66, stdev=182.94, samples=35 iops : min= 5, max= 46, avg=19.29, stdev=11.65, samples=35 write: IOPS=22, BW=363KiB/s (372kB/s)(6544KiB/18009msec); 0 zone resets clat (msec): min=11, max=102, avg=31.56, stdev=15.39 lat (msec): min=11, max=102, avg=31.56, stdev=15.39 clat percentiles (msec): | 1.00th=[ 13], 5.00th=[ 16], 10.00th=[ 18], 20.00th=[ 21], | 30.00th=[ 22], 40.00th=[ 24], 50.00th=[ 27], 60.00th=[ 30], | 70.00th=[ 35], 80.00th=[ 42], 90.00th=[ 53], 95.00th=[ 65], | 99.00th=[ 84], 99.50th=[ 86], 99.90th=[ 103], 99.95th=[ 103], | 99.99th=[ 103] bw ( KiB/s): min= 191, max= 608, per=25.32%, avg=354.23, stdev=149.09, samples=35 iops : min= 11, max= 38, avg=21.77, stdev= 9.56, samples=35 lat (msec) : 4=4.82%, 10=23.05%, 20=19.27%, 50=44.27%, 100=8.33% lat (msec) : 250=0.26% cpu : usr=0.04%, sys=10.06%, ctx=1481, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153812: Tue Oct 7 03:21:11 2025 read: IOPS=22, BW=352KiB/s (360kB/s)(6256KiB/17771msec) clat (usec): min=2883, max=98988, avg=14325.23, stdev=14217.64 lat (usec): min=2883, max=98989, avg=14326.25, stdev=14217.97 clat percentiles (usec): | 1.00th=[ 3261], 5.00th=[ 3720], 10.00th=[ 4146], 20.00th=[ 4948], | 30.00th=[ 5604], 40.00th=[ 6456], 50.00th=[ 8717], 60.00th=[11076], | 70.00th=[14615], 80.00th=[20841], 90.00th=[33817], 95.00th=[44827], | 99.00th=[68682], 99.50th=[76022], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 126, max= 864, per=26.64%, avg=354.37, stdev=206.54, samples=35 iops : min= 7, max= 54, avg=21.77, stdev=13.10, samples=35 write: IOPS=21, BW=339KiB/s (348kB/s)(6032KiB/17771msec); 0 zone resets clat (usec): min=14801, max=98621, avg=32160.33, stdev=15221.86 lat (usec): min=14802, max=98623, avg=32162.09, stdev=15221.96 clat percentiles (usec): | 1.00th=[15139], 5.00th=[17957], 10.00th=[19006], 20.00th=[21103], | 30.00th=[22676], 40.00th=[24773], 50.00th=[26608], 60.00th=[30016], | 70.00th=[34341], 80.00th=[41681], 90.00th=[53740], 95.00th=[62653], | 99.00th=[90702], 99.50th=[92799], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 127, max= 608, per=24.02%, avg=336.09, stdev=146.45, samples=35 iops : min= 7, max= 38, avg=20.63, stdev= 9.45, samples=35 lat (msec) : 4=4.43%, 10=23.57%, 20=19.14%, 50=45.05%, 100=7.81% cpu : usr=0.11%, sys=10.48%, ctx=1491, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153814: Tue Oct 7 03:21:11 2025 read: IOPS=21, BW=338KiB/s (346kB/s)(6032KiB/17846msec) clat (usec): min=2908, max=68596, avg=14519.92, stdev=13020.79 lat (usec): min=2909, max=68596, avg=14520.90, stdev=13020.54 clat percentiles (usec): | 1.00th=[ 3195], 5.00th=[ 3621], 10.00th=[ 4178], 20.00th=[ 5014], | 30.00th=[ 5669], 40.00th=[ 7111], 50.00th=[ 9372], 60.00th=[11600], | 70.00th=[16581], 80.00th=[22676], 90.00th=[33817], 95.00th=[42730], | 99.00th=[61604], 99.50th=[67634], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 96, max= 704, per=25.15%, avg=334.46, stdev=157.26, samples=35 iops : min= 6, max= 44, avg=20.51, stdev=10.07, samples=35 write: IOPS=21, BW=351KiB/s (359kB/s)(6256KiB/17846msec); 0 zone resets clat (msec): min=14, max=116, avg=31.59, stdev=14.34 lat (msec): min=14, max=116, avg=31.59, stdev=14.33 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 18], 10.00th=[ 19], 20.00th=[ 21], | 30.00th=[ 23], 40.00th=[ 25], 50.00th=[ 28], 60.00th=[ 31], | 70.00th=[ 35], 80.00th=[ 41], 90.00th=[ 49], 95.00th=[ 61], | 99.00th=[ 85], 99.50th=[ 88], 99.90th=[ 117], 99.95th=[ 117], | 99.99th=[ 117] bw ( KiB/s): min= 95, max= 704, per=24.87%, avg=347.91, stdev=159.13, samples=35 iops : min= 5, max= 44, avg=21.37, stdev=10.21, samples=35 lat (msec) : 4=3.52%, 10=23.31%, 20=19.27%, 50=48.31%, 100=5.47% lat (msec) : 250=0.13% cpu : usr=0.14%, sys=9.94%, ctx=1460, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1330KiB/s (1362kB/s), 319KiB/s-352KiB/s (327kB/s-360kB/s), io=23.4MiB (24.5MB), run=17771-18009msec WRITE: bw=1399KiB/s (1433kB/s), 339KiB/s-363KiB/s (348kB/s-372kB/s), io=24.6MiB (25.8MB), run=17771-18009msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=153826: Tue Oct 7 03:21:20 2025 read: IOPS=3, BW=3498KiB/s (3582kB/s)(7168KiB/2049msec) clat (usec): min=988, max=260639, avg=146222.92, stdev=77516.85 lat (usec): min=988, max=260640, avg=146224.01, stdev=77517.00 clat percentiles (usec): | 1.00th=[ 988], 5.00th=[ 988], 10.00th=[ 988], 20.00th=[119014], | 30.00th=[149947], 40.00th=[149947], 50.00th=[156238], 60.00th=[164627], | 70.00th=[164627], 80.00th=[173016], 90.00th=[261096], 95.00th=[261096], | 99.00th=[261096], 99.50th=[261096], 99.90th=[261096], 99.95th=[261096], | 99.99th=[261096] bw ( KiB/s): min= 4087, max= 5251, per=65.66%, avg=4669.00, stdev=823.07, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=2, BW=2499KiB/s (2559kB/s)(5120KiB/2049msec); 0 zone resets clat (msec): min=112, max=351, avg=199.58, stdev=89.49 lat (msec): min=112, max=351, avg=199.70, stdev=89.42 clat percentiles (msec): | 1.00th=[ 113], 5.00th=[ 113], 10.00th=[ 113], 20.00th=[ 113], | 30.00th=[ 174], 40.00th=[ 174], 50.00th=[ 176], 60.00th=[ 176], | 70.00th=[ 186], 80.00th=[ 186], 90.00th=[ 351], 95.00th=[ 351], | 99.00th=[ 351], 99.50th=[ 351], 99.90th=[ 351], 99.95th=[ 351], | 99.99th=[ 351] bw ( KiB/s): min= 1750, max= 6083, per=39.34%, avg=3916.50, stdev=3063.89, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 lat (usec) : 1000=8.33% lat (msec) : 250=75.00%, 500=16.67% cpu : usr=0.00%, sys=56.79%, ctx=873, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153827: Tue Oct 7 03:21:20 2025 read: IOPS=0, BW=711KiB/s (728kB/s)(2048KiB/2880msec) clat (usec): min=1028, max=82317, avg=41673.25, stdev=57479.84 lat (usec): min=1029, max=82319, avg=41674.90, stdev=57480.63 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1029], 10.00th=[ 1029], 20.00th=[ 1029], | 30.00th=[ 1029], 40.00th=[ 1029], 50.00th=[ 1029], 60.00th=[82314], | 70.00th=[82314], 80.00th=[82314], 90.00th=[82314], 95.00th=[82314], | 99.00th=[82314], 99.50th=[82314], 99.90th=[82314], 99.95th=[82314], | 99.99th=[82314] bw ( KiB/s): min= 1726, max= 2027, per=26.39%, avg=1876.50, stdev=212.84, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3556KiB/s (3641kB/s)(10.0MiB/2880msec); 0 zone resets clat (msec): min=119, max=587, avg=278.61, stdev=150.50 lat (msec): min=119, max=587, avg=278.67, stdev=150.49 clat percentiles (msec): | 1.00th=[ 121], 5.00th=[ 121], 10.00th=[ 121], 20.00th=[ 165], | 30.00th=[ 174], 40.00th=[ 184], 50.00th=[ 197], 60.00th=[ 241], | 70.00th=[ 309], 80.00th=[ 347], 90.00th=[ 468], 95.00th=[ 592], | 99.00th=[ 592], 99.50th=[ 592], 99.90th=[ 592], 99.95th=[ 592], | 99.99th=[ 592] bw ( KiB/s): min= 2048, max= 6083, per=40.97%, avg=4078.50, stdev=1647.35, samples=4 iops : min= 2, max= 5, avg= 3.50, stdev= 1.29, samples=4 lat (msec) : 2=8.33%, 100=8.33%, 250=50.00%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=44.95%, ctx=414, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153828: Tue Oct 7 03:21:20 2025 read: IOPS=2, BW=2335KiB/s (2391kB/s)(6144KiB/2631msec) clat (usec): min=945, max=189811, avg=120630.49, stdev=64242.14 lat (usec): min=946, max=189811, avg=120631.60, stdev=64241.91 clat percentiles (usec): | 1.00th=[ 947], 5.00th=[ 947], 10.00th=[ 947], 20.00th=[122160], | 30.00th=[122160], 40.00th=[125305], 50.00th=[125305], 60.00th=[127402], | 70.00th=[158335], 80.00th=[158335], 90.00th=[189793], 95.00th=[189793], | 99.00th=[189793], 99.50th=[189793], 99.90th=[189793], 99.95th=[189793], | 99.99th=[189793] bw ( KiB/s): min= 2027, max= 4096, per=35.91%, avg=2553.50, stdev=1028.37, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 write: IOPS=2, BW=2335KiB/s (2391kB/s)(6144KiB/2631msec); 0 zone resets clat (msec): min=131, max=528, avg=311.11, stdev=159.64 lat (msec): min=131, max=528, avg=311.19, stdev=159.64 clat percentiles (msec): | 1.00th=[ 132], 5.00th=[ 132], 10.00th=[ 132], 20.00th=[ 178], | 30.00th=[ 178], 40.00th=[ 222], 50.00th=[ 222], 60.00th=[ 355], | 70.00th=[ 456], 80.00th=[ 456], 90.00th=[ 527], 95.00th=[ 527], | 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527], | 99.99th=[ 527] bw ( KiB/s): min= 2043, max= 4096, per=29.25%, avg=2911.50, stdev=1033.23, samples=4 iops : min= 1, max= 4, avg= 2.50, stdev= 1.29, samples=4 lat (usec) : 1000=8.33% lat (msec) : 250=66.67%, 500=16.67%, 750=8.33% cpu : usr=0.04%, sys=33.76%, ctx=419, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153829: Tue Oct 7 03:21:20 2025 read: IOPS=2, BW=2247KiB/s (2301kB/s)(5120KiB/2279msec) clat (usec): min=944, max=255707, avg=127641.90, stdev=122443.09 lat (usec): min=945, max=255708, avg=127642.92, stdev=122442.86 clat percentiles (usec): | 1.00th=[ 947], 5.00th=[ 947], 10.00th=[ 947], 20.00th=[ 947], | 30.00th=[ 947], 40.00th=[ 947], 50.00th=[147850], 60.00th=[147850], | 70.00th=[233833], 80.00th=[233833], 90.00th=[254804], 95.00th=[254804], | 99.00th=[254804], 99.50th=[254804], 99.90th=[254804], 99.95th=[254804], | 99.99th=[254804] bw ( KiB/s): min= 4055, max= 4096, per=57.31%, avg=4075.50, stdev=28.99, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3145KiB/s (3221kB/s)(7168KiB/2279msec); 0 zone resets clat (msec): min=172, max=408, avg=231.92, stdev=82.06 lat (msec): min=172, max=408, avg=231.99, stdev=82.06 clat percentiles (msec): | 1.00th=[ 174], 5.00th=[ 174], 10.00th=[ 174], 20.00th=[ 184], | 30.00th=[ 184], 40.00th=[ 184], 50.00th=[ 199], 60.00th=[ 236], | 70.00th=[ 236], 80.00th=[ 241], 90.00th=[ 409], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 1765, max= 6131, per=40.15%, avg=3997.33, stdev=2184.67, samples=3 iops : min= 1, max= 5, avg= 3.33, stdev= 2.08, samples=3 lat (usec) : 1000=16.67% lat (msec) : 250=66.67%, 500=16.67% cpu : usr=0.09%, sys=51.23%, ctx=165, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7111KiB/s (7282kB/s), 711KiB/s-3498KiB/s (728kB/s-3582kB/s), io=20.0MiB (20.0MB), run=2049-2880msec WRITE: bw=9956KiB/s (10.2MB/s), 2335KiB/s-3556KiB/s (2391kB/s-3641kB/s), io=28.0MiB (29.4MB), run=2049-2880msec rand-rw: (groupid=0, jobs=1): err= 0: pid=153832: Tue Oct 7 03:21:21 2025 read: IOPS=2, BW=2656KiB/s (2720kB/s)(7168KiB/2699msec) clat (msec): min=22, max=628, avg=213.19, stdev=222.73 lat (msec): min=22, max=628, avg=213.20, stdev=222.73 clat percentiles (msec): | 1.00th=[ 23], 5.00th=[ 23], 10.00th=[ 23], 20.00th=[ 56], | 30.00th=[ 60], 40.00th=[ 60], 50.00th=[ 118], 60.00th=[ 218], | 70.00th=[ 218], 80.00th=[ 393], 90.00th=[ 625], 95.00th=[ 625], | 99.00th=[ 625], 99.50th=[ 625], 99.90th=[ 625], 99.95th=[ 625], | 99.99th=[ 625] bw ( KiB/s): min= 2031, max= 4096, per=47.67%, avg=3396.67, stdev=1182.82, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=1, BW=1897KiB/s (1943kB/s)(5120KiB/2699msec); 0 zone resets clat (msec): min=55, max=525, avg=224.44, stdev=206.92 lat (msec): min=55, max=525, avg=224.53, stdev=206.91 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 56], 10.00th=[ 56], 20.00th=[ 56], | 30.00th=[ 69], 40.00th=[ 69], 50.00th=[ 118], 60.00th=[ 118], | 70.00th=[ 355], 80.00th=[ 355], 90.00th=[ 527], 95.00th=[ 527], | 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527], | 99.99th=[ 527] bw ( KiB/s): min= 2048, max= 4096, per=30.79%, avg=3072.00, stdev=1448.15, samples=2 iops : min= 2, max= 4, avg= 3.00, stdev= 1.41, samples=2 lat (msec) : 50=8.33%, 100=33.33%, 250=25.00%, 500=16.67%, 750=16.67% cpu : usr=0.00%, sys=13.57%, ctx=89, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153833: Tue Oct 7 03:21:21 2025 read: IOPS=0, BW=736KiB/s (754kB/s)(2048KiB/2782msec) clat (msec): min=82, max=269, avg=176.35, stdev=132.21 lat (msec): min=82, max=269, avg=176.35, stdev=132.21 clat percentiles (msec): | 1.00th=[ 84], 5.00th=[ 84], 10.00th=[ 84], 20.00th=[ 84], | 30.00th=[ 84], 40.00th=[ 84], 50.00th=[ 84], 60.00th=[ 271], | 70.00th=[ 271], 80.00th=[ 271], 90.00th=[ 271], 95.00th=[ 271], | 99.00th=[ 271], 99.50th=[ 271], 99.90th=[ 271], 99.95th=[ 271], | 99.99th=[ 271] bw ( KiB/s): min= 1973, max= 2048, per=28.22%, avg=2010.50, stdev=53.03, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3681KiB/s (3769kB/s)(10.0MiB/2782msec); 0 zone resets clat (msec): min=58, max=775, avg=229.33, stdev=210.46 lat (msec): min=58, max=775, avg=229.40, stdev=210.46 clat percentiles (msec): | 1.00th=[ 59], 5.00th=[ 59], 10.00th=[ 59], 20.00th=[ 79], | 30.00th=[ 105], 40.00th=[ 123], 50.00th=[ 148], 60.00th=[ 184], | 70.00th=[ 222], 80.00th=[ 268], 90.00th=[ 334], 95.00th=[ 776], | 99.00th=[ 776], 99.50th=[ 776], 99.90th=[ 776], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 4063, max= 4096, per=40.95%, avg=4085.00, stdev=19.05, samples=3 iops : min= 3, max= 4, avg= 3.67, stdev= 0.58, samples=3 lat (msec) : 100=25.00%, 250=41.67%, 500=25.00%, 1000=8.33% cpu : usr=0.04%, sys=19.09%, ctx=119, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153834: Tue Oct 7 03:21:21 2025 read: IOPS=2, BW=2437KiB/s (2496kB/s)(6144KiB/2521msec) clat (msec): min=60, max=249, avg=116.84, stdev=68.39 lat (msec): min=60, max=249, avg=116.84, stdev=68.39 clat percentiles (msec): | 1.00th=[ 61], 5.00th=[ 61], 10.00th=[ 61], 20.00th=[ 69], | 30.00th=[ 69], 40.00th=[ 100], 50.00th=[ 100], 60.00th=[ 108], | 70.00th=[ 115], 80.00th=[ 115], 90.00th=[ 249], 95.00th=[ 249], | 99.00th=[ 249], 99.50th=[ 249], 99.90th=[ 249], 99.95th=[ 249], | 99.99th=[ 249] bw ( KiB/s): min= 2031, max= 4096, per=47.83%, avg=3407.67, stdev=1192.23, samples=3 iops : min= 1, max= 4, avg= 3.00, stdev= 1.73, samples=3 write: IOPS=2, BW=2437KiB/s (2496kB/s)(6144KiB/2521msec); 0 zone resets clat (msec): min=62, max=598, avg=289.90, stdev=194.58 lat (msec): min=62, max=598, avg=289.96, stdev=194.58 clat percentiles (msec): | 1.00th=[ 63], 5.00th=[ 63], 10.00th=[ 63], 20.00th=[ 140], | 30.00th=[ 140], 40.00th=[ 215], 50.00th=[ 215], 60.00th=[ 326], | 70.00th=[ 405], 80.00th=[ 405], 90.00th=[ 600], 95.00th=[ 600], | 99.00th=[ 600], 99.50th=[ 600], 99.90th=[ 600], 99.95th=[ 600], | 99.99th=[ 600] bw ( KiB/s): min= 2031, max= 4096, per=30.71%, avg=3063.50, stdev=1192.23, samples=4 iops : min= 1, max= 4, avg= 2.50, stdev= 1.73, samples=4 lat (msec) : 100=33.33%, 250=41.67%, 500=16.67%, 750=8.33% cpu : usr=0.00%, sys=15.08%, ctx=412, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153835: Tue Oct 7 03:21:21 2025 read: IOPS=1, BW=1781KiB/s (1824kB/s)(5120KiB/2874msec) clat (msec): min=23, max=362, avg=184.59, stdev=141.89 lat (msec): min=23, max=362, avg=184.59, stdev=141.89 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 24], 10.00th=[ 24], 20.00th=[ 24], | 30.00th=[ 80], 40.00th=[ 80], 50.00th=[ 165], 60.00th=[ 165], | 70.00th=[ 292], 80.00th=[ 292], 90.00th=[ 363], 95.00th=[ 363], | 99.00th=[ 363], 99.50th=[ 363], 99.90th=[ 363], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 2048, max= 4063, per=42.88%, avg=3055.50, stdev=1424.82, samples=2 iops : min= 2, max= 3, avg= 2.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2494KiB/s (2554kB/s)(7168KiB/2874msec); 0 zone resets clat (msec): min=32, max=498, avg=266.53, stdev=198.55 lat (msec): min=32, max=499, avg=266.60, stdev=198.55 clat percentiles (msec): | 1.00th=[ 33], 5.00th=[ 33], 10.00th=[ 33], 20.00th=[ 51], | 30.00th=[ 107], 40.00th=[ 107], 50.00th=[ 334], 60.00th=[ 384], | 70.00th=[ 384], 80.00th=[ 460], 90.00th=[ 498], 95.00th=[ 498], | 99.00th=[ 498], 99.50th=[ 498], 99.90th=[ 498], 99.95th=[ 498], | 99.99th=[ 498] bw ( KiB/s): min= 2015, max= 2048, per=20.40%, avg=2035.50, stdev=15.84, samples=4 iops : min= 1, max= 2, avg= 1.50, stdev= 0.58, samples=4 lat (msec) : 50=16.67%, 100=16.67%, 250=16.67%, 500=50.00% cpu : usr=0.00%, sys=11.21%, ctx=84, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7126KiB/s (7297kB/s), 736KiB/s-2656KiB/s (754kB/s-2720kB/s), io=20.0MiB (20.0MB), run=2521-2874msec WRITE: bw=9976KiB/s (10.2MB/s), 1897KiB/s-3681KiB/s (1943kB/s-3769kB/s), io=28.0MiB (29.4MB), run=2521-2874msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=153841: Tue Oct 7 03:21:30 2025 read: IOPS=2, BW=8802KiB/s (9014kB/s)(12.0MiB/1396msec) clat (msec): min=256, max=861, avg=458.63, stdev=349.05 lat (msec): min=256, max=861, avg=458.63, stdev=349.05 clat percentiles (msec): | 1.00th=[ 257], 5.00th=[ 257], 10.00th=[ 257], 20.00th=[ 257], | 30.00th=[ 257], 40.00th=[ 257], 50.00th=[ 257], 60.00th=[ 257], | 70.00th=[ 860], 80.00th=[ 860], 90.00th=[ 860], 95.00th=[ 860], | 99.00th=[ 860], 99.50th=[ 860], 99.90th=[ 860], 99.95th=[ 860], | 99.99th=[ 860] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 1000=33.33% cpu : usr=0.00%, sys=44.44%, ctx=3175, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153842: Tue Oct 7 03:21:30 2025 read: IOPS=0, BW=1531KiB/s (1567kB/s)(4096KiB/2676msec) clat (nsec): min=1396.9M, max=1396.9M, avg=1396900057.00, stdev= 0.00 lat (nsec): min=1396.9M, max=1396.9M, avg=1396902572.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1401], 5.00th=[ 1401], 10.00th=[ 1401], 20.00th=[ 1401], | 30.00th=[ 1401], 40.00th=[ 1401], 50.00th=[ 1401], 60.00th=[ 1401], | 70.00th=[ 1401], 80.00th=[ 1401], 90.00th=[ 1401], 95.00th=[ 1401], | 99.00th=[ 1401], 99.50th=[ 1401], 99.90th=[ 1401], 99.95th=[ 1401], | 99.99th=[ 1401] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3061KiB/s (3135kB/s)(8192KiB/2676msec); 0 zone resets clat (msec): min=453, max=802, avg=628.10, stdev=247.26 lat (msec): min=453, max=803, avg=628.54, stdev=247.01 clat percentiles (msec): | 1.00th=[ 456], 5.00th=[ 456], 10.00th=[ 456], 20.00th=[ 456], | 30.00th=[ 456], 40.00th=[ 456], 50.00th=[ 456], 60.00th=[ 802], | 70.00th=[ 802], 80.00th=[ 802], 90.00th=[ 802], 95.00th=[ 802], | 99.00th=[ 802], 99.50th=[ 802], 99.90th=[ 802], 99.95th=[ 802], | 99.99th=[ 802] lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=13.91%, ctx=789, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153844: Tue Oct 7 03:21:30 2025 write: IOPS=0, BW=3856KiB/s (3948kB/s)(12.0MiB/3187msec); 0 zone resets clat (msec): min=131, max=2584, avg=1057.51, stdev=1332.23 lat (msec): min=131, max=2584, avg=1057.90, stdev=1332.35 clat percentiles (msec): | 1.00th=[ 132], 5.00th=[ 132], 10.00th=[ 132], 20.00th=[ 132], | 30.00th=[ 132], 40.00th=[ 456], 50.00th=[ 456], 60.00th=[ 456], | 70.00th=[ 2601], 80.00th=[ 2601], 90.00th=[ 2601], 95.00th=[ 2601], | 99.00th=[ 2601], 99.50th=[ 2601], 99.90th=[ 2601], 99.95th=[ 2601], | 99.99th=[ 2601] bw ( KiB/s): min= 8143, max= 8143, per=79.20%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, >=2000=33.33% cpu : usr=0.06%, sys=9.51%, ctx=71, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153846: Tue Oct 7 03:21:30 2025 write: IOPS=1, BW=4111KiB/s (4210kB/s)(12.0MiB/2989msec); 0 zone resets clat (msec): min=145, max=2277, avg=984.75, stdev=1135.79 lat (msec): min=145, max=2277, avg=985.04, stdev=1135.78 clat percentiles (msec): | 1.00th=[ 146], 5.00th=[ 146], 10.00th=[ 146], 20.00th=[ 146], | 30.00th=[ 146], 40.00th=[ 531], 50.00th=[ 531], 60.00th=[ 531], | 70.00th=[ 2265], 80.00th=[ 2265], 90.00th=[ 2265], 95.00th=[ 2265], | 99.00th=[ 2265], 99.50th=[ 2265], 99.90th=[ 2265], 99.95th=[ 2265], | 99.99th=[ 2265] bw ( KiB/s): min= 8143, max= 8143, per=79.20%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, >=2000=33.33% cpu : usr=0.03%, sys=12.68%, ctx=40, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6123KiB/s (6270kB/s), 1531KiB/s-8802KiB/s (1567kB/s-9014kB/s), io=16.0MiB (16.8MB), run=1396-2676msec WRITE: bw=10.0MiB/s (10.5MB/s), 3061KiB/s-4111KiB/s (3135kB/s-4210kB/s), io=32.0MiB (33.6MB), run=2676-3187msec rand-rw: (groupid=0, jobs=1): err= 0: pid=153843: Tue Oct 7 03:21:30 2025 read: IOPS=1, BW=4301KiB/s (4404kB/s)(12.0MiB/2857msec) clat (msec): min=346, max=2061, avg=942.95, stdev=969.59 lat (msec): min=346, max=2061, avg=942.95, stdev=969.59 clat percentiles (msec): | 1.00th=[ 347], 5.00th=[ 347], 10.00th=[ 347], 20.00th=[ 347], | 30.00th=[ 347], 40.00th=[ 422], 50.00th=[ 422], 60.00th=[ 422], | 70.00th=[ 2056], 80.00th=[ 2056], 90.00th=[ 2056], 95.00th=[ 2056], | 99.00th=[ 2056], 99.50th=[ 2056], 99.90th=[ 2056], 99.95th=[ 2056], | 99.99th=[ 2056] bw ( KiB/s): min= 8047, max= 8143, per=100.00%, avg=8095.00, stdev=67.88, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=9.63%, ctx=68, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153845: Tue Oct 7 03:21:30 2025 read: IOPS=0, BW=1310KiB/s (1342kB/s)(4096KiB/3126msec) clat (nsec): min=2245.7M, max=2245.7M, avg=2245687867.00, stdev= 0.00 lat (nsec): min=2245.7M, max=2245.7M, avg=2245689251.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 2232], 5.00th=[ 2232], 10.00th=[ 2232], 20.00th=[ 2232], | 30.00th=[ 2232], 40.00th=[ 2232], 50.00th=[ 2232], 60.00th=[ 2232], | 70.00th=[ 2232], 80.00th=[ 2232], 90.00th=[ 2232], 95.00th=[ 2232], | 99.00th=[ 2232], 99.50th=[ 2232], 99.90th=[ 2232], 99.95th=[ 2232], | 99.99th=[ 2232] bw ( KiB/s): min= 8047, max= 8047, per=100.00%, avg=8047.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2621KiB/s (2683kB/s)(8192KiB/3126msec); 0 zone resets clat (msec): min=378, max=454, avg=416.44, stdev=53.96 lat (msec): min=378, max=454, avg=416.72, stdev=53.91 clat percentiles (msec): | 1.00th=[ 380], 5.00th=[ 380], 10.00th=[ 380], 20.00th=[ 380], | 30.00th=[ 380], 40.00th=[ 380], 50.00th=[ 380], 60.00th=[ 456], | 70.00th=[ 456], 80.00th=[ 456], 90.00th=[ 456], 95.00th=[ 456], | 99.00th=[ 456], 99.50th=[ 456], 99.90th=[ 456], 99.95th=[ 456], | 99.99th=[ 456] lat (msec) : 500=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=8.35%, ctx=71, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153847: Tue Oct 7 03:21:30 2025 write: IOPS=1, BW=4706KiB/s (4819kB/s)(12.0MiB/2611msec); 0 zone resets clat (msec): min=296, max=1776, avg=856.60, stdev=802.75 lat (msec): min=296, max=1776, avg=856.97, stdev=802.65 clat percentiles (msec): | 1.00th=[ 296], 5.00th=[ 296], 10.00th=[ 296], 20.00th=[ 296], | 30.00th=[ 296], 40.00th=[ 498], 50.00th=[ 498], 60.00th=[ 498], | 70.00th=[ 1770], 80.00th=[ 1770], 90.00th=[ 1770], 95.00th=[ 1770], | 99.00th=[ 1770], 99.50th=[ 1770], 99.90th=[ 1770], 99.95th=[ 1770], | 99.99th=[ 1770] bw ( KiB/s): min=16286, max=16286, per=100.00%, avg=16286.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=18.35%, ctx=63, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=153848: Tue Oct 7 03:21:30 2025 write: IOPS=1, BW=4573KiB/s (4683kB/s)(12.0MiB/2687msec); 0 zone resets clat (msec): min=227, max=2004, avg=878.85, stdev=979.03 lat (msec): min=228, max=2005, avg=879.23, stdev=978.93 clat percentiles (msec): | 1.00th=[ 228], 5.00th=[ 228], 10.00th=[ 228], 20.00th=[ 228], | 30.00th=[ 228], 40.00th=[ 405], 50.00th=[ 405], 60.00th=[ 405], | 70.00th=[ 2005], 80.00th=[ 2005], 90.00th=[ 2005], 95.00th=[ 2005], | 99.00th=[ 2005], 99.50th=[ 2005], 99.90th=[ 2005], 99.95th=[ 2005], | 99.99th=[ 2005] bw ( KiB/s): min= 8143, max= 8143, per=77.69%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=11.73%, ctx=92, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5241KiB/s (5367kB/s), 1310KiB/s-4301KiB/s (1342kB/s-4404kB/s), io=16.0MiB (16.8MB), run=2857-3126msec WRITE: bw=10.2MiB/s (10.7MB/s), 2621KiB/s-4706KiB/s (2683kB/s-4819kB/s), io=32.0MiB (33.6MB), run=2611-3126msec