== sanity test 398b: DIO and buffer IO race ============== 11:41:39 (1754408499) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.2032 s, 15.7 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=187363: Tue Aug 5 11:42:40 2025 read: IOPS=30, BW=120KiB/s (123kB/s)(5992KiB/49786msec) clat (usec): min=75, max=703019, avg=18375.50, stdev=19937.61 lat (usec): min=76, max=703019, avg=18377.44, stdev=19937.72 clat percentiles (usec): | 1.00th=[ 314], 5.00th=[ 7635], 10.00th=[ 8979], 20.00th=[ 10552], | 30.00th=[ 11994], 40.00th=[ 14091], 50.00th=[ 16319], 60.00th=[ 18744], | 70.00th=[ 21627], 80.00th=[ 24249], 90.00th=[ 29230], 95.00th=[ 34341], | 99.00th=[ 50070], 99.50th=[ 55313], 99.90th=[ 68682], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 6, max= 199, per=25.62%, avg=120.40, stdev=29.51, samples=99 iops : min= 1, max= 49, avg=29.79, stdev= 7.29, samples=99 write: IOPS=31, BW=126KiB/s (129kB/s)(6296KiB/49786msec); 0 zone resets clat (usec): min=733, max=77363, avg=14015.58, stdev=11654.08 lat (usec): min=735, max=77364, avg=14016.87, stdev=11653.98 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1795], 10.00th=[ 2409], 20.00th=[ 3195], | 30.00th=[ 4948], 40.00th=[ 8848], 50.00th=[11338], 60.00th=[14091], | 70.00th=[18220], 80.00th=[22938], 90.00th=[30016], 95.00th=[36439], | 99.00th=[51119], 99.50th=[58459], 99.90th=[65799], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 63, max= 247, per=25.95%, avg=127.69, stdev=34.87, samples=98 iops : min= 15, max= 61, avg=31.61, stdev= 8.69, samples=98 lat (usec) : 100=0.03%, 250=0.29%, 500=0.78%, 750=0.20%, 1000=0.29% lat (msec) : 2=2.90%, 4=9.99%, 10=15.85%, 20=39.39%, 50=29.10% lat (msec) : 100=1.14%, 750=0.03% cpu : usr=0.11%, sys=23.41%, ctx=8049, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187364: Tue Aug 5 11:42:40 2025 read: IOPS=29, BW=118KiB/s (121kB/s)(5940KiB/50425msec) clat (usec): min=146, max=98908, avg=18092.95, stdev=9931.45 lat (usec): min=147, max=98908, avg=18094.46, stdev=9931.69 clat percentiles (usec): | 1.00th=[ 334], 5.00th=[ 7635], 10.00th=[ 8848], 20.00th=[10421], | 30.00th=[12256], 40.00th=[14222], 50.00th=[16057], 60.00th=[18482], | 70.00th=[21365], 80.00th=[24249], 90.00th=[29754], 95.00th=[35390], | 99.00th=[50594], 99.50th=[56886], 99.90th=[87557], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 6, max= 183, per=25.02%, avg=117.60, stdev=28.28, samples=100 iops : min= 1, max= 45, avg=29.08, stdev= 7.03, samples=100 write: IOPS=31, BW=126KiB/s (129kB/s)(6348KiB/50425msec); 0 zone resets clat (usec): min=802, max=694763, avg=14729.53, stdev=20909.17 lat (usec): min=804, max=694764, avg=14730.88, stdev=20909.11 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 2024], 10.00th=[ 2573], 20.00th=[ 3294], | 30.00th=[ 5211], 40.00th=[ 8848], 50.00th=[ 11600], 60.00th=[ 14222], | 70.00th=[ 18220], 80.00th=[ 23462], 90.00th=[ 30278], 95.00th=[ 37487], | 99.00th=[ 54789], 99.50th=[ 60031], 99.90th=[121111], 99.95th=[692061], | 99.99th=[692061] bw ( KiB/s): min= 64, max= 262, per=25.77%, avg=126.79, stdev=35.09, samples=99 iops : min= 16, max= 65, avg=31.38, stdev= 8.77, samples=99 lat (usec) : 250=0.16%, 500=0.88%, 750=0.20%, 1000=0.36% lat (msec) : 2=2.15%, 4=10.64%, 10=17.09%, 20=38.38%, 50=28.87% lat (msec) : 100=1.20%, 250=0.03%, 750=0.03% cpu : usr=0.13%, sys=23.12%, ctx=8037, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187365: Tue Aug 5 11:42:40 2025 read: IOPS=29, BW=119KiB/s (122kB/s)(6028KiB/50722msec) clat (usec): min=138, max=70619, avg=18270.18, stdev=10064.85 lat (usec): min=138, max=70620, avg=18271.17, stdev=10064.94 clat percentiles (usec): | 1.00th=[ 326], 5.00th=[ 7570], 10.00th=[ 8717], 20.00th=[10290], | 30.00th=[11994], 40.00th=[13829], 50.00th=[16188], 60.00th=[19006], | 70.00th=[21627], 80.00th=[24773], 90.00th=[31589], 95.00th=[36963], | 99.00th=[54264], 99.50th=[61080], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 48, max= 192, per=25.43%, avg=119.52, stdev=30.26, samples=99 iops : min= 12, max= 48, avg=29.57, stdev= 7.53, samples=99 write: IOPS=30, BW=123KiB/s (126kB/s)(6260KiB/50722msec); 0 zone resets clat (usec): min=616, max=700197, avg=14707.20, stdev=20980.48 lat (usec): min=617, max=700199, avg=14708.39, stdev=20980.48 clat percentiles (usec): | 1.00th=[ 1106], 5.00th=[ 1991], 10.00th=[ 2573], 20.00th=[ 3163], | 30.00th=[ 5211], 40.00th=[ 9241], 50.00th=[ 11600], 60.00th=[ 14222], | 70.00th=[ 17957], 80.00th=[ 22938], 90.00th=[ 30540], 95.00th=[ 37487], | 99.00th=[ 53740], 99.50th=[ 59507], 99.90th=[ 81265], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 13, max= 207, per=24.89%, avg=122.46, stdev=33.12, samples=100 iops : min= 3, max= 51, avg=30.30, stdev= 8.25, samples=100 lat (usec) : 250=0.29%, 500=0.75%, 750=0.26%, 1000=0.36% lat (msec) : 2=2.41%, 4=10.42%, 10=15.89%, 20=38.74%, 50=29.52% lat (msec) : 100=1.33%, 750=0.03% cpu : usr=0.15%, sys=23.09%, ctx=8090, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187366: Tue Aug 5 11:42:40 2025 read: IOPS=29, BW=118KiB/s (121kB/s)(6052KiB/51078msec) clat (usec): min=126, max=83812, avg=18012.02, stdev=9693.53 lat (usec): min=127, max=83813, avg=18013.68, stdev=9694.30 clat percentiles (usec): | 1.00th=[ 330], 5.00th=[ 7242], 10.00th=[ 8717], 20.00th=[10421], | 30.00th=[12125], 40.00th=[14222], 50.00th=[16581], 60.00th=[18744], | 70.00th=[21365], 80.00th=[24511], 90.00th=[29230], 95.00th=[34341], | 99.00th=[50594], 99.50th=[57410], 99.90th=[81265], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 55, max= 191, per=25.38%, avg=119.29, stdev=26.54, samples=100 iops : min= 13, max= 47, avg=29.51, stdev= 6.63, samples=100 write: IOPS=30, BW=122KiB/s (125kB/s)(6236KiB/51078msec); 0 zone resets clat (usec): min=556, max=734309, avg=15168.31, stdev=21706.39 lat (usec): min=556, max=734310, avg=15169.51, stdev=21706.39 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 2147], 10.00th=[ 2671], 20.00th=[ 3523], | 30.00th=[ 6128], 40.00th=[ 9634], 50.00th=[ 12125], 60.00th=[ 15139], | 70.00th=[ 19006], 80.00th=[ 23987], 90.00th=[ 31065], 95.00th=[ 35914], | 99.00th=[ 52691], 99.50th=[ 57934], 99.90th=[ 88605], 99.95th=[734004], | 99.99th=[734004] bw ( KiB/s): min= 48, max= 199, per=25.04%, avg=123.19, stdev=28.93, samples=100 iops : min= 12, max= 49, avg=30.49, stdev= 7.27, samples=100 lat (usec) : 250=0.23%, 500=1.17%, 750=0.10%, 1000=0.20% lat (msec) : 2=1.89%, 4=9.83%, 10=16.08%, 20=39.13%, 50=30.18% lat (msec) : 100=1.17%, 750=0.03% cpu : usr=0.17%, sys=22.89%, ctx=8044, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=470KiB/s (481kB/s), 118KiB/s-120KiB/s (121kB/s-123kB/s), io=23.4MiB (24.6MB), run=49786-51078msec WRITE: bw=492KiB/s (504kB/s), 122KiB/s-126KiB/s (125kB/s-129kB/s), io=24.6MiB (25.7MB), run=49786-51078msec rand-rw: (groupid=0, jobs=1): err= 0: pid=187358: Tue Aug 5 11:42:52 2025 read: IOPS=23, BW=95.3KiB/s (97.6kB/s)(5992KiB/62847msec) clat (msec): min=3, max=704, avg=17.28, stdev=22.16 lat (msec): min=3, max=704, avg=17.28, stdev=22.16 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 15], | 70.00th=[ 20], 80.00th=[ 25], 90.00th=[ 36], 95.00th=[ 45], | 99.00th=[ 68], 99.50th=[ 73], 99.90th=[ 110], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 23, max= 304, per=25.35%, avg=96.07, stdev=58.04, samples=124 iops : min= 5, max= 76, avg=23.73, stdev=14.58, samples=124 write: IOPS=25, BW=100KiB/s (103kB/s)(6296KiB/62847msec); 0 zone resets clat (msec): min=5, max=115, avg=23.35, stdev=15.15 lat (msec): min=5, max=115, avg=23.35, stdev=15.15 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 19], 60.00th=[ 24], | 70.00th=[ 29], 80.00th=[ 35], 90.00th=[ 45], 95.00th=[ 52], | 99.00th=[ 73], 99.50th=[ 87], 99.90th=[ 108], 99.95th=[ 116], | 99.99th=[ 116] bw ( KiB/s): min= 32, max= 280, per=25.40%, avg=100.85, stdev=55.82, samples=124 iops : min= 8, max= 70, avg=24.93, stdev=14.02, samples=124 lat (msec) : 4=0.26%, 10=26.63%, 20=36.20%, 50=32.36%, 100=4.43% lat (msec) : 250=0.10%, 750=0.03% cpu : usr=0.11%, sys=10.96%, ctx=4178, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187359: Tue Aug 5 11:42:52 2025 read: IOPS=23, BW=93.9KiB/s (96.1kB/s)(5940KiB/63285msec) clat (msec): min=3, max=698, avg=17.56, stdev=22.42 lat (msec): min=3, max=698, avg=17.56, stdev=22.42 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 15], | 70.00th=[ 19], 80.00th=[ 25], 90.00th=[ 37], 95.00th=[ 47], | 99.00th=[ 66], 99.50th=[ 77], 99.90th=[ 110], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 15, max= 304, per=24.93%, avg=94.47, stdev=59.04, samples=125 iops : min= 3, max= 76, avg=23.34, stdev=14.84, samples=125 write: IOPS=25, BW=100KiB/s (103kB/s)(6348KiB/63285msec); 0 zone resets clat (msec): min=4, max=114, avg=23.31, stdev=15.31 lat (msec): min=4, max=114, avg=23.32, stdev=15.31 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 19], 60.00th=[ 24], | 70.00th=[ 29], 80.00th=[ 35], 90.00th=[ 44], 95.00th=[ 53], | 99.00th=[ 77], 99.50th=[ 84], 99.90th=[ 109], 99.95th=[ 115], | 99.99th=[ 115] bw ( KiB/s): min= 39, max= 264, per=25.42%, avg=100.94, stdev=55.39, samples=125 iops : min= 9, max= 66, avg=24.95, stdev=13.93, samples=125 lat (msec) : 4=0.23%, 10=25.98%, 20=36.98%, 50=32.03%, 100=4.65% lat (msec) : 250=0.10%, 750=0.03% cpu : usr=0.13%, sys=10.96%, ctx=4268, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187360: Tue Aug 5 11:42:52 2025 read: IOPS=23, BW=95.4KiB/s (97.7kB/s)(6028KiB/63177msec) clat (msec): min=3, max=116, avg=17.43, stdev=13.55 lat (msec): min=3, max=116, avg=17.43, stdev=13.55 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 16], | 70.00th=[ 20], 80.00th=[ 26], 90.00th=[ 37], 95.00th=[ 45], | 99.00th=[ 64], 99.50th=[ 73], 99.90th=[ 86], 99.95th=[ 117], | 99.99th=[ 117] bw ( KiB/s): min= 16, max= 288, per=25.25%, avg=95.69, stdev=55.25, samples=124 iops : min= 4, max= 72, avg=23.64, stdev=13.88, samples=124 write: IOPS=24, BW=99.1KiB/s (101kB/s)(6260KiB/63177msec); 0 zone resets clat (msec): min=5, max=709, avg=23.45, stdev=23.25 lat (msec): min=5, max=709, avg=23.45, stdev=23.25 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 18], 60.00th=[ 22], | 70.00th=[ 27], 80.00th=[ 34], 90.00th=[ 44], 95.00th=[ 52], | 99.00th=[ 82], 99.50th=[ 94], 99.90th=[ 134], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 39, max= 280, per=24.95%, avg=99.04, stdev=56.02, samples=124 iops : min= 9, max= 70, avg=24.48, stdev=14.09, samples=124 lat (msec) : 4=0.13%, 10=25.85%, 20=37.86%, 50=31.09%, 100=4.88% lat (msec) : 250=0.16%, 750=0.03% cpu : usr=0.13%, sys=11.04%, ctx=4301, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187361: Tue Aug 5 11:42:52 2025 read: IOPS=23, BW=95.9KiB/s (98.2kB/s)(6052KiB/63086msec) clat (usec): min=3451, max=94241, avg=16982.02, stdev=13535.67 lat (usec): min=3452, max=94242, avg=16983.02, stdev=13535.68 clat percentiles (usec): | 1.00th=[ 4146], 5.00th=[ 5014], 10.00th=[ 5604], 20.00th=[ 6980], | 30.00th=[ 8455], 40.00th=[10290], 50.00th=[12256], 60.00th=[14746], | 70.00th=[18482], 80.00th=[24249], 90.00th=[35914], 95.00th=[46400], | 99.00th=[66323], 99.50th=[71828], 99.90th=[84411], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 8, max= 352, per=25.38%, avg=96.20, stdev=57.64, samples=124 iops : min= 2, max= 88, avg=23.77, stdev=14.48, samples=124 write: IOPS=24, BW=98.8KiB/s (101kB/s)(6236KiB/63086msec); 0 zone resets clat (msec): min=5, max=706, avg=23.85, stdev=23.41 lat (msec): min=5, max=706, avg=23.85, stdev=23.41 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 35], 90.00th=[ 44], 95.00th=[ 53], | 99.00th=[ 87], 99.50th=[ 100], 99.90th=[ 134], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 23, max= 264, per=25.03%, avg=99.36, stdev=56.65, samples=124 iops : min= 5, max= 66, avg=24.56, stdev=14.25, samples=124 lat (msec) : 4=0.39%, 10=26.24%, 20=36.26%, 50=32.42%, 100=4.49% lat (msec) : 250=0.16%, 750=0.03% cpu : usr=0.12%, sys=11.09%, ctx=4193, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=379KiB/s (389kB/s), 93.9KiB/s-95.9KiB/s (96.1kB/s-98.2kB/s), io=23.4MiB (24.6MB), run=62847-63285msec WRITE: bw=397KiB/s (407kB/s), 98.8KiB/s-100KiB/s (101kB/s-103kB/s), io=24.6MiB (25.7MB), run=62847-63285msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=187390: Tue Aug 5 11:43:11 2025 read: IOPS=28, BW=451KiB/s (461kB/s)(5920KiB/13139msec) clat (usec): min=94, max=67721, avg=18422.86, stdev=10003.18 lat (usec): min=95, max=67721, avg=18423.85, stdev=10003.13 clat percentiles (usec): | 1.00th=[ 182], 5.00th=[ 429], 10.00th=[ 9372], 20.00th=[11469], | 30.00th=[12911], 40.00th=[14746], 50.00th=[16909], 60.00th=[19268], | 70.00th=[21890], 80.00th=[24249], 90.00th=[30016], 95.00th=[36963], | 99.00th=[48497], 99.50th=[56886], 99.90th=[67634], 99.95th=[67634], | 99.99th=[67634] bw ( KiB/s): min= 256, max= 608, per=24.87%, avg=451.62, stdev=99.25, samples=26 iops : min= 16, max= 38, avg=27.77, stdev= 6.07, samples=26 write: IOPS=30, BW=485KiB/s (496kB/s)(6368KiB/13139msec); 0 zone resets clat (usec): min=1367, max=66713, avg=15630.57, stdev=12409.19 lat (usec): min=1370, max=66714, avg=15632.29, stdev=12409.19 clat percentiles (usec): | 1.00th=[ 1680], 5.00th=[ 3261], 10.00th=[ 4178], 20.00th=[ 4948], | 30.00th=[ 6194], 40.00th=[ 9896], 50.00th=[12911], 60.00th=[14746], | 70.00th=[17957], 80.00th=[24249], 90.00th=[32900], 95.00th=[41681], | 99.00th=[56361], 99.50th=[64750], 99.90th=[66847], 99.95th=[66847], | 99.99th=[66847] bw ( KiB/s): min= 287, max= 766, per=25.36%, avg=484.69, stdev=108.01, samples=26 iops : min= 17, max= 47, avg=29.85, stdev= 6.66, samples=26 lat (usec) : 100=0.13%, 250=0.52%, 500=2.21%, 750=0.26% lat (msec) : 2=1.17%, 4=3.91%, 10=18.23%, 20=42.45%, 50=29.56% lat (msec) : 100=1.56% cpu : usr=0.15%, sys=26.83%, ctx=2500, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187391: Tue Aug 5 11:43:11 2025 read: IOPS=27, BW=436KiB/s (446kB/s)(5744KiB/13186msec) clat (usec): min=275, max=58670, avg=17836.78, stdev=9365.46 lat (usec): min=275, max=58672, avg=17837.74, stdev=9365.46 clat percentiles (usec): | 1.00th=[ 343], 5.00th=[ 486], 10.00th=[ 8848], 20.00th=[10945], | 30.00th=[12649], 40.00th=[14877], 50.00th=[16712], 60.00th=[18744], | 70.00th=[21103], 80.00th=[24249], 90.00th=[28967], 95.00th=[36439], | 99.00th=[46400], 99.50th=[57934], 99.90th=[58459], 99.95th=[58459], | 99.99th=[58459] bw ( KiB/s): min= 187, max= 608, per=23.86%, avg=433.38, stdev=99.93, samples=26 iops : min= 11, max= 38, avg=26.62, stdev= 6.33, samples=26 write: IOPS=31, BW=496KiB/s (508kB/s)(6544KiB/13186msec); 0 zone resets clat (usec): min=1532, max=56167, avg=16380.43, stdev=11893.39 lat (usec): min=1535, max=56169, avg=16385.04, stdev=11892.43 clat percentiles (usec): | 1.00th=[ 2507], 5.00th=[ 3654], 10.00th=[ 4228], 20.00th=[ 5145], | 30.00th=[ 8094], 40.00th=[11338], 50.00th=[13435], 60.00th=[16319], | 70.00th=[20317], 80.00th=[25297], 90.00th=[34341], 95.00th=[41681], | 99.00th=[51119], 99.50th=[52691], 99.90th=[56361], 99.95th=[56361], | 99.99th=[56361] bw ( KiB/s): min= 320, max= 670, per=25.88%, avg=494.50, stdev=102.39, samples=26 iops : min= 20, max= 41, avg=30.46, stdev= 6.16, samples=26 lat (usec) : 500=2.47%, 750=0.26% lat (msec) : 2=0.78%, 4=4.30%, 10=17.58%, 20=41.54%, 50=32.03% lat (msec) : 100=1.04% cpu : usr=0.16%, sys=26.94%, ctx=2630, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187393: Tue Aug 5 11:43:11 2025 read: IOPS=29, BW=476KiB/s (487kB/s)(6256KiB/13141msec) clat (usec): min=252, max=90002, avg=18297.98, stdev=10502.04 lat (usec): min=253, max=90003, avg=18299.34, stdev=10501.98 clat percentiles (usec): | 1.00th=[ 326], 5.00th=[ 404], 10.00th=[ 8586], 20.00th=[11863], | 30.00th=[13042], 40.00th=[14615], 50.00th=[16581], 60.00th=[19268], | 70.00th=[21365], 80.00th=[24249], 90.00th=[30016], 95.00th=[36439], | 99.00th=[54789], 99.50th=[57934], 99.90th=[89654], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 256, max= 638, per=26.29%, avg=477.35, stdev=86.83, samples=26 iops : min= 16, max= 39, avg=29.38, stdev= 5.28, samples=26 write: IOPS=28, BW=459KiB/s (470kB/s)(6032KiB/13141msec); 0 zone resets clat (usec): min=1448, max=72579, avg=15773.54, stdev=11732.83 lat (usec): min=1450, max=72581, avg=15781.94, stdev=11746.99 clat percentiles (usec): | 1.00th=[ 1713], 5.00th=[ 3458], 10.00th=[ 4080], 20.00th=[ 5407], | 30.00th=[ 8094], 40.00th=[10683], 50.00th=[13042], 60.00th=[14877], | 70.00th=[17957], 80.00th=[24249], 90.00th=[33817], 95.00th=[39060], | 99.00th=[50594], 99.50th=[71828], 99.90th=[72877], 99.95th=[72877], | 99.99th=[72877] bw ( KiB/s): min= 191, max= 704, per=23.69%, avg=452.81, stdev=123.22, samples=26 iops : min= 11, max= 44, avg=27.85, stdev= 7.83, samples=26 lat (usec) : 500=3.12%, 750=0.26% lat (msec) : 2=1.04%, 4=3.52%, 10=16.54%, 20=43.88%, 50=30.08% lat (msec) : 100=1.56% cpu : usr=0.16%, sys=27.15%, ctx=2495, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187395: Tue Aug 5 11:43:11 2025 read: IOPS=29, BW=478KiB/s (489kB/s)(6032KiB/12626msec) clat (usec): min=127, max=79654, avg=17753.66, stdev=11230.13 lat (usec): min=128, max=79655, avg=17754.54, stdev=11230.10 clat percentiles (usec): | 1.00th=[ 206], 5.00th=[ 355], 10.00th=[ 537], 20.00th=[11076], | 30.00th=[13304], 40.00th=[14746], 50.00th=[16712], 60.00th=[18220], | 70.00th=[20579], 80.00th=[23987], 90.00th=[28967], 95.00th=[39584], | 99.00th=[60556], 99.50th=[76022], 99.90th=[79168], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 218, max= 640, per=26.09%, avg=473.72, stdev=113.31, samples=25 iops : min= 13, max= 40, avg=29.16, stdev= 7.05, samples=25 write: IOPS=30, BW=495KiB/s (507kB/s)(6256KiB/12626msec); 0 zone resets clat (usec): min=1344, max=75479, avg=14931.82, stdev=11239.74 lat (usec): min=1346, max=75481, avg=14938.27, stdev=11247.13 clat percentiles (usec): | 1.00th=[ 2040], 5.00th=[ 3785], 10.00th=[ 4293], 20.00th=[ 5145], | 30.00th=[ 6980], 40.00th=[ 9372], 50.00th=[11863], 60.00th=[14091], | 70.00th=[17695], 80.00th=[24511], 90.00th=[29754], 95.00th=[36963], | 99.00th=[49546], 99.50th=[55313], 99.90th=[74974], 99.95th=[74974], | 99.99th=[74974] bw ( KiB/s): min= 287, max= 800, per=25.93%, avg=495.48, stdev=139.67, samples=25 iops : min= 17, max= 50, avg=30.52, stdev= 8.89, samples=25 lat (usec) : 250=0.65%, 500=4.17%, 750=0.52% lat (msec) : 2=0.39%, 4=3.12%, 10=20.05%, 20=43.10%, 50=26.82% lat (msec) : 100=1.17% cpu : usr=0.14%, sys=27.46%, ctx=2484, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1816KiB/s (1860kB/s), 436KiB/s-478KiB/s (446kB/s-489kB/s), io=23.4MiB (24.5MB), run=12626-13186msec WRITE: bw=1911KiB/s (1957kB/s), 459KiB/s-496KiB/s (470kB/s-508kB/s), io=24.6MiB (25.8MB), run=12626-13186msec rand-rw: (groupid=0, jobs=1): err= 0: pid=187392: Tue Aug 5 11:43:14 2025 read: IOPS=22, BW=366KiB/s (375kB/s)(5920KiB/16156msec) clat (usec): min=3883, max=84352, avg=16688.16, stdev=13960.59 lat (usec): min=3884, max=84353, avg=16689.08, stdev=13960.64 clat percentiles (usec): | 1.00th=[ 4359], 5.00th=[ 4948], 10.00th=[ 5342], 20.00th=[ 6456], | 30.00th=[ 7898], 40.00th=[ 9241], 50.00th=[11207], 60.00th=[13829], | 70.00th=[17171], 80.00th=[24511], 90.00th=[39584], 95.00th=[49021], | 99.00th=[64750], 99.50th=[73925], 99.90th=[84411], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 95, max= 1056, per=24.29%, avg=360.03, stdev=244.69, samples=32 iops : min= 5, max= 66, avg=22.00, stdev=15.53, samples=32 write: IOPS=24, BW=394KiB/s (404kB/s)(6368KiB/16156msec); 0 zone resets clat (usec): min=6343, max=98054, avg=24819.99, stdev=17471.11 lat (usec): min=6345, max=98057, avg=24821.97, stdev=17471.15 clat percentiles (usec): | 1.00th=[ 6783], 5.00th=[ 8455], 10.00th=[ 9110], 20.00th=[10421], | 30.00th=[11731], 40.00th=[13960], 50.00th=[17695], 60.00th=[24511], | 70.00th=[30540], 80.00th=[39584], 90.00th=[49546], 95.00th=[56886], | 99.00th=[79168], 99.50th=[95945], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 159, max= 1152, per=25.02%, avg=390.00, stdev=245.95, samples=32 iops : min= 9, max= 72, avg=23.88, stdev=15.62, samples=32 lat (msec) : 4=0.13%, 10=29.04%, 20=34.38%, 50=29.69%, 100=6.77% cpu : usr=0.13%, sys=11.28%, ctx=1075, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187394: Tue Aug 5 11:43:14 2025 read: IOPS=22, BW=362KiB/s (370kB/s)(5744KiB/15878msec) clat (usec): min=3590, max=75367, avg=17153.61, stdev=13161.75 lat (usec): min=3591, max=75369, avg=17154.58, stdev=13161.82 clat percentiles (usec): | 1.00th=[ 4359], 5.00th=[ 5211], 10.00th=[ 5866], 20.00th=[ 7242], | 30.00th=[ 8291], 40.00th=[10421], 50.00th=[12256], 60.00th=[14484], | 70.00th=[19006], 80.00th=[25822], 90.00th=[36963], 95.00th=[45876], | 99.00th=[60556], 99.50th=[72877], 99.90th=[74974], 99.95th=[74974], | 99.99th=[74974] bw ( KiB/s): min= 156, max= 896, per=24.17%, avg=358.23, stdev=204.20, samples=31 iops : min= 9, max= 56, avg=21.87, stdev=12.98, samples=31 write: IOPS=25, BW=412KiB/s (422kB/s)(6544KiB/15878msec); 0 zone resets clat (usec): min=6659, max=98201, avg=23641.91, stdev=15380.58 lat (usec): min=6661, max=98203, avg=23643.72, stdev=15380.56 clat percentiles (usec): | 1.00th=[ 7504], 5.00th=[ 8717], 10.00th=[ 9634], 20.00th=[10814], | 30.00th=[12649], 40.00th=[15008], 50.00th=[18744], 60.00th=[22938], | 70.00th=[28705], 80.00th=[33817], 90.00th=[46400], 95.00th=[52691], | 99.00th=[74974], 99.50th=[82314], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 126, max= 1056, per=25.82%, avg=402.48, stdev=222.82, samples=31 iops : min= 7, max= 66, avg=24.65, stdev=14.12, samples=31 lat (msec) : 4=0.13%, 10=24.09%, 20=37.24%, 50=33.33%, 100=5.21% cpu : usr=0.14%, sys=11.84%, ctx=1102, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187396: Tue Aug 5 11:43:14 2025 read: IOPS=24, BW=394KiB/s (404kB/s)(6256KiB/15870msec) clat (msec): min=3, max=100, avg=17.15, stdev=14.46 lat (msec): min=3, max=100, avg=17.15, stdev=14.46 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 26], 90.00th=[ 41], 95.00th=[ 51], | 99.00th=[ 64], 99.50th=[ 75], 99.90th=[ 101], 99.95th=[ 101], | 99.99th=[ 101] bw ( KiB/s): min= 159, max= 1184, per=26.39%, avg=391.16, stdev=252.62, samples=31 iops : min= 9, max= 74, avg=23.94, stdev=16.01, samples=31 write: IOPS=23, BW=380KiB/s (389kB/s)(6032KiB/15870msec); 0 zone resets clat (usec): min=6140, max=98317, avg=24104.27, stdev=16230.67 lat (usec): min=6143, max=98319, avg=24106.25, stdev=16230.73 clat percentiles (usec): | 1.00th=[ 6980], 5.00th=[ 8455], 10.00th=[ 9503], 20.00th=[10814], | 30.00th=[12649], 40.00th=[14615], 50.00th=[18220], 60.00th=[22414], | 70.00th=[28443], 80.00th=[36439], 90.00th=[50070], 95.00th=[56886], | 99.00th=[73925], 99.50th=[78119], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 127, max= 992, per=23.70%, avg=369.52, stdev=212.44, samples=31 iops : min= 7, max= 62, avg=22.58, stdev=13.49, samples=31 lat (msec) : 4=0.13%, 10=27.99%, 20=36.33%, 50=27.99%, 100=7.42% lat (msec) : 250=0.13% cpu : usr=0.08%, sys=11.92%, ctx=1126, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187397: Tue Aug 5 11:43:14 2025 read: IOPS=23, BW=375KiB/s (384kB/s)(6032KiB/16078msec) clat (usec): min=4122, max=99007, avg=17684.92, stdev=15290.48 lat (usec): min=4123, max=99007, avg=17686.48, stdev=15290.12 clat percentiles (usec): | 1.00th=[ 4490], 5.00th=[ 5342], 10.00th=[ 6128], 20.00th=[ 7046], | 30.00th=[ 8029], 40.00th=[ 9241], 50.00th=[11338], 60.00th=[14484], | 70.00th=[19006], 80.00th=[26608], 90.00th=[42206], 95.00th=[48497], | 99.00th=[73925], 99.50th=[78119], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 159, max= 1184, per=25.10%, avg=372.00, stdev=241.63, samples=32 iops : min= 9, max= 74, avg=22.75, stdev=15.33, samples=32 write: IOPS=24, BW=389KiB/s (398kB/s)(6256KiB/16078msec); 0 zone resets clat (usec): min=6599, max=98929, avg=23750.98, stdev=16123.78 lat (usec): min=6600, max=98932, avg=23753.13, stdev=16123.73 clat percentiles (usec): | 1.00th=[ 7373], 5.00th=[ 8717], 10.00th=[ 9241], 20.00th=[10552], | 30.00th=[11994], 40.00th=[14484], 50.00th=[18220], 60.00th=[22152], | 70.00th=[29230], 80.00th=[36439], 90.00th=[44303], 95.00th=[56361], | 99.00th=[81265], 99.50th=[94897], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 159, max= 928, per=25.01%, avg=389.97, stdev=237.29, samples=32 iops : min= 9, max= 58, avg=23.87, stdev=15.08, samples=32 lat (msec) : 10=29.30%, 20=34.24%, 50=30.60%, 100=5.86% cpu : usr=0.11%, sys=11.56%, ctx=1112, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1483KiB/s (1518kB/s), 362KiB/s-394KiB/s (370kB/s-404kB/s), io=23.4MiB (24.5MB), run=15870-16156msec WRITE: bw=1560KiB/s (1597kB/s), 380KiB/s-412KiB/s (389kB/s-422kB/s), io=24.6MiB (25.8MB), run=15870-16156msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=187404: Tue Aug 5 11:43:23 2025 read: IOPS=2, BW=2910KiB/s (2980kB/s)(7168KiB/2463msec) clat (usec): min=966, max=216178, avg=126765.80, stdev=65585.50 lat (usec): min=967, max=216181, avg=126767.33, stdev=65585.83 clat percentiles (usec): | 1.00th=[ 963], 5.00th=[ 963], 10.00th=[ 963], 20.00th=[117965], | 30.00th=[120062], 40.00th=[120062], 50.00th=[123208], 60.00th=[143655], | 70.00th=[143655], 80.00th=[166724], 90.00th=[217056], 95.00th=[217056], | 99.00th=[217056], 99.50th=[217056], 99.90th=[217056], 99.95th=[217056], | 99.99th=[217056] bw ( KiB/s): min= 2039, max= 5242, per=61.72%, avg=3792.33, stdev=1622.95, samples=3 iops : min= 1, max= 5, avg= 3.33, stdev= 2.08, samples=3 write: IOPS=2, BW=2079KiB/s (2129kB/s)(5120KiB/2463msec); 0 zone resets clat (msec): min=173, max=531, avg=307.97, stdev=133.90 lat (msec): min=173, max=531, avg=308.05, stdev=133.89 clat percentiles (msec): | 1.00th=[ 174], 5.00th=[ 174], 10.00th=[ 174], 20.00th=[ 174], | 30.00th=[ 253], 40.00th=[ 253], 50.00th=[ 292], 60.00th=[ 292], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 531], 95.00th=[ 531], | 99.00th=[ 531], 99.50th=[ 531], 99.90th=[ 531], 99.95th=[ 531], | 99.99th=[ 531] bw ( KiB/s): min= 2039, max= 4096, per=31.71%, avg=2727.67, stdev=1185.02, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 lat (usec) : 1000=8.33% lat (msec) : 250=58.33%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=50.85%, ctx=1004, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187405: Tue Aug 5 11:43:23 2025 read: IOPS=0, BW=614KiB/s (629kB/s)(2048KiB/3333msec) clat (usec): min=803, max=177021, avg=88912.60, stdev=124604.87 lat (usec): min=804, max=177025, avg=88914.98, stdev=124606.59 clat percentiles (usec): | 1.00th=[ 807], 5.00th=[ 807], 10.00th=[ 807], 20.00th=[ 807], | 30.00th=[ 807], 40.00th=[ 807], 50.00th=[ 807], 60.00th=[177210], | 70.00th=[177210], 80.00th=[177210], 90.00th=[177210], 95.00th=[177210], | 99.00th=[177210], 99.50th=[177210], 99.90th=[177210], 99.95th=[177210], | 99.99th=[177210] bw ( KiB/s): min= 1753, max= 2003, per=30.57%, avg=1878.00, stdev=176.78, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3072KiB/s (3146kB/s)(10.0MiB/3333msec); 0 zone resets clat (msec): min=171, max=497, avg=311.20, stdev=121.44 lat (msec): min=171, max=497, avg=311.28, stdev=121.44 clat percentiles (msec): | 1.00th=[ 171], 5.00th=[ 171], 10.00th=[ 171], 20.00th=[ 178], | 30.00th=[ 182], 40.00th=[ 218], 50.00th=[ 300], 60.00th=[ 355], | 70.00th=[ 359], 80.00th=[ 384], 90.00th=[ 472], 95.00th=[ 498], | 99.00th=[ 498], 99.50th=[ 498], 99.90th=[ 498], 99.95th=[ 498], | 99.99th=[ 498] bw ( KiB/s): min= 2039, max= 4096, per=37.87%, avg=3257.20, stdev=1108.55, samples=5 iops : min= 1, max= 4, avg= 2.80, stdev= 1.30, samples=5 lat (usec) : 1000=8.33% lat (msec) : 250=41.67%, 500=50.00% cpu : usr=0.00%, sys=43.61%, ctx=387, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187406: Tue Aug 5 11:43:23 2025 read: IOPS=2, BW=2731KiB/s (2796kB/s)(6144KiB/2250msec) clat (usec): min=764, max=212986, avg=114904.28, stdev=92707.48 lat (usec): min=765, max=212987, avg=114905.43, stdev=92707.49 clat percentiles (usec): | 1.00th=[ 766], 5.00th=[ 766], 10.00th=[ 766], 20.00th=[ 1745], | 30.00th=[ 1745], 40.00th=[128451], 50.00th=[128451], 60.00th=[154141], | 70.00th=[191890], 80.00th=[191890], 90.00th=[212861], 95.00th=[212861], | 99.00th=[212861], 99.50th=[212861], 99.90th=[212861], 99.95th=[212861], | 99.99th=[212861] bw ( KiB/s): min= 2048, max= 4096, per=55.46%, avg=3407.67, stdev=1177.54, samples=3 iops : min= 2, max= 4, avg= 3.00, stdev= 1.00, samples=3 write: IOPS=2, BW=2731KiB/s (2796kB/s)(6144KiB/2250msec); 0 zone resets clat (msec): min=172, max=348, avg=258.45, stdev=67.28 lat (msec): min=172, max=348, avg=258.51, stdev=67.28 clat percentiles (msec): | 1.00th=[ 174], 5.00th=[ 174], 10.00th=[ 174], 20.00th=[ 209], | 30.00th=[ 209], 40.00th=[ 232], 50.00th=[ 232], 60.00th=[ 268], | 70.00th=[ 321], 80.00th=[ 321], 90.00th=[ 347], 95.00th=[ 347], | 99.00th=[ 347], 99.50th=[ 347], 99.90th=[ 347], 99.95th=[ 347], | 99.99th=[ 347] bw ( KiB/s): min= 2039, max= 3512, per=28.04%, avg=2411.75, stdev=733.51, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (usec) : 1000=8.33% lat (msec) : 2=8.33%, 250=58.33%, 500=25.00% cpu : usr=0.00%, sys=52.78%, ctx=436, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187407: Tue Aug 5 11:43:23 2025 read: IOPS=1, BW=1877KiB/s (1922kB/s)(5120KiB/2728msec) clat (usec): min=877, max=252795, avg=129476.85, stdev=117024.98 lat (usec): min=878, max=252796, avg=129477.85, stdev=117024.74 clat percentiles (usec): | 1.00th=[ 881], 5.00th=[ 881], 10.00th=[ 881], 20.00th=[ 881], | 30.00th=[ 7373], 40.00th=[ 7373], 50.00th=[191890], 60.00th=[191890], | 70.00th=[196084], 80.00th=[196084], 90.00th=[252707], 95.00th=[252707], | 99.00th=[252707], 99.50th=[252707], 99.90th=[252707], 99.95th=[252707], | 99.99th=[252707] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=2, BW=2628KiB/s (2691kB/s)(7168KiB/2728msec); 0 zone resets clat (msec): min=166, max=459, avg=288.73, stdev=104.61 lat (msec): min=166, max=459, avg=288.79, stdev=104.62 clat percentiles (msec): | 1.00th=[ 167], 5.00th=[ 167], 10.00th=[ 167], 20.00th=[ 201], | 30.00th=[ 220], 40.00th=[ 220], 50.00th=[ 271], 60.00th=[ 342], | 70.00th=[ 342], 80.00th=[ 363], 90.00th=[ 460], 95.00th=[ 460], | 99.00th=[ 460], 99.50th=[ 460], 99.90th=[ 460], 99.95th=[ 460], | 99.99th=[ 460] bw ( KiB/s): min= 1750, max= 6011, per=40.41%, avg=3476.25, stdev=1985.69, samples=4 iops : min= 1, max= 5, avg= 3.00, stdev= 1.83, samples=4 lat (usec) : 1000=8.33% lat (msec) : 10=8.33%, 250=41.67%, 500=41.67% cpu : usr=0.00%, sys=45.07%, ctx=447, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6145KiB/s (6292kB/s), 614KiB/s-2910KiB/s (629kB/s-2980kB/s), io=20.0MiB (20.0MB), run=2250-3333msec WRITE: bw=8602KiB/s (8809kB/s), 2079KiB/s-3072KiB/s (2129kB/s-3146kB/s), io=28.0MiB (29.4MB), run=2250-3333msec rand-rw: (groupid=0, jobs=1): err= 0: pid=187409: Tue Aug 5 11:43:23 2025 read: IOPS=2, BW=2131KiB/s (2182kB/s)(7168KiB/3364msec) clat (msec): min=126, max=513, avg=232.43, stdev=130.31 lat (msec): min=126, max=513, avg=232.43, stdev=130.31 clat percentiles (msec): | 1.00th=[ 128], 5.00th=[ 128], 10.00th=[ 128], 20.00th=[ 144], | 30.00th=[ 182], 40.00th=[ 182], 50.00th=[ 203], 60.00th=[ 224], | 70.00th=[ 224], 80.00th=[ 239], 90.00th=[ 514], 95.00th=[ 514], | 99.00th=[ 514], 99.50th=[ 514], 99.90th=[ 514], 99.95th=[ 514], | 99.99th=[ 514] bw ( KiB/s): min= 2039, max= 5180, per=62.84%, avg=3768.67, stdev=1594.51, samples=3 iops : min= 1, max= 5, avg= 3.00, stdev= 2.00, samples=3 write: IOPS=1, BW=1522KiB/s (1559kB/s)(5120KiB/3364msec); 0 zone resets clat (msec): min=68, max=691, avg=341.55, stdev=282.91 lat (msec): min=68, max=691, avg=341.63, stdev=282.90 clat percentiles (msec): | 1.00th=[ 69], 5.00th=[ 69], 10.00th=[ 69], 20.00th=[ 69], | 30.00th=[ 155], 40.00th=[ 155], 50.00th=[ 194], 60.00th=[ 194], | 70.00th=[ 600], 80.00th=[ 600], 90.00th=[ 693], 95.00th=[ 693], | 99.00th=[ 693], 99.50th=[ 693], 99.90th=[ 693], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 1777, max= 6119, per=39.48%, avg=3314.67, stdev=2432.40, samples=3 iops : min= 1, max= 5, avg= 2.67, stdev= 2.08, samples=3 lat (msec) : 100=8.33%, 250=66.67%, 750=25.00% cpu : usr=0.03%, sys=17.16%, ctx=147, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187410: Tue Aug 5 11:43:23 2025 read: IOPS=0, BW=600KiB/s (614kB/s)(2048KiB/3415msec) clat (msec): min=73, max=624, avg=348.89, stdev=389.48 lat (msec): min=73, max=624, avg=348.89, stdev=389.48 clat percentiles (msec): | 1.00th=[ 74], 5.00th=[ 74], 10.00th=[ 74], 20.00th=[ 74], | 30.00th=[ 74], 40.00th=[ 74], 50.00th=[ 74], 60.00th=[ 625], | 70.00th=[ 625], 80.00th=[ 625], 90.00th=[ 625], 95.00th=[ 625], | 99.00th=[ 625], 99.50th=[ 625], 99.90th=[ 625], 99.95th=[ 625], | 99.99th=[ 625] bw ( KiB/s): min= 2035, max= 2048, per=34.04%, avg=2041.50, stdev= 9.19, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2999KiB/s (3071kB/s)(10.0MiB/3415msec); 0 zone resets clat (msec): min=34, max=485, avg=259.62, stdev=160.53 lat (msec): min=35, max=485, avg=259.69, stdev=160.54 clat percentiles (msec): | 1.00th=[ 35], 5.00th=[ 35], 10.00th=[ 35], 20.00th=[ 50], | 30.00th=[ 109], 40.00th=[ 174], 50.00th=[ 275], 60.00th=[ 326], | 70.00th=[ 342], 80.00th=[ 351], 90.00th=[ 451], 95.00th=[ 485], | 99.00th=[ 485], 99.50th=[ 485], 99.90th=[ 485], 99.95th=[ 485], | 99.99th=[ 485] bw ( KiB/s): min= 1777, max= 4087, per=28.56%, avg=2397.20, stdev=951.51, samples=5 iops : min= 1, max= 3, avg= 1.60, stdev= 0.89, samples=5 lat (msec) : 50=16.67%, 100=8.33%, 250=16.67%, 500=50.00%, 750=8.33% cpu : usr=0.06%, sys=23.32%, ctx=282, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187411: Tue Aug 5 11:43:23 2025 read: IOPS=1, BW=1818KiB/s (1862kB/s)(6144KiB/3379msec) clat (msec): min=52, max=554, avg=271.07, stdev=229.54 lat (msec): min=52, max=554, avg=271.07, stdev=229.54 clat percentiles (msec): | 1.00th=[ 53], 5.00th=[ 53], 10.00th=[ 53], 20.00th=[ 65], | 30.00th=[ 65], 40.00th=[ 101], 50.00th=[ 101], 60.00th=[ 342], | 70.00th=[ 514], 80.00th=[ 514], 90.00th=[ 558], 95.00th=[ 558], | 99.00th=[ 558], 99.50th=[ 558], 99.90th=[ 558], 99.95th=[ 558], | 99.99th=[ 558] bw ( KiB/s): min= 1777, max= 4096, per=44.00%, avg=2638.67, stdev=1269.08, samples=3 iops : min= 1, max= 4, avg= 2.00, stdev= 1.73, samples=3 write: IOPS=1, BW=1818KiB/s (1862kB/s)(6144KiB/3379msec); 0 zone resets clat (msec): min=54, max=534, avg=250.70, stdev=185.14 lat (msec): min=54, max=534, avg=250.78, stdev=185.14 clat percentiles (msec): | 1.00th=[ 55], 5.00th=[ 55], 10.00th=[ 55], 20.00th=[ 105], | 30.00th=[ 105], 40.00th=[ 142], 50.00th=[ 142], 60.00th=[ 284], | 70.00th=[ 384], 80.00th=[ 384], 90.00th=[ 535], 95.00th=[ 535], | 99.00th=[ 535], 99.50th=[ 535], 99.90th=[ 535], 99.95th=[ 535], | 99.99th=[ 535] bw ( KiB/s): min= 2035, max= 3555, per=28.80%, avg=2418.00, stdev=758.01, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 100=25.00%, 250=25.00%, 500=25.00%, 750=25.00% cpu : usr=0.06%, sys=12.61%, ctx=106, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187412: Tue Aug 5 11:43:23 2025 read: IOPS=1, BW=1590KiB/s (1628kB/s)(5120KiB/3220msec) clat (msec): min=74, max=300, avg=181.21, stdev=97.44 lat (msec): min=74, max=300, avg=181.22, stdev=97.44 clat percentiles (msec): | 1.00th=[ 75], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 75], | 30.00th=[ 128], 40.00th=[ 128], 50.00th=[ 136], 60.00th=[ 136], | 70.00th=[ 268], 80.00th=[ 268], 90.00th=[ 300], 95.00th=[ 300], | 99.00th=[ 300], 99.50th=[ 300], 99.90th=[ 300], 99.95th=[ 300], | 99.99th=[ 300] bw ( KiB/s): min= 4087, max= 4096, per=68.23%, avg=4091.50, stdev= 6.36, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2226KiB/s (2280kB/s)(7168KiB/3220msec); 0 zone resets clat (msec): min=110, max=626, avg=310.05, stdev=178.64 lat (msec): min=110, max=626, avg=310.12, stdev=178.63 clat percentiles (msec): | 1.00th=[ 111], 5.00th=[ 111], 10.00th=[ 111], 20.00th=[ 128], | 30.00th=[ 251], 40.00th=[ 251], 50.00th=[ 284], 60.00th=[ 347], | 70.00th=[ 347], 80.00th=[ 422], 90.00th=[ 625], 95.00th=[ 625], | 99.00th=[ 625], 99.50th=[ 625], 99.90th=[ 625], 99.95th=[ 625], | 99.99th=[ 625] bw ( KiB/s): min= 2035, max= 5333, per=40.19%, avg=3373.75, stdev=1621.31, samples=4 iops : min= 1, max= 5, avg= 2.75, stdev= 1.71, samples=4 lat (msec) : 100=8.33%, 250=33.33%, 500=50.00%, 750=8.33% cpu : usr=0.03%, sys=21.53%, ctx=243, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5997KiB/s (6141kB/s), 600KiB/s-2131KiB/s (614kB/s-2182kB/s), io=20.0MiB (20.0MB), run=3220-3415msec WRITE: bw=8396KiB/s (8597kB/s), 1522KiB/s-2999KiB/s (1559kB/s-3071kB/s), io=28.0MiB (29.4MB), run=3220-3415msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=187418: Tue Aug 5 11:43:34 2025 read: IOPS=1, BW=7046KiB/s (7215kB/s)(12.0MiB/1744msec) clat (msec): min=2, max=1246, avg=574.03, stdev=628.56 lat (msec): min=2, max=1246, avg=574.03, stdev=628.56 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 3], 20.00th=[ 3], | 30.00th=[ 3], 40.00th=[ 472], 50.00th=[ 472], 60.00th=[ 472], | 70.00th=[ 1250], 80.00th=[ 1250], 90.00th=[ 1250], 95.00th=[ 1250], | 99.00th=[ 1250], 99.50th=[ 1250], 99.90th=[ 1250], 99.95th=[ 1250], | 99.99th=[ 1250] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 4=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=30.87%, ctx=1888, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187419: Tue Aug 5 11:43:34 2025 read: IOPS=0, BW=1048KiB/s (1073kB/s)(4096KiB/3910msec) clat (nsec): min=1270.1M, max=1270.1M, avg=1270081359.00, stdev= 0.00 lat (nsec): min=1270.1M, max=1270.1M, avg=1270083324.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1267], 5.00th=[ 1267], 10.00th=[ 1267], 20.00th=[ 1267], | 30.00th=[ 1267], 40.00th=[ 1267], 50.00th=[ 1267], 60.00th=[ 1267], | 70.00th=[ 1267], 80.00th=[ 1267], 90.00th=[ 1267], 95.00th=[ 1267], | 99.00th=[ 1267], 99.50th=[ 1267], 99.90th=[ 1267], 99.95th=[ 1267], | 99.99th=[ 1267] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2095KiB/s (2145kB/s)(8192KiB/3910msec); 0 zone resets clat (msec): min=919, max=1707, avg=1313.34, stdev=557.33 lat (msec): min=919, max=1707, avg=1313.58, stdev=557.20 clat percentiles (msec): | 1.00th=[ 919], 5.00th=[ 919], 10.00th=[ 919], 20.00th=[ 919], | 30.00th=[ 919], 40.00th=[ 919], 50.00th=[ 919], 60.00th=[ 1703], | 70.00th=[ 1703], 80.00th=[ 1703], 90.00th=[ 1703], 95.00th=[ 1703], | 99.00th=[ 1703], 99.50th=[ 1703], 99.90th=[ 1703], 99.95th=[ 1703], | 99.99th=[ 1703] bw ( KiB/s): min= 8110, max= 8110, per=98.64%, avg=8110.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=30.78%, ctx=783, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187420: Tue Aug 5 11:43:34 2025 write: IOPS=0, BW=3084KiB/s (3158kB/s)(12.0MiB/3985msec); 0 zone resets clat (msec): min=1137, max=1578, avg=1312.26, stdev=234.56 lat (msec): min=1137, max=1579, avg=1312.64, stdev=234.66 clat percentiles (msec): | 1.00th=[ 1133], 5.00th=[ 1133], 10.00th=[ 1133], 20.00th=[ 1133], | 30.00th=[ 1133], 40.00th=[ 1217], 50.00th=[ 1217], 60.00th=[ 1217], | 70.00th=[ 1586], 80.00th=[ 1586], 90.00th=[ 1586], 95.00th=[ 1586], | 99.00th=[ 1586], 99.50th=[ 1586], 99.90th=[ 1586], 99.95th=[ 1586], | 99.99th=[ 1586] bw ( KiB/s): min= 8110, max= 8192, per=99.14%, avg=8151.00, stdev=57.98, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 2000=100.00% cpu : usr=0.00%, sys=36.27%, ctx=38, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187421: Tue Aug 5 11:43:34 2025 write: IOPS=0, BW=3780KiB/s (3870kB/s)(12.0MiB/3251msec); 0 zone resets clat (msec): min=431, max=1507, avg=1063.41, stdev=562.09 lat (msec): min=431, max=1508, avg=1063.90, stdev=562.33 clat percentiles (msec): | 1.00th=[ 430], 5.00th=[ 430], 10.00th=[ 430], 20.00th=[ 430], | 30.00th=[ 430], 40.00th=[ 1250], 50.00th=[ 1250], 60.00th=[ 1250], | 70.00th=[ 1502], 80.00th=[ 1502], 90.00th=[ 1502], 95.00th=[ 1502], | 99.00th=[ 1502], 99.50th=[ 1502], 99.90th=[ 1502], 99.95th=[ 1502], | 99.99th=[ 1502] bw ( KiB/s): min=16384, max=16384, per=100.00%, avg=16384.00, stdev= 0.00, samples=1 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.03%, sys=48.06%, ctx=56, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4190KiB/s (4291kB/s), 1048KiB/s-7046KiB/s (1073kB/s-7215kB/s), io=16.0MiB (16.8MB), run=1744-3910msec WRITE: bw=8223KiB/s (8420kB/s), 2095KiB/s-3780KiB/s (2145kB/s-3870kB/s), io=32.0MiB (33.6MB), run=3251-3985msec rand-rw: (groupid=0, jobs=1): err= 0: pid=187423: Tue Aug 5 11:43:35 2025 read: IOPS=0, BW=2766KiB/s (2833kB/s)(12.0MiB/4442msec) clat (msec): min=1068, max=2147, avg=1467.36, stdev=592.00 lat (msec): min=1068, max=2147, avg=1467.37, stdev=592.00 clat percentiles (msec): | 1.00th=[ 1070], 5.00th=[ 1070], 10.00th=[ 1070], 20.00th=[ 1070], | 30.00th=[ 1070], 40.00th=[ 1183], 50.00th=[ 1183], 60.00th=[ 1183], | 70.00th=[ 2165], 80.00th=[ 2165], 90.00th=[ 2165], 95.00th=[ 2165], | 99.00th=[ 2165], 99.50th=[ 2165], 99.90th=[ 2165], 99.95th=[ 2165], | 99.99th=[ 2165] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 2000=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=19.50%, ctx=143, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187424: Tue Aug 5 11:43:35 2025 read: IOPS=0, BW=923KiB/s (946kB/s)(4096KiB/4436msec) clat (nsec): min=2321.7M, max=2321.7M, avg=2321683059.00, stdev= 0.00 lat (nsec): min=2321.7M, max=2321.7M, avg=2321684990.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 2333], 5.00th=[ 2333], 10.00th=[ 2333], 20.00th=[ 2333], | 30.00th=[ 2333], 40.00th=[ 2333], 50.00th=[ 2333], 60.00th=[ 2333], | 70.00th=[ 2333], 80.00th=[ 2333], 90.00th=[ 2333], 95.00th=[ 2333], | 99.00th=[ 2333], 99.50th=[ 2333], 99.90th=[ 2333], 99.95th=[ 2333], | 99.99th=[ 2333] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1847KiB/s (1891kB/s)(8192KiB/4436msec); 0 zone resets clat (msec): min=943, max=1144, avg=1044.42, stdev=142.18 lat (msec): min=944, max=1145, avg=1044.74, stdev=142.15 clat percentiles (msec): | 1.00th=[ 944], 5.00th=[ 944], 10.00th=[ 944], 20.00th=[ 944], | 30.00th=[ 944], 40.00th=[ 944], 50.00th=[ 944], 60.00th=[ 1150], | 70.00th=[ 1150], 80.00th=[ 1150], 90.00th=[ 1150], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 1000=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.02%, sys=7.58%, ctx=31, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187425: Tue Aug 5 11:43:35 2025 write: IOPS=0, BW=2692KiB/s (2757kB/s)(12.0MiB/4564msec); 0 zone resets clat (msec): min=833, max=2490, avg=1510.12, stdev=869.08 lat (msec): min=833, max=2490, avg=1510.47, stdev=869.15 clat percentiles (msec): | 1.00th=[ 835], 5.00th=[ 835], 10.00th=[ 835], 20.00th=[ 835], | 30.00th=[ 835], 40.00th=[ 1200], 50.00th=[ 1200], 60.00th=[ 1200], | 70.00th=[ 2500], 80.00th=[ 2500], 90.00th=[ 2500], 95.00th=[ 2500], | 99.00th=[ 2500], 99.50th=[ 2500], 99.90th=[ 2500], 99.95th=[ 2500], | 99.99th=[ 2500] bw ( KiB/s): min= 8175, max= 8192, per=100.00%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 1000=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.02%, sys=5.96%, ctx=62, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=187426: Tue Aug 5 11:43:35 2025 write: IOPS=0, BW=2592KiB/s (2654kB/s)(12.0MiB/4741msec); 0 zone resets clat (msec): min=220, max=3285, avg=1564.46, stdev=1566.63 lat (msec): min=220, max=3285, avg=1564.75, stdev=1566.62 clat percentiles (msec): | 1.00th=[ 222], 5.00th=[ 222], 10.00th=[ 222], 20.00th=[ 222], | 30.00th=[ 222], 40.00th=[ 1183], 50.00th=[ 1183], 60.00th=[ 1183], | 70.00th=[ 3272], 80.00th=[ 3272], 90.00th=[ 3272], 95.00th=[ 3272], | 99.00th=[ 3272], 99.50th=[ 3272], 99.90th=[ 3272], 99.95th=[ 3272], | 99.99th=[ 3272] bw ( KiB/s): min= 8000, max= 8192, per=100.00%, avg=8096.00, stdev=135.76, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=16.75%, ctx=176, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3688KiB/s (3777kB/s), 923KiB/s-2766KiB/s (946kB/s-2833kB/s), io=16.0MiB (16.8MB), run=4436-4442msec WRITE: bw=6912KiB/s (7078kB/s), 1847KiB/s-2692KiB/s (1891kB/s-2757kB/s), io=32.0MiB (33.6MB), run=4436-4741msec