== sanity test 398b: DIO and buffer IO race ============== 20:04:38 (1753747478) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.93859 s, 17.1 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=181691: Mon Jul 28 20:05:40 2025 read: IOPS=29, BW=116KiB/s (119kB/s)(5992KiB/51614msec) clat (usec): min=123, max=821933, avg=18937.24, stdev=23031.64 lat (usec): min=124, max=821934, avg=18938.08, stdev=23031.66 clat percentiles (usec): | 1.00th=[ 347], 5.00th=[ 7504], 10.00th=[ 8979], 20.00th=[ 10814], | 30.00th=[ 12518], 40.00th=[ 14484], 50.00th=[ 16581], 60.00th=[ 18744], | 70.00th=[ 21103], 80.00th=[ 24773], 90.00th=[ 30540], 95.00th=[ 35914], | 99.00th=[ 54789], 99.50th=[ 66847], 99.90th=[104334], 99.95th=[817890], | 99.99th=[817890] bw ( KiB/s): min= 13, max= 199, per=25.18%, avg=115.59, stdev=28.98, samples=102 iops : min= 3, max= 49, avg=28.44, stdev= 7.23, samples=102 write: IOPS=30, BW=122KiB/s (125kB/s)(6296KiB/51614msec); 0 zone resets clat (usec): min=668, max=78524, avg=14648.96, stdev=11814.99 lat (usec): min=669, max=78526, avg=14650.07, stdev=11814.97 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 2057], 10.00th=[ 2606], 20.00th=[ 3425], | 30.00th=[ 6259], 40.00th=[ 9372], 50.00th=[11863], 60.00th=[14877], | 70.00th=[18744], 80.00th=[23725], 90.00th=[31589], 95.00th=[38011], | 99.00th=[51119], 99.50th=[56361], 99.90th=[66323], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 39, max= 232, per=25.54%, avg=122.61, stdev=30.53, samples=101 iops : min= 9, max= 58, avg=30.20, stdev= 7.71, samples=101 lat (usec) : 250=0.16%, 500=0.62%, 750=0.16%, 1000=0.42% lat (msec) : 2=1.92%, 4=9.51%, 10=16.34%, 20=40.04%, 50=29.52% lat (msec) : 100=1.24%, 250=0.03%, 1000=0.03% cpu : usr=0.11%, sys=23.12%, ctx=7596, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181692: Mon Jul 28 20:05:40 2025 read: IOPS=28, BW=115KiB/s (117kB/s)(5940KiB/51770msec) clat (usec): min=125, max=830035, avg=19321.10, stdev=23417.61 lat (usec): min=125, max=830037, avg=19321.96, stdev=23417.66 clat percentiles (usec): | 1.00th=[ 302], 5.00th=[ 7439], 10.00th=[ 8717], 20.00th=[ 10814], | 30.00th=[ 12649], 40.00th=[ 14353], 50.00th=[ 16909], 60.00th=[ 19530], | 70.00th=[ 22152], 80.00th=[ 25560], 90.00th=[ 31589], 95.00th=[ 38536], | 99.00th=[ 54789], 99.50th=[ 58459], 99.90th=[ 90702], 99.95th=[826278], | 99.99th=[826278] bw ( KiB/s): min= 7, max= 192, per=25.11%, avg=115.24, stdev=28.66, samples=101 iops : min= 1, max= 48, avg=28.36, stdev= 7.17, samples=101 write: IOPS=30, BW=123KiB/s (126kB/s)(6348KiB/51770msec); 0 zone resets clat (usec): min=671, max=101562, avg=14417.45, stdev=12001.21 lat (usec): min=672, max=101563, avg=14418.49, stdev=12001.22 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 2024], 10.00th=[ 2606], 20.00th=[ 3359], | 30.00th=[ 5735], 40.00th=[ 9241], 50.00th=[ 11469], 60.00th=[ 14484], | 70.00th=[ 18482], 80.00th=[ 23200], 90.00th=[ 30802], 95.00th=[ 37487], | 99.00th=[ 51643], 99.50th=[ 56886], 99.90th=[ 92799], 99.95th=[101188], | 99.99th=[101188] bw ( KiB/s): min= 23, max= 239, per=25.41%, avg=121.96, stdev=38.69, samples=101 iops : min= 5, max= 59, avg=30.04, stdev= 9.72, samples=101 lat (usec) : 250=0.23%, 500=0.72%, 750=0.23%, 1000=0.33% lat (msec) : 2=2.18%, 4=9.70%, 10=16.60%, 20=38.25%, 50=30.40% lat (msec) : 100=1.30%, 250=0.03%, 1000=0.03% cpu : usr=0.14%, sys=22.89%, ctx=7446, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181693: Mon Jul 28 20:05:40 2025 read: IOPS=29, BW=117KiB/s (120kB/s)(6028KiB/51335msec) clat (usec): min=119, max=100211, avg=18039.93, stdev=9685.53 lat (usec): min=119, max=100212, avg=18040.90, stdev=9685.57 clat percentiles (usec): | 1.00th=[ 334], 5.00th=[ 6849], 10.00th=[ 8586], 20.00th=[ 10421], | 30.00th=[ 12256], 40.00th=[ 14222], 50.00th=[ 16319], 60.00th=[ 18482], | 70.00th=[ 21103], 80.00th=[ 24773], 90.00th=[ 30016], 95.00th=[ 36439], | 99.00th=[ 47973], 99.50th=[ 54264], 99.90th=[ 70779], 99.95th=[100140], | 99.99th=[100140] bw ( KiB/s): min= 31, max= 192, per=25.84%, avg=118.58, stdev=29.70, samples=101 iops : min= 7, max= 48, avg=29.19, stdev= 7.42, samples=101 write: IOPS=30, BW=122KiB/s (125kB/s)(6260KiB/51335msec); 0 zone resets clat (usec): min=741, max=838831, avg=15315.12, stdev=24266.82 lat (usec): min=742, max=838832, avg=15316.44, stdev=24266.78 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 1991], 10.00th=[ 2507], 20.00th=[ 3130], | 30.00th=[ 5145], 40.00th=[ 9372], 50.00th=[ 11994], 60.00th=[ 15139], | 70.00th=[ 19268], 80.00th=[ 24511], 90.00th=[ 31851], 95.00th=[ 38536], | 99.00th=[ 53740], 99.50th=[ 70779], 99.90th=[ 92799], 99.95th=[834667], | 99.99th=[834667] bw ( KiB/s): min= 31, max= 183, per=25.66%, avg=123.16, stdev=30.69, samples=101 iops : min= 7, max= 45, avg=30.34, stdev= 7.70, samples=101 lat (usec) : 250=0.20%, 500=0.98%, 750=0.26%, 1000=0.39% lat (msec) : 2=2.28%, 4=10.25%, 10=15.79%, 20=38.38%, 50=30.44% lat (msec) : 100=0.98%, 250=0.03%, 1000=0.03% cpu : usr=0.12%, sys=22.83%, ctx=7553, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181694: Mon Jul 28 20:05:40 2025 read: IOPS=28, BW=116KiB/s (119kB/s)(6052KiB/52297msec) clat (usec): min=128, max=79814, avg=18550.26, stdev=10258.03 lat (usec): min=128, max=79815, avg=18556.98, stdev=10268.42 clat percentiles (usec): | 1.00th=[ 302], 5.00th=[ 7308], 10.00th=[ 8586], 20.00th=[10814], | 30.00th=[12518], 40.00th=[14615], 50.00th=[16712], 60.00th=[19006], | 70.00th=[21365], 80.00th=[25297], 90.00th=[30802], 95.00th=[36963], | 99.00th=[55837], 99.50th=[64750], 99.90th=[69731], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 23, max= 200, per=25.45%, avg=116.83, stdev=29.75, samples=103 iops : min= 5, max= 50, avg=28.76, stdev= 7.45, samples=103 write: IOPS=29, BW=119KiB/s (122kB/s)(6236KiB/52297msec); 0 zone resets clat (usec): min=642, max=837839, avg=15417.65, stdev=24325.29 lat (usec): min=642, max=837840, avg=15418.91, stdev=24325.28 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 2008], 10.00th=[ 2540], 20.00th=[ 3556], | 30.00th=[ 6325], 40.00th=[ 9896], 50.00th=[ 12387], 60.00th=[ 15008], | 70.00th=[ 18482], 80.00th=[ 23462], 90.00th=[ 31065], 95.00th=[ 39060], | 99.00th=[ 51119], 99.50th=[ 60556], 99.90th=[170918], 99.95th=[834667], | 99.99th=[834667] bw ( KiB/s): min= 31, max= 224, per=25.08%, avg=120.38, stdev=34.21, samples=103 iops : min= 7, max= 56, avg=29.65, stdev= 8.58, samples=103 lat (usec) : 250=0.26%, 500=1.01%, 750=0.13%, 1000=0.26% lat (msec) : 2=2.18%, 4=8.63%, 10=15.92%, 20=40.53%, 50=29.65% lat (msec) : 100=1.37%, 250=0.03%, 1000=0.03% cpu : usr=0.11%, sys=22.77%, ctx=7685, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=459KiB/s (470kB/s), 115KiB/s-117KiB/s (117kB/s-120kB/s), io=23.4MiB (24.6MB), run=51335-52297msec WRITE: bw=481KiB/s (492kB/s), 119KiB/s-123KiB/s (122kB/s-126kB/s), io=24.6MiB (25.7MB), run=51335-52297msec rand-rw: (groupid=0, jobs=1): err= 0: pid=181686: Mon Jul 28 20:05:52 2025 read: IOPS=23, BW=93.3KiB/s (95.6kB/s)(5992KiB/64215msec) clat (msec): min=3, max=826, avg=18.03, stdev=24.89 lat (msec): min=3, max=826, avg=18.03, stdev=24.89 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 14], 60.00th=[ 16], | 70.00th=[ 20], 80.00th=[ 26], 90.00th=[ 39], 95.00th=[ 47], | 99.00th=[ 66], 99.50th=[ 71], 99.90th=[ 93], 99.95th=[ 827], | 99.99th=[ 827] bw ( KiB/s): min= 23, max= 296, per=25.05%, avg=93.43, stdev=55.63, samples=126 iops : min= 5, max= 74, avg=23.00, stdev=14.02, samples=126 write: IOPS=24, BW=98.0KiB/s (100kB/s)(6296KiB/64215msec); 0 zone resets clat (msec): min=4, max=200, avg=23.50, stdev=16.11 lat (msec): min=4, max=200, avg=23.50, stdev=16.11 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 19], 60.00th=[ 22], | 70.00th=[ 28], 80.00th=[ 36], 90.00th=[ 46], 95.00th=[ 54], | 99.00th=[ 74], 99.50th=[ 80], 99.90th=[ 107], 99.95th=[ 201], | 99.99th=[ 201] bw ( KiB/s): min= 8, max= 278, per=24.98%, avg=97.67, stdev=54.46, samples=126 iops : min= 2, max= 69, avg=24.06, stdev=13.70, samples=126 lat (msec) : 4=0.46%, 10=25.68%, 20=36.46%, 50=32.06%, 100=5.21% lat (msec) : 250=0.10%, 1000=0.03% cpu : usr=0.14%, sys=10.91%, ctx=4159, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181687: Mon Jul 28 20:05:52 2025 read: IOPS=23, BW=92.8KiB/s (95.1kB/s)(5940KiB/63976msec) clat (msec): min=3, max=825, avg=17.95, stdev=24.86 lat (msec): min=3, max=825, avg=17.95, stdev=24.86 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 13], 60.00th=[ 16], | 70.00th=[ 20], 80.00th=[ 26], 90.00th=[ 37], 95.00th=[ 47], | 99.00th=[ 62], 99.50th=[ 71], 99.90th=[ 100], 99.95th=[ 827], | 99.99th=[ 827] bw ( KiB/s): min= 8, max= 318, per=25.01%, avg=93.29, stdev=56.54, samples=126 iops : min= 2, max= 79, avg=22.97, stdev=14.21, samples=126 write: IOPS=24, BW=99.2KiB/s (102kB/s)(6348KiB/63976msec); 0 zone resets clat (msec): min=4, max=104, avg=23.38, stdev=15.52 lat (msec): min=4, max=104, avg=23.38, stdev=15.52 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 19], 60.00th=[ 23], | 70.00th=[ 29], 80.00th=[ 35], 90.00th=[ 46], 95.00th=[ 54], | 99.00th=[ 73], 99.50th=[ 85], 99.90th=[ 102], 99.95th=[ 106], | 99.99th=[ 106] bw ( KiB/s): min= 24, max= 296, per=25.45%, avg=99.52, stdev=55.11, samples=126 iops : min= 6, max= 74, avg=24.52, stdev=13.87, samples=126 lat (msec) : 4=0.33%, 10=25.75%, 20=36.39%, 50=32.06%, 100=5.37% lat (msec) : 250=0.07%, 1000=0.03% cpu : usr=0.11%, sys=10.93%, ctx=4231, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181689: Mon Jul 28 20:05:52 2025 read: IOPS=23, BW=94.5KiB/s (96.8kB/s)(6028KiB/63793msec) clat (msec): min=3, max=100, avg=17.90, stdev=13.76 lat (msec): min=3, max=100, avg=17.90, stdev=13.76 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 21], 80.00th=[ 27], 90.00th=[ 37], 95.00th=[ 47], | 99.00th=[ 67], 99.50th=[ 75], 99.90th=[ 100], 99.95th=[ 101], | 99.99th=[ 101] bw ( KiB/s): min= 15, max= 336, per=25.52%, avg=95.21, stdev=54.66, samples=126 iops : min= 3, max= 84, avg=23.44, stdev=13.78, samples=126 write: IOPS=24, BW=98.1KiB/s (100kB/s)(6260KiB/63793msec); 0 zone resets clat (msec): min=4, max=832, avg=23.39, stdev=25.48 lat (msec): min=4, max=832, avg=23.39, stdev=25.48 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 19], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 34], 90.00th=[ 43], 95.00th=[ 51], | 99.00th=[ 74], 99.50th=[ 86], 99.90th=[ 167], 99.95th=[ 835], | 99.99th=[ 835] bw ( KiB/s): min= 24, max= 295, per=25.26%, avg=98.75, stdev=55.30, samples=126 iops : min= 6, max= 73, avg=24.33, stdev=13.89, samples=126 lat (msec) : 4=0.29%, 10=24.90%, 20=37.01%, 50=33.30%, 100=4.30% lat (msec) : 250=0.16%, 1000=0.03% cpu : usr=0.12%, sys=11.19%, ctx=4173, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181690: Mon Jul 28 20:05:52 2025 read: IOPS=23, BW=94.5KiB/s (96.8kB/s)(6052KiB/64046msec) clat (msec): min=3, max=103, avg=17.85, stdev=14.31 lat (msec): min=3, max=103, avg=17.85, stdev=14.31 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 16], | 70.00th=[ 20], 80.00th=[ 27], 90.00th=[ 40], 95.00th=[ 48], | 99.00th=[ 69], 99.50th=[ 77], 99.90th=[ 90], 99.95th=[ 104], | 99.99th=[ 104] bw ( KiB/s): min= 8, max= 304, per=25.39%, avg=94.70, stdev=55.04, samples=126 iops : min= 2, max= 76, avg=23.32, stdev=13.85, samples=126 write: IOPS=24, BW=97.4KiB/s (99.7kB/s)(6236KiB/64046msec); 0 zone resets clat (msec): min=4, max=834, avg=23.63, stdev=25.86 lat (msec): min=4, max=834, avg=23.63, stdev=25.86 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 16], 50.00th=[ 18], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 34], 90.00th=[ 44], 95.00th=[ 53], | 99.00th=[ 77], 99.50th=[ 96], 99.90th=[ 201], 99.95th=[ 835], | 99.99th=[ 835] bw ( KiB/s): min= 24, max= 270, per=24.98%, avg=97.67, stdev=54.35, samples=126 iops : min= 6, max= 67, avg=24.06, stdev=13.65, samples=126 lat (msec) : 4=0.52%, 10=25.29%, 20=37.92%, 50=30.83%, 100=5.21% lat (msec) : 250=0.20%, 1000=0.03% cpu : usr=0.10%, sys=11.15%, ctx=4155, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=374KiB/s (383kB/s), 92.8KiB/s-94.5KiB/s (95.1kB/s-96.8kB/s), io=23.4MiB (24.6MB), run=63793-64215msec WRITE: bw=391KiB/s (401kB/s), 97.4KiB/s-99.2KiB/s (99.7kB/s-102kB/s), io=24.6MiB (25.7MB), run=63793-64215msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=181718: Mon Jul 28 20:06:12 2025 read: IOPS=26, BW=419KiB/s (429kB/s)(5920KiB/14137msec) clat (usec): min=115, max=68896, avg=18950.03, stdev=9978.60 lat (usec): min=116, max=68897, avg=18950.99, stdev=9978.62 clat percentiles (usec): | 1.00th=[ 318], 5.00th=[ 6521], 10.00th=[ 9503], 20.00th=[11600], | 30.00th=[13566], 40.00th=[15533], 50.00th=[17433], 60.00th=[19530], | 70.00th=[21890], 80.00th=[24511], 90.00th=[31327], 95.00th=[38011], | 99.00th=[52167], 99.50th=[58459], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 254, max= 702, per=24.16%, avg=409.22, stdev=110.43, samples=27 iops : min= 15, max= 43, avg=25.15, stdev= 6.91, samples=27 write: IOPS=28, BW=450KiB/s (461kB/s)(6368KiB/14137msec); 0 zone resets clat (usec): min=1588, max=84178, avg=17809.04, stdev=13455.08 lat (usec): min=1589, max=84180, avg=17810.89, stdev=13455.08 clat percentiles (usec): | 1.00th=[ 2704], 5.00th=[ 4047], 10.00th=[ 4621], 20.00th=[ 7111], | 30.00th=[ 9241], 40.00th=[11600], 50.00th=[14091], 60.00th=[17171], | 70.00th=[20841], 80.00th=[27919], 90.00th=[35390], 95.00th=[44303], | 99.00th=[70779], 99.50th=[83362], 99.90th=[84411], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 191, max= 638, per=25.21%, avg=449.19, stdev=100.36, samples=27 iops : min= 11, max= 39, avg=27.67, stdev= 6.38, samples=27 lat (usec) : 250=0.26%, 500=1.17%, 750=0.52%, 1000=0.26% lat (msec) : 2=0.13%, 4=2.08%, 10=18.36%, 20=43.23%, 50=31.77% lat (msec) : 100=2.21% cpu : usr=0.11%, sys=27.41%, ctx=2405, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181719: Mon Jul 28 20:06:12 2025 read: IOPS=25, BW=408KiB/s (417kB/s)(5744KiB/14089msec) clat (usec): min=176, max=64839, avg=20049.71, stdev=9879.38 lat (usec): min=177, max=64840, avg=20050.87, stdev=9879.57 clat percentiles (usec): | 1.00th=[ 359], 5.00th=[ 898], 10.00th=[ 9634], 20.00th=[12649], | 30.00th=[14877], 40.00th=[17433], 50.00th=[19530], 60.00th=[21103], | 70.00th=[23462], 80.00th=[26870], 90.00th=[31327], 95.00th=[39060], | 99.00th=[53216], 99.50th=[57410], 99.90th=[64750], 99.95th=[64750], | 99.99th=[64750] bw ( KiB/s): min= 191, max= 574, per=24.17%, avg=409.48, stdev=99.38, samples=27 iops : min= 11, max= 35, avg=25.19, stdev= 6.25, samples=27 write: IOPS=29, BW=464KiB/s (476kB/s)(6544KiB/14089msec); 0 zone resets clat (usec): min=1428, max=70349, avg=16685.73, stdev=11697.59 lat (usec): min=1430, max=70350, avg=16687.49, stdev=11697.68 clat percentiles (usec): | 1.00th=[ 2507], 5.00th=[ 3523], 10.00th=[ 4228], 20.00th=[ 5997], | 30.00th=[ 8160], 40.00th=[11338], 50.00th=[13698], 60.00th=[17433], | 70.00th=[21365], 80.00th=[25822], 90.00th=[33162], 95.00th=[40109], | 99.00th=[50070], 99.50th=[54789], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 319, max= 608, per=25.73%, avg=458.56, stdev=78.29, samples=27 iops : min= 19, max= 38, avg=28.26, stdev= 5.04, samples=27 lat (usec) : 250=0.13%, 500=1.43%, 750=0.52%, 1000=0.26% lat (msec) : 2=0.52%, 4=3.91%, 10=17.19%, 20=36.20%, 50=38.67% lat (msec) : 100=1.17% cpu : usr=0.13%, sys=27.68%, ctx=2449, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181720: Mon Jul 28 20:06:12 2025 read: IOPS=28, BW=449KiB/s (460kB/s)(6256KiB/13934msec) clat (usec): min=281, max=70620, avg=19250.75, stdev=9898.36 lat (usec): min=283, max=70621, avg=19251.79, stdev=9898.35 clat percentiles (usec): | 1.00th=[ 355], 5.00th=[ 553], 10.00th=[10028], 20.00th=[12387], | 30.00th=[14353], 40.00th=[16319], 50.00th=[17957], 60.00th=[19792], | 70.00th=[22414], 80.00th=[25560], 90.00th=[30540], 95.00th=[35390], | 99.00th=[52167], 99.50th=[66847], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 318, max= 640, per=26.91%, avg=455.81, stdev=89.57, samples=27 iops : min= 19, max= 40, avg=28.07, stdev= 5.82, samples=27 write: IOPS=27, BW=433KiB/s (443kB/s)(6032KiB/13934msec); 0 zone resets clat (usec): min=1394, max=77459, avg=16826.59, stdev=12145.10 lat (usec): min=1395, max=77461, avg=16828.43, stdev=12145.29 clat percentiles (usec): | 1.00th=[ 1663], 5.00th=[ 4047], 10.00th=[ 4621], 20.00th=[ 6063], | 30.00th=[ 9372], 40.00th=[11338], 50.00th=[13435], 60.00th=[16581], | 70.00th=[20841], 80.00th=[25297], 90.00th=[32637], 95.00th=[39060], | 99.00th=[60031], 99.50th=[77071], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 224, max= 608, per=24.19%, avg=431.04, stdev=112.65, samples=27 iops : min= 14, max= 38, avg=26.52, stdev= 7.09, samples=27 lat (usec) : 500=2.21%, 750=0.52%, 1000=0.13% lat (msec) : 2=0.65%, 4=1.69%, 10=17.32%, 20=42.06%, 50=34.11% lat (msec) : 100=1.30% cpu : usr=0.07%, sys=27.22%, ctx=2562, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181721: Mon Jul 28 20:06:12 2025 read: IOPS=26, BW=430KiB/s (440kB/s)(6032KiB/14043msec) clat (usec): min=128, max=64495, avg=19614.90, stdev=10724.12 lat (usec): min=129, max=64495, avg=19615.87, stdev=10724.15 clat percentiles (usec): | 1.00th=[ 285], 5.00th=[ 400], 10.00th=[ 8160], 20.00th=[11994], | 30.00th=[14091], 40.00th=[16057], 50.00th=[18482], 60.00th=[21365], | 70.00th=[23725], 80.00th=[27657], 90.00th=[32900], 95.00th=[38536], | 99.00th=[51643], 99.50th=[56361], 99.90th=[64750], 99.95th=[64750], | 99.99th=[64750] bw ( KiB/s): min= 286, max= 576, per=25.03%, avg=423.96, stdev=89.20, samples=27 iops : min= 17, max= 36, avg=26.07, stdev= 5.52, samples=27 write: IOPS=27, BW=445KiB/s (456kB/s)(6256KiB/14043msec); 0 zone resets clat (usec): min=1702, max=74725, avg=16870.12, stdev=12012.61 lat (usec): min=1704, max=74727, avg=16872.80, stdev=12011.94 clat percentiles (usec): | 1.00th=[ 2278], 5.00th=[ 3720], 10.00th=[ 4293], 20.00th=[ 5800], | 30.00th=[ 7832], 40.00th=[11338], 50.00th=[14746], 60.00th=[17433], | 70.00th=[21365], 80.00th=[25822], 90.00th=[33817], 95.00th=[39584], | 99.00th=[53216], 99.50th=[61604], 99.90th=[74974], 99.95th=[74974], | 99.99th=[74974] bw ( KiB/s): min= 223, max= 734, per=25.00%, avg=445.48, stdev=121.57, samples=27 iops : min= 13, max= 45, avg=27.44, stdev= 7.67, samples=27 lat (usec) : 250=0.13%, 500=2.73%, 750=0.91%, 1000=0.13% lat (msec) : 2=0.52%, 4=3.52%, 10=17.19%, 20=36.72%, 50=36.72% lat (msec) : 100=1.43% cpu : usr=0.10%, sys=26.99%, ctx=2466, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1694KiB/s (1735kB/s), 408KiB/s-449KiB/s (417kB/s-460kB/s), io=23.4MiB (24.5MB), run=13934-14137msec WRITE: bw=1783KiB/s (1825kB/s), 433KiB/s-464KiB/s (443kB/s-476kB/s), io=24.6MiB (25.8MB), run=13934-14137msec rand-rw: (groupid=0, jobs=1): err= 0: pid=181723: Mon Jul 28 20:06:15 2025 read: IOPS=22, BW=352KiB/s (361kB/s)(5920KiB/16802msec) clat (usec): min=4223, max=79469, avg=18110.69, stdev=13538.21 lat (usec): min=4225, max=79470, avg=18111.63, stdev=13538.28 clat percentiles (usec): | 1.00th=[ 4555], 5.00th=[ 5735], 10.00th=[ 6521], 20.00th=[ 7898], | 30.00th=[ 9372], 40.00th=[11338], 50.00th=[13042], 60.00th=[15795], | 70.00th=[19530], 80.00th=[25822], 90.00th=[39060], 95.00th=[47449], | 99.00th=[60556], 99.50th=[74974], 99.90th=[79168], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 160, max= 1024, per=25.03%, avg=351.21, stdev=191.41, samples=33 iops : min= 10, max= 64, avg=21.58, stdev=12.07, samples=33 write: IOPS=23, BW=379KiB/s (388kB/s)(6368KiB/16802msec); 0 zone resets clat (usec): min=6195, max=83414, avg=25301.79, stdev=14761.29 lat (usec): min=6196, max=83417, avg=25303.78, stdev=14761.31 clat percentiles (usec): | 1.00th=[ 6849], 5.00th=[ 8979], 10.00th=[10421], 20.00th=[12518], | 30.00th=[15270], 40.00th=[17957], 50.00th=[21365], 60.00th=[25297], | 70.00th=[30540], 80.00th=[37487], 90.00th=[45876], 95.00th=[54789], | 99.00th=[70779], 99.50th=[72877], 99.90th=[83362], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 159, max= 992, per=25.69%, avg=379.48, stdev=184.98, samples=33 iops : min= 9, max= 62, avg=23.36, stdev=11.68, samples=33 lat (msec) : 10=19.79%, 20=38.41%, 50=36.07%, 100=5.73% cpu : usr=0.10%, sys=12.18%, ctx=1084, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181724: Mon Jul 28 20:06:15 2025 read: IOPS=21, BW=342KiB/s (351kB/s)(5744KiB/16774msec) clat (usec): min=4148, max=85032, avg=18063.76, stdev=14097.56 lat (usec): min=4149, max=85033, avg=18064.67, stdev=14097.57 clat percentiles (usec): | 1.00th=[ 4490], 5.00th=[ 5342], 10.00th=[ 6259], 20.00th=[ 7832], | 30.00th=[ 9634], 40.00th=[11600], 50.00th=[13042], 60.00th=[15270], | 70.00th=[19006], 80.00th=[25822], 90.00th=[38536], 95.00th=[46924], | 99.00th=[69731], 99.50th=[79168], 99.90th=[85459], 99.95th=[85459], | 99.99th=[85459] bw ( KiB/s): min= 127, max= 896, per=24.38%, avg=342.09, stdev=170.73, samples=33 iops : min= 7, max= 56, avg=21.00, stdev=10.79, samples=33 write: IOPS=24, BW=390KiB/s (399kB/s)(6544KiB/16774msec); 0 zone resets clat (msec): min=6, max=101, avg=24.97, stdev=15.73 lat (msec): min=6, max=101, avg=24.97, stdev=15.73 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 9], 10.00th=[ 11], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 25], | 70.00th=[ 31], 80.00th=[ 37], 90.00th=[ 46], 95.00th=[ 56], | 99.00th=[ 81], 99.50th=[ 84], 99.90th=[ 103], 99.95th=[ 103], | 99.99th=[ 103] bw ( KiB/s): min= 159, max= 1056, per=26.45%, avg=390.73, stdev=205.36, samples=33 iops : min= 9, max= 66, avg=24.06, stdev=12.94, samples=33 lat (msec) : 10=20.05%, 20=40.62%, 50=33.85%, 100=5.34%, 250=0.13% cpu : usr=0.10%, sys=11.94%, ctx=1048, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181725: Mon Jul 28 20:06:15 2025 read: IOPS=22, BW=367KiB/s (375kB/s)(6256KiB/17061msec) clat (usec): min=3687, max=97231, avg=18398.92, stdev=14532.47 lat (usec): min=3688, max=97233, avg=18399.88, stdev=14532.48 clat percentiles (usec): | 1.00th=[ 4178], 5.00th=[ 5473], 10.00th=[ 6259], 20.00th=[ 7832], | 30.00th=[ 9372], 40.00th=[11076], 50.00th=[13304], 60.00th=[15401], | 70.00th=[21103], 80.00th=[27132], 90.00th=[39060], 95.00th=[49021], | 99.00th=[74974], 99.50th=[79168], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 128, max= 1085, per=26.01%, avg=364.91, stdev=223.36, samples=33 iops : min= 8, max= 67, avg=22.42, stdev=14.03, samples=33 write: IOPS=22, BW=354KiB/s (362kB/s)(6032KiB/17061msec); 0 zone resets clat (usec): min=6666, max=87514, avg=25939.41, stdev=16611.15 lat (usec): min=6667, max=87516, avg=25941.49, stdev=16611.12 clat percentiles (usec): | 1.00th=[ 6915], 5.00th=[ 8356], 10.00th=[ 9503], 20.00th=[11338], | 30.00th=[14222], 40.00th=[17171], 50.00th=[21890], 60.00th=[25560], | 70.00th=[30802], 80.00th=[38536], 90.00th=[50070], 95.00th=[61604], | 99.00th=[77071], 99.50th=[83362], 99.90th=[87557], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 127, max= 992, per=23.20%, avg=342.73, stdev=192.48, samples=33 iops : min= 7, max= 62, avg=21.06, stdev=12.18, samples=33 lat (msec) : 4=0.26%, 10=23.18%, 20=34.24%, 50=35.42%, 100=6.90% cpu : usr=0.08%, sys=11.57%, ctx=1052, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181726: Mon Jul 28 20:06:15 2025 read: IOPS=22, BW=355KiB/s (364kB/s)(6032KiB/16986msec) clat (usec): min=3337, max=77040, avg=18126.18, stdev=13918.45 lat (usec): min=3338, max=77041, avg=18127.13, stdev=13918.43 clat percentiles (usec): | 1.00th=[ 4178], 5.00th=[ 5669], 10.00th=[ 6587], 20.00th=[ 7767], | 30.00th=[ 9372], 40.00th=[11076], 50.00th=[12649], 60.00th=[15664], | 70.00th=[20317], 80.00th=[27657], 90.00th=[36963], 95.00th=[50594], | 99.00th=[69731], 99.50th=[72877], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 95, max= 1152, per=24.82%, avg=348.27, stdev=198.91, samples=33 iops : min= 5, max= 72, avg=21.39, stdev=12.59, samples=33 write: IOPS=23, BW=368KiB/s (377kB/s)(6256KiB/16986msec); 0 zone resets clat (usec): min=5507, max=87472, avg=25633.43, stdev=16389.91 lat (usec): min=5509, max=87474, avg=25635.54, stdev=16389.93 clat percentiles (usec): | 1.00th=[ 6849], 5.00th=[ 8717], 10.00th=[10159], 20.00th=[11863], | 30.00th=[14615], 40.00th=[16319], 50.00th=[20841], 60.00th=[24511], | 70.00th=[29230], 80.00th=[37487], 90.00th=[50070], 95.00th=[58983], | 99.00th=[76022], 99.50th=[78119], 99.90th=[87557], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 191, max= 1024, per=24.72%, avg=365.18, stdev=202.79, samples=33 iops : min= 11, max= 64, avg=22.45, stdev=12.75, samples=33 lat (msec) : 4=0.39%, 10=21.09%, 20=37.37%, 50=33.33%, 100=7.81% cpu : usr=0.11%, sys=11.75%, ctx=1095, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1404KiB/s (1438kB/s), 342KiB/s-367KiB/s (351kB/s-375kB/s), io=23.4MiB (24.5MB), run=16774-17061msec WRITE: bw=1477KiB/s (1513kB/s), 354KiB/s-390KiB/s (362kB/s-399kB/s), io=24.6MiB (25.8MB), run=16774-17061msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=181733: Mon Jul 28 20:06:24 2025 read: IOPS=2, BW=2855KiB/s (2923kB/s)(7168KiB/2511msec) clat (msec): min=53, max=469, avg=179.75, stdev=163.74 lat (msec): min=53, max=469, avg=179.75, stdev=163.74 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 54], 10.00th=[ 54], 20.00th=[ 68], | 30.00th=[ 77], 40.00th=[ 77], 50.00th=[ 84], 60.00th=[ 161], | 70.00th=[ 161], 80.00th=[ 347], 90.00th=[ 468], 95.00th=[ 468], | 99.00th=[ 468], 99.50th=[ 468], 99.90th=[ 468], 99.95th=[ 468], | 99.99th=[ 468] bw ( KiB/s): min= 2043, max= 4954, per=45.27%, avg=3015.00, stdev=1679.23, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 write: IOPS=1, BW=2039KiB/s (2088kB/s)(5120KiB/2511msec); 0 zone resets clat (msec): min=79, max=355, avg=247.61, stdev=111.37 lat (msec): min=79, max=355, avg=247.72, stdev=111.37 clat percentiles (msec): | 1.00th=[ 80], 5.00th=[ 80], 10.00th=[ 80], 20.00th=[ 80], | 30.00th=[ 218], 40.00th=[ 218], 50.00th=[ 245], 60.00th=[ 245], | 70.00th=[ 342], 80.00th=[ 342], 90.00th=[ 355], 95.00th=[ 355], | 99.00th=[ 355], 99.50th=[ 355], 99.90th=[ 355], 99.95th=[ 355], | 99.99th=[ 355] bw ( KiB/s): min= 1651, max= 6071, per=41.41%, avg=3861.00, stdev=3125.41, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 lat (msec) : 100=41.67%, 250=25.00%, 500=33.33% cpu : usr=0.04%, sys=16.93%, ctx=116, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181734: Mon Jul 28 20:06:24 2025 read: IOPS=0, BW=684KiB/s (701kB/s)(2048KiB/2993msec) clat (msec): min=82, max=113, avg=98.00, stdev=21.38 lat (msec): min=82, max=113, avg=98.00, stdev=21.38 clat percentiles (msec): | 1.00th=[ 84], 5.00th=[ 84], 10.00th=[ 84], 20.00th=[ 84], | 30.00th=[ 84], 40.00th=[ 84], 50.00th=[ 84], 60.00th=[ 113], | 70.00th=[ 113], 80.00th=[ 113], 90.00th=[ 113], 95.00th=[ 113], | 99.00th=[ 113], 99.50th=[ 113], 99.90th=[ 113], 99.95th=[ 113], | 99.99th=[ 113] bw ( KiB/s): min= 1646, max= 2048, per=27.73%, avg=1847.00, stdev=284.26, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3421KiB/s (3503kB/s)(10.0MiB/2993msec); 0 zone resets clat (msec): min=71, max=522, avg=271.67, stdev=145.37 lat (msec): min=71, max=522, avg=271.77, stdev=145.39 clat percentiles (msec): | 1.00th=[ 71], 5.00th=[ 71], 10.00th=[ 71], 20.00th=[ 92], | 30.00th=[ 159], 40.00th=[ 239], 50.00th=[ 255], 60.00th=[ 257], | 70.00th=[ 305], 80.00th=[ 405], 90.00th=[ 418], 95.00th=[ 523], | 99.00th=[ 523], 99.50th=[ 523], 99.90th=[ 523], 99.95th=[ 523], | 99.99th=[ 523] bw ( KiB/s): min= 1646, max= 4096, per=29.82%, avg=2780.80, stdev=1210.98, samples=5 iops : min= 1, max= 4, avg= 2.20, stdev= 1.64, samples=5 lat (msec) : 100=25.00%, 250=25.00%, 500=41.67%, 750=8.33% cpu : usr=0.03%, sys=21.62%, ctx=129, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181735: Mon Jul 28 20:06:24 2025 read: IOPS=1, BW=1998KiB/s (2046kB/s)(6144KiB/3075msec) clat (msec): min=110, max=378, avg=224.69, stdev=111.33 lat (msec): min=110, max=378, avg=224.70, stdev=111.33 clat percentiles (msec): | 1.00th=[ 111], 5.00th=[ 111], 10.00th=[ 111], 20.00th=[ 124], | 30.00th=[ 124], 40.00th=[ 174], 50.00th=[ 174], 60.00th=[ 228], | 70.00th=[ 334], 80.00th=[ 334], 90.00th=[ 380], 95.00th=[ 380], | 99.00th=[ 380], 99.50th=[ 380], 99.90th=[ 380], 99.95th=[ 380], | 99.99th=[ 380] bw ( KiB/s): min= 2043, max= 4047, per=40.73%, avg=2712.67, stdev=1155.57, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 write: IOPS=1, BW=1998KiB/s (2046kB/s)(6144KiB/3075msec); 0 zone resets clat (msec): min=63, max=612, avg=280.68, stdev=251.74 lat (msec): min=63, max=612, avg=280.78, stdev=251.74 clat percentiles (msec): | 1.00th=[ 64], 5.00th=[ 64], 10.00th=[ 64], 20.00th=[ 75], | 30.00th=[ 75], 40.00th=[ 95], 50.00th=[ 95], 60.00th=[ 271], | 70.00th=[ 567], 80.00th=[ 567], 90.00th=[ 609], 95.00th=[ 609], | 99.00th=[ 609], 99.50th=[ 609], 99.90th=[ 609], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 2043, max= 3303, per=25.32%, avg=2360.50, stdev=628.34, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (msec) : 100=25.00%, 250=33.33%, 500=25.00%, 750=16.67% cpu : usr=0.00%, sys=17.18%, ctx=87, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181736: Mon Jul 28 20:06:24 2025 read: IOPS=2, BW=2129KiB/s (2180kB/s)(5120KiB/2405msec) clat (msec): min=73, max=256, avg=170.79, stdev=85.69 lat (msec): min=73, max=256, avg=170.79, stdev=85.69 clat percentiles (msec): | 1.00th=[ 74], 5.00th=[ 74], 10.00th=[ 74], 20.00th=[ 74], | 30.00th=[ 89], 40.00th=[ 89], 50.00th=[ 190], 60.00th=[ 190], | 70.00th=[ 245], 80.00th=[ 245], 90.00th=[ 257], 95.00th=[ 257], | 99.00th=[ 257], 99.50th=[ 257], 99.90th=[ 257], 99.95th=[ 257], | 99.99th=[ 257] bw ( KiB/s): min= 4087, max= 4096, per=61.43%, avg=4091.50, stdev= 6.36, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2980KiB/s (3052kB/s)(7168KiB/2405msec); 0 zone resets clat (msec): min=92, max=417, avg=213.13, stdev=111.99 lat (msec): min=92, max=417, avg=213.22, stdev=111.99 clat percentiles (msec): | 1.00th=[ 92], 5.00th=[ 92], 10.00th=[ 92], 20.00th=[ 130], | 30.00th=[ 150], 40.00th=[ 150], 50.00th=[ 174], 60.00th=[ 243], | 70.00th=[ 243], 80.00th=[ 288], 90.00th=[ 418], 95.00th=[ 418], | 99.00th=[ 418], 99.50th=[ 418], 99.90th=[ 418], 99.95th=[ 418], | 99.99th=[ 418] bw ( KiB/s): min= 2023, max= 4923, per=29.59%, avg=2759.25, stdev=1442.54, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 lat (msec) : 100=25.00%, 250=50.00%, 500=25.00% cpu : usr=0.00%, sys=19.47%, ctx=268, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6660KiB/s (6820kB/s), 684KiB/s-2855KiB/s (701kB/s-2923kB/s), io=20.0MiB (20.0MB), run=2405-3075msec WRITE: bw=9324KiB/s (9548kB/s), 1998KiB/s-3421KiB/s (2046kB/s-3503kB/s), io=28.0MiB (29.4MB), run=2405-3075msec rand-rw: (groupid=0, jobs=1): err= 0: pid=181738: Mon Jul 28 20:06:24 2025 read: IOPS=2, BW=2663KiB/s (2727kB/s)(7168KiB/2692msec) clat (usec): min=873, max=383563, avg=156541.07, stdev=124596.36 lat (usec): min=874, max=383564, avg=156542.19, stdev=124596.51 clat percentiles (usec): | 1.00th=[ 873], 5.00th=[ 873], 10.00th=[ 873], 20.00th=[ 71828], | 30.00th=[120062], 40.00th=[120062], 50.00th=[125305], 60.00th=[147850], | 70.00th=[147850], 80.00th=[246416], 90.00th=[383779], 95.00th=[383779], | 99.00th=[383779], 99.50th=[383779], 99.90th=[383779], 99.95th=[383779], | 99.99th=[383779] bw ( KiB/s): min= 2007, max= 5242, per=43.62%, avg=2835.00, stdev=1604.77, samples=4 iops : min= 1, max= 5, avg= 2.25, stdev= 1.89, samples=4 write: IOPS=1, BW=1902KiB/s (1948kB/s)(5120KiB/2692msec); 0 zone resets clat (msec): min=154, max=406, avg=300.50, stdev=110.38 lat (msec): min=155, max=406, avg=300.59, stdev=110.36 clat percentiles (msec): | 1.00th=[ 155], 5.00th=[ 155], 10.00th=[ 155], 20.00th=[ 155], | 30.00th=[ 222], 40.00th=[ 222], 50.00th=[ 321], 60.00th=[ 321], | 70.00th=[ 401], 80.00th=[ 401], 90.00th=[ 405], 95.00th=[ 405], | 99.00th=[ 405], 99.50th=[ 405], 99.90th=[ 405], 99.95th=[ 405], | 99.99th=[ 405] bw ( KiB/s): min= 2043, max= 4096, per=29.99%, avg=2729.00, stdev=1183.86, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 lat (usec) : 1000=8.33% lat (msec) : 100=8.33%, 250=50.00%, 500=33.33% cpu : usr=0.07%, sys=39.84%, ctx=589, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181739: Mon Jul 28 20:06:24 2025 read: IOPS=0, BW=650KiB/s (666kB/s)(2048KiB/3151msec) clat (usec): min=740, max=182088, avg=91414.41, stdev=128232.07 lat (usec): min=741, max=182090, avg=91416.04, stdev=128233.23 clat percentiles (usec): | 1.00th=[ 742], 5.00th=[ 742], 10.00th=[ 742], 20.00th=[ 742], | 30.00th=[ 742], 40.00th=[ 742], 50.00th=[ 742], 60.00th=[181404], | 70.00th=[181404], 80.00th=[181404], 90.00th=[181404], 95.00th=[181404], | 99.00th=[181404], 99.50th=[181404], 99.90th=[181404], 99.95th=[181404], | 99.99th=[181404] bw ( KiB/s): min= 1729, max= 2043, per=29.02%, avg=1886.00, stdev=222.03, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3250KiB/s (3328kB/s)(10.0MiB/3151msec); 0 zone resets clat (msec): min=47, max=705, avg=293.93, stdev=213.46 lat (msec): min=47, max=705, avg=294.03, stdev=213.45 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 48], 10.00th=[ 48], 20.00th=[ 51], | 30.00th=[ 56], 40.00th=[ 215], 50.00th=[ 271], 60.00th=[ 330], | 70.00th=[ 376], 80.00th=[ 405], 90.00th=[ 485], 95.00th=[ 709], | 99.00th=[ 709], 99.50th=[ 709], 99.90th=[ 709], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 1729, max= 8192, per=39.70%, avg=3612.60, stdev=2728.09, samples=5 iops : min= 1, max= 8, avg= 3.00, stdev= 2.92, samples=5 lat (usec) : 750=8.33% lat (msec) : 50=8.33%, 100=16.67%, 250=16.67%, 500=41.67%, 750=8.33% cpu : usr=0.00%, sys=36.92%, ctx=407, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181740: Mon Jul 28 20:06:24 2025 read: IOPS=2, BW=2303KiB/s (2358kB/s)(6144KiB/2668msec) clat (usec): min=937, max=256997, avg=124935.89, stdev=109980.37 lat (usec): min=938, max=256999, avg=124936.97, stdev=109980.34 clat percentiles (usec): | 1.00th=[ 938], 5.00th=[ 938], 10.00th=[ 938], 20.00th=[ 1418], | 30.00th=[ 1418], 40.00th=[109577], 50.00th=[109577], 60.00th=[147850], | 70.00th=[233833], 80.00th=[233833], 90.00th=[256902], 95.00th=[256902], | 99.00th=[256902], 99.50th=[256902], 99.90th=[256902], 99.95th=[256902], | 99.99th=[256902] bw ( KiB/s): min= 2043, max= 4096, per=52.08%, avg=3384.67, stdev=1162.62, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=2, BW=2303KiB/s (2358kB/s)(6144KiB/2668msec); 0 zone resets clat (msec): min=162, max=556, avg=300.65, stdev=177.04 lat (msec): min=162, max=556, avg=300.73, stdev=177.03 clat percentiles (msec): | 1.00th=[ 163], 5.00th=[ 163], 10.00th=[ 163], 20.00th=[ 176], | 30.00th=[ 176], 40.00th=[ 203], 50.00th=[ 203], 60.00th=[ 209], | 70.00th=[ 498], 80.00th=[ 498], 90.00th=[ 558], 95.00th=[ 558], | 99.00th=[ 558], 99.50th=[ 558], 99.90th=[ 558], 99.95th=[ 558], | 99.99th=[ 558] bw ( KiB/s): min= 2043, max= 3537, per=26.59%, avg=2419.00, stdev=745.34, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (usec) : 1000=8.33% lat (msec) : 2=8.33%, 250=58.33%, 500=16.67%, 750=8.33% cpu : usr=0.07%, sys=35.28%, ctx=605, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181741: Mon Jul 28 20:06:24 2025 read: IOPS=2, BW=2060KiB/s (2110kB/s)(5120KiB/2485msec) clat (usec): min=965, max=155473, avg=80473.14, stdev=76215.38 lat (usec): min=966, max=155475, avg=80474.30, stdev=76215.32 clat percentiles (usec): | 1.00th=[ 963], 5.00th=[ 963], 10.00th=[ 963], 20.00th=[ 963], | 30.00th=[ 2311], 40.00th=[ 2311], 50.00th=[ 91751], 60.00th=[ 91751], | 70.00th=[152044], 80.00th=[152044], 90.00th=[156238], 95.00th=[156238], | 99.00th=[156238], 99.50th=[156238], 99.90th=[156238], 99.95th=[156238], | 99.99th=[156238] bw ( KiB/s): min= 4015, max= 4096, per=62.40%, avg=4055.50, stdev=57.28, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2885KiB/s (2954kB/s)(7168KiB/2485msec); 0 zone resets clat (msec): min=121, max=576, avg=278.25, stdev=166.39 lat (msec): min=121, max=576, avg=278.33, stdev=166.37 clat percentiles (msec): | 1.00th=[ 123], 5.00th=[ 123], 10.00th=[ 123], 20.00th=[ 161], | 30.00th=[ 186], 40.00th=[ 186], 50.00th=[ 232], 60.00th=[ 234], | 70.00th=[ 234], 80.00th=[ 439], 90.00th=[ 575], 95.00th=[ 575], | 99.00th=[ 575], 99.50th=[ 575], 99.90th=[ 575], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 1756, max= 4096, per=27.22%, avg=2476.75, stdev=1087.19, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 lat (usec) : 1000=8.33% lat (msec) : 4=8.33%, 100=8.33%, 250=58.33%, 500=8.33%, 750=8.33% cpu : usr=0.00%, sys=41.63%, ctx=382, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6500KiB/s (6656kB/s), 650KiB/s-2663KiB/s (666kB/s-2727kB/s), io=20.0MiB (20.0MB), run=2485-3151msec WRITE: bw=9099KiB/s (9318kB/s), 1902KiB/s-3250KiB/s (1948kB/s-3328kB/s), io=28.0MiB (29.4MB), run=2485-3151msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=181749: Mon Jul 28 20:06:34 2025 read: IOPS=1, BW=7807KiB/s (7994kB/s)(12.0MiB/1574msec) clat (msec): min=121, max=571, avg=402.59, stdev=244.71 lat (msec): min=121, max=571, avg=402.60, stdev=244.71 clat percentiles (msec): | 1.00th=[ 123], 5.00th=[ 123], 10.00th=[ 123], 20.00th=[ 123], | 30.00th=[ 123], 40.00th=[ 514], 50.00th=[ 514], 60.00th=[ 514], | 70.00th=[ 575], 80.00th=[ 575], 90.00th=[ 575], 95.00th=[ 575], | 99.00th=[ 575], 99.50th=[ 575], 99.90th=[ 575], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 8159, max= 8175, per=100.00%, avg=8167.00, stdev=11.31, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 750=66.67% cpu : usr=0.00%, sys=32.74%, ctx=363, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181751: Mon Jul 28 20:06:34 2025 read: IOPS=0, BW=1301KiB/s (1332kB/s)(4096KiB/3149msec) clat (nsec): min=567985k, max=567985k, avg=567984809.00, stdev= 0.00 lat (nsec): min=567987k, max=567987k, avg=567987025.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 567], 5.00th=[ 567], 10.00th=[ 567], 20.00th=[ 567], | 30.00th=[ 567], 40.00th=[ 567], 50.00th=[ 567], 60.00th=[ 567], | 70.00th=[ 567], 80.00th=[ 567], 90.00th=[ 567], 95.00th=[ 567], | 99.00th=[ 567], 99.50th=[ 567], 99.90th=[ 567], 99.95th=[ 567], | 99.99th=[ 567] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2601KiB/s (2664kB/s)(8192KiB/3149msec); 0 zone resets clat (msec): min=1056, max=1133, avg=1094.97, stdev=54.73 lat (msec): min=1056, max=1134, avg=1095.49, stdev=54.72 clat percentiles (msec): | 1.00th=[ 1053], 5.00th=[ 1053], 10.00th=[ 1053], 20.00th=[ 1053], | 30.00th=[ 1053], 40.00th=[ 1053], 50.00th=[ 1053], 60.00th=[ 1133], | 70.00th=[ 1133], 80.00th=[ 1133], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 8192, max= 8192, per=81.18%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=31.67%, ctx=277, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181753: Mon Jul 28 20:06:34 2025 write: IOPS=0, BW=3784KiB/s (3875kB/s)(12.0MiB/3247msec); 0 zone resets clat (msec): min=480, max=1301, avg=958.11, stdev=426.40 lat (msec): min=481, max=1301, avg=958.58, stdev=426.29 clat percentiles (msec): | 1.00th=[ 481], 5.00th=[ 481], 10.00th=[ 481], 20.00th=[ 481], | 30.00th=[ 481], 40.00th=[ 1099], 50.00th=[ 1099], 60.00th=[ 1099], | 70.00th=[ 1301], 80.00th=[ 1301], 90.00th=[ 1301], 95.00th=[ 1301], | 99.00th=[ 1301], 99.50th=[ 1301], 99.90th=[ 1301], 99.95th=[ 1301], | 99.99th=[ 1301] bw ( KiB/s): min= 8159, max= 8175, per=80.93%, avg=8167.00, stdev=11.31, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.00%, sys=43.53%, ctx=104, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181755: Mon Jul 28 20:06:34 2025 write: IOPS=1, BW=4303KiB/s (4406kB/s)(12.0MiB/2856msec); 0 zone resets clat (msec): min=609, max=1075, avg=836.69, stdev=233.55 lat (msec): min=609, max=1076, avg=837.24, stdev=233.67 clat percentiles (msec): | 1.00th=[ 609], 5.00th=[ 609], 10.00th=[ 609], 20.00th=[ 609], | 30.00th=[ 609], 40.00th=[ 827], 50.00th=[ 827], 60.00th=[ 827], | 70.00th=[ 1083], 80.00th=[ 1083], 90.00th=[ 1083], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 8159, max= 8159, per=80.85%, avg=8159.00, stdev= 0.00, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=56.57%, ctx=118, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5203KiB/s (5328kB/s), 1301KiB/s-7807KiB/s (1332kB/s-7994kB/s), io=16.0MiB (16.8MB), run=1574-3149msec WRITE: bw=9.85MiB/s (10.3MB/s), 2601KiB/s-4303KiB/s (2664kB/s-4406kB/s), io=32.0MiB (33.6MB), run=2856-3247msec rand-rw: (groupid=0, jobs=1): err= 0: pid=181748: Mon Jul 28 20:06:35 2025 read: IOPS=1, BW=4779KiB/s (4894kB/s)(12.0MiB/2571msec) clat (msec): min=372, max=1388, avg=798.75, stdev=526.98 lat (msec): min=372, max=1388, avg=798.75, stdev=526.98 clat percentiles (msec): | 1.00th=[ 372], 5.00th=[ 372], 10.00th=[ 372], 20.00th=[ 372], | 30.00th=[ 372], 40.00th=[ 634], 50.00th=[ 634], 60.00th=[ 634], | 70.00th=[ 1385], 80.00th=[ 1385], 90.00th=[ 1385], 95.00th=[ 1385], | 99.00th=[ 1385], 99.50th=[ 1385], 99.90th=[ 1385], 99.95th=[ 1385], | 99.99th=[ 1385] bw ( KiB/s): min= 7098, max= 8159, per=100.00%, avg=7628.50, stdev=750.24, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=22.49%, ctx=82, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181750: Mon Jul 28 20:06:35 2025 read: IOPS=0, BW=1098KiB/s (1124kB/s)(4096KiB/3732msec) clat (nsec): min=1515.3M, max=1515.3M, avg=1515289379.00, stdev= 0.00 lat (nsec): min=1515.3M, max=1515.3M, avg=1515302014.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1519], 5.00th=[ 1519], 10.00th=[ 1519], 20.00th=[ 1519], | 30.00th=[ 1519], 40.00th=[ 1519], 50.00th=[ 1519], 60.00th=[ 1519], | 70.00th=[ 1519], 80.00th=[ 1519], 90.00th=[ 1519], 95.00th=[ 1519], | 99.00th=[ 1519], 99.50th=[ 1519], 99.90th=[ 1519], 99.95th=[ 1519], | 99.99th=[ 1519] bw ( KiB/s): min= 8126, max= 8126, per=100.00%, avg=8126.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2195KiB/s (2248kB/s)(8192KiB/3732msec); 0 zone resets clat (msec): min=861, max=902, avg=881.96, stdev=28.71 lat (msec): min=862, max=902, avg=882.39, stdev=28.69 clat percentiles (msec): | 1.00th=[ 860], 5.00th=[ 860], 10.00th=[ 860], 20.00th=[ 860], | 30.00th=[ 860], 40.00th=[ 860], 50.00th=[ 860], 60.00th=[ 902], | 70.00th=[ 902], 80.00th=[ 902], 90.00th=[ 902], 95.00th=[ 902], | 99.00th=[ 902], 99.50th=[ 902], 99.90th=[ 902], 99.95th=[ 902], | 99.99th=[ 902] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 1000=66.67%, 2000=33.33% cpu : usr=0.03%, sys=13.11%, ctx=113, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181752: Mon Jul 28 20:06:35 2025 write: IOPS=0, BW=3070KiB/s (3143kB/s)(12.0MiB/4003msec); 0 zone resets clat (msec): min=313, max=1682, avg=1214.86, stdev=780.79 lat (msec): min=313, max=1683, avg=1215.33, stdev=780.89 clat percentiles (msec): | 1.00th=[ 313], 5.00th=[ 313], 10.00th=[ 313], 20.00th=[ 313], | 30.00th=[ 313], 40.00th=[ 1653], 50.00th=[ 1653], 60.00th=[ 1653], | 70.00th=[ 1687], 80.00th=[ 1687], 90.00th=[ 1687], 95.00th=[ 1687], | 99.00th=[ 1687], 99.50th=[ 1687], 99.90th=[ 1687], 99.95th=[ 1687], | 99.99th=[ 1687] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.07%, sys=12.99%, ctx=91, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181754: Mon Jul 28 20:06:35 2025 write: IOPS=1, BW=5357KiB/s (5485kB/s)(12.0MiB/2294msec); 0 zone resets clat (msec): min=244, max=1526, avg=706.25, stdev=712.60 lat (msec): min=244, max=1527, avg=706.72, stdev=712.62 clat percentiles (msec): | 1.00th=[ 245], 5.00th=[ 245], 10.00th=[ 245], 20.00th=[ 245], | 30.00th=[ 245], 40.00th=[ 347], 50.00th=[ 347], 60.00th=[ 347], | 70.00th=[ 1536], 80.00th=[ 1536], 90.00th=[ 1536], 95.00th=[ 1536], | 99.00th=[ 1536], 99.50th=[ 1536], 99.90th=[ 1536], 99.95th=[ 1536], | 99.99th=[ 1536] bw ( KiB/s): min= 7111, max= 8143, per=93.18%, avg=7627.00, stdev=729.73, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=18.53%, ctx=114, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4390KiB/s (4496kB/s), 1098KiB/s-4779KiB/s (1124kB/s-4894kB/s), io=16.0MiB (16.8MB), run=2571-3732msec WRITE: bw=8186KiB/s (8382kB/s), 2195KiB/s-5357KiB/s (2248kB/s-5485kB/s), io=32.0MiB (33.6MB), run=2294-4003msec