== sanity test 398b: DIO and buffer IO race ============== 14:16:38 (1752689798) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.12936 s, 16.1 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155499: Wed Jul 16 14:17:42 2025 read: IOPS=27, BW=109KiB/s (111kB/s)(5992KiB/55166msec) clat (usec): min=72, max=102673, avg=19235.21, stdev=11565.79 lat (usec): min=73, max=102674, avg=19236.28, stdev=11565.93 clat percentiles (usec): | 1.00th=[ 310], 5.00th=[ 5276], 10.00th=[ 7832], 20.00th=[ 10290], | 30.00th=[ 12780], 40.00th=[ 15139], 50.00th=[ 17171], 60.00th=[ 19792], | 70.00th=[ 22938], 80.00th=[ 27395], 90.00th=[ 33162], 95.00th=[ 38536], | 99.00th=[ 57934], 99.50th=[ 65799], 99.90th=[102237], 99.95th=[102237], | 99.99th=[102237] bw ( KiB/s): min= 19, max= 216, per=24.68%, avg=107.36, stdev=29.39, samples=109 iops : min= 4, max= 54, avg=26.43, stdev= 7.40, samples=109 write: IOPS=28, BW=114KiB/s (117kB/s)(6296KiB/55166msec); 0 zone resets clat (usec): min=687, max=540907, avg=16634.43, stdev=19595.47 lat (usec): min=687, max=540908, avg=16636.04, stdev=19595.38 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 2245], 10.00th=[ 2802], 20.00th=[ 4621], | 30.00th=[ 6718], 40.00th=[ 9634], 50.00th=[ 12780], 60.00th=[ 16319], | 70.00th=[ 20579], 80.00th=[ 26084], 90.00th=[ 34341], 95.00th=[ 42206], | 99.00th=[ 57410], 99.50th=[ 79168], 99.90th=[240124], 99.95th=[541066], | 99.99th=[541066] bw ( KiB/s): min= 47, max= 184, per=24.96%, avg=113.57, stdev=30.59, samples=108 iops : min= 11, max= 46, avg=27.99, stdev= 7.72, samples=108 lat (usec) : 100=0.10%, 250=0.16%, 500=1.01%, 750=0.65%, 1000=0.75% lat (msec) : 2=1.50%, 4=6.77%, 10=19.43%, 20=34.77%, 50=33.07% lat (msec) : 100=1.66%, 250=0.10%, 750=0.03% cpu : usr=0.18%, sys=28.94%, ctx=8706, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155500: Wed Jul 16 14:17:42 2025 read: IOPS=27, BW=109KiB/s (111kB/s)(5940KiB/54566msec) clat (usec): min=121, max=163364, avg=19082.57, stdev=11596.26 lat (usec): min=121, max=163366, avg=19083.61, stdev=11596.29 clat percentiles (usec): | 1.00th=[ 334], 5.00th=[ 5997], 10.00th=[ 8291], 20.00th=[ 10683], | 30.00th=[ 12780], 40.00th=[ 14877], 50.00th=[ 16712], 60.00th=[ 19268], | 70.00th=[ 22938], 80.00th=[ 26346], 90.00th=[ 32113], 95.00th=[ 38011], | 99.00th=[ 56886], 99.50th=[ 70779], 99.90th=[ 87557], 99.95th=[162530], | 99.99th=[162530] bw ( KiB/s): min= 6, max= 215, per=24.93%, avg=108.44, stdev=31.19, samples=108 iops : min= 1, max= 53, avg=26.70, stdev= 7.83, samples=108 write: IOPS=29, BW=116KiB/s (119kB/s)(6348KiB/54566msec); 0 zone resets clat (usec): min=577, max=549439, avg=16414.32, stdev=18860.53 lat (usec): min=577, max=549440, avg=16416.71, stdev=18860.81 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 2278], 10.00th=[ 3195], 20.00th=[ 4948], | 30.00th=[ 7111], 40.00th=[ 10421], 50.00th=[ 13698], 60.00th=[ 16319], | 70.00th=[ 20055], 80.00th=[ 25035], 90.00th=[ 32637], 95.00th=[ 39584], | 99.00th=[ 58983], 99.50th=[ 74974], 99.90th=[170918], 99.95th=[549454], | 99.99th=[549454] bw ( KiB/s): min= 19, max= 207, per=25.37%, avg=115.45, stdev=31.77, samples=108 iops : min= 4, max= 51, avg=28.45, stdev= 8.04, samples=108 lat (usec) : 250=0.10%, 500=0.94%, 750=0.75%, 1000=0.52% lat (msec) : 2=1.53%, 4=5.76%, 10=18.68%, 20=37.92%, 50=31.97% lat (msec) : 100=1.69%, 250=0.10%, 750=0.03% cpu : usr=0.15%, sys=29.36%, ctx=9010, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155501: Wed Jul 16 14:17:42 2025 read: IOPS=27, BW=109KiB/s (112kB/s)(6028KiB/55129msec) clat (usec): min=93, max=549488, avg=19546.81, stdev=18325.46 lat (usec): min=93, max=549488, avg=19548.23, stdev=18325.55 clat percentiles (usec): | 1.00th=[ 281], 5.00th=[ 5866], 10.00th=[ 8291], 20.00th=[ 10552], | 30.00th=[ 12518], 40.00th=[ 14746], 50.00th=[ 16712], 60.00th=[ 19268], | 70.00th=[ 22414], 80.00th=[ 26346], 90.00th=[ 32375], 95.00th=[ 38536], | 99.00th=[ 68682], 99.50th=[ 88605], 99.90th=[122160], 99.95th=[549454], | 99.99th=[549454] bw ( KiB/s): min= 13, max= 192, per=24.87%, avg=108.17, stdev=29.98, samples=109 iops : min= 3, max= 48, avg=26.64, stdev= 7.56, samples=109 write: IOPS=28, BW=114KiB/s (116kB/s)(6260KiB/55129msec); 0 zone resets clat (usec): min=698, max=131327, avg=16293.81, stdev=13294.34 lat (usec): min=698, max=131330, avg=16295.20, stdev=13294.43 clat percentiles (usec): | 1.00th=[ 1205], 5.00th=[ 2376], 10.00th=[ 3064], 20.00th=[ 5080], | 30.00th=[ 7570], 40.00th=[ 10421], 50.00th=[ 13304], 60.00th=[ 16450], | 70.00th=[ 20055], 80.00th=[ 25297], 90.00th=[ 33162], 95.00th=[ 39060], | 99.00th=[ 60556], 99.50th=[ 74974], 99.90th=[112722], 99.95th=[131597], | 99.99th=[131597] bw ( KiB/s): min= 26, max= 208, per=24.62%, avg=112.03, stdev=32.55, samples=109 iops : min= 6, max= 52, avg=27.61, stdev= 8.19, samples=109 lat (usec) : 100=0.03%, 250=0.33%, 500=0.94%, 750=0.62%, 1000=0.33% lat (msec) : 2=1.40%, 4=5.60%, 10=19.17%, 20=37.66%, 50=31.84% lat (msec) : 100=1.89%, 250=0.16%, 750=0.03% cpu : usr=0.16%, sys=29.05%, ctx=8897, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155502: Wed Jul 16 14:17:42 2025 read: IOPS=27, BW=111KiB/s (114kB/s)(6052KiB/54476msec) clat (usec): min=149, max=247681, avg=18909.06, stdev=12341.22 lat (usec): min=150, max=247682, avg=18910.60, stdev=12341.46 clat percentiles (usec): | 1.00th=[ 297], 5.00th=[ 4490], 10.00th=[ 8160], 20.00th=[ 10421], | 30.00th=[ 12518], 40.00th=[ 14746], 50.00th=[ 16909], 60.00th=[ 19530], | 70.00th=[ 22414], 80.00th=[ 26346], 90.00th=[ 32375], 95.00th=[ 37487], | 99.00th=[ 54789], 99.50th=[ 70779], 99.90th=[ 93848], 99.95th=[248513], | 99.99th=[248513] bw ( KiB/s): min= 40, max= 192, per=25.68%, avg=111.71, stdev=28.13, samples=107 iops : min= 10, max= 48, avg=27.52, stdev= 7.07, samples=107 write: IOPS=28, BW=114KiB/s (117kB/s)(6236KiB/54476msec); 0 zone resets clat (usec): min=753, max=582078, avg=16463.39, stdev=19301.04 lat (usec): min=754, max=582080, avg=16465.39, stdev=19300.70 clat percentiles (usec): | 1.00th=[ 979], 5.00th=[ 2278], 10.00th=[ 2999], 20.00th=[ 4817], | 30.00th=[ 7111], 40.00th=[ 10290], 50.00th=[ 13173], 60.00th=[ 16450], | 70.00th=[ 20317], 80.00th=[ 25560], 90.00th=[ 32637], 95.00th=[ 39060], | 99.00th=[ 59507], 99.50th=[ 70779], 99.90th=[117965], 99.95th=[583009], | 99.99th=[583009] bw ( KiB/s): min= 47, max= 215, per=25.29%, avg=115.08, stdev=33.02, samples=107 iops : min= 11, max= 53, avg=28.36, stdev= 8.34, samples=107 lat (usec) : 250=0.16%, 500=1.07%, 750=0.49%, 1000=0.91% lat (msec) : 2=1.37%, 4=6.48%, 10=18.10%, 20=36.85%, 50=32.75% lat (msec) : 100=1.73%, 250=0.07%, 750=0.03% cpu : usr=0.16%, sys=29.11%, ctx=8749, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=435KiB/s (446kB/s), 109KiB/s-111KiB/s (111kB/s-114kB/s), io=23.4MiB (24.6MB), run=54476-55166msec WRITE: bw=456KiB/s (467kB/s), 114KiB/s-116KiB/s (116kB/s-119kB/s), io=24.6MiB (25.7MB), run=54476-55166msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155495: Wed Jul 16 14:18:13 2025 read: IOPS=17, BW=70.1KiB/s (71.8kB/s)(5992KiB/85507msec) clat (msec): min=2, max=617, avg=17.00, stdev=21.08 lat (msec): min=2, max=617, avg=17.00, stdev=21.08 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 8], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 16], | 70.00th=[ 19], 80.00th=[ 24], 90.00th=[ 35], 95.00th=[ 46], | 99.00th=[ 73], 99.50th=[ 87], 99.90th=[ 123], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 8, max= 160, per=25.12%, avg=70.35, stdev=35.67, samples=169 iops : min= 2, max= 40, avg=17.31, stdev= 9.05, samples=169 write: IOPS=18, BW=73.6KiB/s (75.4kB/s)(6296KiB/85507msec); 0 zone resets clat (msec): min=12, max=173, avg=38.03, stdev=18.54 lat (msec): min=12, max=173, avg=38.03, stdev=18.54 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 36], | 70.00th=[ 42], 80.00th=[ 48], 90.00th=[ 61], 95.00th=[ 72], | 99.00th=[ 113], 99.50th=[ 127], 99.90th=[ 159], 99.95th=[ 174], | 99.99th=[ 174] bw ( KiB/s): min= 23, max= 160, per=25.11%, avg=73.81, stdev=30.10, samples=169 iops : min= 5, max= 40, avg=18.18, stdev= 7.71, samples=169 lat (msec) : 4=2.12%, 10=17.32%, 20=19.24%, 50=50.49%, 100=9.96% lat (msec) : 250=0.85%, 750=0.03% cpu : usr=0.12%, sys=9.70%, ctx=4476, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155496: Wed Jul 16 14:18:13 2025 read: IOPS=17, BW=69.7KiB/s (71.4kB/s)(5940KiB/85169msec) clat (msec): min=2, max=562, avg=16.94, stdev=20.63 lat (msec): min=2, max=562, avg=16.94, stdev=20.63 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 8], 40.00th=[ 10], 50.00th=[ 13], 60.00th=[ 15], | 70.00th=[ 18], 80.00th=[ 24], 90.00th=[ 35], 95.00th=[ 48], | 99.00th=[ 78], 99.50th=[ 95], 99.90th=[ 136], 99.95th=[ 567], | 99.99th=[ 567] bw ( KiB/s): min= 15, max= 192, per=24.99%, avg=69.96, stdev=35.69, samples=168 iops : min= 3, max= 48, avg=17.21, stdev= 9.09, samples=168 write: IOPS=18, BW=74.5KiB/s (76.3kB/s)(6348KiB/85169msec); 0 zone resets clat (msec): min=14, max=248, avg=37.70, stdev=18.51 lat (msec): min=14, max=248, avg=37.70, stdev=18.51 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 26], | 30.00th=[ 28], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 36], | 70.00th=[ 41], 80.00th=[ 49], 90.00th=[ 59], 95.00th=[ 70], | 99.00th=[ 113], 99.50th=[ 138], 99.90th=[ 178], 99.95th=[ 249], | 99.99th=[ 249] bw ( KiB/s): min= 23, max= 136, per=25.38%, avg=74.63, stdev=30.22, samples=168 iops : min= 5, max= 34, avg=18.38, stdev= 7.71, samples=168 lat (msec) : 4=1.73%, 10=18.16%, 20=18.78%, 50=49.90%, 100=10.61% lat (msec) : 250=0.78%, 750=0.03% cpu : usr=0.11%, sys=9.76%, ctx=4405, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155497: Wed Jul 16 14:18:13 2025 read: IOPS=17, BW=70.5KiB/s (72.2kB/s)(6028KiB/85492msec) clat (msec): min=2, max=235, avg=17.08, stdev=14.98 lat (msec): min=2, max=235, avg=17.08, stdev=14.98 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 11], 50.00th=[ 14], 60.00th=[ 16], | 70.00th=[ 20], 80.00th=[ 25], 90.00th=[ 35], 95.00th=[ 45], | 99.00th=[ 69], 99.50th=[ 82], 99.90th=[ 110], 99.95th=[ 236], | 99.99th=[ 236] bw ( KiB/s): min= 8, max= 176, per=25.26%, avg=70.73, stdev=34.33, samples=169 iops : min= 2, max= 44, avg=17.41, stdev= 8.74, samples=169 write: IOPS=18, BW=73.2KiB/s (74.0kB/s)(6260KiB/85492msec); 0 zone resets clat (msec): min=13, max=577, avg=38.07, stdev=22.56 lat (msec): min=13, max=577, avg=38.07, stdev=22.56 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 36], | 70.00th=[ 41], 80.00th=[ 49], 90.00th=[ 62], 95.00th=[ 73], | 99.00th=[ 108], 99.50th=[ 129], 99.90th=[ 148], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 16, max= 152, per=24.98%, avg=73.43, stdev=30.51, samples=169 iops : min= 4, max= 38, avg=18.08, stdev= 7.80, samples=169 lat (msec) : 4=1.69%, 10=16.96%, 20=19.08%, 50=50.88%, 100=10.61% lat (msec) : 250=0.75%, 750=0.03% cpu : usr=0.13%, sys=9.74%, ctx=4490, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155498: Wed Jul 16 14:18:13 2025 read: IOPS=17, BW=71.5KiB/s (73.2kB/s)(6052KiB/84690msec) clat (msec): min=2, max=130, avg=16.34, stdev=13.69 lat (msec): min=2, max=130, avg=16.34, stdev=13.69 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 16], | 70.00th=[ 18], 80.00th=[ 23], 90.00th=[ 33], 95.00th=[ 45], | 99.00th=[ 69], 99.50th=[ 74], 99.90th=[ 127], 99.95th=[ 131], | 99.99th=[ 131] bw ( KiB/s): min= 7, max= 200, per=25.60%, avg=71.67, stdev=35.59, samples=167 iops : min= 1, max= 50, avg=17.64, stdev= 9.03, samples=167 write: IOPS=18, BW=73.6KiB/s (75.4kB/s)(6236KiB/84690msec); 0 zone resets clat (msec): min=14, max=577, avg=38.35, stdev=23.13 lat (msec): min=14, max=577, avg=38.35, stdev=23.13 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 42], 80.00th=[ 49], 90.00th=[ 61], 95.00th=[ 74], | 99.00th=[ 109], 99.50th=[ 128], 99.90th=[ 253], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 15, max= 152, per=25.10%, avg=73.78, stdev=29.42, samples=167 iops : min= 3, max= 38, avg=18.17, stdev= 7.54, samples=167 lat (msec) : 4=1.66%, 10=17.90%, 20=19.73%, 50=50.20%, 100=9.73% lat (msec) : 250=0.72%, 500=0.03%, 750=0.03% cpu : usr=0.10%, sys=9.94%, ctx=4481, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=281KiB/s (288kB/s), 69.7KiB/s-71.5KiB/s (71.4kB/s-73.2kB/s), io=23.4MiB (24.6MB), run=84690-85507msec WRITE: bw=294KiB/s (301kB/s), 73.2KiB/s-74.5KiB/s (74.0kB/s-76.3kB/s), io=24.6MiB (25.7MB), run=84690-85507msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155531: Wed Jul 16 14:18:34 2025 read: IOPS=23, BW=380KiB/s (389kB/s)(5920KiB/15585msec) clat (usec): min=191, max=118238, avg=20492.58, stdev=13938.38 lat (usec): min=191, max=118240, avg=20493.58, stdev=13938.41 clat percentiles (usec): | 1.00th=[ 343], 5.00th=[ 545], 10.00th=[ 701], 20.00th=[ 10683], | 30.00th=[ 14091], 40.00th=[ 16319], 50.00th=[ 18220], 60.00th=[ 21365], | 70.00th=[ 25297], 80.00th=[ 29754], 90.00th=[ 35390], 95.00th=[ 42730], | 99.00th=[ 69731], 99.50th=[ 87557], 99.90th=[117965], 99.95th=[117965], | 99.99th=[117965] bw ( KiB/s): min= 188, max= 832, per=25.35%, avg=378.73, stdev=128.63, samples=30 iops : min= 11, max= 52, avg=23.10, stdev= 8.12, samples=30 write: IOPS=25, BW=409KiB/s (418kB/s)(6368KiB/15585msec); 0 zone resets clat (usec): min=1049, max=75683, avg=19972.43, stdev=14079.26 lat (usec): min=1050, max=75687, avg=19974.40, stdev=14079.35 clat percentiles (usec): | 1.00th=[ 1844], 5.00th=[ 4228], 10.00th=[ 5735], 20.00th=[ 7242], | 30.00th=[10028], 40.00th=[13566], 50.00th=[16581], 60.00th=[20317], | 70.00th=[25035], 80.00th=[31327], 90.00th=[39060], 95.00th=[49021], | 99.00th=[68682], 99.50th=[76022], 99.90th=[76022], 99.95th=[76022], | 99.99th=[76022] bw ( KiB/s): min= 256, max= 704, per=26.24%, avg=412.53, stdev=98.58, samples=30 iops : min= 16, max= 44, avg=25.23, stdev= 6.26, samples=30 lat (usec) : 250=0.13%, 500=1.56%, 750=3.52%, 1000=0.39% lat (msec) : 2=0.65%, 4=1.95%, 10=15.36%, 20=34.11%, 50=38.80% lat (msec) : 100=3.39%, 250=0.13% cpu : usr=0.13%, sys=32.21%, ctx=2514, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155532: Wed Jul 16 14:18:34 2025 read: IOPS=22, BW=360KiB/s (369kB/s)(5744KiB/15940msec) clat (usec): min=292, max=119278, avg=22643.17, stdev=14027.26 lat (usec): min=293, max=119280, avg=22644.16, stdev=14027.31 clat percentiles (usec): | 1.00th=[ 412], 5.00th=[ 676], 10.00th=[ 9241], 20.00th=[ 12780], | 30.00th=[ 15664], 40.00th=[ 18220], 50.00th=[ 20579], 60.00th=[ 23200], | 70.00th=[ 26346], 80.00th=[ 30802], 90.00th=[ 38536], 95.00th=[ 44303], | 99.00th=[ 71828], 99.50th=[101188], 99.90th=[119014], 99.95th=[119014], | 99.99th=[119014] bw ( KiB/s): min= 192, max= 544, per=23.96%, avg=357.94, stdev=98.88, samples=31 iops : min= 12, max= 34, avg=21.81, stdev= 6.22, samples=31 write: IOPS=25, BW=411KiB/s (420kB/s)(6544KiB/15940msec); 0 zone resets clat (usec): min=1229, max=102001, avg=18949.13, stdev=13172.32 lat (usec): min=1230, max=102002, avg=18951.23, stdev=13172.57 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 23], 80.00th=[ 30], 90.00th=[ 37], 95.00th=[ 44], | 99.00th=[ 54], 99.50th=[ 57], 99.90th=[ 103], 99.95th=[ 103], | 99.99th=[ 103] bw ( KiB/s): min= 62, max= 704, per=25.72%, avg=404.32, stdev=133.64, samples=31 iops : min= 3, max= 44, avg=24.71, stdev= 8.49, samples=31 lat (usec) : 500=0.78%, 750=1.82%, 1000=0.52% lat (msec) : 2=0.52%, 4=2.47%, 10=15.76%, 20=32.42%, 50=42.97% lat (msec) : 100=2.34%, 250=0.39% cpu : usr=0.13%, sys=32.54%, ctx=2627, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155533: Wed Jul 16 14:18:34 2025 read: IOPS=24, BW=390KiB/s (400kB/s)(6256KiB/16023msec) clat (usec): min=109, max=101501, avg=22550.98, stdev=12969.64 lat (usec): min=110, max=101502, avg=22552.11, stdev=12969.66 clat percentiles (usec): | 1.00th=[ 392], 5.00th=[ 701], 10.00th=[ 10290], 20.00th=[ 13566], | 30.00th=[ 16188], 40.00th=[ 18220], 50.00th=[ 20841], 60.00th=[ 23462], | 70.00th=[ 26870], 80.00th=[ 31327], 90.00th=[ 38011], 95.00th=[ 42206], | 99.00th=[ 63701], 99.50th=[ 91751], 99.90th=[101188], 99.95th=[101188], | 99.99th=[101188] bw ( KiB/s): min= 158, max= 640, per=25.96%, avg=387.84, stdev=113.34, samples=31 iops : min= 9, max= 40, avg=23.68, stdev= 7.32, samples=31 write: IOPS=23, BW=376KiB/s (385kB/s)(6032KiB/16023msec); 0 zone resets clat (usec): min=1572, max=85084, avg=18980.69, stdev=13829.07 lat (usec): min=1574, max=85085, avg=18982.89, stdev=13829.08 clat percentiles (usec): | 1.00th=[ 2737], 5.00th=[ 4686], 10.00th=[ 5866], 20.00th=[ 7504], | 30.00th=[ 9503], 40.00th=[11731], 50.00th=[15533], 60.00th=[19006], | 70.00th=[22152], 80.00th=[28181], 90.00th=[39060], 95.00th=[47449], | 99.00th=[67634], 99.50th=[74974], 99.90th=[85459], 99.95th=[85459], | 99.99th=[85459] bw ( KiB/s): min= 159, max= 640, per=23.35%, avg=367.03, stdev=100.93, samples=31 iops : min= 9, max= 40, avg=22.39, stdev= 6.42, samples=31 lat (usec) : 250=0.39%, 500=0.65%, 750=2.08%, 1000=0.26% lat (msec) : 2=0.39%, 4=1.30%, 10=15.63%, 20=33.85%, 50=42.19% lat (msec) : 100=3.12%, 250=0.13% cpu : usr=0.14%, sys=32.31%, ctx=2702, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155534: Wed Jul 16 14:18:34 2025 read: IOPS=23, BW=381KiB/s (391kB/s)(6032KiB/15812msec) clat (usec): min=134, max=87346, avg=21874.63, stdev=13111.27 lat (usec): min=134, max=87347, avg=21875.65, stdev=13111.31 clat percentiles (usec): | 1.00th=[ 302], 5.00th=[ 529], 10.00th=[ 3752], 20.00th=[12125], | 30.00th=[15270], 40.00th=[17433], 50.00th=[20841], 60.00th=[23200], | 70.00th=[27395], 80.00th=[31851], 90.00th=[39584], 95.00th=[45351], | 99.00th=[62129], 99.50th=[63177], 99.90th=[87557], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 223, max= 640, per=25.35%, avg=378.71, stdev=100.65, samples=31 iops : min= 13, max= 40, avg=23.10, stdev= 6.49, samples=31 write: IOPS=24, BW=396KiB/s (405kB/s)(6256KiB/15812msec); 0 zone resets clat (usec): min=1363, max=70938, avg=19245.36, stdev=13034.58 lat (usec): min=1366, max=70941, avg=19247.50, stdev=13034.72 clat percentiles (usec): | 1.00th=[ 1844], 5.00th=[ 3818], 10.00th=[ 5145], 20.00th=[ 7242], | 30.00th=[ 9896], 40.00th=[13173], 50.00th=[16909], 60.00th=[20317], | 70.00th=[24511], 80.00th=[28967], 90.00th=[36963], 95.00th=[45876], | 99.00th=[61080], 99.50th=[64226], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 222, max= 544, per=25.10%, avg=394.58, stdev=91.96, samples=31 iops : min= 13, max= 34, avg=24.10, stdev= 5.91, samples=31 lat (usec) : 250=0.26%, 500=1.69%, 750=1.69%, 1000=0.65% lat (msec) : 2=1.17%, 4=2.47%, 10=14.58%, 20=30.99%, 50=44.01% lat (msec) : 100=2.47% cpu : usr=0.12%, sys=31.72%, ctx=2699, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1495KiB/s (1531kB/s), 360KiB/s-390KiB/s (369kB/s-400kB/s), io=23.4MiB (24.5MB), run=15585-16023msec WRITE: bw=1573KiB/s (1610kB/s), 376KiB/s-411KiB/s (385kB/s-420kB/s), io=24.6MiB (25.8MB), run=15585-16023msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155536: Wed Jul 16 14:18:42 2025 read: IOPS=15, BW=253KiB/s (259kB/s)(5920KiB/23400msec) clat (msec): min=2, max=107, avg=20.32, stdev=18.56 lat (msec): min=2, max=107, avg=20.32, stdev=18.56 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 21], 80.00th=[ 30], 90.00th=[ 49], 95.00th=[ 63], | 99.00th=[ 87], 99.50th=[ 92], 99.90th=[ 108], 99.95th=[ 108], | 99.99th=[ 108] bw ( KiB/s): min= 32, max= 544, per=24.13%, avg=246.80, stdev=128.17, samples=46 iops : min= 2, max= 34, avg=14.98, stdev= 8.16, samples=46 write: IOPS=17, BW=272KiB/s (279kB/s)(6368KiB/23400msec); 0 zone resets clat (msec): min=14, max=183, avg=39.71, stdev=23.04 lat (msec): min=14, max=183, avg=39.71, stdev=23.04 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 19], 10.00th=[ 21], 20.00th=[ 24], | 30.00th=[ 27], 40.00th=[ 30], 50.00th=[ 34], 60.00th=[ 37], | 70.00th=[ 43], 80.00th=[ 51], 90.00th=[ 70], 95.00th=[ 88], | 99.00th=[ 126], 99.50th=[ 150], 99.90th=[ 184], 99.95th=[ 184], | 99.99th=[ 184] bw ( KiB/s): min= 103, max= 544, per=24.91%, avg=268.04, stdev=133.42, samples=46 iops : min= 6, max= 34, avg=16.30, stdev= 8.58, samples=46 lat (msec) : 4=2.60%, 10=12.89%, 20=21.48%, 50=47.92%, 100=13.41% lat (msec) : 250=1.69% cpu : usr=0.08%, sys=11.16%, ctx=1270, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155537: Wed Jul 16 14:18:42 2025 read: IOPS=15, BW=248KiB/s (254kB/s)(5744KiB/23139msec) clat (msec): min=3, max=123, avg=19.82, stdev=18.25 lat (msec): min=3, max=123, avg=19.82, stdev=18.25 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 20], 80.00th=[ 27], 90.00th=[ 40], 95.00th=[ 61], | 99.00th=[ 106], 99.50th=[ 115], 99.90th=[ 125], 99.95th=[ 125], | 99.99th=[ 125] bw ( KiB/s): min= 51, max= 608, per=24.23%, avg=247.91, stdev=134.53, samples=45 iops : min= 3, max= 38, avg=15.04, stdev= 8.55, samples=45 write: IOPS=17, BW=283KiB/s (290kB/s)(6544KiB/23139msec); 0 zone resets clat (msec): min=14, max=144, avg=38.93, stdev=19.35 lat (msec): min=14, max=144, avg=38.93, stdev=19.35 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 43], 80.00th=[ 52], 90.00th=[ 66], 95.00th=[ 74], | 99.00th=[ 108], 99.50th=[ 124], 99.90th=[ 144], 99.95th=[ 144], | 99.99th=[ 144] bw ( KiB/s): min= 126, max= 544, per=25.94%, avg=279.16, stdev=115.96, samples=45 iops : min= 7, max= 34, avg=17.00, stdev= 7.47, samples=45 lat (msec) : 4=0.39%, 10=13.54%, 20=22.27%, 50=49.22%, 100=13.15% lat (msec) : 250=1.43% cpu : usr=0.09%, sys=11.54%, ctx=1274, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155538: Wed Jul 16 14:18:42 2025 read: IOPS=17, BW=273KiB/s (280kB/s)(6256KiB/22899msec) clat (msec): min=3, max=146, avg=20.05, stdev=18.13 lat (msec): min=3, max=146, avg=20.05, stdev=18.13 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 17], | 70.00th=[ 21], 80.00th=[ 30], 90.00th=[ 44], 95.00th=[ 56], | 99.00th=[ 91], 99.50th=[ 97], 99.90th=[ 146], 99.95th=[ 146], | 99.99th=[ 146] bw ( KiB/s): min= 95, max= 832, per=26.92%, avg=275.36, stdev=169.04, samples=45 iops : min= 5, max= 52, avg=16.76, stdev=10.77, samples=45 write: IOPS=16, BW=263KiB/s (270kB/s)(6032KiB/22899msec); 0 zone resets clat (msec): min=15, max=117, avg=39.57, stdev=18.69 lat (msec): min=15, max=117, avg=39.57, stdev=18.69 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 22], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 31], 50.00th=[ 34], 60.00th=[ 38], | 70.00th=[ 46], 80.00th=[ 53], 90.00th=[ 67], 95.00th=[ 78], | 99.00th=[ 105], 99.50th=[ 109], 99.90th=[ 118], 99.95th=[ 118], | 99.99th=[ 118] bw ( KiB/s): min= 95, max= 480, per=24.26%, avg=261.00, stdev=116.23, samples=45 iops : min= 5, max= 30, avg=15.87, stdev= 7.46, samples=45 lat (msec) : 4=0.26%, 10=16.28%, 20=20.18%, 50=48.31%, 100=14.32% lat (msec) : 250=0.65% cpu : usr=0.12%, sys=11.86%, ctx=1288, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155539: Wed Jul 16 14:18:42 2025 read: IOPS=16, BW=261KiB/s (267kB/s)(6032KiB/23149msec) clat (msec): min=3, max=109, avg=20.78, stdev=19.20 lat (msec): min=3, max=109, avg=20.79, stdev=19.20 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 21], 80.00th=[ 29], 90.00th=[ 50], 95.00th=[ 64], | 99.00th=[ 103], 99.50th=[ 106], 99.90th=[ 110], 99.95th=[ 110], | 99.99th=[ 110] bw ( KiB/s): min= 95, max= 608, per=24.94%, avg=255.18, stdev=119.15, samples=45 iops : min= 5, max= 38, avg=15.49, stdev= 7.61, samples=45 write: IOPS=16, BW=270KiB/s (277kB/s)(6256KiB/23149msec); 0 zone resets clat (msec): min=14, max=192, avg=38.81, stdev=20.89 lat (msec): min=14, max=192, avg=38.81, stdev=20.89 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 25], | 30.00th=[ 27], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 36], | 70.00th=[ 41], 80.00th=[ 50], 90.00th=[ 65], 95.00th=[ 82], | 99.00th=[ 113], 99.50th=[ 155], 99.90th=[ 192], 99.95th=[ 192], | 99.99th=[ 192] bw ( KiB/s): min= 95, max= 576, per=24.90%, avg=267.93, stdev=129.08, samples=45 iops : min= 5, max= 36, avg=16.29, stdev= 8.32, samples=45 lat (msec) : 4=0.65%, 10=13.41%, 20=22.92%, 50=48.44%, 100=12.89% lat (msec) : 250=1.69% cpu : usr=0.11%, sys=11.31%, ctx=1270, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1024KiB/s (1048kB/s), 248KiB/s-273KiB/s (254kB/s-280kB/s), io=23.4MiB (24.5MB), run=22899-23400msec WRITE: bw=1077KiB/s (1103kB/s), 263KiB/s-283KiB/s (270kB/s-290kB/s), io=24.6MiB (25.8MB), run=22899-23400msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155548: Wed Jul 16 14:18:51 2025 read: IOPS=1, BW=1638KiB/s (1678kB/s)(7168KiB/4375msec) clat (msec): min=81, max=947, avg=331.67, stdev=321.33 lat (msec): min=81, max=947, avg=331.68, stdev=321.33 clat percentiles (msec): | 1.00th=[ 82], 5.00th=[ 82], 10.00th=[ 82], 20.00th=[ 96], | 30.00th=[ 107], 40.00th=[ 107], 50.00th=[ 199], 60.00th=[ 330], | 70.00th=[ 330], 80.00th=[ 567], 90.00th=[ 944], 95.00th=[ 944], | 99.00th=[ 944], 99.50th=[ 944], 99.90th=[ 944], 99.95th=[ 944], | 99.99th=[ 944] bw ( KiB/s): min= 1957, max= 5515, per=100.00%, avg=3856.00, stdev=1791.10, samples=3 iops : min= 1, max= 5, avg= 3.33, stdev= 2.08, samples=3 write: IOPS=1, BW=1170KiB/s (1198kB/s)(5120KiB/4375msec); 0 zone resets clat (msec): min=93, max=1301, avg=404.79, stdev=508.68 lat (msec): min=93, max=1302, avg=404.87, stdev=508.70 clat percentiles (msec): | 1.00th=[ 94], 5.00th=[ 94], 10.00th=[ 94], 20.00th=[ 94], | 30.00th=[ 144], 40.00th=[ 144], 50.00th=[ 163], 60.00th=[ 163], | 70.00th=[ 321], 80.00th=[ 321], 90.00th=[ 1301], 95.00th=[ 1301], | 99.00th=[ 1301], 99.50th=[ 1301], 99.90th=[ 1301], 99.95th=[ 1301], | 99.99th=[ 1301] bw ( KiB/s): min= 1838, max= 3992, per=45.52%, avg=2456.50, stdev=1026.99, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 100=25.00%, 250=33.33%, 500=16.67%, 750=8.33%, 1000=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=12.89%, ctx=147, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155549: Wed Jul 16 14:18:51 2025 read: IOPS=0, BW=386KiB/s (395kB/s)(2048KiB/5312msec) clat (msec): min=42, max=962, avg=502.90, stdev=650.55 lat (msec): min=42, max=962, avg=502.90, stdev=650.55 clat percentiles (msec): | 1.00th=[ 43], 5.00th=[ 43], 10.00th=[ 43], 20.00th=[ 43], | 30.00th=[ 43], 40.00th=[ 43], 50.00th=[ 43], 60.00th=[ 961], | 70.00th=[ 961], 80.00th=[ 961], 90.00th=[ 961], 95.00th=[ 961], | 99.00th=[ 961], 99.50th=[ 961], 99.90th=[ 961], 99.95th=[ 961], | 99.99th=[ 961] bw ( KiB/s): min= 1838, max= 2039, per=50.29%, avg=1938.50, stdev=142.13, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1928KiB/s (1974kB/s)(10.0MiB/5312msec); 0 zone resets clat (msec): min=46, max=2152, avg=429.11, stdev=632.14 lat (msec): min=46, max=2152, avg=429.22, stdev=632.13 clat percentiles (msec): | 1.00th=[ 47], 5.00th=[ 47], 10.00th=[ 47], 20.00th=[ 67], | 30.00th=[ 77], 40.00th=[ 104], 50.00th=[ 176], 60.00th=[ 224], | 70.00th=[ 468], 80.00th=[ 489], 90.00th=[ 493], 95.00th=[ 2165], | 99.00th=[ 2165], 99.50th=[ 2165], 99.90th=[ 2165], 99.95th=[ 2165], | 99.99th=[ 2165] bw ( KiB/s): min= 1838, max= 6144, per=52.13%, avg=2813.60, stdev=1863.62, samples=5 iops : min= 1, max= 6, avg= 2.20, stdev= 2.17, samples=5 lat (msec) : 50=16.67%, 100=16.67%, 250=25.00%, 500=25.00%, 1000=8.33% lat (msec) : >=2000=8.33% cpu : usr=0.06%, sys=13.99%, ctx=76, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155550: Wed Jul 16 14:18:51 2025 read: IOPS=1, BW=1922KiB/s (1969kB/s)(6144KiB/3196msec) clat (msec): min=27, max=307, avg=161.69, stdev=107.48 lat (msec): min=27, max=307, avg=161.69, stdev=107.48 clat percentiles (msec): | 1.00th=[ 28], 5.00th=[ 28], 10.00th=[ 28], 20.00th=[ 93], | 30.00th=[ 93], 40.00th=[ 95], 50.00th=[ 95], 60.00th=[ 197], | 70.00th=[ 253], 80.00th=[ 253], 90.00th=[ 309], 95.00th=[ 309], | 99.00th=[ 309], 99.50th=[ 309], 99.90th=[ 309], 99.95th=[ 309], | 99.99th=[ 309] bw ( KiB/s): min= 1957, max= 5988, per=86.12%, avg=3320.00, stdev=2310.74, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 write: IOPS=1, BW=1922KiB/s (1969kB/s)(6144KiB/3196msec); 0 zone resets clat (msec): min=66, max=1091, avg=363.54, stdev=393.48 lat (msec): min=66, max=1091, avg=363.64, stdev=393.48 clat percentiles (msec): | 1.00th=[ 67], 5.00th=[ 67], 10.00th=[ 67], 20.00th=[ 100], | 30.00th=[ 100], 40.00th=[ 140], 50.00th=[ 140], 60.00th=[ 259], | 70.00th=[ 527], 80.00th=[ 527], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 1996, max= 3716, per=45.28%, avg=2443.75, stdev=848.44, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 50=8.33%, 100=33.33%, 250=16.67%, 500=25.00%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.09%, sys=14.77%, ctx=85, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155552: Wed Jul 16 14:18:51 2025 read: IOPS=0, BW=979KiB/s (1002kB/s)(5120KiB/5232msec) clat (msec): min=46, max=958, avg=348.36, stdev=353.47 lat (msec): min=46, max=958, avg=348.37, stdev=353.47 clat percentiles (msec): | 1.00th=[ 47], 5.00th=[ 47], 10.00th=[ 47], 20.00th=[ 47], | 30.00th=[ 192], 40.00th=[ 192], 50.00th=[ 268], 60.00th=[ 268], | 70.00th=[ 275], 80.00th=[ 275], 90.00th=[ 961], 95.00th=[ 961], | 99.00th=[ 961], 99.50th=[ 961], 99.90th=[ 961], 99.95th=[ 961], | 99.99th=[ 961] bw ( KiB/s): min= 2015, max= 4079, per=70.19%, avg=2705.67, stdev=1189.35, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=1, BW=1370KiB/s (1403kB/s)(7168KiB/5232msec); 0 zone resets clat (msec): min=145, max=1247, avg=491.21, stdev=405.98 lat (msec): min=146, max=1247, avg=491.32, stdev=405.97 clat percentiles (msec): | 1.00th=[ 146], 5.00th=[ 146], 10.00th=[ 146], 20.00th=[ 153], | 30.00th=[ 264], 40.00th=[ 264], 50.00th=[ 359], 60.00th=[ 447], | 70.00th=[ 447], 80.00th=[ 827], 90.00th=[ 1250], 95.00th=[ 1250], | 99.00th=[ 1250], 99.50th=[ 1250], 99.90th=[ 1250], 99.95th=[ 1250], | 99.99th=[ 1250] bw ( KiB/s): min= 1910, max= 3992, per=44.42%, avg=2397.60, stdev=892.86, samples=5 iops : min= 1, max= 3, avg= 1.60, stdev= 0.89, samples=5 lat (msec) : 50=8.33%, 250=25.00%, 500=41.67%, 1000=16.67%, 2000=8.33% cpu : usr=0.04%, sys=14.22%, ctx=182, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3855KiB/s (3948kB/s), 386KiB/s-1922KiB/s (395kB/s-1969kB/s), io=20.0MiB (20.0MB), run=3196-5312msec WRITE: bw=5398KiB/s (5527kB/s), 1170KiB/s-1928KiB/s (1198kB/s-1974kB/s), io=28.0MiB (29.4MB), run=3196-5312msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155551: Wed Jul 16 14:18:51 2025 read: IOPS=1, BW=1742KiB/s (1784kB/s)(7168KiB/4114msec) clat (usec): min=1160, max=1054.2k, avg=283199.85, stdev=357842.49 lat (usec): min=1161, max=1054.2k, avg=283201.16, stdev=357842.45 clat percentiles (usec): | 1.00th=[ 1156], 5.00th=[ 1156], 10.00th=[ 1156], | 20.00th=[ 83362], 30.00th=[ 139461], 40.00th=[ 139461], | 50.00th=[ 162530], 60.00th=[ 173016], 70.00th=[ 173016], | 80.00th=[ 367002], 90.00th=[1052771], 95.00th=[1052771], | 99.00th=[1052771], 99.50th=[1052771], 99.90th=[1052771], | 99.95th=[1052771], 99.99th=[1052771] bw ( KiB/s): min= 2015, max= 4047, per=74.10%, avg=2955.00, stdev=1075.29, samples=4 iops : min= 1, max= 3, avg= 2.25, stdev= 0.96, samples=4 write: IOPS=1, BW=1245KiB/s (1274kB/s)(5120KiB/4114msec); 0 zone resets clat (msec): min=84, max=1188, avg=421.02, stdev=437.96 lat (msec): min=84, max=1188, avg=421.08, stdev=437.94 clat percentiles (msec): | 1.00th=[ 85], 5.00th=[ 85], 10.00th=[ 85], 20.00th=[ 85], | 30.00th=[ 226], 40.00th=[ 226], 50.00th=[ 296], 60.00th=[ 296], | 70.00th=[ 309], 80.00th=[ 309], 90.00th=[ 1183], 95.00th=[ 1183], | 99.00th=[ 1183], 99.50th=[ 1183], 99.90th=[ 1183], 99.95th=[ 1183], | 99.99th=[ 1183] bw ( KiB/s): min= 2023, max= 3915, per=47.68%, avg=2662.00, stdev=1085.20, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 lat (msec) : 2=8.33%, 100=16.67%, 250=33.33%, 500=25.00%, 2000=16.67% cpu : usr=0.00%, sys=39.05%, ctx=1048, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155553: Wed Jul 16 14:18:51 2025 read: IOPS=0, BW=399KiB/s (408kB/s)(2048KiB/5135msec) clat (msec): min=162, max=1128, avg=645.46, stdev=682.57 lat (msec): min=162, max=1128, avg=645.46, stdev=682.57 clat percentiles (msec): | 1.00th=[ 163], 5.00th=[ 163], 10.00th=[ 163], 20.00th=[ 163], | 30.00th=[ 163], 40.00th=[ 163], 50.00th=[ 163], 60.00th=[ 1133], | 70.00th=[ 1133], 80.00th=[ 1133], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 1899, max= 2039, per=49.37%, avg=1969.00, stdev=98.99, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1994KiB/s (2042kB/s)(10.0MiB/5135msec); 0 zone resets clat (msec): min=71, max=1766, avg=375.05, stdev=503.05 lat (msec): min=71, max=1766, avg=375.16, stdev=503.07 clat percentiles (msec): | 1.00th=[ 71], 5.00th=[ 71], 10.00th=[ 71], 20.00th=[ 80], | 30.00th=[ 157], 40.00th=[ 157], 50.00th=[ 190], 60.00th=[ 241], | 70.00th=[ 288], 80.00th=[ 355], 90.00th=[ 451], 95.00th=[ 1770], | 99.00th=[ 1770], 99.50th=[ 1770], 99.90th=[ 1770], 99.95th=[ 1770], | 99.99th=[ 1770] bw ( KiB/s): min= 2015, max= 8192, per=82.06%, avg=4581.50, stdev=3069.19, samples=4 iops : min= 1, max= 8, avg= 4.00, stdev= 3.16, samples=4 lat (msec) : 100=16.67%, 250=41.67%, 500=25.00%, 2000=16.67% cpu : usr=0.02%, sys=30.17%, ctx=401, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155554: Wed Jul 16 14:18:51 2025 read: IOPS=1, BW=1457KiB/s (1492kB/s)(6144KiB/4217msec) clat (usec): min=670, max=1071.4k, avg=278058.33, stdev=415117.30 lat (usec): min=671, max=1071.4k, avg=278059.30, stdev=415116.87 clat percentiles (usec): | 1.00th=[ 668], 5.00th=[ 668], 10.00th=[ 668], | 20.00th=[ 988], 30.00th=[ 988], 40.00th=[ 1205], | 50.00th=[ 1205], 60.00th=[ 274727], 70.00th=[ 320865], | 80.00th=[ 320865], 90.00th=[1069548], 95.00th=[1069548], | 99.00th=[1069548], 99.50th=[1069548], 99.90th=[1069548], | 99.95th=[1069548], 99.99th=[1069548] bw ( KiB/s): min= 1957, max= 4079, per=63.15%, avg=2518.50, stdev=1040.75, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=1, BW=1457KiB/s (1492kB/s)(6144KiB/4217msec); 0 zone resets clat (msec): min=61, max=1279, avg=410.68, stdev=448.04 lat (msec): min=61, max=1279, avg=410.79, stdev=448.03 clat percentiles (msec): | 1.00th=[ 62], 5.00th=[ 62], 10.00th=[ 62], 20.00th=[ 197], | 30.00th=[ 197], 40.00th=[ 199], 50.00th=[ 199], 60.00th=[ 239], | 70.00th=[ 489], 80.00th=[ 489], 90.00th=[ 1284], 95.00th=[ 1284], | 99.00th=[ 1284], 99.50th=[ 1284], 99.90th=[ 1284], 99.95th=[ 1284], | 99.99th=[ 1284] bw ( KiB/s): min= 2015, max= 4079, per=53.26%, avg=2973.75, stdev=1109.28, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 lat (usec) : 750=8.33%, 1000=8.33% lat (msec) : 2=8.33%, 100=8.33%, 250=25.00%, 500=25.00%, 2000=16.67% cpu : usr=0.05%, sys=33.92%, ctx=560, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155555: Wed Jul 16 14:18:51 2025 read: IOPS=1, BW=1029KiB/s (1054kB/s)(5120KiB/4974msec) clat (usec): min=805, max=883461, avg=233188.80, stdev=372331.32 lat (usec): min=807, max=883463, avg=233190.13, stdev=372331.73 clat percentiles (usec): | 1.00th=[ 807], 5.00th=[ 807], 10.00th=[ 807], 20.00th=[ 807], | 30.00th=[ 898], 40.00th=[ 898], 50.00th=[ 83362], 60.00th=[ 83362], | 70.00th=[198181], 80.00th=[198181], 90.00th=[884999], 95.00th=[884999], | 99.00th=[884999], 99.50th=[884999], 99.90th=[884999], 99.95th=[884999], | 99.99th=[884999] bw ( KiB/s): min= 2039, max= 6071, per=100.00%, avg=4055.00, stdev=2851.05, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=1, BW=1441KiB/s (1476kB/s)(7168KiB/4974msec); 0 zone resets clat (msec): min=163, max=1608, avg=535.38, stdev=496.67 lat (msec): min=163, max=1608, avg=535.47, stdev=496.68 clat percentiles (msec): | 1.00th=[ 163], 5.00th=[ 163], 10.00th=[ 163], 20.00th=[ 257], | 30.00th=[ 259], 40.00th=[ 259], 50.00th=[ 363], 60.00th=[ 489], | 70.00th=[ 489], 80.00th=[ 609], 90.00th=[ 1603], 95.00th=[ 1603], | 99.00th=[ 1603], 99.50th=[ 1603], 99.90th=[ 1603], 99.95th=[ 1603], | 99.99th=[ 1603] bw ( KiB/s): min= 2015, max= 3915, per=44.78%, avg=2500.25, stdev=943.27, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (usec) : 1000=16.67% lat (msec) : 100=8.33%, 250=16.67%, 500=33.33%, 750=8.33%, 1000=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=25.52%, ctx=222, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3988KiB/s (4084kB/s), 399KiB/s-1742KiB/s (408kB/s-1784kB/s), io=20.0MiB (20.0MB), run=4114-5135msec WRITE: bw=5584KiB/s (5718kB/s), 1245KiB/s-1994KiB/s (1274kB/s-2042kB/s), io=28.0MiB (29.4MB), run=4114-5135msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=155567: Wed Jul 16 14:19:00 2025 read: IOPS=1, BW=6990KiB/s (7158kB/s)(12.0MiB/1758msec) clat (msec): min=450, max=647, avg=568.30, stdev=103.91 lat (msec): min=450, max=647, avg=568.30, stdev=103.91 clat percentiles (msec): | 1.00th=[ 451], 5.00th=[ 451], 10.00th=[ 451], 20.00th=[ 451], | 30.00th=[ 451], 40.00th=[ 609], 50.00th=[ 609], 60.00th=[ 609], | 70.00th=[ 651], 80.00th=[ 651], 90.00th=[ 651], 95.00th=[ 651], | 99.00th=[ 651], 99.50th=[ 651], 99.90th=[ 651], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 7627, max= 7699, per=100.00%, avg=7663.00, stdev=50.91, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.00%, sys=48.04%, ctx=2118, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155568: Wed Jul 16 14:19:00 2025 read: IOPS=0, BW=1307KiB/s (1339kB/s)(4096KiB/3133msec) clat (nsec): min=427000k, max=427000k, avg=427000039.00, stdev= 0.00 lat (nsec): min=427002k, max=427002k, avg=427001849.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 426], 5.00th=[ 426], 10.00th=[ 426], 20.00th=[ 426], | 30.00th=[ 426], 40.00th=[ 426], 50.00th=[ 426], 60.00th=[ 426], | 70.00th=[ 426], 80.00th=[ 426], 90.00th=[ 426], 95.00th=[ 426], | 99.00th=[ 426], 99.50th=[ 426], 99.90th=[ 426], 99.95th=[ 426], | 99.99th=[ 426] bw ( KiB/s): min= 7816, max= 7816, per=100.00%, avg=7816.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2615KiB/s (2678kB/s)(8192KiB/3133msec); 0 zone resets clat (msec): min=881, max=1758, avg=1320.25, stdev=619.93 lat (msec): min=882, max=1758, avg=1320.55, stdev=619.98 clat percentiles (msec): | 1.00th=[ 885], 5.00th=[ 885], 10.00th=[ 885], 20.00th=[ 885], | 30.00th=[ 885], 40.00th=[ 885], 50.00th=[ 885], 60.00th=[ 1754], | 70.00th=[ 1754], 80.00th=[ 1754], 90.00th=[ 1754], 95.00th=[ 1754], | 99.00th=[ 1754], 99.50th=[ 1754], 99.90th=[ 1754], 99.95th=[ 1754], | 99.99th=[ 1754] bw ( KiB/s): min= 8015, max= 8015, per=76.64%, avg=8015.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=33.84%, ctx=945, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155569: Wed Jul 16 14:19:00 2025 write: IOPS=0, BW=4037KiB/s (4134kB/s)(12.0MiB/3044msec); 0 zone resets clat (msec): min=668, max=1543, avg=1006.19, stdev=470.71 lat (msec): min=668, max=1544, avg=1006.49, stdev=470.70 clat percentiles (msec): | 1.00th=[ 667], 5.00th=[ 667], 10.00th=[ 667], 20.00th=[ 667], | 30.00th=[ 667], 40.00th=[ 810], 50.00th=[ 810], 60.00th=[ 810], | 70.00th=[ 1552], 80.00th=[ 1552], 90.00th=[ 1552], 95.00th=[ 1552], | 99.00th=[ 1552], 99.50th=[ 1552], 99.90th=[ 1552], 99.95th=[ 1552], | 99.99th=[ 1552] bw ( KiB/s): min= 7699, max= 8015, per=75.13%, avg=7857.00, stdev=223.45, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.03%, sys=44.81%, ctx=119, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155570: Wed Jul 16 14:19:00 2025 write: IOPS=1, BW=4590KiB/s (4700kB/s)(12.0MiB/2677msec); 0 zone resets clat (msec): min=796, max=1028, avg=880.51, stdev=128.47 lat (msec): min=796, max=1028, avg=880.80, stdev=128.43 clat percentiles (msec): | 1.00th=[ 793], 5.00th=[ 793], 10.00th=[ 793], 20.00th=[ 793], | 30.00th=[ 793], 40.00th=[ 818], 50.00th=[ 818], 60.00th=[ 818], | 70.00th=[ 1028], 80.00th=[ 1028], 90.00th=[ 1028], 95.00th=[ 1028], | 99.00th=[ 1028], 99.50th=[ 1028], 99.90th=[ 1028], 99.95th=[ 1028], | 99.99th=[ 1028] bw ( KiB/s): min= 7314, max= 8015, per=73.29%, avg=7664.50, stdev=495.68, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=66.67%, 2000=33.33% cpu : usr=0.00%, sys=53.89%, ctx=287, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5229KiB/s (5355kB/s), 1307KiB/s-6990KiB/s (1339kB/s-7158kB/s), io=16.0MiB (16.8MB), run=1758-3133msec WRITE: bw=10.2MiB/s (10.7MB/s), 2615KiB/s-4590KiB/s (2678kB/s-4700kB/s), io=32.0MiB (33.6MB), run=2677-3133msec rand-rw: (groupid=0, jobs=1): err= 0: pid=155562: Wed Jul 16 14:19:01 2025 read: IOPS=1, BW=7576KiB/s (7758kB/s)(12.0MiB/1622msec) clat (msec): min=153, max=1169, avg=522.38, stdev=562.40 lat (msec): min=153, max=1169, avg=522.38, stdev=562.40 clat percentiles (msec): | 1.00th=[ 155], 5.00th=[ 155], 10.00th=[ 155], 20.00th=[ 155], | 30.00th=[ 155], 40.00th=[ 245], 50.00th=[ 245], 60.00th=[ 245], | 70.00th=[ 1167], 80.00th=[ 1167], 90.00th=[ 1167], 95.00th=[ 1167], | 99.00th=[ 1167], 99.50th=[ 1167], 99.90th=[ 1167], 99.95th=[ 1167], | 99.99th=[ 1167] bw ( KiB/s): min=14840, max=14840, per=100.00%, avg=14840.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, 2000=33.33% cpu : usr=0.00%, sys=17.95%, ctx=79, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155563: Wed Jul 16 14:19:01 2025 read: IOPS=0, BW=1170KiB/s (1198kB/s)(4096KiB/3500msec) clat (nsec): min=607456k, max=607456k, avg=607456458.00, stdev= 0.00 lat (nsec): min=607458k, max=607458k, avg=607457908.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 609], 5.00th=[ 609], 10.00th=[ 609], 20.00th=[ 609], | 30.00th=[ 609], 40.00th=[ 609], 50.00th=[ 609], 60.00th=[ 609], | 70.00th=[ 609], 80.00th=[ 609], 90.00th=[ 609], 95.00th=[ 609], | 99.00th=[ 609], 99.50th=[ 609], 99.90th=[ 609], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2341KiB/s (2397kB/s)(8192KiB/3500msec); 0 zone resets clat (msec): min=207, max=2628, avg=1418.02, stdev=1711.90 lat (msec): min=207, max=2628, avg=1418.24, stdev=1711.98 clat percentiles (msec): | 1.00th=[ 207], 5.00th=[ 207], 10.00th=[ 207], 20.00th=[ 207], | 30.00th=[ 207], 40.00th=[ 207], 50.00th=[ 207], 60.00th=[ 2635], | 70.00th=[ 2635], 80.00th=[ 2635], 90.00th=[ 2635], 95.00th=[ 2635], | 99.00th=[ 2635], 99.50th=[ 2635], 99.90th=[ 2635], 99.95th=[ 2635], | 99.99th=[ 2635] lat (msec) : 250=33.33%, 750=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=14.78%, ctx=74, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155564: Wed Jul 16 14:19:01 2025 write: IOPS=0, BW=3528KiB/s (3613kB/s)(12.0MiB/3483msec); 0 zone resets clat (msec): min=163, max=2007, avg=1140.62, stdev=926.93 lat (msec): min=164, max=2008, avg=1140.89, stdev=926.89 clat percentiles (msec): | 1.00th=[ 165], 5.00th=[ 165], 10.00th=[ 165], 20.00th=[ 165], | 30.00th=[ 165], 40.00th=[ 1250], 50.00th=[ 1250], 60.00th=[ 1250], | 70.00th=[ 2005], 80.00th=[ 2005], 90.00th=[ 2005], 95.00th=[ 2005], | 99.00th=[ 2005], 99.50th=[ 2005], 99.90th=[ 2005], 99.95th=[ 2005], | 99.99th=[ 2005] bw ( KiB/s): min= 7968, max= 7968, per=85.11%, avg=7968.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=16.28%, ctx=263, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=155565: Wed Jul 16 14:19:01 2025 write: IOPS=1, BW=5466KiB/s (5597kB/s)(12.0MiB/2248msec); 0 zone resets clat (msec): min=324, max=1296, avg=728.99, stdev=505.69 lat (msec): min=325, max=1296, avg=729.31, stdev=505.67 clat percentiles (msec): | 1.00th=[ 326], 5.00th=[ 326], 10.00th=[ 326], 20.00th=[ 326], | 30.00th=[ 326], 40.00th=[ 567], 50.00th=[ 567], 60.00th=[ 567], | 70.00th=[ 1301], 80.00th=[ 1301], 90.00th=[ 1301], 95.00th=[ 1301], | 99.00th=[ 1301], 99.50th=[ 1301], 99.90th=[ 1301], 99.95th=[ 1301], | 99.99th=[ 1301] bw ( KiB/s): min= 7433, max= 8192, per=83.45%, avg=7812.50, stdev=536.69, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=14.15%, ctx=51, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4681KiB/s (4793kB/s), 1170KiB/s-7576KiB/s (1198kB/s-7758kB/s), io=16.0MiB (16.8MB), run=1622-3500msec WRITE: bw=9362KiB/s (9587kB/s), 2341KiB/s-5466KiB/s (2397kB/s-5597kB/s), io=32.0MiB (33.6MB), run=2248-3500msec