== sanity test 398b: DIO and buffer IO race ============== 13:19:11 (1774891151) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.86857 s, 17.5 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147967: Mon Mar 30 13:20:19 2026 read: IOPS=25, BW=101KiB/s (104kB/s)(5992KiB/59100msec) clat (usec): min=136, max=648845, avg=20167.32, stdev=20963.40 lat (usec): min=137, max=648845, avg=20169.37, stdev=20963.67 clat percentiles (usec): | 1.00th=[ 227], 5.00th=[ 545], 10.00th=[ 7898], 20.00th=[ 10421], | 30.00th=[ 12518], 40.00th=[ 14746], 50.00th=[ 16909], 60.00th=[ 20317], | 70.00th=[ 23725], 80.00th=[ 28181], 90.00th=[ 34866], 95.00th=[ 42206], | 99.00th=[ 67634], 99.50th=[ 84411], 99.90th=[183501], 99.95th=[650118], | 99.99th=[650118] bw ( KiB/s): min= 8, max= 184, per=24.78%, avg=100.59, stdev=30.33, samples=117 iops : min= 2, max= 46, avg=24.68, stdev= 7.55, samples=117 write: IOPS=26, BW=107KiB/s (109kB/s)(6296KiB/59100msec); 0 zone resets clat (usec): min=915, max=125703, avg=18186.31, stdev=17211.63 lat (usec): min=916, max=125704, avg=18187.87, stdev=17211.81 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 1319], 10.00th=[ 1975], 20.00th=[ 3326], | 30.00th=[ 5735], 40.00th=[ 10552], 50.00th=[ 14353], 60.00th=[ 18220], | 70.00th=[ 23200], 80.00th=[ 29754], 90.00th=[ 40109], 95.00th=[ 49021], | 99.00th=[ 80217], 99.50th=[100140], 99.90th=[124257], 99.95th=[125305], | 99.99th=[125305] bw ( KiB/s): min= 32, max= 198, per=25.09%, avg=106.61, stdev=28.95, samples=116 iops : min= 8, max= 49, avg=26.19, stdev= 7.23, samples=116 lat (usec) : 250=0.81%, 500=1.37%, 750=0.81%, 1000=0.39% lat (msec) : 2=4.98%, 4=7.42%, 10=12.63%, 20=33.46%, 50=34.60% lat (msec) : 100=3.16%, 250=0.33%, 750=0.03% cpu : usr=0.11%, sys=15.47%, ctx=7917, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147968: Mon Mar 30 13:20:19 2026 read: IOPS=25, BW=102KiB/s (105kB/s)(5940KiB/58069msec) clat (usec): min=147, max=110261, avg=19699.05, stdev=12345.05 lat (usec): min=147, max=110262, avg=19702.29, stdev=12347.42 clat percentiles (usec): | 1.00th=[ 237], 5.00th=[ 627], 10.00th=[ 8291], 20.00th=[ 10945], | 30.00th=[ 12780], 40.00th=[ 15008], 50.00th=[ 17171], 60.00th=[ 20317], | 70.00th=[ 23725], 80.00th=[ 27919], 90.00th=[ 34341], 95.00th=[ 40633], | 99.00th=[ 67634], 99.50th=[ 73925], 99.90th=[100140], 99.95th=[110625], | 99.99th=[110625] bw ( KiB/s): min= 8, max= 167, per=25.12%, avg=102.00, stdev=28.69, samples=115 iops : min= 2, max= 41, avg=25.03, stdev= 7.14, samples=115 write: IOPS=27, BW=109KiB/s (112kB/s)(6348KiB/58069msec); 0 zone resets clat (usec): min=778, max=644332, avg=17986.35, stdev=23366.57 lat (usec): min=779, max=644334, avg=17988.16, stdev=23367.10 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 1549], 10.00th=[ 2073], 20.00th=[ 3294], | 30.00th=[ 5276], 40.00th=[ 9896], 50.00th=[ 13566], 60.00th=[ 17433], | 70.00th=[ 22152], 80.00th=[ 28967], 90.00th=[ 38536], 95.00th=[ 48497], | 99.00th=[ 74974], 99.50th=[111674], 99.90th=[156238], 99.95th=[641729], | 99.99th=[641729] bw ( KiB/s): min= 16, max= 200, per=25.59%, avg=108.77, stdev=31.96, samples=115 iops : min= 4, max= 50, avg=26.74, stdev= 7.98, samples=115 lat (usec) : 250=0.62%, 500=1.27%, 750=0.68%, 1000=0.39% lat (msec) : 2=4.65%, 4=8.14%, 10=12.53%, 20=34.02%, 50=34.51% lat (msec) : 100=2.86%, 250=0.29%, 750=0.03% cpu : usr=0.15%, sys=15.77%, ctx=7801, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147969: Mon Mar 30 13:20:19 2026 read: IOPS=25, BW=104KiB/s (106kB/s)(6028KiB/58158msec) clat (usec): min=150, max=661546, avg=19928.07, stdev=21609.54 lat (usec): min=151, max=661547, avg=19929.16, stdev=21609.52 clat percentiles (usec): | 1.00th=[ 223], 5.00th=[ 611], 10.00th=[ 8029], 20.00th=[ 10552], | 30.00th=[ 12387], 40.00th=[ 14222], 50.00th=[ 16712], 60.00th=[ 19792], | 70.00th=[ 22938], 80.00th=[ 27132], 90.00th=[ 34341], 95.00th=[ 40633], | 99.00th=[ 69731], 99.50th=[ 93848], 99.90th=[164627], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 8, max= 192, per=25.37%, avg=102.98, stdev=31.94, samples=115 iops : min= 2, max= 48, avg=25.28, stdev= 8.00, samples=115 write: IOPS=26, BW=108KiB/s (110kB/s)(6260KiB/58158msec); 0 zone resets clat (usec): min=638, max=197098, avg=17810.07, stdev=16331.43 lat (usec): min=638, max=197099, avg=17811.23, stdev=16331.42 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 1483], 10.00th=[ 2278], 20.00th=[ 3458], | 30.00th=[ 5932], 40.00th=[ 10683], 50.00th=[ 14484], 60.00th=[ 18220], | 70.00th=[ 22938], 80.00th=[ 28967], 90.00th=[ 38011], 95.00th=[ 46400], | 99.00th=[ 68682], 99.50th=[ 81265], 99.90th=[156238], 99.95th=[196084], | 99.99th=[196084] bw ( KiB/s): min= 47, max= 168, per=25.33%, avg=107.66, stdev=29.33, samples=114 iops : min= 11, max= 42, avg=26.45, stdev= 7.39, samples=114 lat (usec) : 250=0.68%, 500=1.46%, 750=0.62%, 1000=0.59% lat (msec) : 2=4.07%, 4=7.58%, 10=13.09%, 20=34.54%, 50=34.18% lat (msec) : 100=2.83%, 250=0.33%, 750=0.03% cpu : usr=0.11%, sys=15.75%, ctx=7940, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147970: Mon Mar 30 13:20:19 2026 read: IOPS=25, BW=103KiB/s (105kB/s)(6052KiB/58778msec) clat (usec): min=134, max=645252, avg=19715.75, stdev=20794.07 lat (usec): min=134, max=645253, avg=19716.87, stdev=20793.98 clat percentiles (usec): | 1.00th=[ 221], 5.00th=[ 506], 10.00th=[ 7504], 20.00th=[ 10028], | 30.00th=[ 12256], 40.00th=[ 14353], 50.00th=[ 16712], 60.00th=[ 19792], | 70.00th=[ 23200], 80.00th=[ 27395], 90.00th=[ 34341], 95.00th=[ 41681], | 99.00th=[ 62653], 99.50th=[ 83362], 99.90th=[139461], 99.95th=[641729], | 99.99th=[641729] bw ( KiB/s): min= 8, max= 192, per=25.08%, avg=101.83, stdev=27.64, samples=116 iops : min= 2, max= 48, avg=24.99, stdev= 6.95, samples=116 write: IOPS=26, BW=106KiB/s (109kB/s)(6236KiB/58778msec); 0 zone resets clat (usec): min=674, max=166028, avg=18397.14, stdev=16875.02 lat (usec): min=674, max=166029, avg=18398.36, stdev=16875.05 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 1385], 10.00th=[ 2114], 20.00th=[ 3359], | 30.00th=[ 5604], 40.00th=[ 10421], 50.00th=[ 14746], 60.00th=[ 19006], | 70.00th=[ 24249], 80.00th=[ 30540], 90.00th=[ 40633], 95.00th=[ 49021], | 99.00th=[ 68682], 99.50th=[ 80217], 99.90th=[154141], 99.95th=[166724], | 99.99th=[166724] bw ( KiB/s): min= 40, max= 176, per=24.96%, avg=106.09, stdev=28.66, samples=115 iops : min= 10, max= 44, avg=26.06, stdev= 7.20, samples=115 lat (usec) : 250=0.62%, 500=1.82%, 750=0.81%, 1000=0.46% lat (msec) : 2=4.46%, 4=7.52%, 10=14.03%, 20=31.61%, 50=35.09% lat (msec) : 100=3.32%, 250=0.23%, 750=0.03% cpu : usr=0.12%, sys=15.49%, ctx=7856, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=406KiB/s (416kB/s), 101KiB/s-104KiB/s (104kB/s-106kB/s), io=23.4MiB (24.6MB), run=58069-59100msec WRITE: bw=425KiB/s (436kB/s), 106KiB/s-109KiB/s (109kB/s-112kB/s), io=24.6MiB (25.7MB), run=58069-59100msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147972: Mon Mar 30 13:21:09 2026 read: IOPS=13, BW=54.9KiB/s (56.2kB/s)(5992KiB/109216msec) clat (msec): min=3, max=603, avg=19.93, stdev=21.53 lat (msec): min=3, max=603, avg=19.93, stdev=21.53 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 13], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 23], 80.00th=[ 28], 90.00th=[ 37], 95.00th=[ 47], | 99.00th=[ 81], 99.50th=[ 110], 99.90th=[ 153], 99.95th=[ 600], | 99.99th=[ 600] bw ( KiB/s): min= 7, max= 168, per=25.04%, avg=54.83, stdev=32.18, samples=215 iops : min= 1, max= 42, avg=13.34, stdev= 8.12, samples=215 write: IOPS=14, BW=57.6KiB/s (59.0kB/s)(6296KiB/109216msec); 0 zone resets clat (msec): min=15, max=241, avg=50.37, stdev=25.72 lat (msec): min=15, max=241, avg=50.37, stdev=25.72 clat percentiles (msec): | 1.00th=[ 19], 5.00th=[ 23], 10.00th=[ 26], 20.00th=[ 30], | 30.00th=[ 34], 40.00th=[ 39], 50.00th=[ 43], 60.00th=[ 50], | 70.00th=[ 58], 80.00th=[ 70], 90.00th=[ 84], 95.00th=[ 100], | 99.00th=[ 134], 99.50th=[ 159], 99.90th=[ 203], 99.95th=[ 241], | 99.99th=[ 241] bw ( KiB/s): min= 15, max= 128, per=24.98%, avg=57.46, stdev=24.36, samples=216 iops : min= 3, max= 32, avg=14.00, stdev= 6.17, samples=216 lat (msec) : 4=0.10%, 10=14.91%, 20=15.72%, 50=47.14%, 100=19.40% lat (msec) : 250=2.70%, 750=0.03% cpu : usr=0.09%, sys=5.05%, ctx=6382, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147973: Mon Mar 30 13:21:09 2026 read: IOPS=13, BW=54.4KiB/s (55.7kB/s)(5940KiB/109285msec) clat (msec): min=3, max=669, avg=20.08, stdev=22.97 lat (msec): min=3, max=669, avg=20.08, stdev=22.97 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 23], 80.00th=[ 28], 90.00th=[ 37], 95.00th=[ 44], | 99.00th=[ 90], 99.50th=[ 114], 99.90th=[ 169], 99.95th=[ 667], | 99.99th=[ 667] bw ( KiB/s): min= 7, max= 151, per=25.07%, avg=54.90, stdev=30.74, samples=213 iops : min= 1, max= 37, avg=13.36, stdev= 7.75, samples=213 write: IOPS=14, BW=58.1KiB/s (59.5kB/s)(6348KiB/109285msec); 0 zone resets clat (msec): min=13, max=205, avg=50.02, stdev=25.29 lat (msec): min=13, max=205, avg=50.03, stdev=25.29 clat percentiles (msec): | 1.00th=[ 19], 5.00th=[ 23], 10.00th=[ 26], 20.00th=[ 30], | 30.00th=[ 34], 40.00th=[ 39], 50.00th=[ 43], 60.00th=[ 51], | 70.00th=[ 57], 80.00th=[ 68], 90.00th=[ 86], 95.00th=[ 101], | 99.00th=[ 132], 99.50th=[ 155], 99.90th=[ 201], 99.95th=[ 205], | 99.99th=[ 205] bw ( KiB/s): min= 7, max= 120, per=25.11%, avg=57.76, stdev=24.60, samples=216 iops : min= 1, max= 30, avg=14.07, stdev= 6.22, samples=216 lat (msec) : 4=0.26%, 10=14.26%, 20=16.47%, 50=46.58%, 100=19.40% lat (msec) : 250=2.99%, 750=0.03% cpu : usr=0.07%, sys=5.07%, ctx=6226, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147974: Mon Mar 30 13:21:09 2026 read: IOPS=13, BW=55.2KiB/s (56.6kB/s)(6028KiB/109145msec) clat (msec): min=3, max=159, avg=20.27, stdev=15.19 lat (msec): min=3, max=159, avg=20.27, stdev=15.19 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 18], 60.00th=[ 21], | 70.00th=[ 24], 80.00th=[ 29], 90.00th=[ 39], 95.00th=[ 48], | 99.00th=[ 74], 99.50th=[ 87], 99.90th=[ 127], 99.95th=[ 159], | 99.99th=[ 159] bw ( KiB/s): min= 7, max= 160, per=25.39%, avg=55.61, stdev=29.73, samples=214 iops : min= 1, max= 40, avg=13.54, stdev= 7.52, samples=214 write: IOPS=14, BW=57.4KiB/s (58.7kB/s)(6260KiB/109145msec); 0 zone resets clat (msec): min=15, max=765, avg=50.17, stdev=31.27 lat (msec): min=15, max=765, avg=50.18, stdev=31.27 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 23], 10.00th=[ 25], 20.00th=[ 30], | 30.00th=[ 33], 40.00th=[ 37], 50.00th=[ 43], 60.00th=[ 49], | 70.00th=[ 58], 80.00th=[ 69], 90.00th=[ 86], 95.00th=[ 100], | 99.00th=[ 133], 99.50th=[ 153], 99.90th=[ 218], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 8, max= 120, per=24.87%, avg=57.20, stdev=25.30, samples=216 iops : min= 2, max= 30, avg=13.94, stdev= 6.39, samples=216 lat (msec) : 4=0.13%, 10=13.87%, 20=15.66%, 50=49.06%, 100=18.72% lat (msec) : 250=2.54%, 1000=0.03% cpu : usr=0.07%, sys=5.12%, ctx=6395, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147975: Mon Mar 30 13:21:09 2026 read: IOPS=13, BW=55.5KiB/s (56.8kB/s)(6052KiB/109045msec) clat (msec): min=3, max=170, avg=19.55, stdev=15.37 lat (msec): min=3, max=170, avg=19.55, stdev=15.37 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 23], 80.00th=[ 28], 90.00th=[ 36], 95.00th=[ 46], | 99.00th=[ 78], 99.50th=[ 101], 99.90th=[ 159], 99.95th=[ 171], | 99.99th=[ 171] bw ( KiB/s): min= 7, max= 159, per=25.56%, avg=55.97, stdev=30.75, samples=213 iops : min= 1, max= 39, avg=13.62, stdev= 7.75, samples=213 write: IOPS=14, BW=57.2KiB/s (58.6kB/s)(6236KiB/109045msec); 0 zone resets clat (msec): min=14, max=761, avg=50.92, stdev=31.92 lat (msec): min=14, max=761, avg=50.92, stdev=31.92 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 23], 10.00th=[ 26], 20.00th=[ 30], | 30.00th=[ 34], 40.00th=[ 39], 50.00th=[ 44], 60.00th=[ 50], | 70.00th=[ 59], 80.00th=[ 68], 90.00th=[ 85], 95.00th=[ 101], | 99.00th=[ 148], 99.50th=[ 169], 99.90th=[ 243], 99.95th=[ 760], | 99.99th=[ 760] bw ( KiB/s): min= 23, max= 120, per=24.82%, avg=57.09, stdev=24.27, samples=215 iops : min= 5, max= 30, avg=13.90, stdev= 6.13, samples=215 lat (msec) : 4=0.07%, 10=14.45%, 20=16.80%, 50=46.74%, 100=19.24% lat (msec) : 250=2.67%, 1000=0.03% cpu : usr=0.07%, sys=5.19%, ctx=6358, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=220KiB/s (225kB/s), 54.4KiB/s-55.5KiB/s (55.7kB/s-56.8kB/s), io=23.4MiB (24.6MB), run=109045-109285msec WRITE: bw=230KiB/s (236kB/s), 57.2KiB/s-58.1KiB/s (58.6kB/s-59.5kB/s), io=24.6MiB (25.7MB), run=109045-109285msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148009: Mon Mar 30 13:21:28 2026 read: IOPS=27, BW=447KiB/s (458kB/s)(5920KiB/13242msec) clat (usec): min=112, max=136203, avg=17934.62, stdev=13127.87 lat (usec): min=112, max=136204, avg=17935.85, stdev=13127.89 clat percentiles (usec): | 1.00th=[ 221], 5.00th=[ 416], 10.00th=[ 562], 20.00th=[ 10159], | 30.00th=[ 12518], 40.00th=[ 14353], 50.00th=[ 16712], 60.00th=[ 18744], | 70.00th=[ 21627], 80.00th=[ 25822], 90.00th=[ 31589], 95.00th=[ 39584], | 99.00th=[ 57410], 99.50th=[ 70779], 99.90th=[135267], 99.95th=[135267], | 99.99th=[135267] bw ( KiB/s): min= 253, max= 672, per=25.29%, avg=445.35, stdev=104.60, samples=26 iops : min= 15, max= 42, avg=27.35, stdev= 6.57, samples=26 write: IOPS=30, BW=481KiB/s (492kB/s)(6368KiB/13242msec); 0 zone resets clat (usec): min=1200, max=92407, avg=16419.72, stdev=15181.69 lat (usec): min=1200, max=92408, avg=16427.25, stdev=15193.00 clat percentiles (usec): | 1.00th=[ 1532], 5.00th=[ 2376], 10.00th=[ 3294], 20.00th=[ 4424], | 30.00th=[ 5473], 40.00th=[ 7373], 50.00th=[11076], 60.00th=[15401], | 70.00th=[21365], 80.00th=[27132], 90.00th=[35914], 95.00th=[46400], | 99.00th=[68682], 99.50th=[90702], 99.90th=[92799], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 224, max= 672, per=26.01%, avg=482.04, stdev=130.52, samples=26 iops : min= 14, max= 42, avg=29.65, stdev= 8.11, samples=26 lat (usec) : 250=0.78%, 500=3.39%, 750=2.34%, 1000=0.65% lat (msec) : 2=1.82%, 4=7.16%, 10=17.84%, 20=31.51%, 50=32.16% lat (msec) : 100=2.21%, 250=0.13% cpu : usr=0.13%, sys=22.30%, ctx=2187, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148010: Mon Mar 30 13:21:28 2026 read: IOPS=27, BW=434KiB/s (445kB/s)(5744KiB/13227msec) clat (usec): min=132, max=99466, avg=19474.90, stdev=12382.39 lat (usec): min=132, max=99468, avg=19475.77, stdev=12382.42 clat percentiles (usec): | 1.00th=[ 206], 5.00th=[ 441], 10.00th=[ 1012], 20.00th=[10945], | 30.00th=[13829], 40.00th=[15795], 50.00th=[17695], 60.00th=[20055], | 70.00th=[23462], 80.00th=[27657], 90.00th=[34341], 95.00th=[41157], | 99.00th=[63177], 99.50th=[72877], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 254, max= 636, per=24.61%, avg=433.31, stdev=116.40, samples=26 iops : min= 15, max= 39, avg=26.62, stdev= 7.17, samples=26 write: IOPS=30, BW=495KiB/s (507kB/s)(6544KiB/13227msec); 0 zone resets clat (usec): min=1293, max=77785, avg=15155.62, stdev=12460.54 lat (usec): min=1294, max=77788, avg=15157.10, stdev=12460.61 clat percentiles (usec): | 1.00th=[ 1549], 5.00th=[ 2311], 10.00th=[ 3064], 20.00th=[ 4359], | 30.00th=[ 5735], 40.00th=[ 8586], 50.00th=[11600], 60.00th=[15401], | 70.00th=[19530], 80.00th=[24511], 90.00th=[32375], 95.00th=[39584], | 99.00th=[52167], 99.50th=[60556], 99.90th=[78119], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 192, max= 768, per=26.68%, avg=494.38, stdev=132.42, samples=26 iops : min= 12, max= 48, avg=30.42, stdev= 8.36, samples=26 lat (usec) : 250=1.04%, 500=1.56%, 750=1.82%, 1000=0.13% lat (msec) : 2=2.21%, 4=6.90%, 10=16.80%, 20=35.55%, 50=32.03% lat (msec) : 100=1.95% cpu : usr=0.14%, sys=22.43%, ctx=2121, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148011: Mon Mar 30 13:21:28 2026 read: IOPS=28, BW=460KiB/s (471kB/s)(6256KiB/13594msec) clat (usec): min=109, max=179350, avg=18629.65, stdev=14254.52 lat (usec): min=110, max=179351, avg=18630.54, stdev=14254.52 clat percentiles (usec): | 1.00th=[ 165], 5.00th=[ 351], 10.00th=[ 562], 20.00th=[ 11076], | 30.00th=[ 13435], 40.00th=[ 14877], 50.00th=[ 16909], 60.00th=[ 19530], | 70.00th=[ 22676], 80.00th=[ 26084], 90.00th=[ 31851], 95.00th=[ 39060], | 99.00th=[ 62653], 99.50th=[ 89654], 99.90th=[179307], 99.95th=[179307], | 99.99th=[179307] bw ( KiB/s): min= 288, max= 766, per=25.71%, avg=452.81, stdev=114.58, samples=26 iops : min= 18, max= 47, avg=27.85, stdev= 6.96, samples=26 write: IOPS=27, BW=444KiB/s (454kB/s)(6032KiB/13594msec); 0 zone resets clat (usec): min=1080, max=174765, avg=16465.20, stdev=16459.39 lat (usec): min=1081, max=174766, avg=16466.80, stdev=16459.42 clat percentiles (usec): | 1.00th=[ 1500], 5.00th=[ 2212], 10.00th=[ 3163], 20.00th=[ 4555], | 30.00th=[ 5866], 40.00th=[ 8291], 50.00th=[ 12256], 60.00th=[ 15008], | 70.00th=[ 19530], 80.00th=[ 25822], 90.00th=[ 37487], 95.00th=[ 42730], | 99.00th=[ 79168], 99.50th=[ 87557], 99.90th=[175113], 99.95th=[175113], | 99.99th=[175113] bw ( KiB/s): min= 191, max= 704, per=22.97%, avg=425.62, stdev=131.51, samples=26 iops : min= 11, max= 44, avg=26.12, stdev= 8.26, samples=26 lat (usec) : 250=1.95%, 500=2.21%, 750=1.95%, 1000=0.52% lat (msec) : 2=2.47%, 4=5.73%, 10=16.02%, 20=35.03%, 50=32.29% lat (msec) : 100=1.56%, 250=0.26% cpu : usr=0.13%, sys=21.38%, ctx=2118, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148012: Mon Mar 30 13:21:28 2026 read: IOPS=28, BW=450KiB/s (461kB/s)(6032KiB/13393msec) clat (usec): min=118, max=147006, avg=18486.97, stdev=13751.10 lat (usec): min=118, max=147007, avg=18497.81, stdev=13757.75 clat percentiles (usec): | 1.00th=[ 163], 5.00th=[ 277], 10.00th=[ 465], 20.00th=[ 10290], | 30.00th=[ 13304], 40.00th=[ 15008], 50.00th=[ 17433], 60.00th=[ 20317], | 70.00th=[ 22938], 80.00th=[ 26346], 90.00th=[ 32375], 95.00th=[ 40109], | 99.00th=[ 62653], 99.50th=[ 71828], 99.90th=[147850], 99.95th=[147850], | 99.99th=[147850] bw ( KiB/s): min= 187, max= 606, per=25.35%, avg=446.42, stdev=101.55, samples=26 iops : min= 11, max= 37, avg=27.42, stdev= 6.41, samples=26 write: IOPS=29, BW=467KiB/s (478kB/s)(6256KiB/13393msec); 0 zone resets clat (usec): min=1233, max=106133, avg=16222.61, stdev=14904.97 lat (usec): min=1235, max=106136, avg=16225.54, stdev=14904.64 clat percentiles (usec): | 1.00th=[ 1500], 5.00th=[ 2212], 10.00th=[ 3064], 20.00th=[ 4080], | 30.00th=[ 5473], 40.00th=[ 7832], 50.00th=[ 10945], 60.00th=[ 16057], | 70.00th=[ 20841], 80.00th=[ 26084], 90.00th=[ 33424], 95.00th=[ 50594], | 99.00th=[ 60556], 99.50th=[ 76022], 99.90th=[106431], 99.95th=[106431], | 99.99th=[106431] bw ( KiB/s): min= 267, max= 734, per=25.12%, avg=465.46, stdev=119.60, samples=26 iops : min= 16, max= 45, avg=28.62, stdev= 7.41, samples=26 lat (usec) : 250=2.21%, 500=3.12%, 750=2.21%, 1000=0.26% lat (msec) : 2=2.47%, 4=8.20%, 10=14.58%, 20=30.47%, 50=32.94% lat (msec) : 100=3.26%, 250=0.26% cpu : usr=0.13%, sys=21.42%, ctx=2136, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1762KiB/s (1804kB/s), 434KiB/s-460KiB/s (445kB/s-471kB/s), io=23.4MiB (24.5MB), run=13227-13594msec WRITE: bw=1854KiB/s (1898kB/s), 444KiB/s-495KiB/s (454kB/s-507kB/s), io=24.6MiB (25.8MB), run=13227-13594msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148004: Mon Mar 30 13:21:43 2026 read: IOPS=13, BW=209KiB/s (214kB/s)(5920KiB/28296msec) clat (usec): min=3810, max=98811, avg=20400.17, stdev=14990.60 lat (usec): min=3811, max=98812, avg=20401.12, stdev=14990.57 clat percentiles (usec): | 1.00th=[ 4490], 5.00th=[ 6194], 10.00th=[ 6980], 20.00th=[ 8717], | 30.00th=[10945], 40.00th=[12911], 50.00th=[15270], 60.00th=[19792], | 70.00th=[23725], 80.00th=[29492], 90.00th=[40633], 95.00th=[49546], | 99.00th=[76022], 99.50th=[90702], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 63, max= 512, per=24.47%, avg=207.00, stdev=89.35, samples=56 iops : min= 3, max= 32, avg=12.41, stdev= 5.67, samples=56 write: IOPS=14, BW=225KiB/s (230kB/s)(6368KiB/28296msec); 0 zone resets clat (msec): min=16, max=174, avg=52.04, stdev=22.89 lat (msec): min=16, max=174, avg=52.04, stdev=22.89 clat percentiles (msec): | 1.00th=[ 21], 5.00th=[ 27], 10.00th=[ 30], 20.00th=[ 35], | 30.00th=[ 40], 40.00th=[ 43], 50.00th=[ 47], 60.00th=[ 52], | 70.00th=[ 58], 80.00th=[ 66], 90.00th=[ 78], 95.00th=[ 96], | 99.00th=[ 144], 99.50th=[ 174], 99.90th=[ 176], 99.95th=[ 176], | 99.99th=[ 176] bw ( KiB/s): min= 63, max= 415, per=25.15%, avg=223.82, stdev=80.57, samples=56 iops : min= 3, max= 25, avg=13.46, stdev= 5.10, samples=56 lat (msec) : 4=0.13%, 10=12.37%, 20=17.06%, 50=44.92%, 100=23.18% lat (msec) : 250=2.34% cpu : usr=0.06%, sys=5.68%, ctx=1624, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148005: Mon Mar 30 13:21:43 2026 read: IOPS=12, BW=204KiB/s (209kB/s)(5744KiB/28196msec) clat (usec): min=4350, max=99090, avg=19362.52, stdev=14546.18 lat (usec): min=4351, max=99091, avg=19363.59, stdev=14546.17 clat percentiles (usec): | 1.00th=[ 5604], 5.00th=[ 6390], 10.00th=[ 7439], 20.00th=[ 8979], | 30.00th=[10552], 40.00th=[12256], 50.00th=[14091], 60.00th=[17171], | 70.00th=[21627], 80.00th=[27657], 90.00th=[39060], 95.00th=[50594], | 99.00th=[79168], 99.50th=[93848], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 31, max= 479, per=24.59%, avg=208.02, stdev=104.11, samples=55 iops : min= 1, max= 29, avg=12.47, stdev= 6.50, samples=55 write: IOPS=14, BW=232KiB/s (238kB/s)(6544KiB/28196msec); 0 zone resets clat (msec): min=16, max=174, avg=51.80, stdev=22.48 lat (msec): min=16, max=174, avg=51.80, stdev=22.48 clat percentiles (msec): | 1.00th=[ 22], 5.00th=[ 27], 10.00th=[ 29], 20.00th=[ 34], | 30.00th=[ 39], 40.00th=[ 43], 50.00th=[ 48], 60.00th=[ 53], | 70.00th=[ 58], 80.00th=[ 67], 90.00th=[ 82], 95.00th=[ 96], | 99.00th=[ 123], 99.50th=[ 146], 99.90th=[ 176], 99.95th=[ 176], | 99.99th=[ 176] bw ( KiB/s): min= 95, max= 480, per=26.01%, avg=231.48, stdev=78.78, samples=56 iops : min= 5, max= 30, avg=13.93, stdev= 5.01, samples=56 lat (msec) : 10=12.76%, 20=18.23%, 50=42.71%, 100=24.22%, 250=2.08% cpu : usr=0.07%, sys=5.73%, ctx=1632, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148007: Mon Mar 30 13:21:43 2026 read: IOPS=14, BW=227KiB/s (232kB/s)(6256KiB/27572msec) clat (usec): min=4814, max=97098, avg=20918.69, stdev=16123.06 lat (usec): min=4815, max=97099, avg=20919.93, stdev=16122.86 clat percentiles (usec): | 1.00th=[ 5538], 5.00th=[ 6390], 10.00th=[ 7504], 20.00th=[ 9110], | 30.00th=[10552], 40.00th=[12649], 50.00th=[15401], 60.00th=[18220], | 70.00th=[22938], 80.00th=[29492], 90.00th=[44303], 95.00th=[57934], | 99.00th=[77071], 99.50th=[94897], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 63, max= 608, per=27.09%, avg=229.17, stdev=117.05, samples=54 iops : min= 3, max= 38, avg=13.80, stdev= 7.40, samples=54 write: IOPS=13, BW=219KiB/s (224kB/s)(6032KiB/27572msec); 0 zone resets clat (msec): min=18, max=154, avg=51.24, stdev=21.43 lat (msec): min=18, max=154, avg=51.24, stdev=21.43 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 28], 10.00th=[ 30], 20.00th=[ 35], | 30.00th=[ 39], 40.00th=[ 43], 50.00th=[ 48], 60.00th=[ 53], | 70.00th=[ 57], 80.00th=[ 64], 90.00th=[ 73], 95.00th=[ 96], | 99.00th=[ 128], 99.50th=[ 133], 99.90th=[ 155], 99.95th=[ 155], | 99.99th=[ 155] bw ( KiB/s): min= 31, max= 352, per=24.35%, avg=216.74, stdev=70.64, samples=54 iops : min= 1, max= 22, avg=13.02, stdev= 4.49, samples=54 lat (msec) : 10=12.37%, 20=19.40%, 50=42.84%, 100=23.05%, 250=2.34% cpu : usr=0.08%, sys=5.94%, ctx=1620, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148008: Mon Mar 30 13:21:43 2026 read: IOPS=13, BW=214KiB/s (220kB/s)(6032KiB/28135msec) clat (msec): min=4, max=104, avg=20.36, stdev=16.31 lat (msec): min=4, max=104, avg=20.36, stdev=16.31 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 28], 90.00th=[ 44], 95.00th=[ 58], | 99.00th=[ 97], 99.50th=[ 101], 99.90th=[ 105], 99.95th=[ 105], | 99.99th=[ 105] bw ( KiB/s): min= 63, max= 447, per=24.66%, avg=208.65, stdev=87.02, samples=55 iops : min= 3, max= 27, avg=12.49, stdev= 5.44, samples=55 write: IOPS=13, BW=222KiB/s (228kB/s)(6256KiB/28135msec); 0 zone resets clat (msec): min=17, max=196, avg=52.21, stdev=22.13 lat (msec): min=17, max=196, avg=52.21, stdev=22.13 clat percentiles (msec): | 1.00th=[ 21], 5.00th=[ 27], 10.00th=[ 30], 20.00th=[ 35], | 30.00th=[ 41], 40.00th=[ 44], 50.00th=[ 48], 60.00th=[ 53], | 70.00th=[ 58], 80.00th=[ 67], 90.00th=[ 80], 95.00th=[ 97], | 99.00th=[ 126], 99.50th=[ 138], 99.90th=[ 197], 99.95th=[ 197], | 99.99th=[ 197] bw ( KiB/s): min= 95, max= 416, per=24.67%, avg=219.55, stdev=75.81, samples=55 iops : min= 5, max= 26, avg=13.18, stdev= 4.79, samples=55 lat (msec) : 10=12.76%, 20=19.92%, 50=41.80%, 100=23.05%, 250=2.47% cpu : usr=0.08%, sys=5.86%, ctx=1656, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=846KiB/s (867kB/s), 204KiB/s-227KiB/s (209kB/s-232kB/s), io=23.4MiB (24.5MB), run=27572-28296msec WRITE: bw=891KiB/s (912kB/s), 219KiB/s-232KiB/s (224kB/s-238kB/s), io=24.6MiB (25.8MB), run=27572-28296msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148022: Mon Mar 30 13:21:52 2026 read: IOPS=2, BW=2152KiB/s (2204kB/s)(7168KiB/3331msec) clat (msec): min=64, max=1120, avg=275.42, stdev=376.89 lat (msec): min=64, max=1120, avg=275.42, stdev=376.89 clat percentiles (msec): | 1.00th=[ 65], 5.00th=[ 65], 10.00th=[ 65], 20.00th=[ 92], | 30.00th=[ 122], 40.00th=[ 122], 50.00th=[ 131], 60.00th=[ 159], | 70.00th=[ 159], 80.00th=[ 241], 90.00th=[ 1116], 95.00th=[ 1116], | 99.00th=[ 1116], 99.50th=[ 1116], 99.90th=[ 1116], 99.95th=[ 1116], | 99.99th=[ 1116] bw ( KiB/s): min= 1783, max= 6131, per=58.44%, avg=2997.00, stdev=2092.76, samples=4 iops : min= 1, max= 5, avg= 2.00, stdev= 2.00, samples=4 write: IOPS=1, BW=1537KiB/s (1574kB/s)(5120KiB/3331msec); 0 zone resets clat (msec): min=95, max=433, avg=278.45, stdev=132.85 lat (msec): min=95, max=433, avg=278.52, stdev=132.87 clat percentiles (msec): | 1.00th=[ 95], 5.00th=[ 95], 10.00th=[ 95], 20.00th=[ 95], | 30.00th=[ 199], 40.00th=[ 199], 50.00th=[ 317], 60.00th=[ 317], | 70.00th=[ 347], 80.00th=[ 347], 90.00th=[ 435], 95.00th=[ 435], | 99.00th=[ 435], 99.50th=[ 435], 99.90th=[ 435], 99.95th=[ 435], | 99.99th=[ 435] bw ( KiB/s): min= 1976, max= 2043, per=28.18%, avg=2023.25, stdev=31.67, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 lat (msec) : 100=25.00%, 250=41.67%, 500=25.00%, 2000=8.33% cpu : usr=0.00%, sys=28.08%, ctx=927, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148023: Mon Mar 30 13:21:52 2026 read: IOPS=0, BW=513KiB/s (525kB/s)(2048KiB/3993msec) clat (msec): min=144, max=147, avg=146.09, stdev= 1.82 lat (msec): min=144, max=147, avg=146.09, stdev= 1.82 clat percentiles (msec): | 1.00th=[ 146], 5.00th=[ 146], 10.00th=[ 146], 20.00th=[ 146], | 30.00th=[ 146], 40.00th=[ 146], 50.00th=[ 146], 60.00th=[ 148], | 70.00th=[ 148], 80.00th=[ 148], 90.00th=[ 148], 95.00th=[ 148], | 99.00th=[ 148], 99.50th=[ 148], 99.90th=[ 148], 99.95th=[ 148], | 99.99th=[ 148] bw ( KiB/s): min= 1777, max= 2035, per=37.17%, avg=1906.00, stdev=182.43, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2564KiB/s (2626kB/s)(10.0MiB/3993msec); 0 zone resets clat (msec): min=100, max=1350, avg=364.56, stdev=377.98 lat (msec): min=100, max=1350, avg=364.64, stdev=377.97 clat percentiles (msec): | 1.00th=[ 101], 5.00th=[ 101], 10.00th=[ 101], 20.00th=[ 126], | 30.00th=[ 157], 40.00th=[ 163], 50.00th=[ 182], 60.00th=[ 211], | 70.00th=[ 342], 80.00th=[ 481], 90.00th=[ 542], 95.00th=[ 1351], | 99.00th=[ 1351], 99.50th=[ 1351], 99.90th=[ 1351], 99.95th=[ 1351], | 99.99th=[ 1351] bw ( KiB/s): min= 1976, max= 6119, per=45.32%, avg=3253.80, stdev=1837.32, samples=5 iops : min= 1, max= 5, avg= 2.40, stdev= 1.95, samples=5 lat (msec) : 250=66.67%, 500=16.67%, 750=8.33%, 2000=8.33% cpu : usr=0.03%, sys=28.71%, ctx=267, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148024: Mon Mar 30 13:21:52 2026 read: IOPS=1, BW=1742KiB/s (1784kB/s)(6144KiB/3527msec) clat (usec): min=994, max=419226, avg=156046.91, stdev=147301.98 lat (usec): min=995, max=419228, avg=156048.23, stdev=147301.94 clat percentiles (usec): | 1.00th=[ 996], 5.00th=[ 996], 10.00th=[ 996], 20.00th=[ 37487], | 30.00th=[ 37487], 40.00th=[133694], 50.00th=[133694], 60.00th=[166724], | 70.00th=[177210], 80.00th=[177210], 90.00th=[417334], 95.00th=[417334], | 99.00th=[417334], 99.50th=[417334], 99.90th=[417334], 99.95th=[417334], | 99.99th=[417334] bw ( KiB/s): min= 2039, max= 5930, per=77.70%, avg=3984.50, stdev=2751.35, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=1, BW=1742KiB/s (1784kB/s)(6144KiB/3527msec); 0 zone resets clat (msec): min=108, max=1331, avg=426.54, stdev=459.74 lat (msec): min=109, max=1331, avg=426.61, stdev=459.74 clat percentiles (msec): | 1.00th=[ 109], 5.00th=[ 109], 10.00th=[ 109], 20.00th=[ 197], | 30.00th=[ 197], 40.00th=[ 224], 50.00th=[ 224], 60.00th=[ 228], | 70.00th=[ 472], 80.00th=[ 472], 90.00th=[ 1334], 95.00th=[ 1334], | 99.00th=[ 1334], 99.50th=[ 1334], 99.90th=[ 1334], 99.95th=[ 1334], | 99.99th=[ 1334] bw ( KiB/s): min= 1976, max= 3599, per=33.60%, avg=2412.25, stdev=791.69, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (usec) : 1000=8.33% lat (msec) : 50=8.33%, 250=58.33%, 500=16.67%, 2000=8.33% cpu : usr=0.06%, sys=23.91%, ctx=424, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148025: Mon Mar 30 13:21:52 2026 read: IOPS=1, BW=1353KiB/s (1386kB/s)(5120KiB/3783msec) clat (usec): min=1875, max=361200, avg=170641.39, stdev=130266.96 lat (usec): min=1876, max=361201, avg=170642.66, stdev=130266.94 clat percentiles (usec): | 1.00th=[ 1876], 5.00th=[ 1876], 10.00th=[ 1876], 20.00th=[ 1876], | 30.00th=[130548], 40.00th=[130548], 50.00th=[154141], 60.00th=[154141], | 70.00th=[204473], 80.00th=[204473], 90.00th=[362808], 95.00th=[362808], | 99.00th=[362808], 99.50th=[362808], 99.90th=[362808], 99.95th=[362808], | 99.99th=[362808] bw ( KiB/s): min= 2039, max= 5930, per=77.70%, avg=3984.50, stdev=2751.35, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=1, BW=1895KiB/s (1940kB/s)(7168KiB/3783msec); 0 zone resets clat (msec): min=128, max=1273, avg=413.14, stdev=415.94 lat (msec): min=128, max=1273, avg=413.22, stdev=415.93 clat percentiles (msec): | 1.00th=[ 129], 5.00th=[ 129], 10.00th=[ 129], 20.00th=[ 197], | 30.00th=[ 201], 40.00th=[ 201], 50.00th=[ 213], 60.00th=[ 239], | 70.00th=[ 239], 80.00th=[ 642], 90.00th=[ 1267], 95.00th=[ 1267], | 99.00th=[ 1267], 99.50th=[ 1267], 99.90th=[ 1267], 99.95th=[ 1267], | 99.99th=[ 1267] bw ( KiB/s): min= 1802, max= 4096, per=41.85%, avg=3005.00, stdev=1253.85, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.50, samples=4 lat (msec) : 2=8.33%, 250=66.67%, 500=8.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=25.09%, ctx=243, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5129KiB/s (5252kB/s), 513KiB/s-2152KiB/s (525kB/s-2204kB/s), io=20.0MiB (20.0MB), run=3331-3993msec WRITE: bw=7181KiB/s (7353kB/s), 1537KiB/s-2564KiB/s (1574kB/s-2626kB/s), io=28.0MiB (29.4MB), run=3331-3993msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148027: Mon Mar 30 13:21:53 2026 read: IOPS=1, BW=1684KiB/s (1724kB/s)(7168KiB/4257msec) clat (msec): min=25, max=1065, avg=458.72, stdev=440.16 lat (msec): min=25, max=1065, avg=458.72, stdev=440.16 clat percentiles (msec): | 1.00th=[ 26], 5.00th=[ 26], 10.00th=[ 26], 20.00th=[ 46], | 30.00th=[ 89], 40.00th=[ 89], 50.00th=[ 363], 60.00th=[ 667], | 70.00th=[ 667], 80.00th=[ 953], 90.00th=[ 1070], 95.00th=[ 1070], | 99.00th=[ 1070], 99.50th=[ 1070], 99.90th=[ 1070], 99.95th=[ 1070], | 99.99th=[ 1070] bw ( KiB/s): min= 2031, max= 4079, per=70.44%, avg=3388.33, stdev=1175.55, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=1, BW=1203KiB/s (1232kB/s)(5120KiB/4257msec); 0 zone resets clat (msec): min=73, max=240, avg=172.19, stdev=62.67 lat (msec): min=73, max=241, avg=172.26, stdev=62.67 clat percentiles (msec): | 1.00th=[ 74], 5.00th=[ 74], 10.00th=[ 74], 20.00th=[ 74], | 30.00th=[ 159], 40.00th=[ 159], 50.00th=[ 188], 60.00th=[ 188], | 70.00th=[ 201], 80.00th=[ 201], 90.00th=[ 241], 95.00th=[ 241], | 99.00th=[ 241], 99.50th=[ 241], 99.90th=[ 241], 99.95th=[ 241], | 99.99th=[ 241] bw ( KiB/s): min= 2048, max= 4096, per=45.61%, avg=3072.00, stdev=1448.15, samples=2 iops : min= 2, max= 4, avg= 3.00, stdev= 1.41, samples=2 lat (msec) : 50=16.67%, 100=16.67%, 250=33.33%, 500=8.33%, 750=8.33% lat (msec) : 1000=8.33%, 2000=8.33% cpu : usr=0.00%, sys=6.44%, ctx=108, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148028: Mon Mar 30 13:21:53 2026 read: IOPS=0, BW=495KiB/s (507kB/s)(2048KiB/4140msec) clat (usec): min=65754, max=81507, avg=73631.27, stdev=11138.93 lat (usec): min=65758, max=81508, avg=73633.40, stdev=11137.34 clat percentiles (usec): | 1.00th=[65799], 5.00th=[65799], 10.00th=[65799], 20.00th=[65799], | 30.00th=[65799], 40.00th=[65799], 50.00th=[65799], 60.00th=[81265], | 70.00th=[81265], 80.00th=[81265], 90.00th=[81265], 95.00th=[81265], | 99.00th=[81265], 99.50th=[81265], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 1793, max= 2048, per=39.93%, avg=1920.50, stdev=180.31, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2473KiB/s (2533kB/s)(10.0MiB/4140msec); 0 zone resets clat (msec): min=90, max=1419, avg=379.20, stdev=423.20 lat (msec): min=90, max=1419, avg=379.29, stdev=423.19 clat percentiles (msec): | 1.00th=[ 91], 5.00th=[ 91], 10.00th=[ 91], 20.00th=[ 108], | 30.00th=[ 113], 40.00th=[ 123], 50.00th=[ 159], 60.00th=[ 224], | 70.00th=[ 334], 80.00th=[ 451], 90.00th=[ 776], 95.00th=[ 1418], | 99.00th=[ 1418], 99.50th=[ 1418], 99.90th=[ 1418], 99.95th=[ 1418], | 99.99th=[ 1418] bw ( KiB/s): min= 2011, max= 8192, per=68.13%, avg=4588.50, stdev=2592.03, samples=4 iops : min= 1, max= 8, avg= 4.00, stdev= 2.94, samples=4 lat (msec) : 100=25.00%, 250=41.67%, 500=16.67%, 1000=8.33%, 2000=8.33% cpu : usr=0.00%, sys=10.29%, ctx=125, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148029: Mon Mar 30 13:21:53 2026 read: IOPS=1, BW=1536KiB/s (1572kB/s)(6144KiB/4001msec) clat (msec): min=47, max=329, avg=140.90, stdev=98.52 lat (msec): min=47, max=329, avg=140.90, stdev=98.52 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 48], 10.00th=[ 48], 20.00th=[ 81], | 30.00th=[ 81], 40.00th=[ 118], 50.00th=[ 118], 60.00th=[ 127], | 70.00th=[ 144], 80.00th=[ 144], 90.00th=[ 330], 95.00th=[ 330], | 99.00th=[ 330], 99.50th=[ 330], 99.90th=[ 330], 99.95th=[ 330], | 99.99th=[ 330] bw ( KiB/s): min= 4055, max= 4096, per=84.73%, avg=4075.50, stdev=28.99, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1536KiB/s (1572kB/s)(6144KiB/4001msec); 0 zone resets clat (msec): min=74, max=1710, avg=492.54, stdev=644.50 lat (msec): min=74, max=1711, avg=492.61, stdev=644.49 clat percentiles (msec): | 1.00th=[ 75], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 108], | 30.00th=[ 108], 40.00th=[ 146], 50.00th=[ 146], 60.00th=[ 186], | 70.00th=[ 726], 80.00th=[ 726], 90.00th=[ 1703], 95.00th=[ 1703], | 99.00th=[ 1703], 99.50th=[ 1703], 99.90th=[ 1703], 99.95th=[ 1703], | 99.99th=[ 1703] bw ( KiB/s): min= 2039, max= 4055, per=47.91%, avg=3226.67, stdev=1054.94, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 lat (msec) : 50=8.33%, 100=16.67%, 250=50.00%, 500=8.33%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.05%, sys=9.68%, ctx=120, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148030: Mon Mar 30 13:21:53 2026 read: IOPS=1, BW=1214KiB/s (1243kB/s)(5120KiB/4217msec) clat (msec): min=100, max=676, avg=251.62, stdev=241.84 lat (msec): min=100, max=676, avg=251.62, stdev=241.84 clat percentiles (msec): | 1.00th=[ 101], 5.00th=[ 101], 10.00th=[ 101], 20.00th=[ 101], | 30.00th=[ 121], 40.00th=[ 121], 50.00th=[ 140], 60.00th=[ 140], | 70.00th=[ 224], 80.00th=[ 224], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 2039, max= 4096, per=56.71%, avg=2727.67, stdev=1185.02, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 write: IOPS=1, BW=1700KiB/s (1741kB/s)(7168KiB/4217msec); 0 zone resets clat (msec): min=86, max=1580, avg=396.11, stdev=531.44 lat (msec): min=86, max=1580, avg=396.20, stdev=531.42 clat percentiles (msec): | 1.00th=[ 87], 5.00th=[ 87], 10.00th=[ 87], 20.00th=[ 126], | 30.00th=[ 155], 40.00th=[ 155], 50.00th=[ 174], 60.00th=[ 275], | 70.00th=[ 275], 80.00th=[ 376], 90.00th=[ 1586], 95.00th=[ 1586], | 99.00th=[ 1586], 99.50th=[ 1586], 99.90th=[ 1586], 99.95th=[ 1586], | 99.99th=[ 1586] bw ( KiB/s): min= 1838, max= 4096, per=44.54%, avg=3000.00, stdev=1244.00, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.50, samples=4 lat (msec) : 100=8.33%, 250=58.33%, 500=16.67%, 750=8.33%, 2000=8.33% cpu : usr=0.07%, sys=8.78%, ctx=164, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4811KiB/s (4926kB/s), 495KiB/s-1684KiB/s (507kB/s-1724kB/s), io=20.0MiB (20.0MB), run=4001-4257msec WRITE: bw=6735KiB/s (6897kB/s), 1203KiB/s-2473KiB/s (1232kB/s-2533kB/s), io=28.0MiB (29.4MB), run=4001-4257msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148036: Mon Mar 30 13:22:00 2026 read: IOPS=1, BW=7772KiB/s (7959kB/s)(12.0MiB/1581msec) clat (msec): min=457, max=577, avg=516.27, stdev=60.25 lat (msec): min=457, max=577, avg=516.27, stdev=60.25 clat percentiles (msec): | 1.00th=[ 460], 5.00th=[ 460], 10.00th=[ 460], 20.00th=[ 460], | 30.00th=[ 460], 40.00th=[ 514], 50.00th=[ 514], 60.00th=[ 514], | 70.00th=[ 575], 80.00th=[ 575], 90.00th=[ 575], 95.00th=[ 575], | 99.00th=[ 575], 99.50th=[ 575], 99.90th=[ 575], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 7236, max= 8159, per=93.36%, avg=7697.50, stdev=652.66, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.00%, sys=34.49%, ctx=1356, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148037: Mon Mar 30 13:22:00 2026 read: IOPS=0, BW=2061KiB/s (2111kB/s)(4096KiB/1987msec) clat (nsec): min=401813k, max=401813k, avg=401813210.00, stdev= 0.00 lat (nsec): min=401816k, max=401816k, avg=401815762.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 401], 5.00th=[ 401], 10.00th=[ 401], 20.00th=[ 401], | 30.00th=[ 401], 40.00th=[ 401], 50.00th=[ 401], 60.00th=[ 401], | 70.00th=[ 401], 80.00th=[ 401], 90.00th=[ 401], 95.00th=[ 401], | 99.00th=[ 401], 99.50th=[ 401], 99.90th=[ 401], 99.95th=[ 401], | 99.99th=[ 401] bw ( KiB/s): min= 7543, max= 7543, per=91.49%, avg=7543.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4123KiB/s (4222kB/s)(8192KiB/1987msec); 0 zone resets clat (msec): min=485, max=1033, avg=759.54, stdev=387.05 lat (msec): min=486, max=1033, avg=760.06, stdev=386.56 clat percentiles (msec): | 1.00th=[ 485], 5.00th=[ 485], 10.00th=[ 485], 20.00th=[ 485], | 30.00th=[ 485], 40.00th=[ 485], 50.00th=[ 485], 60.00th=[ 1036], | 70.00th=[ 1036], 80.00th=[ 1036], 90.00th=[ 1036], 95.00th=[ 1036], | 99.00th=[ 1036], 99.50th=[ 1036], 99.90th=[ 1036], 99.95th=[ 1036], | 99.99th=[ 1036] bw ( KiB/s): min= 8159, max= 8159, per=49.48%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=8.36%, ctx=374, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148038: Mon Mar 30 13:22:00 2026 write: IOPS=1, BW=7186KiB/s (7358kB/s)(12.0MiB/1710msec); 0 zone resets clat (msec): min=540, max=555, avg=546.91, stdev= 7.56 lat (msec): min=540, max=555, avg=547.17, stdev= 7.51 clat percentiles (msec): | 1.00th=[ 542], 5.00th=[ 542], 10.00th=[ 542], 20.00th=[ 542], | 30.00th=[ 542], 40.00th=[ 550], 50.00th=[ 550], 60.00th=[ 550], | 70.00th=[ 558], 80.00th=[ 558], 90.00th=[ 558], 95.00th=[ 558], | 99.00th=[ 558], 99.50th=[ 558], 99.90th=[ 558], 99.95th=[ 558], | 99.99th=[ 558] bw ( KiB/s): min= 8143, max= 8159, per=49.43%, avg=8151.00, stdev=11.31, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=100.00% cpu : usr=0.00%, sys=20.54%, ctx=173, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148039: Mon Mar 30 13:22:00 2026 write: IOPS=2, BW=9518KiB/s (9747kB/s)(12.0MiB/1291msec); 0 zone resets clat (msec): min=213, max=770, avg=402.00, stdev=319.10 lat (msec): min=213, max=770, avg=402.33, stdev=319.24 clat percentiles (msec): | 1.00th=[ 213], 5.00th=[ 213], 10.00th=[ 213], 20.00th=[ 213], | 30.00th=[ 213], 40.00th=[ 224], 50.00th=[ 224], 60.00th=[ 224], | 70.00th=[ 768], 80.00th=[ 768], 90.00th=[ 768], 95.00th=[ 768], | 99.00th=[ 768], 99.50th=[ 768], 99.90th=[ 768], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min=10995, max=10995, per=66.67%, avg=10995.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, 1000=33.33% cpu : usr=0.00%, sys=15.27%, ctx=54, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8246KiB/s (8443kB/s), 2061KiB/s-7772KiB/s (2111kB/s-7959kB/s), io=16.0MiB (16.8MB), run=1581-1987msec WRITE: bw=16.1MiB/s (16.9MB/s), 4123KiB/s-9518KiB/s (4222kB/s-9747kB/s), io=32.0MiB (33.6MB), run=1291-1987msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148041: Mon Mar 30 13:22:00 2026 read: IOPS=1, BW=6314KiB/s (6466kB/s)(12.0MiB/1946msec) clat (msec): min=303, max=875, avg=632.97, stdev=295.89 lat (msec): min=303, max=875, avg=632.97, stdev=295.89 clat percentiles (msec): | 1.00th=[ 305], 5.00th=[ 305], 10.00th=[ 305], 20.00th=[ 305], | 30.00th=[ 305], 40.00th=[ 718], 50.00th=[ 718], 60.00th=[ 718], | 70.00th=[ 877], 80.00th=[ 877], 90.00th=[ 877], 95.00th=[ 877], | 99.00th=[ 877], 99.50th=[ 877], 99.90th=[ 877], 99.95th=[ 877], | 99.99th=[ 877] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=7.81%, ctx=85, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148042: Mon Mar 30 13:22:00 2026 read: IOPS=0, BW=1876KiB/s (1921kB/s)(4096KiB/2183msec) clat (nsec): min=872014k, max=872014k, avg=872013818.00, stdev= 0.00 lat (nsec): min=872016k, max=872016k, avg=872015941.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 869], 5.00th=[ 869], 10.00th=[ 869], 20.00th=[ 869], | 30.00th=[ 869], 40.00th=[ 869], 50.00th=[ 869], 60.00th=[ 869], | 70.00th=[ 869], 80.00th=[ 869], 90.00th=[ 869], 95.00th=[ 869], | 99.00th=[ 869], 99.50th=[ 869], 99.90th=[ 869], 99.95th=[ 869], | 99.99th=[ 869] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3753KiB/s (3843kB/s)(8192KiB/2183msec); 0 zone resets clat (msec): min=404, max=786, avg=595.39, stdev=269.77 lat (msec): min=404, max=786, avg=595.78, stdev=269.93 clat percentiles (msec): | 1.00th=[ 405], 5.00th=[ 405], 10.00th=[ 405], 20.00th=[ 405], | 30.00th=[ 405], 40.00th=[ 405], 50.00th=[ 405], 60.00th=[ 785], | 70.00th=[ 785], 80.00th=[ 785], 90.00th=[ 785], 95.00th=[ 785], | 99.00th=[ 785], 99.50th=[ 785], 99.90th=[ 785], 99.95th=[ 785], | 99.99th=[ 785] lat (msec) : 500=33.33%, 1000=66.67% cpu : usr=0.00%, sys=9.21%, ctx=73, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148043: Mon Mar 30 13:22:00 2026 write: IOPS=1, BW=5553KiB/s (5686kB/s)(12.0MiB/2213msec); 0 zone resets clat (msec): min=160, max=1172, avg=717.03, stdev=513.87 lat (msec): min=160, max=1173, avg=717.76, stdev=514.27 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 161], | 30.00th=[ 161], 40.00th=[ 818], 50.00th=[ 818], 60.00th=[ 818], | 70.00th=[ 1167], 80.00th=[ 1167], 90.00th=[ 1167], 95.00th=[ 1167], | 99.00th=[ 1167], 99.50th=[ 1167], 99.90th=[ 1167], 99.95th=[ 1167], | 99.99th=[ 1167] bw ( KiB/s): min= 8175, max= 8175, per=55.21%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=5.15%, ctx=39, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148044: Mon Mar 30 13:22:00 2026 write: IOPS=1, BW=7288KiB/s (7463kB/s)(12.0MiB/1686msec); 0 zone resets clat (msec): min=233, max=869, avg=510.96, stdev=325.48 lat (msec): min=234, max=872, avg=512.35, stdev=327.23 clat percentiles (msec): | 1.00th=[ 234], 5.00th=[ 234], 10.00th=[ 234], 20.00th=[ 234], | 30.00th=[ 234], 40.00th=[ 430], 50.00th=[ 430], 60.00th=[ 430], | 70.00th=[ 869], 80.00th=[ 869], 90.00th=[ 869], 95.00th=[ 869], | 99.00th=[ 869], 99.50th=[ 869], 99.90th=[ 869], 99.95th=[ 869], | 99.99th=[ 869] bw ( KiB/s): min= 8175, max= 8175, per=55.21%, avg=8175.00, stdev= 0.00, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 500=33.33%, 1000=33.33% cpu : usr=0.06%, sys=11.51%, ctx=79, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7505KiB/s (7685kB/s), 1876KiB/s-6314KiB/s (1921kB/s-6466kB/s), io=16.0MiB (16.8MB), run=1946-2183msec WRITE: bw=14.5MiB/s (15.2MB/s), 3753KiB/s-7288KiB/s (3843kB/s-7463kB/s), io=32.0MiB (33.6MB), run=1686-2213msec