== sanity test 398b: DIO and buffer IO race ============== 12:03:11 (1782489791) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 8.26016 s, 6.1 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=184981: Fri Jun 26 12:06:14 2026 read: IOPS=9, BW=38.4KiB/s (39.3kB/s)(5992KiB/156158msec) clat (usec): min=45, max=680252, avg=59150.16, stdev=52416.39 lat (usec): min=47, max=680253, avg=59153.19, stdev=52415.95 clat percentiles (usec): | 1.00th=[ 53], 5.00th=[ 11863], 10.00th=[ 18482], 20.00th=[ 26346], | 30.00th=[ 33424], 40.00th=[ 40633], 50.00th=[ 46924], 60.00th=[ 55837], | 70.00th=[ 66847], 80.00th=[ 83362], 90.00th=[105382], 95.00th=[135267], | 99.00th=[291505], 99.50th=[362808], 99.90th=[557843], 99.95th=[683672], | 99.99th=[683672] bw ( KiB/s): min= 7, max= 135, per=25.39%, avg=38.34, stdev=19.74, samples=303 iops : min= 1, max= 33, avg= 8.93, stdev= 4.97, samples=303 write: IOPS=10, BW=40.3KiB/s (41.3kB/s)(6296KiB/156158msec); 0 zone resets clat (usec): min=964, max=465044, avg=42769.66, stdev=48350.60 lat (usec): min=965, max=465046, avg=42773.47, stdev=48349.32 clat percentiles (usec): | 1.00th=[ 1188], 5.00th=[ 1369], 10.00th=[ 1713], 20.00th=[ 4228], | 30.00th=[ 10159], 40.00th=[ 19530], 50.00th=[ 28705], 60.00th=[ 40109], | 70.00th=[ 52691], 80.00th=[ 69731], 90.00th=[102237], 95.00th=[129500], | 99.00th=[219153], 99.50th=[278922], 99.90th=[434111], 99.95th=[463471], | 99.99th=[463471] bw ( KiB/s): min= 7, max= 134, per=25.56%, avg=40.38, stdev=21.24, samples=302 iops : min= 1, max= 33, avg= 9.45, stdev= 5.36, samples=302 lat (usec) : 50=0.10%, 100=1.76%, 250=0.03%, 750=0.03%, 1000=0.13% lat (msec) : 2=5.83%, 4=3.97%, 10=5.57%, 20=8.95%, 50=34.47% lat (msec) : 100=27.77%, 250=10.45%, 500=0.88%, 750=0.07% cpu : usr=0.06%, sys=4.10%, ctx=6284, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=184982: Fri Jun 26 12:06:14 2026 read: IOPS=9, BW=37.5KiB/s (38.4kB/s)(5940KiB/158241msec) clat (usec): min=43, max=455552, avg=58102.10, stdev=45366.35 lat (usec): min=44, max=455553, avg=58113.34, stdev=45365.05 clat percentiles (usec): | 1.00th=[ 56], 5.00th=[ 13435], 10.00th=[ 19006], 20.00th=[ 26346], | 30.00th=[ 33162], 40.00th=[ 39060], 50.00th=[ 47449], 60.00th=[ 55837], | 70.00th=[ 66847], 80.00th=[ 81265], 90.00th=[106431], 95.00th=[137364], | 99.00th=[227541], 99.50th=[299893], 99.90th=[429917], 99.95th=[455082], | 99.99th=[455082] bw ( KiB/s): min= 6, max= 102, per=25.04%, avg=37.81, stdev=17.75, samples=304 iops : min= 1, max= 25, avg= 8.80, stdev= 4.48, samples=304 write: IOPS=10, BW=40.1KiB/s (41.1kB/s)(6348KiB/158241msec); 0 zone resets clat (usec): min=844, max=872600, avg=45164.46, stdev=56477.82 lat (usec): min=846, max=872602, avg=45182.93, stdev=56493.26 clat percentiles (usec): | 1.00th=[ 1221], 5.00th=[ 1352], 10.00th=[ 1778], 20.00th=[ 4359], | 30.00th=[ 9241], 40.00th=[ 19268], 50.00th=[ 30016], 60.00th=[ 42206], | 70.00th=[ 55837], 80.00th=[ 74974], 90.00th=[110625], 95.00th=[135267], | 99.00th=[219153], 99.50th=[270533], 99.90th=[717226], 99.95th=[876610], | 99.99th=[876610] bw ( KiB/s): min= 7, max= 136, per=25.30%, avg=39.98, stdev=21.52, samples=307 iops : min= 1, max= 34, avg= 9.34, stdev= 5.43, samples=307 lat (usec) : 50=0.16%, 100=1.11%, 250=0.10%, 1000=0.07% lat (msec) : 2=5.79%, 4=4.17%, 10=6.18%, 20=8.59%, 50=33.33% lat (msec) : 100=28.71%, 250=11.07%, 500=0.59%, 750=0.10%, 1000=0.03% cpu : usr=0.07%, sys=3.94%, ctx=6432, majf=0, minf=19 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=184983: Fri Jun 26 12:06:14 2026 read: IOPS=9, BW=38.6KiB/s (39.5kB/s)(6028KiB/156284msec) clat (usec): min=42, max=695339, avg=57384.91, stdev=47804.47 lat (usec): min=44, max=695341, avg=57390.73, stdev=47807.90 clat percentiles (usec): | 1.00th=[ 52], 5.00th=[ 13042], 10.00th=[ 19530], 20.00th=[ 26870], | 30.00th=[ 33424], 40.00th=[ 40109], 50.00th=[ 46924], 60.00th=[ 54789], | 70.00th=[ 65799], 80.00th=[ 79168], 90.00th=[101188], 95.00th=[132645], | 99.00th=[227541], 99.50th=[283116], 99.90th=[549454], 99.95th=[692061], | 99.99th=[692061] bw ( KiB/s): min= 7, max= 100, per=25.48%, avg=38.48, stdev=18.65, samples=303 iops : min= 1, max= 25, avg= 8.97, stdev= 4.72, samples=303 write: IOPS=10, BW=40.1KiB/s (41.0kB/s)(6260KiB/156284msec); 0 zone resets clat (usec): min=975, max=430257, avg=44475.43, stdev=50722.41 lat (usec): min=977, max=430258, avg=44484.17, stdev=50717.87 clat percentiles (usec): | 1.00th=[ 1205], 5.00th=[ 1352], 10.00th=[ 1516], 20.00th=[ 4424], | 30.00th=[ 9503], 40.00th=[ 19268], 50.00th=[ 28443], 60.00th=[ 40109], | 70.00th=[ 55313], 80.00th=[ 76022], 90.00th=[106431], 95.00th=[139461], | 99.00th=[227541], 99.50th=[278922], 99.90th=[396362], 99.95th=[429917], | 99.99th=[429917] bw ( KiB/s): min= 7, max= 103, per=25.64%, avg=40.52, stdev=20.79, samples=299 iops : min= 1, max= 25, avg= 9.47, stdev= 5.24, samples=299 lat (usec) : 50=0.29%, 100=1.60%, 250=0.07%, 500=0.03%, 750=0.03% lat (usec) : 1000=0.03% lat (msec) : 2=6.32%, 4=3.32%, 10=6.05%, 20=8.04%, 50=34.96% lat (msec) : 100=27.96%, 250=10.55%, 500=0.68%, 750=0.07% cpu : usr=0.06%, sys=3.95%, ctx=6338, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=184984: Fri Jun 26 12:06:14 2026 read: IOPS=9, BW=38.3KiB/s (39.2kB/s)(6052KiB/158004msec) clat (usec): min=38, max=483057, avg=57728.96, stdev=43533.37 lat (usec): min=40, max=483059, avg=57735.24, stdev=43532.80 clat percentiles (usec): | 1.00th=[ 55], 5.00th=[ 13173], 10.00th=[ 19530], 20.00th=[ 27395], | 30.00th=[ 34341], 40.00th=[ 41681], 50.00th=[ 48497], 60.00th=[ 56886], | 70.00th=[ 66847], 80.00th=[ 81265], 90.00th=[102237], 95.00th=[131597], | 99.00th=[212861], 99.50th=[265290], 99.90th=[408945], 99.95th=[484443], | 99.99th=[484443] bw ( KiB/s): min= 6, max= 103, per=25.57%, avg=38.61, stdev=18.47, samples=305 iops : min= 1, max= 25, avg= 9.00, stdev= 4.66, samples=305 write: IOPS=9, BW=39.5KiB/s (40.4kB/s)(6236KiB/158004msec); 0 zone resets clat (usec): min=847, max=784195, avg=45229.37, stdev=57288.23 lat (usec): min=848, max=784196, avg=45234.34, stdev=57288.57 clat percentiles (usec): | 1.00th=[ 1188], 5.00th=[ 1336], 10.00th=[ 1598], 20.00th=[ 4146], | 30.00th=[ 9765], 40.00th=[ 20579], 50.00th=[ 30540], 60.00th=[ 42206], | 70.00th=[ 55313], 80.00th=[ 73925], 90.00th=[105382], 95.00th=[135267], | 99.00th=[210764], 99.50th=[329253], 99.90th=[767558], 99.95th=[784335], | 99.99th=[784335] bw ( KiB/s): min= 6, max= 103, per=24.94%, avg=39.40, stdev=20.62, samples=308 iops : min= 1, max= 25, avg= 9.19, stdev= 5.19, samples=308 lat (usec) : 50=0.13%, 100=1.40%, 250=0.16%, 500=0.10%, 1000=0.07% lat (msec) : 2=5.99%, 4=3.81%, 10=5.79%, 20=7.45%, 50=34.57% lat (msec) : 100=29.26%, 250=10.51%, 500=0.65%, 750=0.03%, 1000=0.07% cpu : usr=0.06%, sys=3.98%, ctx=6377, majf=0, minf=19 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=152KiB/s (155kB/s), 37.5KiB/s-38.6KiB/s (38.4kB/s-39.5kB/s), io=23.4MiB (24.6MB), run=156158-158241msec WRITE: bw=159KiB/s (163kB/s), 39.5KiB/s-40.3KiB/s (40.4kB/s-41.3kB/s), io=24.6MiB (25.7MB), run=156158-158241msec rand-rw: (groupid=0, jobs=1): err= 0: pid=184976: Fri Jun 26 12:06:54 2026 read: IOPS=7, BW=30.1KiB/s (30.8kB/s)(5992KiB/199383msec) clat (msec): min=5, max=803, avg=52.48, stdev=62.55 lat (msec): min=5, max=803, avg=52.48, stdev=62.55 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 14], 20.00th=[ 17], | 30.00th=[ 21], 40.00th=[ 26], 50.00th=[ 32], 60.00th=[ 41], | 70.00th=[ 54], 80.00th=[ 75], 90.00th=[ 120], 95.00th=[ 159], | 99.00th=[ 279], 99.50th=[ 347], 99.90th=[ 751], 99.95th=[ 802], | 99.99th=[ 802] bw ( KiB/s): min= 7, max= 136, per=26.27%, avg=31.53, stdev=25.10, samples=367 iops : min= 1, max= 34, avg= 7.20, stdev= 6.31, samples=367 write: IOPS=7, BW=31.6KiB/s (32.3kB/s)(6296KiB/199383msec); 0 zone resets clat (msec): min=11, max=544, avg=76.35, stdev=64.22 lat (msec): min=11, max=544, avg=76.36, stdev=64.22 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 19], 10.00th=[ 23], 20.00th=[ 29], | 30.00th=[ 35], 40.00th=[ 44], 50.00th=[ 54], 60.00th=[ 69], | 70.00th=[ 88], 80.00th=[ 118], 90.00th=[ 165], 95.00th=[ 203], | 99.00th=[ 309], 99.50th=[ 355], 99.90th=[ 523], 99.95th=[ 542], | 99.99th=[ 542] bw ( KiB/s): min= 6, max= 159, per=25.86%, avg=32.58, stdev=25.07, samples=373 iops : min= 1, max= 39, avg= 7.47, stdev= 6.31, samples=373 lat (msec) : 10=1.79%, 20=14.75%, 50=40.46%, 100=23.93%, 250=17.12% lat (msec) : 500=1.73%, 750=0.16%, 1000=0.07% cpu : usr=0.05%, sys=1.66%, ctx=4958, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=184977: Fri Jun 26 12:06:54 2026 read: IOPS=7, BW=29.9KiB/s (30.6kB/s)(5940KiB/198883msec) clat (msec): min=5, max=760, avg=51.53, stdev=60.39 lat (msec): min=5, max=760, avg=51.53, stdev=60.39 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 13], 20.00th=[ 16], | 30.00th=[ 21], 40.00th=[ 24], 50.00th=[ 30], 60.00th=[ 39], | 70.00th=[ 52], 80.00th=[ 74], 90.00th=[ 120], 95.00th=[ 161], | 99.00th=[ 275], 99.50th=[ 317], 99.90th=[ 735], 99.95th=[ 760], | 99.99th=[ 760] bw ( KiB/s): min= 7, max= 158, per=26.48%, avg=31.77, stdev=26.46, samples=361 iops : min= 1, max= 39, avg= 7.27, stdev= 6.67, samples=361 write: IOPS=7, BW=31.9KiB/s (32.7kB/s)(6348KiB/198883msec); 0 zone resets clat (msec): min=9, max=1087, avg=76.73, stdev=70.34 lat (msec): min=9, max=1087, avg=76.74, stdev=70.34 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 20], 10.00th=[ 23], 20.00th=[ 29], | 30.00th=[ 36], 40.00th=[ 44], 50.00th=[ 54], 60.00th=[ 67], | 70.00th=[ 89], 80.00th=[ 115], 90.00th=[ 159], 95.00th=[ 194], | 99.00th=[ 342], 99.50th=[ 368], 99.90th=[ 684], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 7, max= 119, per=26.03%, avg=32.79, stdev=24.61, samples=373 iops : min= 1, max= 29, avg= 7.53, stdev= 6.19, samples=373 lat (msec) : 10=2.28%, 20=15.36%, 50=39.78%, 100=23.08%, 250=17.55% lat (msec) : 500=1.73%, 750=0.16%, 1000=0.03%, 2000=0.03% cpu : usr=0.06%, sys=1.70%, ctx=4905, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=184978: Fri Jun 26 12:06:54 2026 read: IOPS=7, BW=30.2KiB/s (30.0kB/s)(6028KiB/199375msec) clat (msec): min=6, max=364, avg=50.86, stdev=50.35 lat (msec): min=6, max=364, avg=50.86, stdev=50.35 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 14], 20.00th=[ 17], | 30.00th=[ 21], 40.00th=[ 26], 50.00th=[ 32], 60.00th=[ 41], | 70.00th=[ 54], 80.00th=[ 74], 90.00th=[ 120], 95.00th=[ 161], | 99.00th=[ 245], 99.50th=[ 279], 99.90th=[ 355], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 4, max= 135, per=26.20%, avg=31.44, stdev=24.35, samples=370 iops : min= 1, max= 33, avg= 7.18, stdev= 6.11, samples=370 write: IOPS=7, BW=31.4KiB/s (32.2kB/s)(6260KiB/199375msec); 0 zone resets clat (msec): min=10, max=819, avg=78.05, stdev=73.33 lat (msec): min=10, max=819, avg=78.06, stdev=73.33 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 20], 10.00th=[ 23], 20.00th=[ 29], | 30.00th=[ 36], 40.00th=[ 45], 50.00th=[ 53], 60.00th=[ 68], | 70.00th=[ 88], 80.00th=[ 113], 90.00th=[ 163], 95.00th=[ 207], | 99.00th=[ 355], 99.50th=[ 414], 99.90th=[ 802], 99.95th=[ 818], | 99.99th=[ 818] bw ( KiB/s): min= 4, max= 133, per=25.23%, avg=31.79, stdev=24.83, samples=380 iops : min= 1, max= 33, avg= 7.27, stdev= 6.25, samples=380 lat (msec) : 10=1.89%, 20=14.71%, 50=40.66%, 100=23.83%, 250=16.89% lat (msec) : 500=1.86%, 750=0.10%, 1000=0.07% cpu : usr=0.06%, sys=1.72%, ctx=4830, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=184979: Fri Jun 26 12:06:54 2026 read: IOPS=7, BW=30.5KiB/s (31.2kB/s)(6052KiB/198331msec) clat (msec): min=4, max=490, avg=50.69, stdev=50.46 lat (msec): min=4, max=490, avg=50.69, stdev=50.46 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 13], 20.00th=[ 17], | 30.00th=[ 21], 40.00th=[ 27], 50.00th=[ 33], 60.00th=[ 41], | 70.00th=[ 55], 80.00th=[ 80], 90.00th=[ 116], 95.00th=[ 157], | 99.00th=[ 247], 99.50th=[ 279], 99.90th=[ 430], 99.95th=[ 489], | 99.99th=[ 489] bw ( KiB/s): min= 4, max= 175, per=26.93%, avg=32.31, stdev=25.04, samples=362 iops : min= 1, max= 43, avg= 7.40, stdev= 6.29, samples=362 write: IOPS=7, BW=31.4KiB/s (32.2kB/s)(6236KiB/198331msec); 0 zone resets clat (msec): min=10, max=897, avg=77.65, stdev=72.16 lat (msec): min=10, max=897, avg=77.65, stdev=72.16 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 20], 10.00th=[ 24], 20.00th=[ 31], | 30.00th=[ 37], 40.00th=[ 43], 50.00th=[ 54], 60.00th=[ 68], | 70.00th=[ 88], 80.00th=[ 115], 90.00th=[ 159], 95.00th=[ 201], | 99.00th=[ 368], 99.50th=[ 468], 99.90th=[ 743], 99.95th=[ 894], | 99.99th=[ 894] bw ( KiB/s): min= 7, max= 119, per=24.98%, avg=31.48, stdev=24.08, samples=382 iops : min= 1, max= 29, avg= 7.19, stdev= 6.06, samples=382 lat (msec) : 10=2.67%, 20=14.03%, 50=40.56%, 100=23.76%, 250=17.22% lat (msec) : 500=1.60%, 750=0.13%, 1000=0.03% cpu : usr=0.04%, sys=1.61%, ctx=4773, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=120KiB/s (123kB/s), 29.9KiB/s-30.5KiB/s (30.6kB/s-31.2kB/s), io=23.4MiB (24.6MB), run=198331-199383msec WRITE: bw=126KiB/s (129kB/s), 31.4KiB/s-31.9KiB/s (32.2kB/s-32.7kB/s), io=24.6MiB (25.7MB), run=198331-199383msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=185039: Fri Jun 26 12:07:38 2026 read: IOPS=11, BW=182KiB/s (186kB/s)(5920KiB/32551msec) clat (usec): min=42, max=167052, avg=46739.20, stdev=31024.52 lat (usec): min=42, max=167053, avg=46740.59, stdev=31024.65 clat percentiles (usec): | 1.00th=[ 55], 5.00th=[ 69], 10.00th=[ 11994], 20.00th=[ 23200], | 30.00th=[ 29230], 40.00th=[ 35390], 50.00th=[ 42730], 60.00th=[ 49546], | 70.00th=[ 56361], 80.00th=[ 68682], 90.00th=[ 87557], 95.00th=[101188], | 99.00th=[147850], 99.50th=[162530], 99.90th=[166724], 99.95th=[166724], | 99.99th=[166724] bw ( KiB/s): min= 31, max= 383, per=25.32%, avg=181.30, stdev=73.16, samples=64 iops : min= 1, max= 23, avg=10.52, stdev= 4.60, samples=64 write: IOPS=12, BW=196KiB/s (200kB/s)(6368KiB/32551msec); 0 zone resets clat (usec): min=1187, max=198315, avg=37539.84, stdev=38172.59 lat (usec): min=1188, max=198317, avg=37546.01, stdev=38170.00 clat percentiles (usec): | 1.00th=[ 1582], 5.00th=[ 1975], 10.00th=[ 2671], 20.00th=[ 5735], | 30.00th=[ 10552], 40.00th=[ 17171], 50.00th=[ 25297], 60.00th=[ 34866], | 70.00th=[ 46400], 80.00th=[ 65274], 90.00th=[ 88605], 95.00th=[121111], | 99.00th=[162530], 99.50th=[168821], 99.90th=[198181], 99.95th=[198181], | 99.99th=[198181] bw ( KiB/s): min= 27, max= 473, per=26.00%, avg=195.77, stdev=96.67, samples=64 iops : min= 1, max= 29, avg=11.44, stdev= 6.14, samples=64 lat (usec) : 50=0.39%, 100=2.99%, 250=0.65%, 500=0.39% lat (msec) : 2=2.86%, 4=4.82%, 10=7.68%, 20=10.81%, 50=35.94% lat (msec) : 100=26.95%, 250=6.51% cpu : usr=0.06%, sys=5.35%, ctx=1698, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185040: Fri Jun 26 12:07:38 2026 read: IOPS=10, BW=172KiB/s (176kB/s)(5744KiB/33394msec) clat (usec): min=40, max=223275, avg=47552.09, stdev=32079.93 lat (usec): min=41, max=223276, avg=47553.25, stdev=32079.89 clat percentiles (usec): | 1.00th=[ 49], 5.00th=[ 64], 10.00th=[ 16909], 20.00th=[ 24249], | 30.00th=[ 30540], 40.00th=[ 35914], 50.00th=[ 42206], 60.00th=[ 49546], | 70.00th=[ 56361], 80.00th=[ 65799], 90.00th=[ 86508], 95.00th=[105382], | 99.00th=[160433], 99.50th=[208667], 99.90th=[223347], 99.95th=[223347], | 99.99th=[223347] bw ( KiB/s): min= 31, max= 416, per=23.93%, avg=171.31, stdev=77.65, samples=65 iops : min= 1, max= 26, avg= 9.89, stdev= 4.93, samples=65 write: IOPS=12, BW=196KiB/s (201kB/s)(6544KiB/33394msec); 0 zone resets clat (usec): min=1587, max=215800, avg=39595.53, stdev=39113.87 lat (usec): min=1590, max=215801, avg=39612.12, stdev=39118.83 clat percentiles (usec): | 1.00th=[ 1811], 5.00th=[ 2147], 10.00th=[ 3097], 20.00th=[ 6194], | 30.00th=[ 11207], 40.00th=[ 18482], 50.00th=[ 26346], 60.00th=[ 36963], | 70.00th=[ 51643], 80.00th=[ 70779], 90.00th=[ 93848], 95.00th=[124257], | 99.00th=[166724], 99.50th=[179307], 99.90th=[214959], 99.95th=[214959], | 99.99th=[214959] bw ( KiB/s): min= 31, max= 448, per=25.88%, avg=194.85, stdev=89.12, samples=65 iops : min= 1, max= 28, avg=11.38, stdev= 5.65, samples=65 lat (usec) : 50=0.65%, 100=2.34%, 250=0.52% lat (msec) : 2=2.34%, 4=5.08%, 10=7.42%, 20=10.16%, 50=36.85% lat (msec) : 100=27.08%, 250=7.55% cpu : usr=0.05%, sys=5.31%, ctx=1794, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185041: Fri Jun 26 12:07:38 2026 read: IOPS=11, BW=187KiB/s (192kB/s)(6256KiB/33424msec) clat (usec): min=51, max=262084, avg=48286.81, stdev=35148.07 lat (usec): min=52, max=262086, avg=48301.67, stdev=35148.62 clat percentiles (usec): | 1.00th=[ 55], 5.00th=[ 67], 10.00th=[ 12125], 20.00th=[ 23725], | 30.00th=[ 29230], 40.00th=[ 33817], 50.00th=[ 41157], 60.00th=[ 48497], | 70.00th=[ 57934], 80.00th=[ 71828], 90.00th=[ 90702], 95.00th=[112722], | 99.00th=[175113], 99.50th=[189793], 99.90th=[261096], 99.95th=[261096], | 99.99th=[261096] bw ( KiB/s): min= 27, max= 351, per=26.25%, avg=187.98, stdev=69.93, samples=65 iops : min= 1, max= 21, avg=10.94, stdev= 4.43, samples=65 write: IOPS=11, BW=180KiB/s (185kB/s)(6032KiB/33424msec); 0 zone resets clat (usec): min=1330, max=210652, avg=37675.21, stdev=35403.95 lat (usec): min=1332, max=210655, avg=37701.06, stdev=35395.70 clat percentiles (usec): | 1.00th=[ 1778], 5.00th=[ 2147], 10.00th=[ 3261], 20.00th=[ 7504], | 30.00th=[ 11600], 40.00th=[ 20579], 50.00th=[ 28967], 60.00th=[ 36963], | 70.00th=[ 45876], 80.00th=[ 64226], 90.00th=[ 86508], 95.00th=[109577], | 99.00th=[158335], 99.50th=[189793], 99.90th=[210764], 99.95th=[210764], | 99.99th=[210764] bw ( KiB/s): min= 30, max= 407, per=24.01%, avg=180.80, stdev=93.77, samples=64 iops : min= 1, max= 25, avg=10.48, stdev= 5.97, samples=64 lat (usec) : 100=4.04%, 250=0.39% lat (msec) : 2=2.08%, 4=4.04%, 10=7.29%, 20=8.98%, 50=40.23% lat (msec) : 100=25.91%, 250=6.90%, 500=0.13% cpu : usr=0.07%, sys=4.88%, ctx=1747, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185042: Fri Jun 26 12:07:38 2026 read: IOPS=11, BW=186KiB/s (191kB/s)(6032KiB/32410msec) clat (usec): min=39, max=242628, avg=47237.66, stdev=36836.30 lat (usec): min=39, max=242629, avg=47238.80, stdev=36836.36 clat percentiles (usec): | 1.00th=[ 48], 5.00th=[ 61], 10.00th=[ 79], 20.00th=[ 21365], | 30.00th=[ 27395], 40.00th=[ 33162], 50.00th=[ 40633], 60.00th=[ 49021], | 70.00th=[ 58459], 80.00th=[ 70779], 90.00th=[ 91751], 95.00th=[120062], | 99.00th=[179307], 99.50th=[208667], 99.90th=[242222], 99.95th=[242222], | 99.99th=[242222] bw ( KiB/s): min= 27, max= 408, per=25.82%, avg=184.86, stdev=80.59, samples=63 iops : min= 1, max= 25, avg=10.75, stdev= 5.11, samples=63 write: IOPS=12, BW=193KiB/s (198kB/s)(6256KiB/32410msec); 0 zone resets clat (usec): min=1439, max=214970, avg=36770.92, stdev=36200.62 lat (usec): min=1440, max=214972, avg=36773.10, stdev=36200.61 clat percentiles (usec): | 1.00th=[ 1696], 5.00th=[ 2245], 10.00th=[ 3064], 20.00th=[ 6587], | 30.00th=[ 10945], 40.00th=[ 16909], 50.00th=[ 24773], 60.00th=[ 34341], | 70.00th=[ 47449], 80.00th=[ 64226], 90.00th=[ 88605], 95.00th=[112722], | 99.00th=[141558], 99.50th=[183501], 99.90th=[214959], 99.95th=[214959], | 99.99th=[214959] bw ( KiB/s): min= 62, max= 479, per=25.37%, avg=191.05, stdev=87.49, samples=63 iops : min= 3, max= 29, avg=11.13, stdev= 5.55, samples=63 lat (usec) : 50=0.52%, 100=4.69%, 250=0.91%, 500=0.26%, 750=0.13% lat (usec) : 1000=0.13% lat (msec) : 2=2.21%, 4=4.69%, 10=7.94%, 20=10.55%, 50=35.16% lat (msec) : 100=25.13%, 250=7.68% cpu : usr=0.05%, sys=4.79%, ctx=1627, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=717KiB/s (734kB/s), 172KiB/s-187KiB/s (176kB/s-192kB/s), io=23.4MiB (24.5MB), run=32410-33424msec WRITE: bw=754KiB/s (772kB/s), 180KiB/s-196KiB/s (185kB/s-201kB/s), io=24.6MiB (25.8MB), run=32410-33424msec rand-rw: (groupid=0, jobs=1): err= 0: pid=185034: Fri Jun 26 12:07:51 2026 read: IOPS=8, BW=128KiB/s (131kB/s)(5920KiB/46117msec) clat (msec): min=5, max=324, avg=50.34, stdev=45.36 lat (msec): min=5, max=324, avg=50.34, stdev=45.36 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 15], 20.00th=[ 19], | 30.00th=[ 23], 40.00th=[ 29], 50.00th=[ 34], 60.00th=[ 44], | 70.00th=[ 55], 80.00th=[ 73], 90.00th=[ 116], 95.00th=[ 140], | 99.00th=[ 218], 99.50th=[ 321], 99.90th=[ 326], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 31, max= 448, per=24.69%, avg=127.14, stdev=85.66, samples=90 iops : min= 1, max= 28, avg= 7.17, stdev= 5.43, samples=90 write: IOPS=8, BW=138KiB/s (141kB/s)(6368KiB/46117msec); 0 zone resets clat (msec): min=11, max=293, avg=68.27, stdev=45.70 lat (msec): min=11, max=293, avg=68.27, stdev=45.70 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 22], 10.00th=[ 26], 20.00th=[ 33], | 30.00th=[ 40], 40.00th=[ 48], 50.00th=[ 56], 60.00th=[ 64], | 70.00th=[ 79], 80.00th=[ 97], 90.00th=[ 128], 95.00th=[ 165], | 99.00th=[ 236], 99.50th=[ 255], 99.90th=[ 296], 99.95th=[ 296], | 99.99th=[ 296] bw ( KiB/s): min= 21, max= 413, per=25.46%, avg=137.99, stdev=84.67, samples=90 iops : min= 1, max= 25, avg= 7.84, stdev= 5.34, samples=90 lat (msec) : 10=1.43%, 20=11.07%, 50=41.41%, 100=30.08%, 250=15.36% lat (msec) : 500=0.65% cpu : usr=0.06%, sys=1.79%, ctx=1209, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185035: Fri Jun 26 12:07:51 2026 read: IOPS=7, BW=125KiB/s (128kB/s)(5744KiB/46074msec) clat (msec): min=7, max=240, avg=49.65, stdev=41.69 lat (msec): min=7, max=240, avg=49.65, stdev=41.69 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 14], 20.00th=[ 19], | 30.00th=[ 25], 40.00th=[ 31], 50.00th=[ 36], 60.00th=[ 44], | 70.00th=[ 52], 80.00th=[ 73], 90.00th=[ 110], 95.00th=[ 140], | 99.00th=[ 215], 99.50th=[ 224], 99.90th=[ 241], 99.95th=[ 241], | 99.99th=[ 241] bw ( KiB/s): min= 31, max= 477, per=25.21%, avg=129.85, stdev=84.38, samples=86 iops : min= 1, max= 29, avg= 7.34, stdev= 5.36, samples=86 write: IOPS=8, BW=142KiB/s (145kB/s)(6544KiB/46074msec); 0 zone resets clat (msec): min=14, max=327, avg=68.83, stdev=47.83 lat (msec): min=14, max=327, avg=68.83, stdev=47.83 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 22], 10.00th=[ 25], 20.00th=[ 32], | 30.00th=[ 39], 40.00th=[ 45], 50.00th=[ 56], 60.00th=[ 66], | 70.00th=[ 78], 80.00th=[ 99], 90.00th=[ 136], 95.00th=[ 171], | 99.00th=[ 245], 99.50th=[ 255], 99.90th=[ 330], 99.95th=[ 330], | 99.99th=[ 330] bw ( KiB/s): min= 31, max= 479, per=26.22%, avg=142.10, stdev=91.20, samples=90 iops : min= 1, max= 29, avg= 8.10, stdev= 5.76, samples=90 lat (msec) : 10=1.43%, 20=10.81%, 50=43.23%, 100=28.52%, 250=15.49% lat (msec) : 500=0.52% cpu : usr=0.05%, sys=1.73%, ctx=1149, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185036: Fri Jun 26 12:07:51 2026 read: IOPS=8, BW=135KiB/s (138kB/s)(6256KiB/46330msec) clat (msec): min=6, max=346, avg=49.61, stdev=41.37 lat (msec): min=6, max=346, avg=49.61, stdev=41.37 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 19], | 30.00th=[ 24], 40.00th=[ 29], 50.00th=[ 40], 60.00th=[ 45], | 70.00th=[ 54], 80.00th=[ 72], 90.00th=[ 108], 95.00th=[ 136], | 99.00th=[ 190], 99.50th=[ 213], 99.90th=[ 347], 99.95th=[ 347], | 99.99th=[ 347] bw ( KiB/s): min= 31, max= 447, per=27.23%, avg=140.22, stdev=95.02, samples=87 iops : min= 1, max= 27, avg= 8.00, stdev= 5.99, samples=87 write: IOPS=8, BW=130KiB/s (133kB/s)(6032KiB/46330msec); 0 zone resets clat (msec): min=14, max=325, avg=70.91, stdev=50.19 lat (msec): min=14, max=325, avg=70.91, stdev=50.19 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 22], 10.00th=[ 23], 20.00th=[ 34], | 30.00th=[ 40], 40.00th=[ 46], 50.00th=[ 55], 60.00th=[ 67], | 70.00th=[ 85], 80.00th=[ 104], 90.00th=[ 142], 95.00th=[ 165], | 99.00th=[ 257], 99.50th=[ 305], 99.90th=[ 326], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 31, max= 477, per=23.70%, avg=128.43, stdev=83.67, samples=90 iops : min= 1, max= 29, avg= 7.24, stdev= 5.31, samples=90 lat (msec) : 10=1.56%, 20=12.24%, 50=42.84%, 100=25.91%, 250=16.80% lat (msec) : 500=0.65% cpu : usr=0.06%, sys=1.94%, ctx=1313, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185037: Fri Jun 26 12:07:51 2026 read: IOPS=8, BW=130KiB/s (133kB/s)(6032KiB/46422msec) clat (msec): min=7, max=333, avg=51.49, stdev=49.87 lat (msec): min=7, max=333, avg=51.49, stdev=49.87 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 18], | 30.00th=[ 23], 40.00th=[ 27], 50.00th=[ 33], 60.00th=[ 41], | 70.00th=[ 53], 80.00th=[ 78], 90.00th=[ 120], 95.00th=[ 167], | 99.00th=[ 262], 99.50th=[ 321], 99.90th=[ 334], 99.95th=[ 334], | 99.99th=[ 334] bw ( KiB/s): min= 21, max= 538, per=25.62%, avg=131.96, stdev=89.90, samples=89 iops : min= 1, max= 33, avg= 7.47, stdev= 5.68, samples=89 write: IOPS=8, BW=135KiB/s (138kB/s)(6256KiB/46422msec); 0 zone resets clat (msec): min=13, max=317, avg=68.39, stdev=44.99 lat (msec): min=13, max=317, avg=68.40, stdev=44.99 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 21], 10.00th=[ 24], 20.00th=[ 33], | 30.00th=[ 39], 40.00th=[ 50], 50.00th=[ 57], 60.00th=[ 66], | 70.00th=[ 80], 80.00th=[ 101], 90.00th=[ 130], 95.00th=[ 150], | 99.00th=[ 236], 99.50th=[ 279], 99.90th=[ 317], 99.95th=[ 317], | 99.99th=[ 317] bw ( KiB/s): min= 31, max= 447, per=25.41%, avg=137.73, stdev=84.53, samples=89 iops : min= 1, max= 27, avg= 7.83, stdev= 5.33, samples=89 lat (msec) : 10=1.82%, 20=12.50%, 50=39.97%, 100=28.65%, 250=16.15% lat (msec) : 500=0.91% cpu : usr=0.04%, sys=1.87%, ctx=1190, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=516KiB/s (528kB/s), 125KiB/s-135KiB/s (128kB/s-138kB/s), io=23.4MiB (24.5MB), run=46074-46422msec WRITE: bw=543KiB/s (556kB/s), 130KiB/s-142KiB/s (133kB/s-145kB/s), io=24.6MiB (25.8MB), run=46074-46422msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=185054: Fri Jun 26 12:08:10 2026 read: IOPS=1, BW=1269KiB/s (1300kB/s)(7168KiB/5648msec) clat (usec): min=482, max=2757.3k, avg=613660.00, stdev=955266.85 lat (usec): min=483, max=2757.3k, avg=613661.36, stdev=955266.80 clat percentiles (usec): | 1.00th=[ 482], 5.00th=[ 482], 10.00th=[ 482], | 20.00th=[ 185598], 30.00th=[ 270533], 40.00th=[ 270533], | 50.00th=[ 295699], 60.00th=[ 341836], 70.00th=[ 341836], | 80.00th=[ 442500], 90.00th=[2768241], 95.00th=[2768241], | 99.00th=[2768241], 99.50th=[2768241], 99.90th=[2768241], | 99.95th=[2768241], 99.99th=[2768241] bw ( KiB/s): min= 1715, max= 4071, per=89.10%, avg=2400.25, stdev=1120.07, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=0, BW=907KiB/s (928kB/s)(5120KiB/5648msec); 0 zone resets clat (msec): min=159, max=314, avg=237.75, stdev=68.14 lat (msec): min=159, max=314, avg=237.82, stdev=68.13 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 161], | 30.00th=[ 194], 40.00th=[ 194], 50.00th=[ 218], 60.00th=[ 218], | 70.00th=[ 305], 80.00th=[ 305], 90.00th=[ 317], 95.00th=[ 317], | 99.00th=[ 317], 99.50th=[ 317], 99.90th=[ 317], 99.95th=[ 317], | 99.99th=[ 317] bw ( KiB/s): min= 2035, max= 3631, per=68.07%, avg=2567.00, stdev=921.45, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (usec) : 500=8.33% lat (msec) : 250=33.33%, 500=50.00%, >=2000=8.33% cpu : usr=0.02%, sys=7.44%, ctx=565, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185055: Fri Jun 26 12:08:10 2026 read: IOPS=0, BW=270KiB/s (276kB/s)(2048KiB/7586msec) clat (usec): min=665, max=386517, avg=193591.28, stdev=272838.67 lat (usec): min=666, max=386519, avg=193593.22, stdev=272839.64 clat percentiles (usec): | 1.00th=[ 668], 5.00th=[ 668], 10.00th=[ 668], 20.00th=[ 668], | 30.00th=[ 668], 40.00th=[ 668], 50.00th=[ 668], 60.00th=[387974], | 70.00th=[387974], 80.00th=[387974], 90.00th=[387974], 95.00th=[387974], | 99.00th=[387974], 99.50th=[387974], 99.90th=[387974], 99.95th=[387974], | 99.99th=[387974] bw ( KiB/s): min= 1796, max= 2019, per=70.81%, avg=1907.50, stdev=157.68, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1350KiB/s (1382kB/s)(10.0MiB/7586msec); 0 zone resets clat (msec): min=166, max=3213, avg=708.47, stdev=936.03 lat (msec): min=166, max=3213, avg=708.54, stdev=936.02 clat percentiles (msec): | 1.00th=[ 167], 5.00th=[ 167], 10.00th=[ 167], 20.00th=[ 176], | 30.00th=[ 205], 40.00th=[ 284], 50.00th=[ 326], 60.00th=[ 359], | 70.00th=[ 435], 80.00th=[ 709], 90.00th=[ 1217], 95.00th=[ 3205], | 99.00th=[ 3205], 99.50th=[ 3205], 99.90th=[ 3205], 99.95th=[ 3205], | 99.99th=[ 3205] bw ( KiB/s): min= 1996, max= 4039, per=83.19%, avg=3137.00, stdev=1043.34, samples=5 iops : min= 1, max= 3, avg= 2.20, stdev= 1.10, samples=5 lat (usec) : 750=8.33% lat (msec) : 250=25.00%, 500=41.67%, 750=8.33%, 2000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=6.25%, ctx=97, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185056: Fri Jun 26 12:08:10 2026 read: IOPS=0, BW=922KiB/s (944kB/s)(6144KiB/6665msec) clat (usec): min=955, max=417537, avg=226420.49, stdev=187256.42 lat (usec): min=957, max=417538, avg=226421.78, stdev=187256.33 clat percentiles (usec): | 1.00th=[ 955], 5.00th=[ 955], 10.00th=[ 955], 20.00th=[ 5997], | 30.00th=[ 5997], 40.00th=[204473], 50.00th=[204473], 60.00th=[346031], | 70.00th=[383779], 80.00th=[383779], 90.00th=[417334], 95.00th=[417334], | 99.00th=[417334], 99.50th=[417334], 99.90th=[417334], 99.95th=[417334], | 99.99th=[417334] bw ( KiB/s): min= 1980, max= 3631, per=89.69%, avg=2416.25, stdev=810.16, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=0, BW=922KiB/s (944kB/s)(6144KiB/6665msec); 0 zone resets clat (msec): min=228, max=3249, avg=874.68, stdev=1184.62 lat (msec): min=228, max=3249, avg=874.77, stdev=1184.62 clat percentiles (msec): | 1.00th=[ 228], 5.00th=[ 228], 10.00th=[ 228], 20.00th=[ 243], | 30.00th=[ 243], 40.00th=[ 243], 50.00th=[ 243], 60.00th=[ 481], | 70.00th=[ 810], 80.00th=[ 810], 90.00th=[ 3239], 95.00th=[ 3239], | 99.00th=[ 3239], 99.50th=[ 3239], 99.90th=[ 3239], 99.95th=[ 3239], | 99.99th=[ 3239] bw ( KiB/s): min= 2000, max= 3488, per=63.26%, avg=2385.50, stdev=735.14, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (usec) : 1000=8.33% lat (msec) : 10=8.33%, 250=33.33%, 500=33.33%, 1000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=6.17%, ctx=99, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185057: Fri Jun 26 12:08:10 2026 read: IOPS=0, BW=674KiB/s (690kB/s)(5120KiB/7602msec) clat (usec): min=307, max=353255, avg=70987.32, stdev=157792.54 lat (usec): min=308, max=353256, avg=70988.20, stdev=157792.67 clat percentiles (usec): | 1.00th=[ 310], 5.00th=[ 310], 10.00th=[ 310], 20.00th=[ 310], | 30.00th=[ 347], 40.00th=[ 347], 50.00th=[ 490], 60.00th=[ 490], | 70.00th=[ 537], 80.00th=[ 537], 90.00th=[354419], 95.00th=[354419], | 99.00th=[354419], 99.50th=[354419], 99.90th=[354419], 99.95th=[354419], | 99.99th=[354419] bw ( KiB/s): min= 7922, max= 7922, per=100.00%, avg=7922.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=0, BW=943KiB/s (966kB/s)(7168KiB/7602msec); 0 zone resets clat (msec): min=322, max=3888, avg=1017.58, stdev=1293.90 lat (msec): min=322, max=3888, avg=1018.49, stdev=1293.58 clat percentiles (msec): | 1.00th=[ 321], 5.00th=[ 321], 10.00th=[ 321], 20.00th=[ 359], | 30.00th=[ 380], 40.00th=[ 380], 50.00th=[ 443], 60.00th=[ 642], | 70.00th=[ 642], 80.00th=[ 1099], 90.00th=[ 3876], 95.00th=[ 3876], | 99.00th=[ 3876], 99.50th=[ 3876], 99.90th=[ 3876], 99.95th=[ 3876], | 99.99th=[ 3876] bw ( KiB/s): min= 1815, max= 2035, per=51.49%, avg=1941.60, stdev=90.47, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (usec) : 500=25.00%, 750=8.33% lat (msec) : 500=41.67%, 750=8.33%, 2000=8.33%, >=2000=8.33% cpu : usr=0.01%, sys=5.09%, ctx=98, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2694KiB/s (2759kB/s), 270KiB/s-1269KiB/s (276kB/s-1300kB/s), io=20.0MiB (20.0MB), run=5648-7602msec WRITE: bw=3772KiB/s (3862kB/s), 907KiB/s-1350KiB/s (928kB/s-1382kB/s), io=28.0MiB (29.4MB), run=5648-7602msec rand-rw: (groupid=0, jobs=1): err= 0: pid=185059: Fri Jun 26 12:08:12 2026 read: IOPS=0, BW=923KiB/s (945kB/s)(7168KiB/7768msec) clat (msec): min=140, max=2622, avg=924.55, stdev=1054.65 lat (msec): min=140, max=2622, avg=924.55, stdev=1054.65 clat percentiles (msec): | 1.00th=[ 140], 5.00th=[ 140], 10.00th=[ 140], 20.00th=[ 163], | 30.00th=[ 186], 40.00th=[ 186], 50.00th=[ 300], 60.00th=[ 852], | 70.00th=[ 852], 80.00th=[ 2198], 90.00th=[ 2635], 95.00th=[ 2635], | 99.00th=[ 2635], 99.50th=[ 2635], 99.90th=[ 2635], 99.95th=[ 2635], | 99.99th=[ 2635] bw ( KiB/s): min= 1932, max= 4055, per=100.00%, avg=2519.50, stdev=1025.07, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 write: IOPS=0, BW=659KiB/s (675kB/s)(5120KiB/7768msec); 0 zone resets clat (msec): min=164, max=453, avg=244.27, stdev=118.49 lat (msec): min=164, max=453, avg=244.65, stdev=118.34 clat percentiles (msec): | 1.00th=[ 165], 5.00th=[ 165], 10.00th=[ 165], 20.00th=[ 165], | 30.00th=[ 190], 40.00th=[ 190], 50.00th=[ 192], 60.00th=[ 192], | 70.00th=[ 224], 80.00th=[ 224], 90.00th=[ 456], 95.00th=[ 456], | 99.00th=[ 456], 99.50th=[ 456], 99.90th=[ 456], 99.95th=[ 456], | 99.99th=[ 456] bw ( KiB/s): min= 2027, max= 4096, per=80.71%, avg=2716.67, stdev=1194.54, samples=3 iops : min= 1, max= 4, avg= 2.00, stdev= 1.73, samples=3 lat (msec) : 250=58.33%, 500=16.67%, 1000=8.33%, >=2000=16.67% cpu : usr=0.00%, sys=1.91%, ctx=56, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185060: Fri Jun 26 12:08:12 2026 read: IOPS=0, BW=252KiB/s (258kB/s)(2048KiB/8119msec) clat (msec): min=311, max=333, avg=322.72, stdev=15.38 lat (msec): min=311, max=333, avg=322.72, stdev=15.38 clat percentiles (msec): | 1.00th=[ 313], 5.00th=[ 313], 10.00th=[ 313], 20.00th=[ 313], | 30.00th=[ 313], 40.00th=[ 313], 50.00th=[ 313], 60.00th=[ 334], | 70.00th=[ 334], 80.00th=[ 334], 90.00th=[ 334], 95.00th=[ 334], | 99.00th=[ 334], 99.50th=[ 334], 99.90th=[ 334], 99.95th=[ 334], | 99.99th=[ 334] bw ( KiB/s): min= 1648, max= 2048, per=76.87%, avg=1848.00, stdev=282.84, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1261KiB/s (1292kB/s)(10.0MiB/8119msec); 0 zone resets clat (msec): min=111, max=4058, avg=742.16, stdev=1207.96 lat (msec): min=111, max=4058, avg=742.55, stdev=1207.82 clat percentiles (msec): | 1.00th=[ 112], 5.00th=[ 112], 10.00th=[ 112], 20.00th=[ 201], | 30.00th=[ 201], 40.00th=[ 218], 50.00th=[ 243], 60.00th=[ 284], | 70.00th=[ 384], 80.00th=[ 506], 90.00th=[ 1217], 95.00th=[ 4044], | 99.00th=[ 4044], 99.50th=[ 4044], 99.90th=[ 4044], 99.95th=[ 4044], | 99.99th=[ 4044] bw ( KiB/s): min= 1681, max= 6047, per=93.54%, avg=3148.40, stdev=1878.45, samples=5 iops : min= 1, max= 5, avg= 2.20, stdev= 1.79, samples=5 lat (msec) : 250=41.67%, 500=33.33%, 750=8.33%, 2000=8.33%, >=2000=8.33% cpu : usr=0.01%, sys=1.97%, ctx=95, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185061: Fri Jun 26 12:08:12 2026 read: IOPS=0, BW=744KiB/s (761kB/s)(6144KiB/8263msec) clat (msec): min=103, max=855, avg=419.20, stdev=318.37 lat (msec): min=103, max=855, avg=419.20, stdev=318.37 clat percentiles (msec): | 1.00th=[ 105], 5.00th=[ 105], 10.00th=[ 105], 20.00th=[ 130], | 30.00th=[ 130], 40.00th=[ 205], 50.00th=[ 205], 60.00th=[ 527], | 70.00th=[ 693], 80.00th=[ 693], 90.00th=[ 860], 95.00th=[ 860], | 99.00th=[ 860], 99.50th=[ 860], 99.90th=[ 860], 99.95th=[ 860], | 99.99th=[ 860] bw ( KiB/s): min= 2015, max= 3864, per=100.00%, avg=2635.33, stdev=1064.07, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=0, BW=744KiB/s (761kB/s)(6144KiB/8263msec); 0 zone resets clat (msec): min=120, max=4148, avg=937.58, stdev=1579.53 lat (msec): min=120, max=4149, avg=937.72, stdev=1579.67 clat percentiles (msec): | 1.00th=[ 121], 5.00th=[ 121], 10.00th=[ 121], 20.00th=[ 159], | 30.00th=[ 159], 40.00th=[ 300], 50.00th=[ 300], 60.00th=[ 401], | 70.00th=[ 498], 80.00th=[ 498], 90.00th=[ 4144], 95.00th=[ 4144], | 99.00th=[ 4144], 99.50th=[ 4144], 99.90th=[ 4144], 99.95th=[ 4144], | 99.99th=[ 4144] bw ( KiB/s): min= 1681, max= 3002, per=66.45%, avg=2236.67, stdev=685.00, samples=3 iops : min= 1, max= 2, avg= 1.33, stdev= 0.58, samples=3 lat (msec) : 250=41.67%, 500=25.00%, 750=16.67%, 1000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=2.29%, ctx=81, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185062: Fri Jun 26 12:08:12 2026 read: IOPS=0, BW=601KiB/s (616kB/s)(5120KiB/8518msec) clat (msec): min=93, max=774, avg=381.63, stdev=270.25 lat (msec): min=93, max=774, avg=381.63, stdev=270.25 clat percentiles (msec): | 1.00th=[ 94], 5.00th=[ 94], 10.00th=[ 94], 20.00th=[ 94], | 30.00th=[ 163], 40.00th=[ 163], 50.00th=[ 430], 60.00th=[ 430], | 70.00th=[ 447], 80.00th=[ 447], 90.00th=[ 776], 95.00th=[ 776], | 99.00th=[ 776], 99.50th=[ 776], 99.90th=[ 776], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 2015, max= 2048, per=84.58%, avg=2033.25, stdev=15.11, samples=4 iops : min= 1, max= 2, avg= 1.25, stdev= 0.50, samples=4 write: IOPS=0, BW=842KiB/s (862kB/s)(7168KiB/8518msec); 0 zone resets clat (msec): min=114, max=4001, avg=930.51, stdev=1400.96 lat (msec): min=114, max=4001, avg=931.25, stdev=1400.76 clat percentiles (msec): | 1.00th=[ 114], 5.00th=[ 114], 10.00th=[ 114], 20.00th=[ 121], | 30.00th=[ 169], 40.00th=[ 169], 50.00th=[ 376], 60.00th=[ 617], | 70.00th=[ 617], 80.00th=[ 1116], 90.00th=[ 4010], 95.00th=[ 4010], | 99.00th=[ 4010], 99.50th=[ 4010], 99.90th=[ 4010], 99.95th=[ 4010], | 99.99th=[ 4010] bw ( KiB/s): min= 1560, max= 4087, per=67.06%, avg=2257.40, stdev=1039.83, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 lat (msec) : 100=8.33%, 250=33.33%, 500=25.00%, 750=8.33%, 1000=8.33% lat (msec) : 2000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=2.13%, ctx=103, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2404KiB/s (2462kB/s), 252KiB/s-923KiB/s (258kB/s-945kB/s), io=20.0MiB (20.0MB), run=7768-8518msec WRITE: bw=3366KiB/s (3447kB/s), 659KiB/s-1261KiB/s (675kB/s-1292kB/s), io=28.0MiB (29.4MB), run=7768-8518msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=185074: Fri Jun 26 12:08:31 2026 read: IOPS=1, BW=4504KiB/s (4613kB/s)(12.0MiB/2728msec) clat (msec): min=650, max=1042, avg=908.99, stdev=224.03 lat (msec): min=650, max=1042, avg=908.99, stdev=224.03 clat percentiles (msec): | 1.00th=[ 651], 5.00th=[ 651], 10.00th=[ 651], 20.00th=[ 651], | 30.00th=[ 651], 40.00th=[ 1036], 50.00th=[ 1036], 60.00th=[ 1036], | 70.00th=[ 1045], 80.00th=[ 1045], 90.00th=[ 1045], 95.00th=[ 1045], | 99.00th=[ 1045], 99.50th=[ 1045], 99.90th=[ 1045], 99.95th=[ 1045], | 99.99th=[ 1045] bw ( KiB/s): min= 5657, max= 8031, per=100.00%, avg=6844.00, stdev=1678.67, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=11.55%, ctx=248, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185075: Fri Jun 26 12:08:31 2026 read: IOPS=0, BW=1124KiB/s (1151kB/s)(4096KiB/3643msec) clat (nsec): min=807419k, max=807419k, avg=807419463.00, stdev= 0.00 lat (nsec): min=807422k, max=807422k, avg=807421541.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 810], 5.00th=[ 810], 10.00th=[ 810], 20.00th=[ 810], | 30.00th=[ 810], 40.00th=[ 810], 50.00th=[ 810], 60.00th=[ 810], | 70.00th=[ 810], 80.00th=[ 810], 90.00th=[ 810], 95.00th=[ 810], | 99.00th=[ 810], 99.50th=[ 810], 99.90th=[ 810], 99.95th=[ 810], | 99.99th=[ 810] bw ( KiB/s): min= 8110, max= 8110, per=100.00%, avg=8110.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2249KiB/s (2303kB/s)(8192KiB/3643msec); 0 zone resets clat (msec): min=994, max=1834, avg=1414.45, stdev=593.50 lat (msec): min=994, max=1834, avg=1414.73, stdev=593.65 clat percentiles (msec): | 1.00th=[ 995], 5.00th=[ 995], 10.00th=[ 995], 20.00th=[ 995], | 30.00th=[ 995], 40.00th=[ 995], 50.00th=[ 995], 60.00th=[ 1838], | 70.00th=[ 1838], 80.00th=[ 1838], 90.00th=[ 1838], 95.00th=[ 1838], | 99.00th=[ 1838], 99.50th=[ 1838], 99.90th=[ 1838], 99.95th=[ 1838], | 99.99th=[ 1838] bw ( KiB/s): min= 8078, max= 8078, per=92.23%, avg=8078.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 1000=66.67%, 2000=33.33% cpu : usr=0.00%, sys=2.86%, ctx=66, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185076: Fri Jun 26 12:08:31 2026 write: IOPS=0, BW=3285KiB/s (3364kB/s)(12.0MiB/3741msec); 0 zone resets clat (msec): min=350, max=1757, avg=1244.27, stdev=776.45 lat (msec): min=351, max=1757, avg=1246.61, stdev=777.95 clat percentiles (msec): | 1.00th=[ 351], 5.00th=[ 351], 10.00th=[ 351], 20.00th=[ 351], | 30.00th=[ 351], 40.00th=[ 1620], 50.00th=[ 1620], 60.00th=[ 1620], | 70.00th=[ 1754], 80.00th=[ 1754], 90.00th=[ 1754], 95.00th=[ 1754], | 99.00th=[ 1754], 99.50th=[ 1754], 99.90th=[ 1754], 99.95th=[ 1754], | 99.99th=[ 1754] bw ( KiB/s): min= 8000, max= 8031, per=91.51%, avg=8015.50, stdev=21.92, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.27%, sys=2.57%, ctx=36, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185077: Fri Jun 26 12:08:31 2026 write: IOPS=0, BW=3725KiB/s (3814kB/s)(12.0MiB/3299msec); 0 zone resets clat (msec): min=459, max=2186, avg=1095.00, stdev=949.60 lat (msec): min=459, max=2189, avg=1099.40, stdev=948.80 clat percentiles (msec): | 1.00th=[ 460], 5.00th=[ 460], 10.00th=[ 460], 20.00th=[ 460], | 30.00th=[ 460], 40.00th=[ 642], 50.00th=[ 642], 60.00th=[ 642], | 70.00th=[ 2198], 80.00th=[ 2198], 90.00th=[ 2198], 95.00th=[ 2198], | 99.00th=[ 2198], 99.50th=[ 2198], 99.90th=[ 2198], 99.95th=[ 2198], | 99.99th=[ 2198] bw ( KiB/s): min= 5902, max= 8062, per=79.71%, avg=6982.00, stdev=1527.35, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=33.33%, >=2000=33.33% cpu : usr=0.03%, sys=4.06%, ctx=52, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4497KiB/s (4605kB/s), 1124KiB/s-4504KiB/s (1151kB/s-4613kB/s), io=16.0MiB (16.8MB), run=2728-3643msec WRITE: bw=8759KiB/s (8969kB/s), 2249KiB/s-3725KiB/s (2303kB/s-3814kB/s), io=32.0MiB (33.6MB), run=3299-3741msec rand-rw: (groupid=0, jobs=1): err= 0: pid=185069: Fri Jun 26 12:08:32 2026 read: IOPS=0, BW=3131KiB/s (3206kB/s)(12.0MiB/3925msec) clat (msec): min=912, max=1713, avg=1271.48, stdev=407.14 lat (msec): min=912, max=1713, avg=1271.48, stdev=407.14 clat percentiles (msec): | 1.00th=[ 911], 5.00th=[ 911], 10.00th=[ 911], 20.00th=[ 911], | 30.00th=[ 911], 40.00th=[ 1183], 50.00th=[ 1183], 60.00th=[ 1183], | 70.00th=[ 1720], 80.00th=[ 1720], 90.00th=[ 1720], 95.00th=[ 1720], | 99.00th=[ 1720], 99.50th=[ 1720], 99.90th=[ 1720], 99.95th=[ 1720], | 99.99th=[ 1720] bw ( KiB/s): min= 8062, max= 8159, per=100.00%, avg=8110.50, stdev=68.59, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=1.89%, ctx=47, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185070: Fri Jun 26 12:08:32 2026 read: IOPS=0, BW=887KiB/s (908kB/s)(4096KiB/4619msec) clat (nsec): min=1586.1M, max=1586.1M, avg=1586050978.00, stdev= 0.00 lat (nsec): min=1586.1M, max=1586.1M, avg=1586053547.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1586], 5.00th=[ 1586], 10.00th=[ 1586], 20.00th=[ 1586], | 30.00th=[ 1586], 40.00th=[ 1586], 50.00th=[ 1586], 60.00th=[ 1586], | 70.00th=[ 1586], 80.00th=[ 1586], 90.00th=[ 1586], 95.00th=[ 1586], | 99.00th=[ 1586], 99.50th=[ 1586], 99.90th=[ 1586], 99.95th=[ 1586], | 99.99th=[ 1586] bw ( KiB/s): min= 8031, max= 8031, per=100.00%, avg=8031.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1774KiB/s (1816kB/s)(8192KiB/4619msec); 0 zone resets clat (msec): min=1279, max=1709, avg=1494.21, stdev=303.84 lat (msec): min=1279, max=1709, avg=1494.51, stdev=303.70 clat percentiles (msec): | 1.00th=[ 1284], 5.00th=[ 1284], 10.00th=[ 1284], 20.00th=[ 1284], | 30.00th=[ 1284], 40.00th=[ 1284], 50.00th=[ 1284], 60.00th=[ 1703], | 70.00th=[ 1703], 80.00th=[ 1703], 90.00th=[ 1703], 95.00th=[ 1703], | 99.00th=[ 1703], 99.50th=[ 1703], 99.90th=[ 1703], 99.95th=[ 1703], | 99.99th=[ 1703] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 2000=100.00% cpu : usr=0.00%, sys=2.40%, ctx=44, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185071: Fri Jun 26 12:08:32 2026 write: IOPS=1, BW=5364KiB/s (5492kB/s)(12.0MiB/2291msec); 0 zone resets clat (msec): min=466, max=1097, avg=717.04, stdev=335.20 lat (msec): min=467, max=1098, avg=717.49, stdev=335.12 clat percentiles (msec): | 1.00th=[ 468], 5.00th=[ 468], 10.00th=[ 468], 20.00th=[ 468], | 30.00th=[ 468], 40.00th=[ 584], 50.00th=[ 584], 60.00th=[ 584], | 70.00th=[ 1099], 80.00th=[ 1099], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 5483, max= 8062, per=95.47%, avg=6772.50, stdev=1823.63, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=4.54%, ctx=54, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=185072: Fri Jun 26 12:08:32 2026 write: IOPS=1, BW=5090KiB/s (5212kB/s)(12.0MiB/2414msec); 0 zone resets clat (msec): min=417, max=1185, avg=799.40, stdev=384.01 lat (msec): min=427, max=1189, avg=804.11, stdev=381.49 clat percentiles (msec): | 1.00th=[ 418], 5.00th=[ 418], 10.00th=[ 418], 20.00th=[ 418], | 30.00th=[ 418], 40.00th=[ 793], 50.00th=[ 793], 60.00th=[ 793], | 70.00th=[ 1183], 80.00th=[ 1183], 90.00th=[ 1183], 95.00th=[ 1183], | 99.00th=[ 1183], 99.50th=[ 1183], 99.90th=[ 1183], 99.95th=[ 1183], | 99.99th=[ 1183] bw ( KiB/s): min= 7198, max= 8000, per=100.00%, avg=7599.00, stdev=567.10, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=4.14%, ctx=52, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3547KiB/s (3632kB/s), 887KiB/s-3131KiB/s (908kB/s-3206kB/s), io=16.0MiB (16.8MB), run=3925-4619msec WRITE: bw=7094KiB/s (7264kB/s), 1774KiB/s-5364KiB/s (1816kB/s-5492kB/s), io=32.0MiB (33.6MB), run=2291-4619msec