== sanity test 398b: DIO and buffer IO race ============== 17:51:47 (1778363507) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.44796 s, 20.6 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=196150: Sat May 9 17:52:41 2026 read: IOPS=32, BW=132KiB/s (135kB/s)(5992KiB/45440msec) clat (usec): min=44, max=564346, avg=16722.36, stdev=16760.17 lat (usec): min=45, max=564347, avg=16723.08, stdev=16760.18 clat percentiles (usec): | 1.00th=[ 95], 5.00th=[ 6456], 10.00th=[ 7898], 20.00th=[ 9503], | 30.00th=[ 11076], 40.00th=[ 12911], 50.00th=[ 14746], 60.00th=[ 16909], | 70.00th=[ 19006], 80.00th=[ 22152], 90.00th=[ 27132], 95.00th=[ 32375], | 99.00th=[ 49021], 99.50th=[ 51643], 99.90th=[ 90702], 99.95th=[566232], | 99.99th=[566232] bw ( KiB/s): min= 56, max= 207, per=25.00%, avg=131.48, stdev=31.31, samples=90 iops : min= 14, max= 51, avg=32.53, stdev= 7.80, samples=90 write: IOPS=34, BW=139KiB/s (142kB/s)(6296KiB/45440msec); 0 zone resets clat (usec): min=794, max=62717, avg=12839.80, stdev=9926.39 lat (usec): min=795, max=62718, avg=12840.97, stdev=9926.32 clat percentiles (usec): | 1.00th=[ 1450], 5.00th=[ 2245], 10.00th=[ 2638], 20.00th=[ 3261], | 30.00th=[ 5211], 40.00th=[ 8848], 50.00th=[10814], 60.00th=[13173], | 70.00th=[16057], 80.00th=[20579], 90.00th=[26870], 95.00th=[31851], | 99.00th=[43254], 99.50th=[47973], 99.90th=[52691], 99.95th=[62653], | 99.99th=[62653] bw ( KiB/s): min= 87, max= 247, per=25.11%, avg=138.37, stdev=32.77, samples=90 iops : min= 21, max= 61, avg=34.26, stdev= 8.17, samples=90 lat (usec) : 50=0.07%, 100=0.49%, 250=0.75%, 1000=0.07% lat (msec) : 2=1.50%, 4=11.59%, 10=20.38%, 20=41.24%, 50=23.40% lat (msec) : 100=0.49%, 750=0.03% cpu : usr=0.11%, sys=23.75%, ctx=8698, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196151: Sat May 9 17:52:41 2026 read: IOPS=32, BW=130KiB/s (133kB/s)(5940KiB/45577msec) clat (usec): min=26, max=553139, avg=16798.32, stdev=16457.28 lat (usec): min=27, max=553139, avg=16799.02, stdev=16457.29 clat percentiles (usec): | 1.00th=[ 83], 5.00th=[ 6915], 10.00th=[ 8094], 20.00th=[ 9503], | 30.00th=[ 10945], 40.00th=[ 12649], 50.00th=[ 14746], 60.00th=[ 16909], | 70.00th=[ 19268], 80.00th=[ 21890], 90.00th=[ 26870], 95.00th=[ 32900], | 99.00th=[ 48497], 99.50th=[ 56886], 99.90th=[ 64750], 99.95th=[549454], | 99.99th=[549454] bw ( KiB/s): min= 79, max= 184, per=24.68%, avg=129.83, stdev=26.44, samples=90 iops : min= 19, max= 46, avg=32.12, stdev= 6.60, samples=90 write: IOPS=34, BW=139KiB/s (143kB/s)(6348KiB/45577msec); 0 zone resets clat (usec): min=705, max=65607, avg=12885.82, stdev=10416.63 lat (usec): min=706, max=65608, avg=12886.71, stdev=10416.64 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 1991], 10.00th=[ 2573], 20.00th=[ 3130], | 30.00th=[ 4621], 40.00th=[ 8586], 50.00th=[10814], 60.00th=[13042], | 70.00th=[16188], 80.00th=[21103], 90.00th=[27395], 95.00th=[32900], | 99.00th=[45351], 99.50th=[52691], 99.90th=[61080], 99.95th=[65799], | 99.99th=[65799] bw ( KiB/s): min= 56, max= 279, per=25.06%, avg=138.06, stdev=39.32, samples=90 iops : min= 14, max= 69, avg=34.18, stdev= 9.82, samples=90 lat (usec) : 50=0.20%, 100=0.39%, 250=0.33%, 750=0.10%, 1000=0.33% lat (msec) : 2=2.21%, 4=11.75%, 10=19.34%, 20=40.76%, 50=23.83% lat (msec) : 100=0.75%, 750=0.03% cpu : usr=0.14%, sys=23.40%, ctx=8762, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196152: Sat May 9 17:52:41 2026 read: IOPS=33, BW=133KiB/s (136kB/s)(6028KiB/45423msec) clat (usec): min=50, max=66262, avg=16256.27, stdev=8369.67 lat (usec): min=50, max=66263, avg=16257.12, stdev=8369.64 clat percentiles (usec): | 1.00th=[ 99], 5.00th=[ 6390], 10.00th=[ 8094], 20.00th=[ 9634], | 30.00th=[11338], 40.00th=[12911], 50.00th=[14877], 60.00th=[16909], | 70.00th=[19268], 80.00th=[21890], 90.00th=[26608], 95.00th=[31851], | 99.00th=[44303], 99.50th=[49546], 99.90th=[65274], 99.95th=[66323], | 99.99th=[66323] bw ( KiB/s): min= 72, max= 200, per=25.20%, avg=132.53, stdev=27.42, samples=90 iops : min= 18, max= 50, avg=32.80, stdev= 6.81, samples=90 write: IOPS=34, BW=138KiB/s (141kB/s)(6260KiB/45423msec); 0 zone resets clat (usec): min=817, max=555162, avg=13248.57, stdev=16940.15 lat (usec): min=818, max=555162, avg=13250.13, stdev=16939.86 clat percentiles (usec): | 1.00th=[ 1385], 5.00th=[ 2212], 10.00th=[ 2671], 20.00th=[ 3261], | 30.00th=[ 5211], 40.00th=[ 8586], 50.00th=[ 11076], 60.00th=[ 13304], | 70.00th=[ 16581], 80.00th=[ 20579], 90.00th=[ 26870], 95.00th=[ 31851], | 99.00th=[ 43254], 99.50th=[ 46924], 99.90th=[ 71828], 99.95th=[557843], | 99.99th=[557843] bw ( KiB/s): min= 71, max= 240, per=24.91%, avg=137.28, stdev=30.99, samples=90 iops : min= 17, max= 60, avg=33.98, stdev= 7.70, samples=90 lat (usec) : 100=0.55%, 250=0.78%, 500=0.03%, 1000=0.10% lat (msec) : 2=1.63%, 4=11.13%, 10=19.47%, 20=42.09%, 50=23.76% lat (msec) : 100=0.42%, 750=0.03% cpu : usr=0.12%, sys=23.51%, ctx=8617, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196153: Sat May 9 17:52:41 2026 read: IOPS=33, BW=135KiB/s (138kB/s)(6052KiB/44933msec) clat (usec): min=29, max=61616, avg=15870.56, stdev=8391.29 lat (usec): min=30, max=61617, avg=15871.38, stdev=8391.34 clat percentiles (usec): | 1.00th=[ 96], 5.00th=[ 6652], 10.00th=[ 7635], 20.00th=[ 9241], | 30.00th=[10552], 40.00th=[12125], 50.00th=[14353], 60.00th=[16450], | 70.00th=[18482], 80.00th=[21365], 90.00th=[26346], 95.00th=[31851], | 99.00th=[44827], 99.50th=[48497], 99.90th=[58459], 99.95th=[61604], | 99.99th=[61604] bw ( KiB/s): min= 64, max= 191, per=25.56%, avg=134.43, stdev=26.43, samples=89 iops : min= 16, max= 47, avg=33.27, stdev= 6.53, samples=89 write: IOPS=34, BW=139KiB/s (142kB/s)(6236KiB/44933msec); 0 zone resets clat (usec): min=715, max=558072, avg=13300.93, stdev=17291.14 lat (usec): min=717, max=558072, avg=13303.66, stdev=17290.49 clat percentiles (usec): | 1.00th=[ 1106], 5.00th=[ 1991], 10.00th=[ 2606], 20.00th=[ 3163], | 30.00th=[ 4555], 40.00th=[ 8455], 50.00th=[ 10814], 60.00th=[ 13435], | 70.00th=[ 16319], 80.00th=[ 20841], 90.00th=[ 27395], 95.00th=[ 34341], | 99.00th=[ 45351], 99.50th=[ 50594], 99.90th=[ 66847], 99.95th=[557843], | 99.99th=[557843] bw ( KiB/s): min= 56, max= 221, per=25.12%, avg=138.42, stdev=35.75, samples=89 iops : min= 14, max= 55, avg=34.27, stdev= 8.99, samples=89 lat (usec) : 50=0.10%, 100=0.46%, 250=0.65%, 500=0.03%, 750=0.07% lat (usec) : 1000=0.23% lat (msec) : 2=2.28%, 4=11.39%, 10=20.70%, 20=40.79%, 50=22.82% lat (msec) : 100=0.46%, 750=0.03% cpu : usr=0.11%, sys=23.59%, ctx=8546, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=527KiB/s (539kB/s), 130KiB/s-135KiB/s (133kB/s-138kB/s), io=23.4MiB (24.6MB), run=44933-45577msec WRITE: bw=552KiB/s (565kB/s), 138KiB/s-139KiB/s (141kB/s-143kB/s), io=24.6MiB (25.7MB), run=44933-45577msec rand-rw: (groupid=0, jobs=1): err= 0: pid=196155: Sat May 9 17:52:49 2026 read: IOPS=28, BW=113KiB/s (116kB/s)(5992KiB/52951msec) clat (msec): min=2, max=592, avg=14.85, stdev=18.60 lat (msec): min=2, max=592, avg=14.85, stdev=18.60 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 8], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 31], 95.00th=[ 38], | 99.00th=[ 56], 99.50th=[ 63], 99.90th=[ 79], 99.95th=[ 592], | 99.99th=[ 592] bw ( KiB/s): min= 39, max= 368, per=25.14%, avg=113.63, stdev=73.78, samples=104 iops : min= 9, max= 92, avg=28.08, stdev=18.57, samples=104 write: IOPS=29, BW=119KiB/s (122kB/s)(6296KiB/52951msec); 0 zone resets clat (usec): min=4729, max=88059, avg=19478.63, stdev=13088.87 lat (usec): min=4730, max=88060, avg=19479.55, stdev=13088.88 clat percentiles (usec): | 1.00th=[ 5407], 5.00th=[ 6259], 10.00th=[ 6980], 20.00th=[ 8291], | 30.00th=[10028], 40.00th=[12125], 50.00th=[15139], 60.00th=[19006], | 70.00th=[24249], 80.00th=[30016], 90.00th=[37487], 95.00th=[44827], | 99.00th=[61604], 99.50th=[69731], 99.90th=[84411], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 40, max= 336, per=25.23%, avg=119.32, stdev=71.85, samples=104 iops : min= 10, max= 84, avg=29.50, stdev=18.10, samples=104 lat (msec) : 4=1.40%, 10=35.55%, 20=32.55%, 50=28.35%, 100=2.12% lat (msec) : 750=0.03% cpu : usr=0.09%, sys=12.05%, ctx=5674, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196156: Sat May 9 17:52:49 2026 read: IOPS=28, BW=112KiB/s (115kB/s)(5940KiB/52847msec) clat (msec): min=2, max=573, avg=14.71, stdev=18.37 lat (msec): min=2, max=573, avg=14.71, stdev=18.37 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 31], 95.00th=[ 39], | 99.00th=[ 52], 99.50th=[ 60], 99.90th=[ 84], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 32, max= 392, per=25.02%, avg=113.10, stdev=75.10, samples=104 iops : min= 8, max= 98, avg=27.94, stdev=18.91, samples=104 write: IOPS=30, BW=120KiB/s (123kB/s)(6348KiB/52847msec); 0 zone resets clat (usec): min=4620, max=86783, avg=19504.93, stdev=13259.09 lat (usec): min=4621, max=86784, avg=19505.75, stdev=13259.14 clat percentiles (usec): | 1.00th=[ 5342], 5.00th=[ 6259], 10.00th=[ 6915], 20.00th=[ 8225], | 30.00th=[10290], 40.00th=[12518], 50.00th=[15533], 60.00th=[19006], | 70.00th=[23987], 80.00th=[29492], 90.00th=[36963], 95.00th=[45876], | 99.00th=[63701], 99.50th=[69731], 99.90th=[84411], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 32, max= 376, per=25.52%, avg=120.70, stdev=73.96, samples=104 iops : min= 8, max= 94, avg=29.85, stdev=18.62, samples=104 lat (msec) : 4=1.37%, 10=36.49%, 20=31.67%, 50=27.93%, 100=2.51% lat (msec) : 750=0.03% cpu : usr=0.08%, sys=12.07%, ctx=5672, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196157: Sat May 9 17:52:49 2026 read: IOPS=28, BW=114KiB/s (116kB/s)(6028KiB/53046msec) clat (usec): min=2590, max=73414, avg=14843.76, stdev=11712.48 lat (usec): min=2590, max=73416, avg=14844.46, stdev=11712.52 clat percentiles (usec): | 1.00th=[ 3425], 5.00th=[ 4178], 10.00th=[ 4621], 20.00th=[ 5538], | 30.00th=[ 7111], 40.00th=[ 9241], 50.00th=[11207], 60.00th=[12911], | 70.00th=[16450], 80.00th=[22152], 90.00th=[31851], 95.00th=[40109], | 99.00th=[54789], 99.50th=[57934], 99.90th=[71828], 99.95th=[73925], | 99.99th=[73925] bw ( KiB/s): min= 26, max= 368, per=25.11%, avg=113.49, stdev=74.55, samples=105 iops : min= 6, max= 92, avg=27.99, stdev=18.80, samples=105 write: IOPS=29, BW=118KiB/s (121kB/s)(6260KiB/53046msec); 0 zone resets clat (msec): min=4, max=561, avg=19.56, stdev=18.68 lat (msec): min=4, max=561, avg=19.57, stdev=18.68 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 30], 90.00th=[ 38], 95.00th=[ 44], | 99.00th=[ 63], 99.50th=[ 69], 99.90th=[ 86], 99.95th=[ 558], | 99.99th=[ 558] bw ( KiB/s): min= 26, max= 320, per=24.86%, avg=117.60, stdev=72.50, samples=105 iops : min= 6, max= 80, avg=29.02, stdev=18.29, samples=105 lat (msec) : 4=1.89%, 10=34.93%, 20=32.26%, 50=28.65%, 100=2.25% lat (msec) : 750=0.03% cpu : usr=0.10%, sys=12.06%, ctx=5601, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196158: Sat May 9 17:52:49 2026 read: IOPS=28, BW=114KiB/s (117kB/s)(6052KiB/52989msec) clat (usec): min=3127, max=76629, avg=14761.36, stdev=11894.56 lat (usec): min=3127, max=76630, avg=14761.98, stdev=11894.59 clat percentiles (usec): | 1.00th=[ 3523], 5.00th=[ 4228], 10.00th=[ 4686], 20.00th=[ 5473], | 30.00th=[ 7046], 40.00th=[ 9241], 50.00th=[10814], 60.00th=[12649], | 70.00th=[16188], 80.00th=[21627], 90.00th=[31851], 95.00th=[40633], | 99.00th=[56361], 99.50th=[62129], 99.90th=[76022], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 24, max= 352, per=25.29%, avg=114.31, stdev=70.81, samples=104 iops : min= 6, max= 88, avg=28.25, stdev=17.84, samples=104 write: IOPS=29, BW=118KiB/s (121kB/s)(6236KiB/52989msec); 0 zone resets clat (msec): min=4, max=557, avg=19.62, stdev=18.61 lat (msec): min=4, max=557, avg=19.62, stdev=18.61 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 29], 90.00th=[ 37], 95.00th=[ 44], | 99.00th=[ 62], 99.50th=[ 70], 99.90th=[ 85], 99.95th=[ 558], | 99.99th=[ 558] bw ( KiB/s): min= 47, max= 360, per=24.98%, avg=118.16, stdev=73.73, samples=104 iops : min= 11, max= 90, avg=29.21, stdev=18.58, samples=104 lat (msec) : 4=1.86%, 10=35.58%, 20=31.32%, 50=28.74%, 100=2.47% lat (msec) : 750=0.03% cpu : usr=0.11%, sys=12.09%, ctx=5471, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=453KiB/s (464kB/s), 112KiB/s-114KiB/s (115kB/s-117kB/s), io=23.4MiB (24.6MB), run=52847-53046msec WRITE: bw=474KiB/s (485kB/s), 118KiB/s-120KiB/s (121kB/s-123kB/s), io=24.6MiB (25.7MB), run=52847-53046msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=196178: Sat May 9 17:53:07 2026 read: IOPS=29, BW=473KiB/s (485kB/s)(5920KiB/12506msec) clat (usec): min=35, max=64891, avg=17066.61, stdev=9449.97 lat (usec): min=35, max=64891, avg=17067.24, stdev=9449.97 clat percentiles (usec): | 1.00th=[ 81], 5.00th=[ 119], 10.00th=[ 7767], 20.00th=[10421], | 30.00th=[12125], 40.00th=[14484], 50.00th=[16319], 60.00th=[18220], | 70.00th=[20579], 80.00th=[23725], 90.00th=[27657], 95.00th=[30802], | 99.00th=[48497], 99.50th=[63177], 99.90th=[64750], 99.95th=[64750], | 99.99th=[64750] bw ( KiB/s): min= 287, max= 672, per=25.01%, avg=466.92, stdev=97.78, samples=24 iops : min= 17, max= 42, avg=28.75, stdev= 6.28, samples=24 write: IOPS=31, BW=509KiB/s (521kB/s)(6368KiB/12506msec); 0 zone resets clat (usec): min=1245, max=55647, avg=15363.05, stdev=11495.69 lat (usec): min=1246, max=55648, avg=15364.34, stdev=11495.87 clat percentiles (usec): | 1.00th=[ 1876], 5.00th=[ 3294], 10.00th=[ 3884], 20.00th=[ 4817], | 30.00th=[ 6783], 40.00th=[10028], 50.00th=[12911], 60.00th=[15401], | 70.00th=[18482], 80.00th=[24773], 90.00th=[32900], 95.00th=[39060], | 99.00th=[52691], 99.50th=[53740], 99.90th=[55837], 99.95th=[55837], | 99.99th=[55837] bw ( KiB/s): min= 160, max= 736, per=25.92%, avg=509.00, stdev=133.75, samples=24 iops : min= 10, max= 46, avg=31.42, stdev= 8.43, samples=24 lat (usec) : 50=0.13%, 100=1.30%, 250=2.21% lat (msec) : 2=0.65%, 4=5.60%, 10=19.40%, 20=40.76%, 50=28.65% lat (msec) : 100=1.30% cpu : usr=0.11%, sys=25.36%, ctx=2524, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196179: Sat May 9 17:53:07 2026 read: IOPS=27, BW=448KiB/s (458kB/s)(5744KiB/12829msec) clat (usec): min=70, max=59350, avg=18566.63, stdev=10078.08 lat (usec): min=71, max=59351, avg=18567.27, stdev=10078.07 clat percentiles (usec): | 1.00th=[ 86], 5.00th=[ 6259], 10.00th=[ 9241], 20.00th=[11338], | 30.00th=[12911], 40.00th=[14746], 50.00th=[16319], 60.00th=[18482], | 70.00th=[21365], 80.00th=[25035], 90.00th=[31851], 95.00th=[37487], | 99.00th=[55313], 99.50th=[57934], 99.90th=[59507], 99.95th=[59507], | 99.99th=[59507] bw ( KiB/s): min= 286, max= 608, per=23.81%, avg=444.52, stdev=82.48, samples=25 iops : min= 17, max= 38, avg=27.36, stdev= 5.17, samples=25 write: IOPS=31, BW=510KiB/s (522kB/s)(6544KiB/12829msec); 0 zone resets clat (usec): min=974, max=52776, avg=14882.66, stdev=9758.42 lat (usec): min=976, max=52777, avg=14883.95, stdev=9758.44 clat percentiles (usec): | 1.00th=[ 1778], 5.00th=[ 3359], 10.00th=[ 4080], 20.00th=[ 5014], | 30.00th=[ 7898], 40.00th=[11207], 50.00th=[13566], 60.00th=[15795], | 70.00th=[18744], 80.00th=[22676], 90.00th=[28181], 95.00th=[33424], | 99.00th=[42206], 99.50th=[45351], 99.90th=[52691], 99.95th=[52691], | 99.99th=[52691] bw ( KiB/s): min= 318, max= 702, per=25.53%, avg=501.32, stdev=107.78, samples=25 iops : min= 19, max= 43, avg=30.88, stdev= 6.61, samples=25 lat (usec) : 100=0.91%, 250=1.04%, 1000=0.13% lat (msec) : 2=0.52%, 4=4.30%, 10=17.84%, 20=45.18%, 50=29.04% lat (msec) : 100=1.04% cpu : usr=0.12%, sys=25.55%, ctx=2604, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196180: Sat May 9 17:53:07 2026 read: IOPS=31, BW=507KiB/s (519kB/s)(6256KiB/12343msec) clat (usec): min=34, max=78125, avg=16971.12, stdev=9106.90 lat (usec): min=35, max=78126, avg=16971.82, stdev=9106.89 clat percentiles (usec): | 1.00th=[ 65], 5.00th=[ 113], 10.00th=[ 8979], 20.00th=[10814], | 30.00th=[12518], 40.00th=[14091], 50.00th=[16319], 60.00th=[18220], | 70.00th=[20317], 80.00th=[22938], 90.00th=[26870], 95.00th=[30802], | 99.00th=[46924], 99.50th=[52691], 99.90th=[78119], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 351, max= 704, per=27.27%, avg=509.21, stdev=95.36, samples=24 iops : min= 21, max= 44, avg=31.42, stdev= 6.04, samples=24 write: IOPS=30, BW=489KiB/s (500kB/s)(6032KiB/12343msec); 0 zone resets clat (usec): min=1341, max=51555, avg=14998.09, stdev=10832.54 lat (usec): min=1343, max=51557, avg=15003.43, stdev=10829.85 clat percentiles (usec): | 1.00th=[ 1565], 5.00th=[ 3392], 10.00th=[ 4080], 20.00th=[ 4948], | 30.00th=[ 7111], 40.00th=[10290], 50.00th=[12256], 60.00th=[14877], | 70.00th=[18744], 80.00th=[23462], 90.00th=[30540], 95.00th=[38011], | 99.00th=[47973], 99.50th=[51643], 99.90th=[51643], 99.95th=[51643], | 99.99th=[51643] bw ( KiB/s): min= 191, max= 669, per=24.61%, avg=483.29, stdev=126.06, samples=24 iops : min= 11, max= 41, avg=29.79, stdev= 7.96, samples=24 lat (usec) : 50=0.26%, 100=1.30%, 250=1.95% lat (msec) : 2=0.78%, 4=3.52%, 10=19.79%, 20=42.84%, 50=28.78% lat (msec) : 100=0.78% cpu : usr=0.11%, sys=25.81%, ctx=2565, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196181: Sat May 9 17:53:07 2026 read: IOPS=30, BW=482KiB/s (493kB/s)(6032KiB/12519msec) clat (usec): min=46, max=73268, avg=17432.55, stdev=9662.75 lat (usec): min=46, max=73269, avg=17433.25, stdev=9662.73 clat percentiles (usec): | 1.00th=[ 87], 5.00th=[ 123], 10.00th=[ 8979], 20.00th=[10945], | 30.00th=[12518], 40.00th=[14484], 50.00th=[16581], 60.00th=[18744], | 70.00th=[21365], 80.00th=[23725], 90.00th=[28181], 95.00th=[30802], | 99.00th=[56886], 99.50th=[70779], 99.90th=[72877], 99.95th=[72877], | 99.99th=[72877] bw ( KiB/s): min= 320, max= 606, per=25.50%, avg=476.00, stdev=71.73, samples=24 iops : min= 20, max= 37, avg=29.33, stdev= 4.40, samples=24 write: IOPS=31, BW=500KiB/s (512kB/s)(6256KiB/12519msec); 0 zone resets clat (usec): min=1678, max=59359, avg=15161.90, stdev=11352.29 lat (usec): min=1680, max=59360, avg=15163.25, stdev=11352.30 clat percentiles (usec): | 1.00th=[ 1942], 5.00th=[ 3359], 10.00th=[ 3884], 20.00th=[ 4948], | 30.00th=[ 6718], 40.00th=[ 9503], 50.00th=[11863], 60.00th=[14615], | 70.00th=[19006], 80.00th=[24511], 90.00th=[31851], 95.00th=[38011], | 99.00th=[54789], 99.50th=[56361], 99.90th=[59507], 99.95th=[59507], | 99.99th=[59507] bw ( KiB/s): min= 319, max= 768, per=25.48%, avg=500.42, stdev=95.46, samples=24 iops : min= 19, max= 48, avg=30.88, stdev= 6.15, samples=24 lat (usec) : 50=0.13%, 100=1.17%, 250=2.21%, 500=0.13%, 1000=0.13% lat (msec) : 2=0.52%, 4=5.73%, 10=19.40%, 20=38.41%, 50=30.99% lat (msec) : 100=1.17% cpu : usr=0.14%, sys=25.38%, ctx=2640, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1867KiB/s (1912kB/s), 448KiB/s-507KiB/s (458kB/s-519kB/s), io=23.4MiB (24.5MB), run=12343-12829msec WRITE: bw=1964KiB/s (2011kB/s), 489KiB/s-510KiB/s (500kB/s-522kB/s), io=24.6MiB (25.8MB), run=12343-12829msec rand-rw: (groupid=0, jobs=1): err= 0: pid=196183: Sat May 9 17:53:09 2026 read: IOPS=25, BW=407KiB/s (417kB/s)(5920KiB/14539msec) clat (usec): min=3371, max=64675, avg=15621.48, stdev=12673.76 lat (usec): min=3372, max=64675, avg=15622.06, stdev=12673.78 clat percentiles (usec): | 1.00th=[ 4047], 5.00th=[ 4817], 10.00th=[ 5276], 20.00th=[ 5866], | 30.00th=[ 7046], 40.00th=[ 9241], 50.00th=[11076], 60.00th=[13304], | 70.00th=[17171], 80.00th=[22414], 90.00th=[33817], 95.00th=[45351], | 99.00th=[60031], 99.50th=[61604], 99.90th=[64750], 99.95th=[64750], | 99.99th=[64750] bw ( KiB/s): min= 160, max= 1309, per=23.87%, avg=389.79, stdev=268.35, samples=28 iops : min= 10, max= 81, avg=23.82, stdev=16.82, samples=28 write: IOPS=27, BW=438KiB/s (449kB/s)(6368KiB/14539msec); 0 zone resets clat (usec): min=5646, max=77188, avg=21824.52, stdev=14447.54 lat (usec): min=5647, max=77189, avg=21825.93, stdev=14447.67 clat percentiles (usec): | 1.00th=[ 5866], 5.00th=[ 6652], 10.00th=[ 7439], 20.00th=[ 8848], | 30.00th=[11207], 40.00th=[14746], 50.00th=[17957], 60.00th=[21627], | 70.00th=[26346], 80.00th=[32900], 90.00th=[44303], 95.00th=[51119], | 99.00th=[67634], 99.50th=[74974], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 159, max= 1344, per=24.51%, avg=421.14, stdev=250.17, samples=28 iops : min= 9, max= 84, avg=25.79, stdev=15.70, samples=28 lat (msec) : 4=0.39%, 10=33.33%, 20=32.68%, 50=29.56%, 100=4.04% cpu : usr=0.11%, sys=12.25%, ctx=1444, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196184: Sat May 9 17:53:09 2026 read: IOPS=24, BW=392KiB/s (402kB/s)(5744KiB/14644msec) clat (usec): min=3945, max=60980, avg=16261.84, stdev=12525.14 lat (usec): min=3945, max=60982, avg=16262.45, stdev=12525.18 clat percentiles (usec): | 1.00th=[ 4490], 5.00th=[ 5014], 10.00th=[ 5473], 20.00th=[ 6325], | 30.00th=[ 7963], 40.00th=[10159], 50.00th=[11600], 60.00th=[13566], | 70.00th=[18482], 80.00th=[23462], 90.00th=[35390], 95.00th=[46400], | 99.00th=[56361], 99.50th=[61080], 99.90th=[61080], 99.95th=[61080], | 99.99th=[61080] bw ( KiB/s): min= 158, max= 1053, per=23.03%, avg=376.14, stdev=216.45, samples=28 iops : min= 9, max= 65, avg=22.96, stdev=13.60, samples=28 write: IOPS=27, BW=447KiB/s (458kB/s)(6544KiB/14644msec); 0 zone resets clat (usec): min=5564, max=95362, avg=21306.11, stdev=14233.85 lat (usec): min=5565, max=95365, avg=21311.43, stdev=14232.08 clat percentiles (usec): | 1.00th=[ 6325], 5.00th=[ 6980], 10.00th=[ 7570], 20.00th=[ 8586], | 30.00th=[10945], 40.00th=[13435], 50.00th=[16909], 60.00th=[21627], | 70.00th=[27132], 80.00th=[33162], 90.00th=[41157], 95.00th=[46400], | 99.00th=[67634], 99.50th=[77071], 99.90th=[94897], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 223, max= 1213, per=24.34%, avg=418.14, stdev=238.88, samples=28 iops : min= 13, max= 75, avg=25.61, stdev=15.00, samples=28 lat (msec) : 4=0.13%, 10=32.55%, 20=30.86%, 50=33.33%, 100=3.12% cpu : usr=0.08%, sys=12.08%, ctx=1551, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196185: Sat May 9 17:53:09 2026 read: IOPS=26, BW=427KiB/s (437kB/s)(6256KiB/14667msec) clat (usec): min=3413, max=69805, avg=16395.66, stdev=12531.10 lat (usec): min=3414, max=69806, avg=16396.36, stdev=12531.15 clat percentiles (usec): | 1.00th=[ 3884], 5.00th=[ 4621], 10.00th=[ 5211], 20.00th=[ 6128], | 30.00th=[ 7832], 40.00th=[10159], 50.00th=[12125], 60.00th=[15270], | 70.00th=[18744], 80.00th=[25035], 90.00th=[34341], 95.00th=[42730], | 99.00th=[58983], 99.50th=[63701], 99.90th=[69731], 99.95th=[69731], | 99.99th=[69731] bw ( KiB/s): min= 160, max= 1437, per=26.22%, avg=428.21, stdev=291.88, samples=29 iops : min= 10, max= 89, avg=26.24, stdev=18.31, samples=29 write: IOPS=25, BW=411KiB/s (421kB/s)(6032KiB/14667msec); 0 zone resets clat (usec): min=5488, max=84710, avg=21659.21, stdev=14576.00 lat (usec): min=5490, max=84712, avg=21660.99, stdev=14575.77 clat percentiles (usec): | 1.00th=[ 5800], 5.00th=[ 6652], 10.00th=[ 7242], 20.00th=[ 8717], | 30.00th=[10683], 40.00th=[13566], 50.00th=[17695], 60.00th=[22152], | 70.00th=[26084], 80.00th=[32375], 90.00th=[44303], 95.00th=[51119], | 99.00th=[65274], 99.50th=[74974], 99.90th=[84411], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 159, max= 1216, per=23.79%, avg=408.69, stdev=263.85, samples=29 iops : min= 9, max= 76, avg=25.03, stdev=16.60, samples=29 lat (msec) : 4=0.78%, 10=32.42%, 20=31.38%, 50=31.64%, 100=3.78% cpu : usr=0.07%, sys=11.99%, ctx=1506, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196186: Sat May 9 17:53:09 2026 read: IOPS=25, BW=412KiB/s (422kB/s)(6032KiB/14643msec) clat (usec): min=3452, max=63928, avg=16796.62, stdev=13409.27 lat (usec): min=3453, max=63928, avg=16797.48, stdev=13409.15 clat percentiles (usec): | 1.00th=[ 3949], 5.00th=[ 4752], 10.00th=[ 5276], 20.00th=[ 5932], | 30.00th=[ 7111], 40.00th=[ 9634], 50.00th=[11731], 60.00th=[14615], | 70.00th=[18220], 80.00th=[26346], 90.00th=[40109], 95.00th=[46400], | 99.00th=[55837], 99.50th=[61604], 99.90th=[63701], 99.95th=[63701], | 99.99th=[63701] bw ( KiB/s): min= 191, max= 1277, per=23.53%, avg=384.21, stdev=241.68, samples=28 iops : min= 11, max= 79, avg=23.50, stdev=15.14, samples=28 write: IOPS=26, BW=427KiB/s (437kB/s)(6256KiB/14643msec); 0 zone resets clat (usec): min=5141, max=89430, avg=21047.14, stdev=14060.19 lat (usec): min=5142, max=89433, avg=21049.04, stdev=14060.84 clat percentiles (usec): | 1.00th=[ 5800], 5.00th=[ 6718], 10.00th=[ 7373], 20.00th=[ 8717], | 30.00th=[10945], 40.00th=[13566], 50.00th=[16581], 60.00th=[21365], | 70.00th=[26870], 80.00th=[31589], 90.00th=[40109], 95.00th=[45876], | 99.00th=[71828], 99.50th=[80217], 99.90th=[89654], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 191, max= 1120, per=23.56%, avg=404.68, stdev=244.64, samples=28 iops : min= 11, max= 70, avg=24.75, stdev=15.40, samples=28 lat (msec) : 4=0.65%, 10=32.81%, 20=30.73%, 50=33.20%, 100=2.60% cpu : usr=0.09%, sys=11.87%, ctx=1469, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1633KiB/s (1672kB/s), 392KiB/s-427KiB/s (402kB/s-437kB/s), io=23.4MiB (24.5MB), run=14539-14667msec WRITE: bw=1718KiB/s (1759kB/s), 411KiB/s-447KiB/s (421kB/s-458kB/s), io=24.6MiB (25.8MB), run=14539-14667msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=196193: Sat May 9 17:53:17 2026 read: IOPS=3, BW=3394KiB/s (3475kB/s)(7168KiB/2112msec) clat (msec): min=53, max=124, avg=97.76, stdev=27.46 lat (msec): min=53, max=124, avg=97.76, stdev=27.46 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 54], 10.00th=[ 54], 20.00th=[ 69], | 30.00th=[ 99], 40.00th=[ 99], 50.00th=[ 101], 60.00th=[ 118], | 70.00th=[ 118], 80.00th=[ 123], 90.00th=[ 126], 95.00th=[ 126], | 99.00th=[ 126], 99.50th=[ 126], 99.90th=[ 126], 99.95th=[ 126], | 99.99th=[ 126] bw ( KiB/s): min= 4096, max= 5189, per=57.83%, avg=4642.50, stdev=772.87, samples=2 iops : min= 4, max= 5, avg= 4.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2424KiB/s (2482kB/s)(5120KiB/2112msec); 0 zone resets clat (msec): min=80, max=407, avg=278.98, stdev=121.59 lat (msec): min=80, max=407, avg=279.04, stdev=121.58 clat percentiles (msec): | 1.00th=[ 81], 5.00th=[ 81], 10.00th=[ 81], 20.00th=[ 81], | 30.00th=[ 279], 40.00th=[ 279], 50.00th=[ 296], 60.00th=[ 296], | 70.00th=[ 334], 80.00th=[ 334], 90.00th=[ 409], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 2048, max= 4087, per=27.29%, avg=3067.50, stdev=1441.79, samples=2 iops : min= 2, max= 3, avg= 2.50, stdev= 0.71, samples=2 lat (msec) : 100=41.67%, 250=25.00%, 500=33.33% cpu : usr=0.00%, sys=21.36%, ctx=152, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196194: Sat May 9 17:53:17 2026 read: IOPS=0, BW=803KiB/s (822kB/s)(2048KiB/2551msec) clat (msec): min=63, max=328, avg=196.05, stdev=187.05 lat (msec): min=63, max=328, avg=196.06, stdev=187.05 clat percentiles (msec): | 1.00th=[ 64], 5.00th=[ 64], 10.00th=[ 64], 20.00th=[ 64], | 30.00th=[ 64], 40.00th=[ 64], 50.00th=[ 64], 60.00th=[ 330], | 70.00th=[ 330], 80.00th=[ 330], 90.00th=[ 330], 95.00th=[ 330], | 99.00th=[ 330], 99.50th=[ 330], 99.90th=[ 330], 99.95th=[ 330], | 99.99th=[ 330] bw ( KiB/s): min= 1729, max= 1950, per=22.91%, avg=1839.50, stdev=156.27, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=4014KiB/s (4110kB/s)(10.0MiB/2551msec); 0 zone resets clat (msec): min=40, max=517, avg=211.32, stdev=176.08 lat (msec): min=40, max=517, avg=211.39, stdev=176.08 clat percentiles (msec): | 1.00th=[ 41], 5.00th=[ 41], 10.00th=[ 41], 20.00th=[ 72], | 30.00th=[ 101], 40.00th=[ 105], 50.00th=[ 116], 60.00th=[ 118], | 70.00th=[ 188], 80.00th=[ 405], 90.00th=[ 451], 95.00th=[ 518], | 99.00th=[ 518], 99.50th=[ 518], 99.90th=[ 518], 99.95th=[ 518], | 99.99th=[ 518] bw ( KiB/s): min= 1729, max= 4096, per=26.18%, avg=2942.00, stdev=1228.69, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.50, samples=4 lat (msec) : 50=8.33%, 100=16.67%, 250=41.67%, 500=25.00%, 750=8.33% cpu : usr=0.04%, sys=17.53%, ctx=90, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196195: Sat May 9 17:53:17 2026 read: IOPS=3, BW=3120KiB/s (3195kB/s)(6144KiB/1969msec) clat (msec): min=91, max=520, avg=221.77, stdev=157.19 lat (msec): min=91, max=520, avg=221.77, stdev=157.19 clat percentiles (msec): | 1.00th=[ 91], 5.00th=[ 91], 10.00th=[ 91], 20.00th=[ 100], | 30.00th=[ 100], 40.00th=[ 186], 50.00th=[ 186], 60.00th=[ 197], | 70.00th=[ 239], 80.00th=[ 239], 90.00th=[ 523], 95.00th=[ 523], | 99.00th=[ 523], 99.50th=[ 523], 99.90th=[ 523], 99.95th=[ 523], | 99.99th=[ 523] bw ( KiB/s): min= 3453, max= 4087, per=46.96%, avg=3770.00, stdev=448.31, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3120KiB/s (3195kB/s)(6144KiB/1969msec); 0 zone resets clat (msec): min=48, max=177, avg=101.68, stdev=50.58 lat (msec): min=48, max=177, avg=101.76, stdev=50.57 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 48], 10.00th=[ 48], 20.00th=[ 59], | 30.00th=[ 59], 40.00th=[ 79], 50.00th=[ 79], 60.00th=[ 103], | 70.00th=[ 146], 80.00th=[ 146], 90.00th=[ 178], 95.00th=[ 178], | 99.00th=[ 178], 99.50th=[ 178], 99.90th=[ 178], 99.95th=[ 178], | 99.99th=[ 178] bw ( KiB/s): min= 2043, max= 5180, per=27.50%, avg=3090.33, stdev=1809.71, samples=3 iops : min= 1, max= 5, avg= 2.67, stdev= 2.08, samples=3 lat (msec) : 50=8.33%, 100=33.33%, 250=50.00%, 750=8.33% cpu : usr=0.05%, sys=16.31%, ctx=82, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196196: Sat May 9 17:53:17 2026 read: IOPS=2, BW=2647KiB/s (2711kB/s)(5120KiB/1934msec) clat (msec): min=59, max=273, avg=127.86, stdev=84.87 lat (msec): min=59, max=273, avg=127.86, stdev=84.87 clat percentiles (msec): | 1.00th=[ 61], 5.00th=[ 61], 10.00th=[ 61], 20.00th=[ 61], | 30.00th=[ 79], 40.00th=[ 79], 50.00th=[ 109], 60.00th=[ 109], | 70.00th=[ 120], 80.00th=[ 120], 90.00th=[ 275], 95.00th=[ 275], | 99.00th=[ 275], 99.50th=[ 275], 99.90th=[ 275], 99.95th=[ 275], | 99.99th=[ 275] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=3, BW=3706KiB/s (3795kB/s)(7168KiB/1934msec); 0 zone resets clat (msec): min=50, max=394, avg=183.04, stdev=130.79 lat (msec): min=50, max=394, avg=183.11, stdev=130.79 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 51], 10.00th=[ 51], 20.00th=[ 89], | 30.00th=[ 100], 40.00th=[ 100], 50.00th=[ 102], 60.00th=[ 247], | 70.00th=[ 247], 80.00th=[ 300], 90.00th=[ 393], 95.00th=[ 393], | 99.00th=[ 393], 99.50th=[ 393], 99.90th=[ 393], 99.95th=[ 393], | 99.99th=[ 393] bw ( KiB/s): min= 2048, max= 5189, per=33.59%, avg=3774.67, stdev=1593.62, samples=3 iops : min= 2, max= 5, avg= 3.33, stdev= 1.53, samples=3 lat (msec) : 100=41.67%, 250=33.33%, 500=25.00% cpu : usr=0.05%, sys=18.42%, ctx=107, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8028KiB/s (8221kB/s), 803KiB/s-3394KiB/s (822kB/s-3475kB/s), io=20.0MiB (20.0MB), run=1934-2551msec WRITE: bw=10.0MiB/s (11.5MB/s), 2424KiB/s-4014KiB/s (2482kB/s-4110kB/s), io=28.0MiB (29.4MB), run=1934-2551msec rand-rw: (groupid=0, jobs=1): err= 0: pid=196198: Sat May 9 17:53:17 2026 read: IOPS=3, BW=3524KiB/s (3609kB/s)(7168KiB/2034msec) clat (usec): min=408, max=196673, avg=78420.47, stdev=88447.57 lat (usec): min=409, max=196674, avg=78421.58, stdev=88447.87 clat percentiles (usec): | 1.00th=[ 408], 5.00th=[ 408], 10.00th=[ 408], 20.00th=[ 461], | 30.00th=[ 578], 40.00th=[ 578], 50.00th=[ 35914], 60.00th=[143655], | 70.00th=[143655], 80.00th=[170918], 90.00th=[196084], 95.00th=[196084], | 99.00th=[196084], 99.50th=[196084], 99.90th=[196084], 99.95th=[196084], | 99.99th=[196084] bw ( KiB/s): min= 4096, max= 4517, per=52.40%, avg=4306.50, stdev=297.69, samples=2 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2517KiB/s (2578kB/s)(5120KiB/2034msec); 0 zone resets clat (msec): min=122, max=364, avg=264.06, stdev=92.07 lat (msec): min=122, max=364, avg=264.11, stdev=92.05 clat percentiles (msec): | 1.00th=[ 123], 5.00th=[ 123], 10.00th=[ 123], 20.00th=[ 123], | 30.00th=[ 236], 40.00th=[ 236], 50.00th=[ 284], 60.00th=[ 284], | 70.00th=[ 313], 80.00th=[ 313], 90.00th=[ 363], 95.00th=[ 363], | 99.00th=[ 363], 99.50th=[ 363], 99.90th=[ 363], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 2048, max= 3992, per=26.25%, avg=3020.00, stdev=1374.62, samples=2 iops : min= 2, max= 3, avg= 2.50, stdev= 0.71, samples=2 lat (usec) : 500=16.67%, 750=8.33% lat (msec) : 50=8.33%, 250=41.67%, 500=25.00% cpu : usr=0.00%, sys=39.30%, ctx=572, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196199: Sat May 9 17:53:17 2026 read: IOPS=0, BW=822KiB/s (842kB/s)(2048KiB/2492msec) clat (msec): min=153, max=266, avg=210.22, stdev=79.70 lat (msec): min=153, max=266, avg=210.22, stdev=79.70 clat percentiles (msec): | 1.00th=[ 155], 5.00th=[ 155], 10.00th=[ 155], 20.00th=[ 155], | 30.00th=[ 155], 40.00th=[ 155], 50.00th=[ 155], 60.00th=[ 268], | 70.00th=[ 268], 80.00th=[ 268], 90.00th=[ 268], 95.00th=[ 268], | 99.00th=[ 268], 99.50th=[ 268], 99.90th=[ 268], 99.95th=[ 268], | 99.99th=[ 268] bw ( KiB/s): min= 1508, max= 2048, per=21.64%, avg=1778.00, stdev=381.84, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=4, BW=4109KiB/s (4208kB/s)(10.0MiB/2492msec); 0 zone resets clat (msec): min=60, max=453, avg=199.17, stdev=134.09 lat (msec): min=60, max=453, avg=199.30, stdev=134.07 clat percentiles (msec): | 1.00th=[ 61], 5.00th=[ 61], 10.00th=[ 61], 20.00th=[ 94], | 30.00th=[ 99], 40.00th=[ 127], 50.00th=[ 150], 60.00th=[ 161], | 70.00th=[ 165], 80.00th=[ 292], 90.00th=[ 393], 95.00th=[ 456], | 99.00th=[ 456], 99.50th=[ 456], 99.90th=[ 456], 99.95th=[ 456], | 99.99th=[ 456] bw ( KiB/s): min= 1996, max= 6144, per=35.45%, avg=4078.67, stdev=2074.05, samples=3 iops : min= 1, max= 6, avg= 3.67, stdev= 2.52, samples=3 lat (msec) : 100=25.00%, 250=41.67%, 500=33.33% cpu : usr=0.00%, sys=46.33%, ctx=366, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196200: Sat May 9 17:53:17 2026 read: IOPS=2, BW=3057KiB/s (3130kB/s)(6144KiB/2010msec) clat (usec): min=496, max=188872, avg=106208.23, stdev=78220.14 lat (usec): min=497, max=188873, avg=106209.18, stdev=78220.15 clat percentiles (usec): | 1.00th=[ 498], 5.00th=[ 498], 10.00th=[ 498], 20.00th=[ 22676], | 30.00th=[ 22676], 40.00th=[115868], 50.00th=[115868], 60.00th=[135267], | 70.00th=[175113], 80.00th=[175113], 90.00th=[189793], 95.00th=[189793], | 99.00th=[189793], 99.50th=[189793], 99.90th=[189793], 99.95th=[189793], | 99.99th=[189793] bw ( KiB/s): min= 1521, max= 3992, per=30.67%, avg=2520.33, stdev=1301.45, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 write: IOPS=2, BW=3057KiB/s (3130kB/s)(6144KiB/2010msec); 0 zone resets clat (msec): min=127, max=409, avg=211.33, stdev=108.52 lat (msec): min=127, max=409, avg=211.38, stdev=108.51 clat percentiles (msec): | 1.00th=[ 128], 5.00th=[ 128], 10.00th=[ 128], 20.00th=[ 138], | 30.00th=[ 138], 40.00th=[ 161], 50.00th=[ 161], 60.00th=[ 171], | 70.00th=[ 264], 80.00th=[ 264], 90.00th=[ 409], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 3992, max= 4564, per=37.18%, avg=4278.00, stdev=404.47, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 lat (usec) : 500=8.33% lat (msec) : 50=8.33%, 250=66.67%, 500=16.67% cpu : usr=0.00%, sys=47.64%, ctx=253, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=196201: Sat May 9 17:53:17 2026 read: IOPS=2, BW=2780KiB/s (2846kB/s)(5120KiB/1842msec) clat (usec): min=549, max=179480, avg=88034.91, stdev=84174.41 lat (usec): min=550, max=179481, avg=88035.86, stdev=84174.44 clat percentiles (usec): | 1.00th=[ 553], 5.00th=[ 553], 10.00th=[ 553], 20.00th=[ 553], | 30.00th=[ 562], 40.00th=[ 562], 50.00th=[105382], 60.00th=[105382], | 70.00th=[154141], 80.00th=[154141], 90.00th=[179307], 95.00th=[179307], | 99.00th=[179307], 99.50th=[179307], 99.90th=[179307], 99.95th=[179307], | 99.99th=[179307] bw ( KiB/s): min= 3992, max= 4096, per=49.21%, avg=4044.00, stdev=73.54, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3891KiB/s (3985kB/s)(7168KiB/1842msec); 0 zone resets clat (msec): min=129, max=306, avg=196.75, stdev=60.82 lat (msec): min=130, max=306, avg=196.83, stdev=60.82 clat percentiles (msec): | 1.00th=[ 130], 5.00th=[ 130], 10.00th=[ 130], 20.00th=[ 140], | 30.00th=[ 161], 40.00th=[ 161], 50.00th=[ 207], 60.00th=[ 215], | 70.00th=[ 215], 80.00th=[ 222], 90.00th=[ 309], 95.00th=[ 309], | 99.00th=[ 309], 99.50th=[ 309], 99.90th=[ 309], 99.95th=[ 309], | 99.99th=[ 309] bw ( KiB/s): min= 1996, max= 4504, per=24.77%, avg=2849.33, stdev=1433.22, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 lat (usec) : 750=16.67% lat (msec) : 250=75.00%, 500=8.33% cpu : usr=0.00%, sys=55.30%, ctx=229, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8218KiB/s (8416kB/s), 822KiB/s-3524KiB/s (842kB/s-3609kB/s), io=20.0MiB (20.0MB), run=1842-2492msec WRITE: bw=11.2MiB/s (11.8MB/s), 2517KiB/s-4109KiB/s (2578kB/s-4208kB/s), io=28.0MiB (29.4MB), run=1842-2492msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes