== sanity test 398b: DIO and buffer IO race ============== 20:11:09 (1772759469) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.74901 s, 18.3 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=158994: Thu Mar 5 20:12:06 2026 read: IOPS=31, BW=124KiB/s (127kB/s)(5992KiB/48263msec) clat (usec): min=133, max=656147, avg=17235.07, stdev=19013.96 lat (usec): min=133, max=656147, avg=17235.62, stdev=19013.97 clat percentiles (usec): | 1.00th=[ 537], 5.00th=[ 5538], 10.00th=[ 8225], 20.00th=[ 10028], | 30.00th=[ 11731], 40.00th=[ 13173], 50.00th=[ 15008], 60.00th=[ 17433], | 70.00th=[ 19792], 80.00th=[ 22414], 90.00th=[ 27657], 95.00th=[ 33817], | 99.00th=[ 51119], 99.50th=[ 58983], 99.90th=[ 84411], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 7, max= 232, per=24.84%, avg=123.44, stdev=33.88, samples=96 iops : min= 1, max= 58, avg=30.45, stdev= 8.46, samples=96 write: IOPS=32, BW=130KiB/s (134kB/s)(6296KiB/48263msec); 0 zone resets clat (usec): min=747, max=129857, avg=14143.44, stdev=10898.27 lat (usec): min=747, max=129857, avg=14144.82, stdev=10898.24 clat percentiles (usec): | 1.00th=[ 1123], 5.00th=[ 2835], 10.00th=[ 3326], 20.00th=[ 4359], | 30.00th=[ 6915], 40.00th=[ 9765], 50.00th=[ 11994], 60.00th=[ 14353], | 70.00th=[ 17433], 80.00th=[ 21890], 90.00th=[ 28443], 95.00th=[ 33424], | 99.00th=[ 49021], 99.50th=[ 52691], 99.90th=[ 77071], 99.95th=[129500], | 99.99th=[129500] bw ( KiB/s): min= 55, max= 280, per=25.25%, avg=131.31, stdev=34.03, samples=95 iops : min= 13, max= 70, avg=32.42, stdev= 8.58, samples=95 lat (usec) : 250=0.10%, 500=0.23%, 750=1.24%, 1000=0.55% lat (msec) : 2=1.14%, 4=7.94%, 10=19.63%, 20=42.90%, 50=25.29% lat (msec) : 100=0.91%, 250=0.03%, 750=0.03% cpu : usr=0.08%, sys=28.96%, ctx=9743, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158996: Thu Mar 5 20:12:06 2026 read: IOPS=30, BW=124KiB/s (127kB/s)(5940KiB/47954msec) clat (usec): min=188, max=125961, avg=16522.34, stdev=9304.67 lat (usec): min=188, max=125962, avg=16522.94, stdev=9304.71 clat percentiles (usec): | 1.00th=[ 553], 5.00th=[ 6390], 10.00th=[ 8356], 20.00th=[ 9896], | 30.00th=[ 11338], 40.00th=[ 12780], 50.00th=[ 14615], 60.00th=[ 16909], | 70.00th=[ 19530], 80.00th=[ 22938], 90.00th=[ 26870], 95.00th=[ 31065], | 99.00th=[ 47973], 99.50th=[ 53216], 99.90th=[ 86508], 99.95th=[126354], | 99.99th=[126354] bw ( KiB/s): min= 7, max= 184, per=24.80%, avg=123.23, stdev=35.79, samples=95 iops : min= 1, max= 46, avg=30.39, stdev= 9.02, samples=95 write: IOPS=33, BW=132KiB/s (136kB/s)(6348KiB/47954msec); 0 zone resets clat (usec): min=787, max=655245, avg=14643.83, stdev=19626.38 lat (usec): min=787, max=655246, avg=14644.53, stdev=19626.37 clat percentiles (usec): | 1.00th=[ 1303], 5.00th=[ 2868], 10.00th=[ 3359], 20.00th=[ 4228], | 30.00th=[ 6456], 40.00th=[ 9503], 50.00th=[ 11863], 60.00th=[ 14484], | 70.00th=[ 17695], 80.00th=[ 21627], 90.00th=[ 28967], 95.00th=[ 35390], | 99.00th=[ 53740], 99.50th=[ 67634], 99.90th=[106431], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 63, max= 223, per=25.48%, avg=132.50, stdev=32.43, samples=94 iops : min= 15, max= 55, avg=32.72, stdev= 8.16, samples=94 lat (usec) : 250=0.07%, 500=0.20%, 750=1.17%, 1000=0.46% lat (msec) : 2=1.04%, 4=8.11%, 10=20.77%, 20=42.35%, 50=24.84% lat (msec) : 100=0.91%, 250=0.07%, 750=0.03% cpu : usr=0.08%, sys=28.86%, ctx=9502, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158998: Thu Mar 5 20:12:06 2026 read: IOPS=31, BW=126KiB/s (129kB/s)(6028KiB/47897msec) clat (usec): min=138, max=81130, avg=16475.19, stdev=9294.07 lat (usec): min=138, max=81131, avg=16475.84, stdev=9294.14 clat percentiles (usec): | 1.00th=[ 490], 5.00th=[ 1106], 10.00th=[ 7898], 20.00th=[ 9765], | 30.00th=[11207], 40.00th=[12911], 50.00th=[14877], 60.00th=[17171], | 70.00th=[19792], 80.00th=[22676], 90.00th=[26870], 95.00th=[32113], | 99.00th=[50070], 99.50th=[58983], 99.90th=[68682], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 63, max= 224, per=25.45%, avg=126.49, stdev=30.94, samples=94 iops : min= 15, max= 56, avg=31.21, stdev= 7.76, samples=94 write: IOPS=32, BW=131KiB/s (134kB/s)(6260KiB/47897msec); 0 zone resets clat (usec): min=796, max=656711, avg=14622.55, stdev=19708.62 lat (usec): min=797, max=656711, avg=14623.25, stdev=19708.62 clat percentiles (usec): | 1.00th=[ 1188], 5.00th=[ 2671], 10.00th=[ 3294], 20.00th=[ 4146], | 30.00th=[ 6063], 40.00th=[ 9372], 50.00th=[ 11994], 60.00th=[ 14484], | 70.00th=[ 17957], 80.00th=[ 21890], 90.00th=[ 28443], 95.00th=[ 35914], | 99.00th=[ 51643], 99.50th=[ 61080], 99.90th=[ 92799], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 15, max= 223, per=24.93%, avg=129.61, stdev=35.07, samples=95 iops : min= 3, max= 55, avg=31.99, stdev= 8.74, samples=95 lat (usec) : 250=0.16%, 500=0.39%, 750=1.56%, 1000=0.75% lat (msec) : 2=1.11%, 4=7.98%, 10=20.41%, 20=40.69%, 50=25.78% lat (msec) : 100=1.14%, 750=0.03% cpu : usr=0.09%, sys=28.94%, ctx=9446, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159000: Thu Mar 5 20:12:06 2026 read: IOPS=31, BW=128KiB/s (131kB/s)(6052KiB/47442msec) clat (usec): min=132, max=80777, avg=16388.57, stdev=8833.39 lat (usec): min=133, max=80778, avg=16389.22, stdev=8833.42 clat percentiles (usec): | 1.00th=[ 537], 5.00th=[ 3720], 10.00th=[ 8356], 20.00th=[ 9896], | 30.00th=[11338], 40.00th=[12911], 50.00th=[14746], 60.00th=[17171], | 70.00th=[19530], 80.00th=[22414], 90.00th=[26608], 95.00th=[32113], | 99.00th=[43779], 99.50th=[51643], 99.90th=[72877], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 16, max= 192, per=25.78%, avg=128.12, stdev=30.18, samples=93 iops : min= 4, max= 48, avg=31.62, stdev= 7.56, samples=93 write: IOPS=32, BW=131KiB/s (135kB/s)(6236KiB/47442msec); 0 zone resets clat (usec): min=661, max=686962, avg=14401.11, stdev=20339.75 lat (usec): min=662, max=686963, avg=14401.84, stdev=20339.75 clat percentiles (usec): | 1.00th=[ 1139], 5.00th=[ 2737], 10.00th=[ 3326], 20.00th=[ 4080], | 30.00th=[ 5669], 40.00th=[ 8979], 50.00th=[ 11863], 60.00th=[ 14484], | 70.00th=[ 17433], 80.00th=[ 21890], 90.00th=[ 27919], 95.00th=[ 34341], | 99.00th=[ 53216], 99.50th=[ 63177], 99.90th=[ 91751], 99.95th=[683672], | 99.99th=[683672] bw ( KiB/s): min= 32, max= 216, per=25.45%, avg=132.35, stdev=37.75, samples=93 iops : min= 8, max= 54, avg=32.68, stdev= 9.50, samples=93 lat (usec) : 250=0.10%, 500=0.23%, 750=1.56%, 1000=0.78% lat (msec) : 2=0.85%, 4=8.66%, 10=20.08%, 20=41.63%, 50=25.20% lat (msec) : 100=0.88%, 750=0.03% cpu : usr=0.09%, sys=29.11%, ctx=9427, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=498KiB/s (509kB/s), 124KiB/s-128KiB/s (127kB/s-131kB/s), io=23.4MiB (24.6MB), run=47442-48263msec WRITE: bw=521KiB/s (533kB/s), 130KiB/s-132KiB/s (134kB/s-136kB/s), io=24.6MiB (25.7MB), run=47442-48263msec rand-rw: (groupid=0, jobs=1): err= 0: pid=158993: Thu Mar 5 20:12:27 2026 read: IOPS=21, BW=87.4KiB/s (89.5kB/s)(5992KiB/68533msec) clat (msec): min=2, max=661, avg=13.96, stdev=21.24 lat (msec): min=2, max=661, avg=13.96, stdev=21.24 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 9], 60.00th=[ 12], | 70.00th=[ 15], 80.00th=[ 21], 90.00th=[ 32], 95.00th=[ 41], | 99.00th=[ 61], 99.50th=[ 83], 99.90th=[ 130], 99.95th=[ 659], | 99.99th=[ 659] bw ( KiB/s): min= 23, max= 248, per=25.26%, avg=87.66, stdev=48.93, samples=135 iops : min= 5, max= 62, avg=21.66, stdev=12.37, samples=135 write: IOPS=22, BW=91.9KiB/s (94.1kB/s)(6296KiB/68533msec); 0 zone resets clat (msec): min=12, max=142, avg=30.14, stdev=14.71 lat (msec): min=12, max=142, avg=30.14, stdev=14.71 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 19], | 30.00th=[ 21], 40.00th=[ 23], 50.00th=[ 26], 60.00th=[ 29], | 70.00th=[ 35], 80.00th=[ 41], 90.00th=[ 50], 95.00th=[ 59], | 99.00th=[ 84], 99.50th=[ 93], 99.90th=[ 115], 99.95th=[ 142], | 99.99th=[ 142] bw ( KiB/s): min= 24, max= 184, per=25.30%, avg=92.10, stdev=44.89, samples=135 iops : min= 6, max= 46, avg=22.77, stdev=11.38, samples=135 lat (msec) : 4=5.83%, 10=21.06%, 20=26.07%, 50=41.02%, 100=5.79% lat (msec) : 250=0.20%, 750=0.03% cpu : usr=0.07%, sys=11.49%, ctx=6101, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158995: Thu Mar 5 20:12:27 2026 read: IOPS=21, BW=86.0KiB/s (88.1kB/s)(5940KiB/69045msec) clat (msec): min=2, max=652, avg=13.69, stdev=20.63 lat (msec): min=2, max=652, avg=13.69, stdev=20.63 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 9], 60.00th=[ 12], | 70.00th=[ 15], 80.00th=[ 21], 90.00th=[ 30], 95.00th=[ 39], | 99.00th=[ 59], 99.50th=[ 72], 99.90th=[ 99], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 8, max= 256, per=24.84%, avg=86.19, stdev=50.05, samples=136 iops : min= 2, max= 64, avg=21.29, stdev=12.63, samples=136 write: IOPS=22, BW=91.9KiB/s (94.1kB/s)(6348KiB/69045msec); 0 zone resets clat (msec): min=11, max=127, avg=30.58, stdev=15.69 lat (msec): min=11, max=127, avg=30.58, stdev=15.69 clat percentiles (msec): | 1.00th=[ 14], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 19], | 30.00th=[ 21], 40.00th=[ 24], 50.00th=[ 26], 60.00th=[ 29], | 70.00th=[ 35], 80.00th=[ 41], 90.00th=[ 52], 95.00th=[ 63], | 99.00th=[ 86], 99.50th=[ 93], 99.90th=[ 127], 99.95th=[ 128], | 99.99th=[ 128] bw ( KiB/s): min= 31, max= 184, per=25.23%, avg=91.84, stdev=44.85, samples=136 iops : min= 7, max= 46, avg=22.71, stdev=11.37, samples=136 lat (msec) : 4=6.25%, 10=20.74%, 20=25.07%, 50=41.15%, 100=6.54% lat (msec) : 250=0.23%, 750=0.03% cpu : usr=0.08%, sys=11.23%, ctx=6117, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158997: Thu Mar 5 20:12:27 2026 read: IOPS=21, BW=87.4KiB/s (89.5kB/s)(6028KiB/68943msec) clat (msec): min=2, max=120, avg=13.51, stdev=12.48 lat (msec): min=2, max=120, avg=13.51, stdev=12.48 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 9], 60.00th=[ 12], | 70.00th=[ 16], 80.00th=[ 21], 90.00th=[ 31], 95.00th=[ 39], | 99.00th=[ 58], 99.50th=[ 69], 99.90th=[ 91], 99.95th=[ 122], | 99.99th=[ 122] bw ( KiB/s): min= 15, max= 256, per=25.30%, avg=87.78, stdev=48.59, samples=136 iops : min= 3, max= 64, avg=21.69, stdev=12.28, samples=136 write: IOPS=22, BW=90.8KiB/s (92.0kB/s)(6260KiB/68943msec); 0 zone resets clat (msec): min=12, max=673, avg=30.93, stdev=22.42 lat (msec): min=12, max=673, avg=30.93, stdev=22.42 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 16], 10.00th=[ 17], 20.00th=[ 19], | 30.00th=[ 21], 40.00th=[ 23], 50.00th=[ 26], 60.00th=[ 29], | 70.00th=[ 35], 80.00th=[ 42], 90.00th=[ 51], 95.00th=[ 59], | 99.00th=[ 87], 99.50th=[ 102], 99.90th=[ 153], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 23, max= 176, per=24.99%, avg=90.96, stdev=44.69, samples=136 iops : min= 5, max= 44, avg=22.49, stdev=11.33, samples=136 lat (msec) : 4=5.63%, 10=21.26%, 20=25.36%, 50=41.21%, 100=6.22% lat (msec) : 250=0.29%, 750=0.03% cpu : usr=0.08%, sys=11.27%, ctx=6070, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=158999: Thu Mar 5 20:12:27 2026 read: IOPS=21, BW=87.7KiB/s (89.8kB/s)(6052KiB/68976msec) clat (msec): min=2, max=106, avg=13.34, stdev=13.05 lat (msec): min=2, max=106, avg=13.34, stdev=13.05 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 9], 60.00th=[ 12], | 70.00th=[ 15], 80.00th=[ 20], 90.00th=[ 31], 95.00th=[ 41], | 99.00th=[ 63], 99.50th=[ 81], 99.90th=[ 105], 99.95th=[ 107], | 99.99th=[ 107] bw ( KiB/s): min= 7, max= 224, per=25.28%, avg=87.72, stdev=48.80, samples=136 iops : min= 1, max= 56, avg=21.68, stdev=12.35, samples=136 write: IOPS=22, BW=90.4KiB/s (92.6kB/s)(6236KiB/68976msec); 0 zone resets clat (msec): min=12, max=682, avg=31.19, stdev=22.72 lat (msec): min=12, max=682, avg=31.19, stdev=22.72 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 16], 10.00th=[ 18], 20.00th=[ 20], | 30.00th=[ 21], 40.00th=[ 24], 50.00th=[ 26], 60.00th=[ 29], | 70.00th=[ 35], 80.00th=[ 42], 90.00th=[ 52], 95.00th=[ 61], | 99.00th=[ 83], 99.50th=[ 116], 99.90th=[ 150], 99.95th=[ 684], | 99.99th=[ 684] bw ( KiB/s): min= 15, max= 176, per=24.87%, avg=90.54, stdev=43.67, samples=136 iops : min= 3, max= 44, avg=22.38, stdev=11.08, samples=136 lat (msec) : 4=6.05%, 10=21.26%, 20=25.52%, 50=40.59%, 100=6.25% lat (msec) : 250=0.29%, 750=0.03% cpu : usr=0.07%, sys=11.29%, ctx=6020, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=348KiB/s (356kB/s), 86.0KiB/s-87.7KiB/s (88.1kB/s-89.8kB/s), io=23.4MiB (24.6MB), run=68533-69045msec WRITE: bw=364KiB/s (373kB/s), 90.4KiB/s-91.9KiB/s (92.6kB/s-94.1kB/s), io=24.6MiB (25.7MB), run=68533-69045msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159024: Thu Mar 5 20:12:46 2026 read: IOPS=28, BW=457KiB/s (468kB/s)(5920KiB/12940msec) clat (usec): min=331, max=98351, avg=18156.69, stdev=10858.09 lat (usec): min=332, max=98352, avg=18157.29, stdev=10858.11 clat percentiles (usec): | 1.00th=[ 441], 5.00th=[ 660], 10.00th=[ 7635], 20.00th=[11207], | 30.00th=[12649], 40.00th=[14746], 50.00th=[16909], 60.00th=[19530], | 70.00th=[21627], 80.00th=[25035], 90.00th=[29492], 95.00th=[35914], | 99.00th=[55313], 99.50th=[60031], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 253, max= 795, per=25.34%, avg=451.88, stdev=123.44, samples=25 iops : min= 15, max= 49, avg=27.76, stdev= 7.72, samples=25 write: IOPS=30, BW=492KiB/s (504kB/s)(6368KiB/12940msec); 0 zone resets clat (usec): min=1233, max=91905, avg=15579.36, stdev=11388.60 lat (usec): min=1233, max=91906, avg=15580.52, stdev=11388.56 clat percentiles (usec): | 1.00th=[ 3097], 5.00th=[ 4424], 10.00th=[ 5014], 20.00th=[ 5997], | 30.00th=[ 7701], 40.00th=[ 9896], 50.00th=[13042], 60.00th=[15926], | 70.00th=[19268], 80.00th=[23200], 90.00th=[30278], 95.00th=[35390], | 99.00th=[54789], 99.50th=[88605], 99.90th=[91751], 99.95th=[91751], | 99.99th=[91751] bw ( KiB/s): min= 285, max= 667, per=26.16%, avg=490.68, stdev=111.36, samples=25 iops : min= 17, max= 41, avg=30.20, stdev= 7.01, samples=25 lat (usec) : 500=0.65%, 750=2.86%, 1000=0.26% lat (msec) : 2=0.52%, 4=1.17%, 10=22.79%, 20=39.58%, 50=30.60% lat (msec) : 100=1.56% cpu : usr=0.09%, sys=32.32%, ctx=2848, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159025: Thu Mar 5 20:12:46 2026 read: IOPS=26, BW=428KiB/s (438kB/s)(5744KiB/13432msec) clat (usec): min=140, max=81409, avg=19037.17, stdev=11166.20 lat (usec): min=140, max=81410, avg=19037.74, stdev=11166.20 clat percentiles (usec): | 1.00th=[ 445], 5.00th=[ 668], 10.00th=[ 6915], 20.00th=[11207], | 30.00th=[13566], 40.00th=[15401], 50.00th=[18220], 60.00th=[20579], | 70.00th=[22676], 80.00th=[25560], 90.00th=[32375], 95.00th=[39584], | 99.00th=[58983], 99.50th=[63177], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 128, max= 576, per=23.01%, avg=410.19, stdev=111.43, samples=26 iops : min= 8, max= 36, avg=25.15, stdev= 7.07, samples=26 write: IOPS=30, BW=487KiB/s (499kB/s)(6544KiB/13432msec); 0 zone resets clat (usec): min=1123, max=82018, avg=16029.54, stdev=11811.21 lat (usec): min=1124, max=82020, avg=16030.90, stdev=11810.96 clat percentiles (usec): | 1.00th=[ 1221], 5.00th=[ 3556], 10.00th=[ 4359], 20.00th=[ 5866], | 30.00th=[ 8586], 40.00th=[10945], 50.00th=[12780], 60.00th=[15533], | 70.00th=[19530], 80.00th=[23987], 90.00th=[31065], 95.00th=[38536], | 99.00th=[56361], 99.50th=[57410], 99.90th=[82314], 99.95th=[82314], | 99.99th=[82314] bw ( KiB/s): min= 286, max= 768, per=24.66%, avg=462.65, stdev=115.85, samples=26 iops : min= 17, max= 48, avg=28.42, stdev= 7.39, samples=26 lat (usec) : 250=0.26%, 500=0.65%, 750=1.95%, 1000=0.65% lat (msec) : 2=1.17%, 4=2.47%, 10=18.88%, 20=38.02%, 50=34.11% lat (msec) : 100=1.82% cpu : usr=0.10%, sys=31.23%, ctx=2880, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159026: Thu Mar 5 20:12:46 2026 read: IOPS=29, BW=475KiB/s (487kB/s)(6256KiB/13157msec) clat (usec): min=121, max=80066, avg=17953.58, stdev=10626.84 lat (usec): min=121, max=80066, avg=17954.20, stdev=10626.85 clat percentiles (usec): | 1.00th=[ 453], 5.00th=[ 644], 10.00th=[ 3490], 20.00th=[10945], | 30.00th=[12911], 40.00th=[14615], 50.00th=[16712], 60.00th=[19006], | 70.00th=[21890], 80.00th=[24773], 90.00th=[30016], 95.00th=[36963], | 99.00th=[49021], 99.50th=[53740], 99.90th=[80217], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 285, max= 672, per=26.65%, avg=475.16, stdev=104.78, samples=25 iops : min= 17, max= 42, avg=29.20, stdev= 6.60, samples=25 write: IOPS=28, BW=458KiB/s (469kB/s)(6032KiB/13157msec); 0 zone resets clat (usec): min=1173, max=75318, avg=16190.84, stdev=11704.04 lat (usec): min=1173, max=75319, avg=16192.02, stdev=11704.07 clat percentiles (usec): | 1.00th=[ 1663], 5.00th=[ 4555], 10.00th=[ 4948], 20.00th=[ 5932], | 30.00th=[ 8455], 40.00th=[11469], 50.00th=[13435], 60.00th=[15795], | 70.00th=[19006], 80.00th=[23987], 90.00th=[30802], 95.00th=[40109], | 99.00th=[62653], 99.50th=[66323], 99.90th=[74974], 99.95th=[74974], | 99.99th=[74974] bw ( KiB/s): min= 223, max= 672, per=23.55%, avg=441.88, stdev=128.79, samples=25 iops : min= 13, max= 42, avg=27.12, stdev= 8.07, samples=25 lat (usec) : 250=0.13%, 500=0.39%, 750=2.86%, 1000=1.30% lat (msec) : 2=1.04%, 4=0.91%, 10=17.71%, 20=42.97%, 50=31.51% lat (msec) : 100=1.17% cpu : usr=0.10%, sys=30.97%, ctx=2885, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159027: Thu Mar 5 20:12:46 2026 read: IOPS=28, BW=462KiB/s (473kB/s)(6032KiB/13052msec) clat (usec): min=338, max=76381, avg=18076.64, stdev=10441.11 lat (usec): min=338, max=76383, avg=18077.26, stdev=10441.13 clat percentiles (usec): | 1.00th=[ 490], 5.00th=[ 594], 10.00th=[ 914], 20.00th=[12256], | 30.00th=[13698], 40.00th=[15139], 50.00th=[17171], 60.00th=[19006], | 70.00th=[21627], 80.00th=[24249], 90.00th=[30016], 95.00th=[36439], | 99.00th=[51643], 99.50th=[67634], 99.90th=[76022], 99.95th=[76022], | 99.99th=[76022] bw ( KiB/s): min= 192, max= 608, per=25.70%, avg=458.32, stdev=98.80, samples=25 iops : min= 12, max= 38, avg=28.16, stdev= 6.32, samples=25 write: IOPS=29, BW=479KiB/s (491kB/s)(6256KiB/13052msec); 0 zone resets clat (usec): min=1184, max=72445, avg=15773.75, stdev=11373.22 lat (usec): min=1184, max=72447, avg=15774.91, stdev=11373.18 clat percentiles (usec): | 1.00th=[ 2442], 5.00th=[ 4490], 10.00th=[ 4948], 20.00th=[ 6390], | 30.00th=[ 8094], 40.00th=[10290], 50.00th=[12780], 60.00th=[15401], | 70.00th=[18744], 80.00th=[23462], 90.00th=[31065], 95.00th=[40109], | 99.00th=[56886], 99.50th=[59507], 99.90th=[72877], 99.95th=[72877], | 99.99th=[72877] bw ( KiB/s): min= 221, max= 795, per=25.65%, avg=481.28, stdev=121.81, samples=25 iops : min= 13, max= 49, avg=29.60, stdev= 7.69, samples=25 lat (usec) : 500=0.52%, 750=3.91%, 1000=0.65% lat (msec) : 2=0.52%, 4=1.04%, 10=20.70%, 20=41.28%, 50=30.08% lat (msec) : 100=1.30% cpu : usr=0.08%, sys=31.72%, ctx=2906, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1783KiB/s (1826kB/s), 428KiB/s-475KiB/s (438kB/s-487kB/s), io=23.4MiB (24.5MB), run=12940-13432msec WRITE: bw=1876KiB/s (1921kB/s), 458KiB/s-492KiB/s (469kB/s-504kB/s), io=24.6MiB (25.8MB), run=12940-13432msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159029: Thu Mar 5 20:12:51 2026 read: IOPS=19, BW=315KiB/s (323kB/s)(5920KiB/18770msec) clat (usec): min=3079, max=92904, avg=16477.90, stdev=14440.25 lat (usec): min=3080, max=92905, avg=16478.52, stdev=14440.23 clat percentiles (usec): | 1.00th=[ 3359], 5.00th=[ 3916], 10.00th=[ 4424], 20.00th=[ 5866], | 30.00th=[ 8225], 40.00th=[ 9896], 50.00th=[11469], 60.00th=[13829], | 70.00th=[17433], 80.00th=[23200], 90.00th=[37487], 95.00th=[47449], | 99.00th=[65799], 99.50th=[85459], 99.90th=[92799], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 159, max= 832, per=24.69%, avg=313.27, stdev=147.64, samples=37 iops : min= 9, max= 52, avg=19.08, stdev= 9.48, samples=37 write: IOPS=21, BW=339KiB/s (347kB/s)(6368KiB/18770msec); 0 zone resets clat (msec): min=14, max=137, avg=31.65, stdev=16.39 lat (msec): min=14, max=137, avg=31.65, stdev=16.39 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 18], 10.00th=[ 19], 20.00th=[ 21], | 30.00th=[ 22], 40.00th=[ 24], 50.00th=[ 27], 60.00th=[ 30], | 70.00th=[ 35], 80.00th=[ 41], 90.00th=[ 52], 95.00th=[ 62], | 99.00th=[ 112], 99.50th=[ 115], 99.90th=[ 138], 99.95th=[ 138], | 99.99th=[ 138] bw ( KiB/s): min= 127, max= 672, per=25.37%, avg=338.65, stdev=156.32, samples=37 iops : min= 7, max= 42, avg=20.65, stdev=10.07, samples=37 lat (msec) : 4=2.60%, 10=16.93%, 20=25.78%, 50=47.01%, 100=7.03% lat (msec) : 250=0.65% cpu : usr=0.07%, sys=12.89%, ctx=1823, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159030: Thu Mar 5 20:12:51 2026 read: IOPS=19, BW=304KiB/s (312kB/s)(5744KiB/18871msec) clat (usec): min=2973, max=86264, avg=16135.11, stdev=13733.96 lat (usec): min=2973, max=86264, avg=16135.73, stdev=13733.98 clat percentiles (usec): | 1.00th=[ 3195], 5.00th=[ 3982], 10.00th=[ 4817], 20.00th=[ 6128], | 30.00th=[ 7701], 40.00th=[ 9503], 50.00th=[11863], 60.00th=[14746], | 70.00th=[17433], 80.00th=[22676], 90.00th=[36439], 95.00th=[44303], | 99.00th=[70779], 99.50th=[84411], 99.90th=[86508], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 95, max= 736, per=23.87%, avg=302.92, stdev=158.78, samples=37 iops : min= 5, max= 46, avg=18.43, stdev=10.17, samples=37 write: IOPS=21, BW=347KiB/s (355kB/s)(6544KiB/18871msec); 0 zone resets clat (msec): min=11, max=126, avg=31.90, stdev=16.51 lat (msec): min=11, max=126, avg=31.90, stdev=16.51 clat percentiles (msec): | 1.00th=[ 13], 5.00th=[ 16], 10.00th=[ 18], 20.00th=[ 21], | 30.00th=[ 23], 40.00th=[ 24], 50.00th=[ 27], 60.00th=[ 30], | 70.00th=[ 34], 80.00th=[ 42], 90.00th=[ 54], 95.00th=[ 68], | 99.00th=[ 91], 99.50th=[ 97], 99.90th=[ 128], 99.95th=[ 128], | 99.99th=[ 128] bw ( KiB/s): min= 159, max= 640, per=25.52%, avg=340.70, stdev=141.15, samples=37 iops : min= 9, max= 40, avg=20.78, stdev= 9.11, samples=37 lat (msec) : 4=2.34%, 10=17.32%, 20=24.74%, 50=47.01%, 100=8.33% lat (msec) : 250=0.26% cpu : usr=0.06%, sys=12.67%, ctx=1766, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159031: Thu Mar 5 20:12:51 2026 read: IOPS=20, BW=334KiB/s (342kB/s)(6256KiB/18743msec) clat (msec): min=2, max=147, avg=15.70, stdev=14.97 lat (msec): min=2, max=147, avg=15.70, stdev=14.97 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 8], 40.00th=[ 10], 50.00th=[ 11], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 23], 90.00th=[ 34], 95.00th=[ 43], | 99.00th=[ 73], 99.50th=[ 102], 99.90th=[ 148], 99.95th=[ 148], | 99.99th=[ 148] bw ( KiB/s): min= 95, max= 928, per=26.46%, avg=335.76, stdev=199.76, samples=37 iops : min= 5, max= 58, avg=20.49, stdev=12.75, samples=37 write: IOPS=20, BW=322KiB/s (330kB/s)(6032KiB/18743msec); 0 zone resets clat (msec): min=14, max=111, avg=33.26, stdev=17.38 lat (msec): min=14, max=111, avg=33.27, stdev=17.38 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 18], 10.00th=[ 18], 20.00th=[ 21], | 30.00th=[ 23], 40.00th=[ 25], 50.00th=[ 27], 60.00th=[ 31], | 70.00th=[ 37], 80.00th=[ 43], 90.00th=[ 58], 95.00th=[ 70], | 99.00th=[ 104], 99.50th=[ 107], 99.90th=[ 111], 99.95th=[ 111], | 99.99th=[ 111] bw ( KiB/s): min= 63, max= 672, per=23.97%, avg=320.03, stdev=149.79, samples=37 iops : min= 3, max= 42, avg=19.49, stdev= 9.65, samples=37 lat (msec) : 4=2.99%, 10=20.31%, 20=23.96%, 50=44.01%, 100=7.94% lat (msec) : 250=0.78% cpu : usr=0.07%, sys=12.98%, ctx=1863, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159032: Thu Mar 5 20:12:51 2026 read: IOPS=20, BW=326KiB/s (334kB/s)(6032KiB/18480msec) clat (usec): min=3199, max=92422, avg=16742.01, stdev=13610.98 lat (usec): min=3199, max=92422, avg=16742.59, stdev=13610.98 clat percentiles (usec): | 1.00th=[ 3490], 5.00th=[ 4047], 10.00th=[ 4752], 20.00th=[ 6128], | 30.00th=[ 8291], 40.00th=[10159], 50.00th=[11863], 60.00th=[14615], | 70.00th=[18744], 80.00th=[25560], 90.00th=[36439], 95.00th=[46400], | 99.00th=[59507], 99.50th=[74974], 99.90th=[92799], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 127, max= 960, per=25.49%, avg=323.44, stdev=167.84, samples=36 iops : min= 7, max= 60, avg=19.69, stdev=10.76, samples=36 write: IOPS=21, BW=339KiB/s (347kB/s)(6256KiB/18480msec); 0 zone resets clat (msec): min=13, max=128, avg=30.90, stdev=15.08 lat (msec): min=13, max=128, avg=30.90, stdev=15.08 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 17], 10.00th=[ 18], 20.00th=[ 20], | 30.00th=[ 22], 40.00th=[ 24], 50.00th=[ 27], 60.00th=[ 30], | 70.00th=[ 34], 80.00th=[ 40], 90.00th=[ 49], 95.00th=[ 63], | 99.00th=[ 93], 99.50th=[ 108], 99.90th=[ 129], 99.95th=[ 129], | 99.99th=[ 129] bw ( KiB/s): min= 159, max= 672, per=25.19%, avg=336.22, stdev=153.10, samples=36 iops : min= 9, max= 42, avg=20.50, stdev= 9.80, samples=36 lat (msec) : 4=2.08%, 10=16.93%, 20=27.86%, 50=46.74%, 100=5.99% lat (msec) : 250=0.39% cpu : usr=0.05%, sys=13.14%, ctx=1853, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1269KiB/s (1300kB/s), 304KiB/s-334KiB/s (312kB/s-342kB/s), io=23.4MiB (24.5MB), run=18480-18871msec WRITE: bw=1335KiB/s (1367kB/s), 322KiB/s-347KiB/s (330kB/s-355kB/s), io=24.6MiB (25.8MB), run=18480-18871msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159045: Thu Mar 5 20:13:00 2026 read: IOPS=2, BW=3036KiB/s (3109kB/s)(7168KiB/2361msec) clat (usec): min=975, max=228663, avg=131343.39, stdev=67575.86 lat (usec): min=976, max=228664, avg=131344.31, stdev=67575.68 clat percentiles (usec): | 1.00th=[ 979], 5.00th=[ 979], 10.00th=[ 979], 20.00th=[117965], | 30.00th=[135267], 40.00th=[135267], 50.00th=[135267], 60.00th=[149947], | 70.00th=[149947], 80.00th=[152044], 90.00th=[229639], 95.00th=[229639], | 99.00th=[229639], 99.50th=[229639], 99.90th=[229639], 99.95th=[229639], | 99.99th=[229639] bw ( KiB/s): min= 3930, max= 5678, per=70.19%, avg=4804.00, stdev=1236.02, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=2, BW=2169KiB/s (2221kB/s)(5120KiB/2361msec); 0 zone resets clat (msec): min=110, max=475, avg=287.95, stdev=154.35 lat (msec): min=110, max=476, avg=288.00, stdev=154.35 clat percentiles (msec): | 1.00th=[ 111], 5.00th=[ 111], 10.00th=[ 111], 20.00th=[ 111], | 30.00th=[ 174], 40.00th=[ 174], 50.00th=[ 271], 60.00th=[ 271], | 70.00th=[ 409], 80.00th=[ 409], 90.00th=[ 477], 95.00th=[ 477], | 99.00th=[ 477], 99.50th=[ 477], 99.90th=[ 477], 99.95th=[ 477], | 99.99th=[ 477] bw ( KiB/s): min= 1965, max= 4096, per=28.21%, avg=2703.00, stdev=1207.09, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 lat (usec) : 1000=8.33% lat (msec) : 250=66.67%, 500=25.00% cpu : usr=0.00%, sys=45.42%, ctx=577, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159046: Thu Mar 5 20:13:00 2026 read: IOPS=0, BW=684KiB/s (701kB/s)(2048KiB/2992msec) clat (msec): min=67, max=202, avg=135.28, stdev=95.59 lat (msec): min=67, max=202, avg=135.28, stdev=95.59 clat percentiles (msec): | 1.00th=[ 68], 5.00th=[ 68], 10.00th=[ 68], 20.00th=[ 68], | 30.00th=[ 68], 40.00th=[ 68], 50.00th=[ 68], 60.00th=[ 203], | 70.00th=[ 203], 80.00th=[ 203], 90.00th=[ 203], 95.00th=[ 203], | 99.00th=[ 203], 99.50th=[ 203], 99.90th=[ 203], 99.95th=[ 203], | 99.99th=[ 203] bw ( KiB/s): min= 1818, max= 2048, per=28.24%, avg=1933.00, stdev=162.63, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3422KiB/s (3505kB/s)(10.0MiB/2992msec); 0 zone resets clat (msec): min=65, max=787, avg=271.90, stdev=203.82 lat (msec): min=66, max=787, avg=271.97, stdev=203.82 clat percentiles (msec): | 1.00th=[ 66], 5.00th=[ 66], 10.00th=[ 66], 20.00th=[ 109], | 30.00th=[ 132], 40.00th=[ 213], 50.00th=[ 218], 60.00th=[ 220], | 70.00th=[ 300], 80.00th=[ 334], 90.00th=[ 347], 95.00th=[ 785], | 99.00th=[ 785], 99.50th=[ 785], 99.90th=[ 785], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 1818, max= 4096, per=36.37%, avg=3485.00, stdev=1114.08, samples=4 iops : min= 1, max= 4, avg= 3.00, stdev= 1.41, samples=4 lat (msec) : 100=16.67%, 250=50.00%, 500=25.00%, 1000=8.33% cpu : usr=0.07%, sys=38.05%, ctx=344, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159047: Thu Mar 5 20:13:00 2026 read: IOPS=2, BW=2781KiB/s (2848kB/s)(6144KiB/2209msec) clat (usec): min=840, max=256930, avg=109921.13, stdev=116128.99 lat (usec): min=841, max=256931, avg=109921.84, stdev=116129.03 clat percentiles (usec): | 1.00th=[ 840], 5.00th=[ 840], 10.00th=[ 840], 20.00th=[ 938], | 30.00th=[ 938], 40.00th=[ 23462], 50.00th=[ 23462], 60.00th=[156238], | 70.00th=[221250], 80.00th=[221250], 90.00th=[256902], 95.00th=[256902], | 99.00th=[256902], 99.50th=[256902], 99.90th=[256902], 99.95th=[256902], | 99.99th=[256902] bw ( KiB/s): min= 2048, max= 4096, per=49.06%, avg=3358.00, stdev=1137.53, samples=3 iops : min= 2, max= 4, avg= 3.00, stdev= 1.00, samples=3 write: IOPS=2, BW=2781KiB/s (2848kB/s)(6144KiB/2209msec); 0 zone resets clat (msec): min=168, max=465, avg=256.62, stdev=107.60 lat (msec): min=168, max=465, avg=256.66, stdev=107.60 clat percentiles (msec): | 1.00th=[ 169], 5.00th=[ 169], 10.00th=[ 169], 20.00th=[ 197], | 30.00th=[ 197], 40.00th=[ 205], 50.00th=[ 205], 60.00th=[ 251], | 70.00th=[ 257], 80.00th=[ 257], 90.00th=[ 464], 95.00th=[ 464], | 99.00th=[ 464], 99.50th=[ 464], 99.90th=[ 464], 99.95th=[ 464], | 99.99th=[ 464] bw ( KiB/s): min= 1965, max= 3657, per=25.35%, avg=2429.50, stdev=819.27, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (usec) : 1000=16.67% lat (msec) : 50=8.33%, 250=41.67%, 500=33.33% cpu : usr=0.00%, sys=47.10%, ctx=806, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159048: Thu Mar 5 20:13:00 2026 read: IOPS=1, BW=1886KiB/s (1931kB/s)(5120KiB/2715msec) clat (usec): min=788, max=246997, avg=108179.14, stdev=106966.52 lat (usec): min=789, max=246997, avg=108179.78, stdev=106966.39 clat percentiles (usec): | 1.00th=[ 791], 5.00th=[ 791], 10.00th=[ 791], 20.00th=[ 791], | 30.00th=[ 996], 40.00th=[ 996], 50.00th=[128451], 60.00th=[128451], | 70.00th=[162530], 80.00th=[162530], 90.00th=[246416], 95.00th=[246416], | 99.00th=[246416], 99.50th=[246416], 99.90th=[246416], 99.95th=[246416], | 99.99th=[246416] bw ( KiB/s): min= 2048, max= 6144, per=59.85%, avg=4096.00, stdev=2896.31, samples=2 iops : min= 2, max= 6, avg= 4.00, stdev= 2.83, samples=2 write: IOPS=2, BW=2640KiB/s (2704kB/s)(7168KiB/2715msec); 0 zone resets clat (msec): min=113, max=506, avg=307.07, stdev=147.73 lat (msec): min=113, max=506, avg=307.13, stdev=147.72 clat percentiles (msec): | 1.00th=[ 114], 5.00th=[ 114], 10.00th=[ 114], 20.00th=[ 144], | 30.00th=[ 249], 40.00th=[ 249], 50.00th=[ 309], 60.00th=[ 393], | 70.00th=[ 393], 80.00th=[ 435], 90.00th=[ 506], 95.00th=[ 506], | 99.00th=[ 506], 99.50th=[ 506], 99.90th=[ 506], 99.95th=[ 506], | 99.99th=[ 506] bw ( KiB/s): min= 1831, max= 3930, per=25.72%, avg=2464.25, stdev=982.51, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (usec) : 1000=16.67% lat (msec) : 250=50.00%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=40.16%, ctx=651, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6845KiB/s (7009kB/s), 684KiB/s-3036KiB/s (701kB/s-3109kB/s), io=20.0MiB (20.0MB), run=2209-2992msec WRITE: bw=9583KiB/s (9813kB/s), 2169KiB/s-3422KiB/s (2221kB/s-3505kB/s), io=28.0MiB (29.4MB), run=2209-2992msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159040: Thu Mar 5 20:13:00 2026 read: IOPS=3, BW=3399KiB/s (3480kB/s)(7168KiB/2109msec) clat (msec): min=48, max=376, avg=151.54, stdev=136.06 lat (msec): min=48, max=376, avg=151.54, stdev=136.06 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 50], 20.00th=[ 54], | 30.00th=[ 61], 40.00th=[ 61], 50.00th=[ 61], 60.00th=[ 157], | 70.00th=[ 157], 80.00th=[ 305], 90.00th=[ 376], 95.00th=[ 376], | 99.00th=[ 376], 99.50th=[ 376], 99.90th=[ 376], 99.95th=[ 376], | 99.99th=[ 376] bw ( KiB/s): min= 1988, max= 5269, per=43.56%, avg=2832.00, stdev=1624.85, samples=4 iops : min= 1, max= 5, avg= 2.25, stdev= 1.89, samples=4 write: IOPS=2, BW=2428KiB/s (2486kB/s)(5120KiB/2109msec); 0 zone resets clat (msec): min=96, max=350, avg=199.98, stdev=106.74 lat (msec): min=96, max=350, avg=200.05, stdev=106.74 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 96], 10.00th=[ 96], 20.00th=[ 96], | 30.00th=[ 99], 40.00th=[ 99], 50.00th=[ 215], 60.00th=[ 215], | 70.00th=[ 241], 80.00th=[ 241], 90.00th=[ 351], 95.00th=[ 351], | 99.00th=[ 351], 99.50th=[ 351], 99.90th=[ 351], 99.95th=[ 351], | 99.99th=[ 351] bw ( KiB/s): min= 1756, max= 4096, per=35.99%, avg=3276.00, stdev=1317.73, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 lat (msec) : 50=8.33%, 100=41.67%, 250=25.00%, 500=25.00% cpu : usr=0.00%, sys=16.37%, ctx=95, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159041: Thu Mar 5 20:13:00 2026 read: IOPS=0, BW=650KiB/s (666kB/s)(2048KiB/3150msec) clat (msec): min=53, max=140, avg=96.91, stdev=61.37 lat (msec): min=53, max=140, avg=96.91, stdev=61.36 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 54], 10.00th=[ 54], 20.00th=[ 54], | 30.00th=[ 54], 40.00th=[ 54], 50.00th=[ 54], 60.00th=[ 140], | 70.00th=[ 140], 80.00th=[ 140], 90.00th=[ 140], 95.00th=[ 140], | 99.00th=[ 140], 99.50th=[ 140], 99.90th=[ 140], 99.95th=[ 140], | 99.99th=[ 140] bw ( KiB/s): min= 1756, max= 1988, per=28.80%, avg=1872.00, stdev=164.05, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3251KiB/s (3329kB/s)(10.0MiB/3150msec); 0 zone resets clat (msec): min=76, max=813, avg=294.22, stdev=251.84 lat (msec): min=76, max=813, avg=294.28, stdev=251.84 clat percentiles (msec): | 1.00th=[ 77], 5.00th=[ 77], 10.00th=[ 77], 20.00th=[ 85], | 30.00th=[ 91], 40.00th=[ 114], 50.00th=[ 144], 60.00th=[ 271], | 70.00th=[ 368], 80.00th=[ 372], 90.00th=[ 609], 95.00th=[ 810], | 99.00th=[ 810], 99.50th=[ 810], 99.90th=[ 810], 99.95th=[ 810], | 99.99th=[ 810] bw ( KiB/s): min= 1988, max= 4096, per=38.92%, avg=3542.80, stdev=902.44, samples=5 iops : min= 1, max= 4, avg= 2.80, stdev= 1.10, samples=5 lat (msec) : 100=33.33%, 250=25.00%, 500=25.00%, 750=8.33%, 1000=8.33% cpu : usr=0.03%, sys=16.04%, ctx=94, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159042: Thu Mar 5 20:13:00 2026 read: IOPS=2, BW=2127KiB/s (2178kB/s)(6144KiB/2889msec) clat (msec): min=62, max=350, avg=179.11, stdev=100.09 lat (msec): min=62, max=350, avg=179.11, stdev=100.09 clat percentiles (msec): | 1.00th=[ 63], 5.00th=[ 63], 10.00th=[ 63], 20.00th=[ 100], | 30.00th=[ 100], 40.00th=[ 176], 50.00th=[ 176], 60.00th=[ 186], | 70.00th=[ 201], 80.00th=[ 201], 90.00th=[ 351], 95.00th=[ 351], | 99.00th=[ 351], 99.50th=[ 351], 99.90th=[ 351], 99.95th=[ 351], | 99.99th=[ 351] bw ( KiB/s): min= 2035, max= 4096, per=52.19%, avg=3392.67, stdev=1176.03, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=2, BW=2127KiB/s (2178kB/s)(6144KiB/2889msec); 0 zone resets clat (msec): min=71, max=476, avg=297.19, stdev=179.21 lat (msec): min=71, max=476, avg=297.25, stdev=179.21 clat percentiles (msec): | 1.00th=[ 72], 5.00th=[ 72], 10.00th=[ 72], 20.00th=[ 80], | 30.00th=[ 80], 40.00th=[ 317], 50.00th=[ 317], 60.00th=[ 414], | 70.00th=[ 426], 80.00th=[ 426], 90.00th=[ 477], 95.00th=[ 477], | 99.00th=[ 477], 99.50th=[ 477], 99.90th=[ 477], 99.95th=[ 477], | 99.99th=[ 477] bw ( KiB/s): min= 1988, max= 3512, per=26.25%, avg=2389.50, stdev=748.60, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 100=33.33%, 250=25.00%, 500=41.67% cpu : usr=0.03%, sys=14.47%, ctx=127, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159043: Thu Mar 5 20:13:00 2026 read: IOPS=1, BW=1809KiB/s (1852kB/s)(5120KiB/2831msec) clat (msec): min=56, max=556, avg=217.96, stdev=221.94 lat (msec): min=56, max=556, avg=217.96, stdev=221.94 clat percentiles (msec): | 1.00th=[ 57], 5.00th=[ 57], 10.00th=[ 57], 20.00th=[ 57], | 30.00th=[ 63], 40.00th=[ 63], 50.00th=[ 81], 60.00th=[ 81], | 70.00th=[ 334], 80.00th=[ 334], 90.00th=[ 558], 95.00th=[ 558], | 99.00th=[ 558], 99.50th=[ 558], 99.90th=[ 558], 99.95th=[ 558], | 99.99th=[ 558] bw ( KiB/s): min= 3976, max= 4096, per=62.08%, avg=4036.00, stdev=84.85, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2532KiB/s (2593kB/s)(7168KiB/2831msec); 0 zone resets clat (msec): min=63, max=433, avg=243.75, stdev=150.24 lat (msec): min=63, max=433, avg=243.82, stdev=150.23 clat percentiles (msec): | 1.00th=[ 64], 5.00th=[ 64], 10.00th=[ 64], 20.00th=[ 95], | 30.00th=[ 111], 40.00th=[ 111], 50.00th=[ 300], 60.00th=[ 338], | 70.00th=[ 338], 80.00th=[ 372], 90.00th=[ 435], 95.00th=[ 435], | 99.00th=[ 435], 99.50th=[ 435], 99.90th=[ 435], 99.95th=[ 435], | 99.99th=[ 435] bw ( KiB/s): min= 2035, max= 4047, per=35.23%, avg=3206.33, stdev=1045.96, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 lat (msec) : 100=41.67%, 250=8.33%, 500=41.67%, 750=8.33% cpu : usr=0.00%, sys=12.69%, ctx=108, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6502KiB/s (6658kB/s), 650KiB/s-3399KiB/s (666kB/s-3480kB/s), io=20.0MiB (20.0MB), run=2109-3150msec WRITE: bw=9102KiB/s (9321kB/s), 2127KiB/s-3251KiB/s (2178kB/s-3329kB/s), io=28.0MiB (29.4MB), run=2109-3150msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159054: Thu Mar 5 20:13:08 2026 read: IOPS=1, BW=7699KiB/s (7884kB/s)(12.0MiB/1596msec) clat (msec): min=464, max=570, avg=501.28, stdev=59.85 lat (msec): min=464, max=570, avg=501.28, stdev=59.85 clat percentiles (msec): | 1.00th=[ 464], 5.00th=[ 464], 10.00th=[ 464], 20.00th=[ 464], | 30.00th=[ 464], 40.00th=[ 468], 50.00th=[ 468], 60.00th=[ 468], | 70.00th=[ 567], 80.00th=[ 567], 90.00th=[ 567], 95.00th=[ 567], | 99.00th=[ 567], 99.50th=[ 567], 99.90th=[ 567], 99.95th=[ 567], | 99.99th=[ 567] bw ( KiB/s): min= 8175, max= 8192, per=91.81%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.00%, sys=52.54%, ctx=2650, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159055: Thu Mar 5 20:13:08 2026 read: IOPS=0, BW=2229KiB/s (2282kB/s)(4096KiB/1838msec) clat (nsec): min=570578k, max=570578k, avg=570577801.00, stdev= 0.00 lat (nsec): min=570579k, max=570579k, avg=570578951.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 575], 5.00th=[ 575], 10.00th=[ 575], 20.00th=[ 575], | 30.00th=[ 575], 40.00th=[ 575], 50.00th=[ 575], 60.00th=[ 575], | 70.00th=[ 575], 80.00th=[ 575], 90.00th=[ 575], 95.00th=[ 575], | 99.00th=[ 575], 99.50th=[ 575], 99.90th=[ 575], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 8192, max= 8192, per=91.90%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4457KiB/s (4564kB/s)(8192KiB/1838msec); 0 zone resets clat (msec): min=359, max=824, avg=592.15, stdev=329.28 lat (msec): min=359, max=825, avg=592.32, stdev=329.20 clat percentiles (msec): | 1.00th=[ 359], 5.00th=[ 359], 10.00th=[ 359], 20.00th=[ 359], | 30.00th=[ 359], 40.00th=[ 359], 50.00th=[ 359], 60.00th=[ 827], | 70.00th=[ 827], 80.00th=[ 827], 90.00th=[ 827], 95.00th=[ 827], | 99.00th=[ 827], 99.50th=[ 827], 99.90th=[ 827], 99.95th=[ 827], | 99.99th=[ 827] bw ( KiB/s): min= 8175, max= 8175, per=50.85%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=20.30%, ctx=771, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159056: Thu Mar 5 20:13:08 2026 write: IOPS=1, BW=6029KiB/s (6174kB/s)(12.0MiB/2038msec); 0 zone resets clat (msec): min=154, max=1289, avg=649.43, stdev=581.23 lat (msec): min=154, max=1289, avg=649.67, stdev=581.28 clat percentiles (msec): | 1.00th=[ 155], 5.00th=[ 155], 10.00th=[ 155], 20.00th=[ 155], | 30.00th=[ 155], 40.00th=[ 506], 50.00th=[ 506], 60.00th=[ 506], | 70.00th=[ 1284], 80.00th=[ 1284], 90.00th=[ 1284], 95.00th=[ 1284], | 99.00th=[ 1284], 99.50th=[ 1284], 99.90th=[ 1284], 99.95th=[ 1284], | 99.99th=[ 1284] bw ( KiB/s): min= 8192, max= 8192, per=50.95%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=17.72%, ctx=88, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159057: Thu Mar 5 20:13:08 2026 write: IOPS=1, BW=7502KiB/s (7682kB/s)(12.0MiB/1638msec); 0 zone resets clat (msec): min=144, max=1099, avg=529.33, stdev=503.52 lat (msec): min=144, max=1099, avg=529.60, stdev=503.51 clat percentiles (msec): | 1.00th=[ 144], 5.00th=[ 144], 10.00th=[ 144], 20.00th=[ 144], | 30.00th=[ 144], 40.00th=[ 347], 50.00th=[ 347], 60.00th=[ 347], | 70.00th=[ 1099], 80.00th=[ 1099], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min=11237, max=11237, per=69.89%, avg=11237.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=17.59%, ctx=97, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8914KiB/s (9128kB/s), 2229KiB/s-7699KiB/s (2282kB/s-7884kB/s), io=16.0MiB (16.8MB), run=1596-1838msec WRITE: bw=15.7MiB/s (16.5MB/s), 4457KiB/s-7502KiB/s (4564kB/s-7682kB/s), io=32.0MiB (33.6MB), run=1638-2038msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159058: Thu Mar 5 20:13:08 2026 read: IOPS=1, BW=6974KiB/s (7141kB/s)(12.0MiB/1762msec) clat (msec): min=211, max=841, avg=578.43, stdev=327.53 lat (msec): min=211, max=841, avg=578.43, stdev=327.53 clat percentiles (msec): | 1.00th=[ 211], 5.00th=[ 211], 10.00th=[ 211], 20.00th=[ 211], | 30.00th=[ 211], 40.00th=[ 684], 50.00th=[ 684], 60.00th=[ 684], | 70.00th=[ 844], 80.00th=[ 844], 90.00th=[ 844], 95.00th=[ 844], | 99.00th=[ 844], 99.50th=[ 844], 99.90th=[ 844], 99.95th=[ 844], | 99.99th=[ 844] bw ( KiB/s): min= 8110, max= 8192, per=100.00%, avg=8151.00, stdev=57.98, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=15.16%, ctx=74, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159059: Thu Mar 5 20:13:08 2026 read: IOPS=0, BW=1975KiB/s (2022kB/s)(4096KiB/2074msec) clat (nsec): min=1048.6M, max=1048.6M, avg=1048576654.00, stdev= 0.00 lat (nsec): min=1048.6M, max=1048.6M, avg=1048577922.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1053], 5.00th=[ 1053], 10.00th=[ 1053], 20.00th=[ 1053], | 30.00th=[ 1053], 40.00th=[ 1053], 50.00th=[ 1053], 60.00th=[ 1053], | 70.00th=[ 1053], 80.00th=[ 1053], 90.00th=[ 1053], 95.00th=[ 1053], | 99.00th=[ 1053], 99.50th=[ 1053], 99.90th=[ 1053], 99.95th=[ 1053], | 99.99th=[ 1053] bw ( KiB/s): min= 8110, max= 8110, per=100.00%, avg=8110.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3950KiB/s (4045kB/s)(8192KiB/2074msec); 0 zone resets clat (msec): min=464, max=521, avg=492.62, stdev=40.23 lat (msec): min=464, max=521, avg=492.93, stdev=40.16 clat percentiles (msec): | 1.00th=[ 464], 5.00th=[ 464], 10.00th=[ 464], 20.00th=[ 464], | 30.00th=[ 464], 40.00th=[ 464], 50.00th=[ 464], 60.00th=[ 523], | 70.00th=[ 523], 80.00th=[ 523], 90.00th=[ 523], 95.00th=[ 523], | 99.00th=[ 523], 99.50th=[ 523], 99.90th=[ 523], 99.95th=[ 523], | 99.99th=[ 523] bw ( KiB/s): min= 8192, max= 8192, per=52.10%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=15.53%, ctx=89, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159060: Thu Mar 5 20:13:08 2026 write: IOPS=1, BW=5896KiB/s (6038kB/s)(12.0MiB/2084msec); 0 zone resets clat (msec): min=141, max=1324, avg=676.67, stdev=599.66 lat (msec): min=141, max=1325, avg=676.91, stdev=599.71 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 142], | 30.00th=[ 142], 40.00th=[ 567], 50.00th=[ 567], 60.00th=[ 567], | 70.00th=[ 1318], 80.00th=[ 1318], 90.00th=[ 1318], 95.00th=[ 1318], | 99.00th=[ 1318], 99.50th=[ 1318], 99.90th=[ 1318], 99.95th=[ 1318], | 99.99th=[ 1318] bw ( KiB/s): min= 8192, max= 8192, per=52.10%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.05%, sys=13.11%, ctx=64, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159061: Thu Mar 5 20:13:08 2026 write: IOPS=1, BW=6978KiB/s (7145kB/s)(12.0MiB/1761msec); 0 zone resets clat (msec): min=464, max=800, avg=579.67, stdev=191.55 lat (msec): min=464, max=801, avg=579.93, stdev=191.54 clat percentiles (msec): | 1.00th=[ 464], 5.00th=[ 464], 10.00th=[ 464], 20.00th=[ 464], | 30.00th=[ 464], 40.00th=[ 472], 50.00th=[ 472], 60.00th=[ 472], | 70.00th=[ 802], 80.00th=[ 802], 90.00th=[ 802], 95.00th=[ 802], | 99.00th=[ 802], 99.50th=[ 802], 99.90th=[ 802], 99.95th=[ 802], | 99.99th=[ 802] bw ( KiB/s): min= 5565, max= 8110, per=43.49%, avg=6837.50, stdev=1799.59, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, 1000=33.33% cpu : usr=0.00%, sys=16.02%, ctx=61, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7900KiB/s (8089kB/s), 1975KiB/s-6974KiB/s (2022kB/s-7141kB/s), io=16.0MiB (16.8MB), run=1762-2074msec WRITE: bw=15.4MiB/s (16.1MB/s), 3950KiB/s-6978KiB/s (4045kB/s-7145kB/s), io=32.0MiB (33.6MB), run=1761-2084msec