== sanity test 398b: DIO and buffer IO race ============== 01:01:30 (1763877690) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.8717 s, 17.5 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159298: Sun Nov 23 01:02:38 2025 read: IOPS=25, BW=101KiB/s (103kB/s)(5992KiB/59569msec) clat (usec): min=108, max=635917, avg=20754.68, stdev=19348.90 lat (usec): min=109, max=635918, avg=20755.38, stdev=19348.91 clat percentiles (usec): | 1.00th=[ 445], 5.00th=[ 6063], 10.00th=[ 8717], 20.00th=[ 11469], | 30.00th=[ 13960], 40.00th=[ 16450], 50.00th=[ 18482], 60.00th=[ 21365], | 70.00th=[ 24773], 80.00th=[ 28443], 90.00th=[ 34341], 95.00th=[ 39584], | 99.00th=[ 59507], 99.50th=[ 65274], 99.90th=[ 79168], 99.95th=[633340], | 99.99th=[633340] bw ( KiB/s): min= 6, max= 200, per=24.79%, avg=99.89, stdev=31.16, samples=118 iops : min= 1, max= 50, avg=24.60, stdev= 7.88, samples=118 write: IOPS=26, BW=106KiB/s (108kB/s)(6296KiB/59569msec); 0 zone resets clat (usec): min=763, max=135240, avg=17956.76, stdev=15287.95 lat (usec): min=764, max=135241, avg=17957.70, stdev=15288.02 clat percentiles (usec): | 1.00th=[ 1123], 5.00th=[ 2540], 10.00th=[ 3392], 20.00th=[ 5538], | 30.00th=[ 7898], 40.00th=[ 11076], 50.00th=[ 14877], 60.00th=[ 18220], | 70.00th=[ 21890], 80.00th=[ 27657], 90.00th=[ 36963], 95.00th=[ 44303], | 99.00th=[ 72877], 99.50th=[ 90702], 99.90th=[135267], 99.95th=[135267], | 99.99th=[135267] bw ( KiB/s): min= 47, max= 200, per=24.99%, avg=105.46, stdev=33.10, samples=117 iops : min= 11, max= 50, avg=26.00, stdev= 8.35, samples=117 lat (usec) : 250=0.10%, 500=0.59%, 750=0.46%, 1000=0.94% lat (msec) : 2=1.14%, 4=4.98%, 10=17.42%, 20=35.25%, 50=36.91% lat (msec) : 100=1.95%, 250=0.23%, 750=0.03% cpu : usr=0.10%, sys=28.99%, ctx=9481, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159299: Sun Nov 23 01:02:38 2025 read: IOPS=25, BW=101KiB/s (104kB/s)(5940KiB/58590msec) clat (usec): min=83, max=102592, avg=20420.75, stdev=11832.01 lat (usec): min=83, max=102593, avg=20421.43, stdev=11832.05 clat percentiles (usec): | 1.00th=[ 486], 5.00th=[ 6652], 10.00th=[ 8848], 20.00th=[ 11600], | 30.00th=[ 13829], 40.00th=[ 15795], 50.00th=[ 17957], 60.00th=[ 20317], | 70.00th=[ 23987], 80.00th=[ 28443], 90.00th=[ 34866], 95.00th=[ 40633], | 99.00th=[ 58983], 99.50th=[ 71828], 99.90th=[ 91751], 99.95th=[102237], | 99.99th=[102237] bw ( KiB/s): min= 6, max= 200, per=25.15%, avg=101.34, stdev=30.27, samples=116 iops : min= 1, max= 50, avg=24.97, stdev= 7.65, samples=116 write: IOPS=27, BW=108KiB/s (111kB/s)(6348KiB/58590msec); 0 zone resets clat (usec): min=661, max=624916, avg=17683.68, stdev=21064.47 lat (usec): min=662, max=624916, avg=17684.54, stdev=21064.50 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 2311], 10.00th=[ 3261], 20.00th=[ 5342], | 30.00th=[ 7570], 40.00th=[ 10683], 50.00th=[ 14091], 60.00th=[ 17171], | 70.00th=[ 21627], 80.00th=[ 27657], 90.00th=[ 35390], 95.00th=[ 43779], | 99.00th=[ 63701], 99.50th=[ 86508], 99.90th=[141558], 99.95th=[624952], | 99.99th=[624952] bw ( KiB/s): min= 24, max= 192, per=25.84%, avg=109.06, stdev=36.87, samples=115 iops : min= 6, max= 48, avg=26.90, stdev= 9.30, samples=115 lat (usec) : 100=0.03%, 250=0.13%, 500=0.33%, 750=0.88%, 1000=0.65% lat (msec) : 2=1.33%, 4=5.37%, 10=17.87%, 20=36.10%, 50=34.64% lat (msec) : 100=2.44%, 250=0.20%, 750=0.03% cpu : usr=0.11%, sys=28.93%, ctx=9434, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159300: Sun Nov 23 01:02:38 2025 read: IOPS=25, BW=102KiB/s (105kB/s)(6028KiB/58984msec) clat (usec): min=88, max=117742, avg=19920.19, stdev=11469.49 lat (usec): min=89, max=117743, avg=19921.05, stdev=11469.46 clat percentiles (usec): | 1.00th=[ 437], 5.00th=[ 5473], 10.00th=[ 8586], 20.00th=[ 11207], | 30.00th=[ 13698], 40.00th=[ 15926], 50.00th=[ 17957], 60.00th=[ 20579], | 70.00th=[ 23200], 80.00th=[ 27657], 90.00th=[ 34341], 95.00th=[ 39060], | 99.00th=[ 55313], 99.50th=[ 63177], 99.90th=[106431], 99.95th=[117965], | 99.99th=[117965] bw ( KiB/s): min= 39, max= 200, per=25.54%, avg=102.94, stdev=32.70, samples=116 iops : min= 9, max= 50, avg=25.37, stdev= 8.27, samples=116 write: IOPS=26, BW=106KiB/s (109kB/s)(6260KiB/58984msec); 0 zone resets clat (usec): min=671, max=704154, avg=18378.07, stdev=23022.27 lat (usec): min=671, max=704155, avg=18379.03, stdev=23022.29 clat percentiles (usec): | 1.00th=[ 914], 5.00th=[ 2606], 10.00th=[ 3490], 20.00th=[ 5604], | 30.00th=[ 7832], 40.00th=[ 11076], 50.00th=[ 14484], 60.00th=[ 18482], | 70.00th=[ 22938], 80.00th=[ 28705], 90.00th=[ 35390], 95.00th=[ 43254], | 99.00th=[ 70779], 99.50th=[ 90702], 99.90th=[145753], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 24, max= 232, per=25.29%, avg=106.74, stdev=33.38, samples=116 iops : min= 6, max= 58, avg=26.32, stdev= 8.42, samples=116 lat (usec) : 100=0.03%, 250=0.10%, 500=0.72%, 750=1.01%, 1000=0.72% lat (msec) : 2=0.91%, 4=4.88%, 10=17.15%, 20=35.19%, 50=36.91% lat (msec) : 100=2.05%, 250=0.29%, 750=0.03% cpu : usr=0.09%, sys=28.78%, ctx=9413, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159301: Sun Nov 23 01:02:38 2025 read: IOPS=25, BW=102KiB/s (104kB/s)(6052KiB/59486msec) clat (usec): min=158, max=194686, avg=20413.72, stdev=11738.68 lat (usec): min=159, max=194687, avg=20414.48, stdev=11738.69 clat percentiles (usec): | 1.00th=[ 433], 5.00th=[ 7177], 10.00th=[ 8979], 20.00th=[ 11731], | 30.00th=[ 14091], 40.00th=[ 16188], 50.00th=[ 18482], 60.00th=[ 21103], | 70.00th=[ 23987], 80.00th=[ 27919], 90.00th=[ 34341], 95.00th=[ 39584], | 99.00th=[ 54789], 99.50th=[ 63701], 99.90th=[100140], 99.95th=[193987], | 99.99th=[193987] bw ( KiB/s): min= 16, max= 192, per=25.37%, avg=102.26, stdev=29.93, samples=117 iops : min= 4, max= 48, avg=25.20, stdev= 7.54, samples=117 write: IOPS=26, BW=105KiB/s (107kB/s)(6236KiB/59486msec); 0 zone resets clat (usec): min=648, max=700539, avg=18213.59, stdev=22858.20 lat (usec): min=649, max=700541, avg=18214.49, stdev=22858.22 clat percentiles (usec): | 1.00th=[ 873], 5.00th=[ 2704], 10.00th=[ 3589], 20.00th=[ 5473], | 30.00th=[ 7701], 40.00th=[ 11076], 50.00th=[ 14484], 60.00th=[ 17695], | 70.00th=[ 22414], 80.00th=[ 27919], 90.00th=[ 35914], 95.00th=[ 44827], | 99.00th=[ 70779], 99.50th=[ 84411], 99.90th=[160433], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 55, max= 200, per=24.96%, avg=105.32, stdev=32.51, samples=117 iops : min= 13, max= 50, avg=25.97, stdev= 8.23, samples=117 lat (usec) : 250=0.07%, 500=0.59%, 750=0.91%, 1000=0.68% lat (msec) : 2=0.85%, 4=4.69%, 10=17.35%, 20=35.35%, 50=36.98% lat (msec) : 100=2.41%, 250=0.10%, 750=0.03% cpu : usr=0.10%, sys=29.02%, ctx=9541, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=403KiB/s (413kB/s), 101KiB/s-102KiB/s (103kB/s-105kB/s), io=23.4MiB (24.6MB), run=58590-59569msec WRITE: bw=422KiB/s (432kB/s), 105KiB/s-108KiB/s (107kB/s-111kB/s), io=24.6MiB (25.7MB), run=58590-59569msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159303: Sun Nov 23 01:03:10 2025 read: IOPS=16, BW=65.7KiB/s (67.2kB/s)(5992KiB/91258msec) clat (msec): min=2, max=637, avg=19.93, stdev=22.75 lat (msec): min=2, max=637, avg=19.93, stdev=22.75 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 21], 80.00th=[ 26], 90.00th=[ 40], 95.00th=[ 52], | 99.00th=[ 87], 99.50th=[ 101], 99.90th=[ 148], 99.95th=[ 634], | 99.99th=[ 634] bw ( KiB/s): min= 7, max= 168, per=24.92%, avg=65.53, stdev=32.52, samples=180 iops : min= 1, max= 42, avg=16.07, stdev= 8.29, samples=180 write: IOPS=17, BW=68.0KiB/s (70.6kB/s)(6296KiB/91258msec); 0 zone resets clat (msec): min=13, max=187, avg=38.92, stdev=19.32 lat (msec): min=13, max=187, avg=38.92, stdev=19.32 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 31], 50.00th=[ 33], 60.00th=[ 38], | 70.00th=[ 43], 80.00th=[ 51], 90.00th=[ 63], 95.00th=[ 75], | 99.00th=[ 113], 99.50th=[ 127], 99.90th=[ 169], 99.95th=[ 188], | 99.99th=[ 188] bw ( KiB/s): min= 15, max= 152, per=25.01%, avg=68.77, stdev=28.57, samples=180 iops : min= 3, max= 38, avg=16.88, stdev= 7.31, samples=180 lat (msec) : 4=1.50%, 10=11.43%, 20=23.57%, 50=50.42%, 100=11.98% lat (msec) : 250=1.07%, 750=0.03% cpu : usr=0.08%, sys=10.23%, ctx=6524, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159304: Sun Nov 23 01:03:10 2025 read: IOPS=16, BW=65.1KiB/s (66.7kB/s)(5940KiB/91214msec) clat (msec): min=2, max=680, avg=20.04, stdev=23.29 lat (msec): min=2, max=680, avg=20.04, stdev=23.29 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 9], | 30.00th=[ 12], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 27], 90.00th=[ 39], 95.00th=[ 51], | 99.00th=[ 85], 99.50th=[ 97], 99.90th=[ 142], 99.95th=[ 684], | 99.99th=[ 684] bw ( KiB/s): min= 7, max= 160, per=24.76%, avg=65.13, stdev=32.03, samples=180 iops : min= 1, max= 40, avg=15.97, stdev= 8.17, samples=180 write: IOPS=17, BW=69.6KiB/s (71.3kB/s)(6348KiB/91214msec); 0 zone resets clat (msec): min=13, max=206, avg=38.63, stdev=18.95 lat (msec): min=13, max=206, avg=38.64, stdev=18.95 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 31], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 42], 80.00th=[ 51], 90.00th=[ 64], 95.00th=[ 78], | 99.00th=[ 105], 99.50th=[ 123], 99.90th=[ 167], 99.95th=[ 207], | 99.99th=[ 207] bw ( KiB/s): min= 8, max= 160, per=25.22%, avg=69.35, stdev=28.93, samples=180 iops : min= 2, max= 40, avg=17.02, stdev= 7.39, samples=180 lat (msec) : 4=1.63%, 10=11.20%, 20=22.36%, 50=52.08%, 100=11.78% lat (msec) : 250=0.91%, 750=0.03% cpu : usr=0.07%, sys=10.22%, ctx=6508, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159305: Sun Nov 23 01:03:10 2025 read: IOPS=16, BW=66.5KiB/s (68.1kB/s)(6028KiB/90687msec) clat (msec): min=2, max=148, avg=19.50, stdev=16.36 lat (msec): min=2, max=148, avg=19.50, stdev=16.36 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 21], 80.00th=[ 28], 90.00th=[ 39], 95.00th=[ 51], | 99.00th=[ 84], 99.50th=[ 99], 99.90th=[ 146], 99.95th=[ 148], | 99.99th=[ 148] bw ( KiB/s): min= 7, max= 200, per=25.45%, avg=66.94, stdev=33.40, samples=178 iops : min= 1, max= 50, avg=16.42, stdev= 8.51, samples=178 write: IOPS=17, BW=69.0KiB/s (70.7kB/s)(6260KiB/90687msec); 0 zone resets clat (msec): min=13, max=697, avg=39.08, stdev=24.82 lat (msec): min=13, max=697, avg=39.08, stdev=24.82 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 26], | 30.00th=[ 28], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 43], 80.00th=[ 50], 90.00th=[ 64], 95.00th=[ 75], | 99.00th=[ 110], 99.50th=[ 127], 99.90th=[ 174], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 23, max= 152, per=25.08%, avg=68.98, stdev=29.02, samples=179 iops : min= 5, max= 38, avg=16.93, stdev= 7.43, samples=179 lat (msec) : 4=1.73%, 10=11.69%, 20=21.61%, 50=52.57%, 100=11.52% lat (msec) : 250=0.85%, 750=0.03% cpu : usr=0.09%, sys=10.26%, ctx=6528, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159306: Sun Nov 23 01:03:10 2025 read: IOPS=16, BW=66.9KiB/s (68.5kB/s)(6052KiB/90471msec) clat (msec): min=2, max=136, avg=19.11, stdev=14.52 lat (msec): min=2, max=136, avg=19.11, stdev=14.52 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 9], | 30.00th=[ 12], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 21], 80.00th=[ 27], 90.00th=[ 37], 95.00th=[ 48], | 99.00th=[ 71], 99.50th=[ 89], 99.90th=[ 132], 99.95th=[ 136], | 99.99th=[ 136] bw ( KiB/s): min= 7, max= 208, per=25.51%, avg=67.10, stdev=34.56, samples=179 iops : min= 1, max= 52, avg=16.46, stdev= 8.78, samples=179 write: IOPS=17, BW=68.9KiB/s (70.6kB/s)(6236KiB/90471msec); 0 zone resets clat (msec): min=14, max=694, avg=39.39, stdev=25.37 lat (msec): min=14, max=694, avg=39.39, stdev=25.37 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 31], 50.00th=[ 33], 60.00th=[ 37], | 70.00th=[ 44], 80.00th=[ 52], 90.00th=[ 64], 95.00th=[ 73], | 99.00th=[ 118], 99.50th=[ 132], 99.90th=[ 207], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 23, max= 152, per=25.13%, avg=69.12, stdev=27.94, samples=179 iops : min= 5, max= 38, avg=16.96, stdev= 7.17, samples=179 lat (msec) : 4=1.60%, 10=10.94%, 20=22.62%, 50=51.66%, 100=12.30% lat (msec) : 250=0.85%, 750=0.03% cpu : usr=0.08%, sys=10.32%, ctx=6539, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=263KiB/s (269kB/s), 65.1KiB/s-66.9KiB/s (66.7kB/s-68.5kB/s), io=23.4MiB (24.6MB), run=90471-91258msec WRITE: bw=275KiB/s (282kB/s), 68.9KiB/s-69.6KiB/s (70.6kB/s-71.3kB/s), io=24.6MiB (25.7MB), run=90471-91258msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159338: Sun Nov 23 01:03:37 2025 read: IOPS=17, BW=277KiB/s (284kB/s)(5920KiB/21360msec) clat (usec): min=149, max=118457, avg=28006.98, stdev=16077.51 lat (usec): min=150, max=118458, avg=28007.68, stdev=16077.53 clat percentiles (usec): | 1.00th=[ 502], 5.00th=[ 742], 10.00th=[ 12911], 20.00th=[ 16909], | 30.00th=[ 19792], 40.00th=[ 22938], 50.00th=[ 25822], 60.00th=[ 29754], | 70.00th=[ 33162], 80.00th=[ 38536], 90.00th=[ 47449], 95.00th=[ 54789], | 99.00th=[ 73925], 99.50th=[110625], 99.90th=[117965], 99.95th=[117965], | 99.99th=[117965] bw ( KiB/s): min= 95, max= 478, per=24.84%, avg=276.71, stdev=91.43, samples=42 iops : min= 5, max= 29, avg=16.67, stdev= 5.83, samples=42 write: IOPS=18, BW=298KiB/s (305kB/s)(6368KiB/21360msec); 0 zone resets clat (usec): min=1228, max=88091, avg=27398.19, stdev=16736.71 lat (usec): min=1229, max=88092, avg=27399.61, stdev=16736.70 clat percentiles (usec): | 1.00th=[ 3163], 5.00th=[ 7439], 10.00th=[ 8717], 20.00th=[13173], | 30.00th=[16712], 40.00th=[19792], 50.00th=[22938], 60.00th=[28181], | 70.00th=[32637], 80.00th=[40633], 90.00th=[52167], 95.00th=[60031], | 99.00th=[79168], 99.50th=[85459], 99.90th=[88605], 99.95th=[88605], | 99.99th=[88605] bw ( KiB/s): min= 190, max= 479, per=25.45%, avg=298.26, stdev=78.18, samples=42 iops : min= 11, max= 29, avg=18.02, stdev= 4.79, samples=42 lat (usec) : 250=0.26%, 500=0.13%, 750=2.08%, 1000=1.04% lat (msec) : 2=0.78%, 4=0.39%, 10=6.12%, 20=25.13%, 50=54.17% lat (msec) : 100=9.64%, 250=0.26% cpu : usr=0.09%, sys=31.88%, ctx=3133, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159339: Sun Nov 23 01:03:37 2025 read: IOPS=16, BW=267KiB/s (274kB/s)(5744KiB/21493msec) clat (usec): min=309, max=115779, avg=30051.83, stdev=15253.58 lat (usec): min=310, max=115780, avg=30052.62, stdev=15253.66 clat percentiles (usec): | 1.00th=[ 519], 5.00th=[ 1074], 10.00th=[ 15401], 20.00th=[ 18744], | 30.00th=[ 21365], 40.00th=[ 25035], 50.00th=[ 28705], 60.00th=[ 33162], | 70.00th=[ 36963], 80.00th=[ 41681], 90.00th=[ 46924], 95.00th=[ 53740], | 99.00th=[ 90702], 99.50th=[ 96994], 99.90th=[115868], 99.95th=[115868], | 99.99th=[115868] bw ( KiB/s): min= 127, max= 415, per=23.88%, avg=266.05, stdev=80.71, samples=42 iops : min= 7, max= 25, avg=16.00, stdev= 5.09, samples=42 write: IOPS=19, BW=304KiB/s (312kB/s)(6544KiB/21493msec); 0 zone resets clat (usec): min=1118, max=151363, avg=25860.77, stdev=17192.48 lat (usec): min=1120, max=151364, avg=25862.15, stdev=17192.49 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 7], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 16], 40.00th=[ 20], 50.00th=[ 23], 60.00th=[ 26], | 70.00th=[ 32], 80.00th=[ 39], 90.00th=[ 48], 95.00th=[ 57], | 99.00th=[ 82], 99.50th=[ 85], 99.90th=[ 153], 99.95th=[ 153], | 99.99th=[ 153] bw ( KiB/s): min= 157, max= 480, per=25.75%, avg=301.76, stdev=80.51, samples=42 iops : min= 9, max= 30, avg=18.24, stdev= 5.08, samples=42 lat (usec) : 500=0.39%, 750=0.78%, 1000=0.78% lat (msec) : 2=1.04%, 4=0.39%, 10=9.24%, 20=22.14%, 50=57.42% lat (msec) : 100=7.42%, 250=0.39% cpu : usr=0.08%, sys=32.64%, ctx=3233, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159340: Sun Nov 23 01:03:37 2025 read: IOPS=18, BW=292KiB/s (299kB/s)(6256KiB/21459msec) clat (usec): min=415, max=95916, avg=29141.91, stdev=15122.15 lat (usec): min=416, max=95917, avg=29142.71, stdev=15122.17 clat percentiles (usec): | 1.00th=[ 627], 5.00th=[ 832], 10.00th=[11600], 20.00th=[18482], | 30.00th=[21627], 40.00th=[25822], 50.00th=[28967], 60.00th=[31589], | 70.00th=[35390], 80.00th=[39060], 90.00th=[46400], 95.00th=[55837], | 99.00th=[76022], 99.50th=[83362], 99.90th=[95945], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 127, max= 544, per=26.25%, avg=292.45, stdev=91.71, samples=42 iops : min= 7, max= 34, avg=17.64, stdev= 5.85, samples=42 write: IOPS=17, BW=281KiB/s (288kB/s)(6032KiB/21459msec); 0 zone resets clat (usec): min=1157, max=146591, avg=26486.89, stdev=18489.00 lat (usec): min=1158, max=146592, avg=26490.91, stdev=18486.75 clat percentiles (usec): | 1.00th=[ 1614], 5.00th=[ 6521], 10.00th=[ 8029], 20.00th=[ 11076], | 30.00th=[ 14877], 40.00th=[ 18744], 50.00th=[ 21627], 60.00th=[ 25560], | 70.00th=[ 32113], 80.00th=[ 39584], 90.00th=[ 49546], 95.00th=[ 64226], | 99.00th=[ 84411], 99.50th=[ 98042], 99.90th=[145753], 99.95th=[145753], | 99.99th=[145753] bw ( KiB/s): min= 94, max= 480, per=23.64%, avg=277.05, stdev=77.03, samples=42 iops : min= 5, max= 30, avg=16.69, stdev= 4.85, samples=42 lat (usec) : 500=0.13%, 750=1.43%, 1000=1.82% lat (msec) : 2=1.30%, 4=0.52%, 10=7.42%, 20=21.48%, 50=57.03% lat (msec) : 100=8.72%, 250=0.13% cpu : usr=0.08%, sys=31.62%, ctx=3109, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159341: Sun Nov 23 01:03:37 2025 read: IOPS=17, BW=284KiB/s (290kB/s)(6032KiB/21265msec) clat (usec): min=219, max=128573, avg=28382.63, stdev=16656.76 lat (usec): min=219, max=128573, avg=28386.38, stdev=16657.04 clat percentiles (usec): | 1.00th=[ 478], 5.00th=[ 775], 10.00th=[ 1057], 20.00th=[ 17433], | 30.00th=[ 20317], 40.00th=[ 23987], 50.00th=[ 27132], 60.00th=[ 31327], | 70.00th=[ 34866], 80.00th=[ 39584], 90.00th=[ 47973], 95.00th=[ 56886], | 99.00th=[ 81265], 99.50th=[ 90702], 99.90th=[128451], 99.95th=[128451], | 99.99th=[128451] bw ( KiB/s): min= 159, max= 416, per=25.23%, avg=281.02, stdev=71.49, samples=42 iops : min= 9, max= 26, avg=16.95, stdev= 4.47, samples=42 write: IOPS=18, BW=294KiB/s (301kB/s)(6256KiB/21265msec); 0 zone resets clat (usec): min=1412, max=160714, avg=26919.84, stdev=17270.72 lat (usec): min=1413, max=160716, avg=26921.35, stdev=17270.65 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 16], 40.00th=[ 21], 50.00th=[ 24], 60.00th=[ 28], | 70.00th=[ 33], 80.00th=[ 40], 90.00th=[ 51], 95.00th=[ 59], | 99.00th=[ 74], 99.50th=[ 84], 99.90th=[ 161], 99.95th=[ 161], | 99.99th=[ 161] bw ( KiB/s): min= 159, max= 571, per=25.05%, avg=293.57, stdev=81.68, samples=42 iops : min= 9, max= 35, avg=17.74, stdev= 5.26, samples=42 lat (usec) : 250=0.13%, 500=0.39%, 750=1.69%, 1000=2.47% lat (msec) : 2=0.78%, 4=0.39%, 10=6.25%, 20=22.40%, 50=56.25% lat (msec) : 100=8.98%, 250=0.26% cpu : usr=0.09%, sys=32.43%, ctx=3223, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1114KiB/s (1141kB/s), 267KiB/s-292KiB/s (274kB/s-299kB/s), io=23.4MiB (24.5MB), run=21265-21493msec WRITE: bw=1172KiB/s (1201kB/s), 281KiB/s-304KiB/s (288kB/s-312kB/s), io=24.6MiB (25.8MB), run=21265-21493msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159333: Sun Nov 23 01:03:42 2025 read: IOPS=13, BW=223KiB/s (228kB/s)(5920KiB/26563msec) clat (msec): min=2, max=126, avg=23.67, stdev=21.54 lat (msec): min=2, max=126, avg=23.67, stdev=21.54 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 14], 50.00th=[ 18], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 35], 90.00th=[ 55], 95.00th=[ 75], | 99.00th=[ 95], 99.50th=[ 116], 99.90th=[ 127], 99.95th=[ 127], | 99.99th=[ 127] bw ( KiB/s): min= 31, max= 640, per=24.08%, avg=216.44, stdev=123.89, samples=52 iops : min= 1, max= 40, avg=12.98, stdev= 8.02, samples=52 write: IOPS=14, BW=240KiB/s (245kB/s)(6368KiB/26563msec); 0 zone resets clat (msec): min=14, max=173, avg=44.62, stdev=24.61 lat (msec): min=14, max=173, avg=44.62, stdev=24.61 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 21], 10.00th=[ 24], 20.00th=[ 28], | 30.00th=[ 31], 40.00th=[ 34], 50.00th=[ 37], 60.00th=[ 43], | 70.00th=[ 50], 80.00th=[ 59], 90.00th=[ 75], 95.00th=[ 90], | 99.00th=[ 157], 99.50th=[ 165], 99.90th=[ 174], 99.95th=[ 174], | 99.99th=[ 174] bw ( KiB/s): min= 64, max= 544, per=24.94%, avg=235.90, stdev=115.29, samples=52 iops : min= 4, max= 34, avg=14.19, stdev= 7.42, samples=52 lat (msec) : 4=2.34%, 10=13.93%, 20=11.59%, 50=51.43%, 100=18.23% lat (msec) : 250=2.47% cpu : usr=0.07%, sys=11.77%, ctx=1795, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159334: Sun Nov 23 01:03:42 2025 read: IOPS=13, BW=216KiB/s (221kB/s)(5744KiB/26621msec) clat (msec): min=3, max=190, avg=24.80, stdev=25.12 lat (msec): min=3, max=190, avg=24.80, stdev=25.12 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 7], | 30.00th=[ 10], 40.00th=[ 14], 50.00th=[ 18], 60.00th=[ 21], | 70.00th=[ 27], 80.00th=[ 36], 90.00th=[ 62], 95.00th=[ 77], | 99.00th=[ 118], 99.50th=[ 159], 99.90th=[ 192], 99.95th=[ 192], | 99.99th=[ 192] bw ( KiB/s): min= 63, max= 640, per=23.82%, avg=214.15, stdev=133.02, samples=52 iops : min= 3, max= 40, avg=12.83, stdev= 8.52, samples=52 write: IOPS=15, BW=246KiB/s (252kB/s)(6544KiB/26621msec); 0 zone resets clat (msec): min=13, max=159, avg=43.24, stdev=23.45 lat (msec): min=13, max=159, avg=43.25, stdev=23.45 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 21], 10.00th=[ 23], 20.00th=[ 25], | 30.00th=[ 29], 40.00th=[ 32], 50.00th=[ 36], 60.00th=[ 42], | 70.00th=[ 50], 80.00th=[ 58], 90.00th=[ 72], 95.00th=[ 96], | 99.00th=[ 124], 99.50th=[ 128], 99.90th=[ 161], 99.95th=[ 161], | 99.99th=[ 161] bw ( KiB/s): min= 63, max= 544, per=25.28%, avg=239.19, stdev=119.61, samples=52 iops : min= 3, max= 34, avg=14.40, stdev= 7.73, samples=52 lat (msec) : 4=1.82%, 10=13.54%, 20=14.06%, 50=48.70%, 100=19.01% lat (msec) : 250=2.86% cpu : usr=0.08%, sys=12.34%, ctx=1885, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159335: Sun Nov 23 01:03:42 2025 read: IOPS=14, BW=236KiB/s (242kB/s)(6256KiB/26506msec) clat (msec): min=3, max=196, avg=24.61, stdev=25.19 lat (msec): min=3, max=196, avg=24.61, stdev=25.19 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 14], 50.00th=[ 18], 60.00th=[ 22], | 70.00th=[ 27], 80.00th=[ 36], 90.00th=[ 56], 95.00th=[ 74], | 99.00th=[ 120], 99.50th=[ 190], 99.90th=[ 197], 99.95th=[ 197], | 99.99th=[ 197] bw ( KiB/s): min= 63, max= 768, per=26.18%, avg=235.33, stdev=144.49, samples=52 iops : min= 3, max= 48, avg=14.15, stdev= 9.22, samples=52 write: IOPS=14, BW=228KiB/s (233kB/s)(6032KiB/26506msec); 0 zone resets clat (msec): min=17, max=192, avg=44.60, stdev=23.78 lat (msec): min=17, max=192, avg=44.60, stdev=23.78 clat percentiles (msec): | 1.00th=[ 19], 5.00th=[ 22], 10.00th=[ 23], 20.00th=[ 27], | 30.00th=[ 29], 40.00th=[ 33], 50.00th=[ 37], 60.00th=[ 44], | 70.00th=[ 52], 80.00th=[ 61], 90.00th=[ 77], 95.00th=[ 94], | 99.00th=[ 131], 99.50th=[ 159], 99.90th=[ 192], 99.95th=[ 192], | 99.99th=[ 192] bw ( KiB/s): min= 63, max= 544, per=23.62%, avg=223.48, stdev=116.68, samples=52 iops : min= 3, max= 34, avg=13.40, stdev= 7.56, samples=52 lat (msec) : 4=1.82%, 10=15.36%, 20=11.85%, 50=48.83%, 100=19.79% lat (msec) : 250=2.34% cpu : usr=0.06%, sys=12.27%, ctx=1899, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159336: Sun Nov 23 01:03:42 2025 read: IOPS=14, BW=229KiB/s (234kB/s)(6032KiB/26390msec) clat (msec): min=3, max=152, avg=24.38, stdev=22.53 lat (msec): min=3, max=152, avg=24.38, stdev=22.53 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 7], | 30.00th=[ 10], 40.00th=[ 16], 50.00th=[ 19], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 37], 90.00th=[ 57], 95.00th=[ 69], | 99.00th=[ 97], 99.50th=[ 144], 99.90th=[ 153], 99.95th=[ 153], | 99.99th=[ 153] bw ( KiB/s): min= 31, max= 608, per=25.09%, avg=225.56, stdev=125.81, samples=52 iops : min= 1, max= 38, avg=13.54, stdev= 8.15, samples=52 write: IOPS=14, BW=237KiB/s (243kB/s)(6256KiB/26390msec); 0 zone resets clat (msec): min=17, max=192, avg=43.85, stdev=23.73 lat (msec): min=17, max=192, avg=43.86, stdev=23.73 clat percentiles (msec): | 1.00th=[ 21], 5.00th=[ 23], 10.00th=[ 24], 20.00th=[ 28], | 30.00th=[ 30], 40.00th=[ 33], 50.00th=[ 36], 60.00th=[ 41], | 70.00th=[ 48], 80.00th=[ 57], 90.00th=[ 73], 95.00th=[ 93], | 99.00th=[ 138], 99.50th=[ 188], 99.90th=[ 192], 99.95th=[ 192], | 99.99th=[ 192] bw ( KiB/s): min= 95, max= 544, per=24.87%, avg=235.31, stdev=113.88, samples=52 iops : min= 5, max= 34, avg=14.15, stdev= 7.34, samples=52 lat (msec) : 4=2.21%, 10=13.67%, 20=11.07%, 50=52.47%, 100=18.62% lat (msec) : 250=1.95% cpu : usr=0.05%, sys=12.17%, ctx=1841, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=900KiB/s (921kB/s), 216KiB/s-236KiB/s (221kB/s-242kB/s), io=23.4MiB (24.5MB), run=26390-26621msec WRITE: bw=947KiB/s (969kB/s), 228KiB/s-246KiB/s (233kB/s-252kB/s), io=24.6MiB (25.8MB), run=26390-26621msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159356: Sun Nov 23 01:03:54 2025 read: IOPS=1, BW=1734KiB/s (1776kB/s)(7168KiB/4133msec) clat (usec): min=1099, max=356869, avg=233164.79, stdev=111052.14 lat (usec): min=1101, max=356870, avg=233166.04, stdev=111051.99 clat percentiles (usec): | 1.00th=[ 1106], 5.00th=[ 1106], 10.00th=[ 1106], 20.00th=[223347], | 30.00th=[242222], 40.00th=[242222], 50.00th=[254804], 60.00th=[267387], | 70.00th=[267387], 80.00th=[287310], 90.00th=[358613], 95.00th=[358613], | 99.00th=[358613], 99.50th=[358613], 99.90th=[358613], 99.95th=[358613], | 99.99th=[358613] bw ( KiB/s): min= 1633, max= 4096, per=60.96%, avg=2453.00, stdev=1112.14, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 write: IOPS=1, BW=1239KiB/s (1269kB/s)(5120KiB/4133msec); 0 zone resets clat (msec): min=297, max=717, avg=473.63, stdev=152.91 lat (msec): min=298, max=718, avg=473.70, stdev=152.91 clat percentiles (msec): | 1.00th=[ 300], 5.00th=[ 300], 10.00th=[ 300], 20.00th=[ 300], | 30.00th=[ 435], 40.00th=[ 435], 50.00th=[ 439], 60.00th=[ 439], | 70.00th=[ 481], 80.00th=[ 481], 90.00th=[ 718], 95.00th=[ 718], | 99.00th=[ 718], 99.50th=[ 718], 99.90th=[ 718], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 2007, max= 2048, per=36.07%, avg=2032.25, stdev=17.69, samples=4 iops : min= 1, max= 2, avg= 1.25, stdev= 0.50, samples=4 lat (msec) : 2=8.33%, 250=16.67%, 500=66.67%, 750=8.33% cpu : usr=0.00%, sys=50.65%, ctx=1362, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159357: Sun Nov 23 01:03:54 2025 read: IOPS=0, BW=402KiB/s (412kB/s)(2048KiB/5089msec) clat (usec): min=1588, max=337370, avg=169479.82, stdev=237433.65 lat (usec): min=1590, max=337372, avg=169481.44, stdev=237433.98 clat percentiles (usec): | 1.00th=[ 1582], 5.00th=[ 1582], 10.00th=[ 1582], 20.00th=[ 1582], | 30.00th=[ 1582], 40.00th=[ 1582], 50.00th=[ 1582], 60.00th=[337642], | 70.00th=[337642], 80.00th=[337642], 90.00th=[337642], 95.00th=[337642], | 99.00th=[337642], 99.50th=[337642], 99.90th=[337642], 99.95th=[337642], | 99.99th=[337642] bw ( KiB/s): min= 1627, max= 2035, per=45.50%, avg=1831.00, stdev=288.50, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=2012KiB/s (2060kB/s)(10.0MiB/5089msec); 0 zone resets clat (msec): min=170, max=713, avg=462.36, stdev=183.63 lat (msec): min=171, max=713, avg=462.44, stdev=183.63 clat percentiles (msec): | 1.00th=[ 171], 5.00th=[ 171], 10.00th=[ 171], 20.00th=[ 313], | 30.00th=[ 326], 40.00th=[ 326], 50.00th=[ 372], 60.00th=[ 567], | 70.00th=[ 584], 80.00th=[ 584], 90.00th=[ 684], 95.00th=[ 718], | 99.00th=[ 718], 99.50th=[ 718], 99.90th=[ 718], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 2035, max= 3778, per=40.65%, avg=2290.14, stdev=656.11, samples=7 iops : min= 1, max= 3, avg= 1.71, stdev= 0.76, samples=7 lat (msec) : 2=8.33%, 250=8.33%, 500=41.67%, 750=41.67% cpu : usr=0.02%, sys=47.17%, ctx=416, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159358: Sun Nov 23 01:03:54 2025 read: IOPS=1, BW=1538KiB/s (1575kB/s)(6144KiB/3994msec) clat (usec): min=711, max=411574, avg=122436.10, stdev=190264.76 lat (usec): min=712, max=411576, avg=122437.20, stdev=190264.81 clat percentiles (usec): | 1.00th=[ 709], 5.00th=[ 709], 10.00th=[ 709], 20.00th=[ 1123], | 30.00th=[ 1123], 40.00th=[ 1188], 50.00th=[ 1188], 60.00th=[ 1369], | 70.00th=[316670], 80.00th=[316670], 90.00th=[413139], 95.00th=[413139], | 99.00th=[413139], 99.50th=[413139], 99.90th=[413139], 99.95th=[413139], | 99.99th=[413139] bw ( KiB/s): min= 2007, max= 4071, per=63.07%, avg=2538.00, stdev=1022.10, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=1, BW=1538KiB/s (1575kB/s)(6144KiB/3994msec); 0 zone resets clat (msec): min=119, max=1444, avg=541.78, stdev=468.03 lat (msec): min=119, max=1444, avg=541.83, stdev=468.03 clat percentiles (msec): | 1.00th=[ 121], 5.00th=[ 121], 10.00th=[ 121], 20.00th=[ 279], | 30.00th=[ 279], 40.00th=[ 368], 50.00th=[ 368], 60.00th=[ 489], | 70.00th=[ 550], 80.00th=[ 550], 90.00th=[ 1452], 95.00th=[ 1452], | 99.00th=[ 1452], 99.50th=[ 1452], 99.90th=[ 1452], 99.95th=[ 1452], | 99.99th=[ 1452] bw ( KiB/s): min= 2007, max= 3230, per=41.32%, avg=2327.75, stdev=601.67, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (usec) : 750=8.33% lat (msec) : 2=25.00%, 250=8.33%, 500=41.67%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=41.70%, ctx=245, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159359: Sun Nov 23 01:03:54 2025 read: IOPS=1, BW=1171KiB/s (1199kB/s)(5120KiB/4374msec) clat (usec): min=864, max=376004, avg=120355.33, stdev=172064.36 lat (usec): min=864, max=376005, avg=120356.55, stdev=172064.54 clat percentiles (usec): | 1.00th=[ 865], 5.00th=[ 865], 10.00th=[ 865], 20.00th=[ 865], | 30.00th=[ 1172], 40.00th=[ 1172], 50.00th=[ 1319], 60.00th=[ 1319], | 70.00th=[223347], 80.00th=[223347], 90.00th=[375391], 95.00th=[375391], | 99.00th=[375391], 99.50th=[375391], 99.90th=[375391], 99.95th=[375391], | 99.99th=[375391] bw ( KiB/s): min= 2039, max= 6107, per=100.00%, avg=4073.00, stdev=2876.51, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=1, BW=1639KiB/s (1678kB/s)(7168KiB/4374msec); 0 zone resets clat (msec): min=247, max=1171, avg=533.13, stdev=383.29 lat (msec): min=247, max=1171, avg=533.19, stdev=383.27 clat percentiles (msec): | 1.00th=[ 247], 5.00th=[ 247], 10.00th=[ 247], 20.00th=[ 264], | 30.00th=[ 330], 40.00th=[ 330], 50.00th=[ 334], 60.00th=[ 388], | 70.00th=[ 388], 80.00th=[ 1003], 90.00th=[ 1167], 95.00th=[ 1167], | 99.00th=[ 1167], 99.50th=[ 1167], 99.90th=[ 1167], 99.95th=[ 1167], | 99.99th=[ 1167] bw ( KiB/s): min= 1638, max= 4096, per=52.60%, avg=2963.25, stdev=1304.38, samples=4 iops : min= 1, max= 4, avg= 2.50, stdev= 1.29, samples=4 lat (usec) : 1000=8.33% lat (msec) : 2=16.67%, 250=16.67%, 500=41.67%, 1000=8.33%, 2000=8.33% cpu : usr=0.00%, sys=41.46%, ctx=305, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4024KiB/s (4121kB/s), 402KiB/s-1734KiB/s (412kB/s-1776kB/s), io=20.0MiB (20.0MB), run=3994-5089msec WRITE: bw=5634KiB/s (5769kB/s), 1239KiB/s-2012KiB/s (1269kB/s-2060kB/s), io=28.0MiB (29.4MB), run=3994-5089msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159361: Sun Nov 23 01:03:55 2025 read: IOPS=2, BW=2156KiB/s (2208kB/s)(7168KiB/3325msec) clat (msec): min=114, max=369, avg=193.72, stdev=98.39 lat (msec): min=114, max=369, avg=193.72, stdev=98.39 clat percentiles (msec): | 1.00th=[ 114], 5.00th=[ 114], 10.00th=[ 114], 20.00th=[ 129], | 30.00th=[ 129], 40.00th=[ 129], 50.00th=[ 130], 60.00th=[ 203], | 70.00th=[ 203], 80.00th=[ 284], 90.00th=[ 372], 95.00th=[ 372], | 99.00th=[ 372], 99.50th=[ 372], 99.90th=[ 372], 99.95th=[ 372], | 99.99th=[ 372] bw ( KiB/s): min= 1954, max= 4079, per=74.82%, avg=2847.75, stdev=1066.82, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 write: IOPS=1, BW=1540KiB/s (1577kB/s)(5120KiB/3325msec); 0 zone resets clat (msec): min=114, max=760, avg=374.93, stdev=323.57 lat (msec): min=114, max=760, avg=375.02, stdev=323.58 clat percentiles (msec): | 1.00th=[ 114], 5.00th=[ 114], 10.00th=[ 114], 20.00th=[ 114], | 30.00th=[ 123], 40.00th=[ 123], 50.00th=[ 184], 60.00th=[ 184], | 70.00th=[ 693], 80.00th=[ 693], 90.00th=[ 760], 95.00th=[ 760], | 99.00th=[ 760], 99.50th=[ 760], 99.90th=[ 760], 99.95th=[ 760], | 99.99th=[ 760] bw ( KiB/s): min= 1939, max= 2039, per=36.90%, avg=1966.40, stdev=41.26, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (msec) : 250=66.67%, 500=16.67%, 750=8.33%, 1000=8.33% cpu : usr=0.03%, sys=20.70%, ctx=247, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159362: Sun Nov 23 01:03:55 2025 read: IOPS=0, BW=387KiB/s (397kB/s)(2048KiB/5286msec) clat (msec): min=302, max=606, avg=454.41, stdev=215.42 lat (msec): min=302, max=606, avg=454.41, stdev=215.42 clat percentiles (msec): | 1.00th=[ 305], 5.00th=[ 305], 10.00th=[ 305], 20.00th=[ 305], | 30.00th=[ 305], 40.00th=[ 305], 50.00th=[ 305], 60.00th=[ 609], | 70.00th=[ 609], 80.00th=[ 609], 90.00th=[ 609], 95.00th=[ 609], | 99.00th=[ 609], 99.50th=[ 609], 99.90th=[ 609], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 2039, max= 2048, per=53.69%, avg=2043.50, stdev= 6.36, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1937KiB/s (1984kB/s)(10.0MiB/5286msec); 0 zone resets clat (msec): min=93, max=1239, avg=420.02, stdev=386.02 lat (msec): min=93, max=1239, avg=420.08, stdev=386.01 clat percentiles (msec): | 1.00th=[ 93], 5.00th=[ 93], 10.00th=[ 93], 20.00th=[ 107], | 30.00th=[ 153], 40.00th=[ 253], 50.00th=[ 259], 60.00th=[ 268], | 70.00th=[ 351], 80.00th=[ 510], 90.00th=[ 969], 95.00th=[ 1234], | 99.00th=[ 1234], 99.50th=[ 1234], 99.90th=[ 1234], 99.95th=[ 1234], | 99.99th=[ 1234] bw ( KiB/s): min= 1939, max= 6144, per=56.43%, avg=3007.33, stdev=1716.69, samples=6 iops : min= 1, max= 6, avg= 2.50, stdev= 1.87, samples=6 lat (msec) : 100=8.33%, 250=16.67%, 500=41.67%, 750=16.67%, 1000=8.33% lat (msec) : 2000=8.33% cpu : usr=0.02%, sys=19.53%, ctx=228, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159363: Sun Nov 23 01:03:55 2025 read: IOPS=1, BW=1254KiB/s (1284kB/s)(6144KiB/4900msec) clat (msec): min=158, max=270, avg=213.77, stdev=43.93 lat (msec): min=158, max=270, avg=213.77, stdev=43.93 clat percentiles (msec): | 1.00th=[ 159], 5.00th=[ 159], 10.00th=[ 159], 20.00th=[ 165], | 30.00th=[ 165], 40.00th=[ 226], 50.00th=[ 226], 60.00th=[ 230], | 70.00th=[ 234], 80.00th=[ 234], 90.00th=[ 271], 95.00th=[ 271], | 99.00th=[ 271], 99.50th=[ 271], 99.90th=[ 271], 99.95th=[ 271], | 99.99th=[ 271] bw ( KiB/s): min= 2035, max= 3878, per=65.74%, avg=2502.25, stdev=917.19, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 write: IOPS=1, BW=1254KiB/s (1284kB/s)(6144KiB/4900msec); 0 zone resets clat (msec): min=123, max=1635, avg=589.00, stdev=551.04 lat (msec): min=123, max=1635, avg=589.07, stdev=551.05 clat percentiles (msec): | 1.00th=[ 124], 5.00th=[ 124], 10.00th=[ 124], 20.00th=[ 288], | 30.00th=[ 288], 40.00th=[ 317], 50.00th=[ 317], 60.00th=[ 443], | 70.00th=[ 735], 80.00th=[ 735], 90.00th=[ 1636], 95.00th=[ 1636], | 99.00th=[ 1636], 99.50th=[ 1636], 99.90th=[ 1636], 99.95th=[ 1636], | 99.99th=[ 1636] bw ( KiB/s): min= 1954, max= 4079, per=45.65%, avg=2432.80, stdev=921.09, samples=5 iops : min= 1, max= 3, avg= 1.80, stdev= 0.84, samples=5 lat (msec) : 250=50.00%, 500=33.33%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=17.98%, ctx=226, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159364: Sun Nov 23 01:03:55 2025 read: IOPS=0, BW=952KiB/s (975kB/s)(5120KiB/5380msec) clat (msec): min=24, max=1032, avg=383.33, stdev=381.44 lat (msec): min=24, max=1032, avg=383.33, stdev=381.44 clat percentiles (msec): | 1.00th=[ 25], 5.00th=[ 25], 10.00th=[ 25], 20.00th=[ 25], | 30.00th=[ 243], 40.00th=[ 243], 50.00th=[ 292], 60.00th=[ 292], | 70.00th=[ 326], 80.00th=[ 326], 90.00th=[ 1036], 95.00th=[ 1036], | 99.00th=[ 1036], 99.50th=[ 1036], 99.90th=[ 1036], 99.95th=[ 1036], | 99.99th=[ 1036] bw ( KiB/s): min= 2035, max= 4096, per=71.63%, avg=2726.33, stdev=1186.18, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 write: IOPS=1, BW=1332KiB/s (1364kB/s)(7168KiB/5380msec); 0 zone resets clat (msec): min=36, max=1495, avg=484.62, stdev=518.68 lat (msec): min=36, max=1495, avg=484.70, stdev=518.69 clat percentiles (msec): | 1.00th=[ 37], 5.00th=[ 37], 10.00th=[ 37], 20.00th=[ 55], | 30.00th=[ 112], 40.00th=[ 112], 50.00th=[ 388], 60.00th=[ 609], | 70.00th=[ 609], 80.00th=[ 693], 90.00th=[ 1502], 95.00th=[ 1502], | 99.00th=[ 1502], 99.50th=[ 1502], 99.90th=[ 1502], 99.95th=[ 1502], | 99.99th=[ 1502] bw ( KiB/s): min= 2035, max= 3908, per=49.93%, avg=2660.67, stdev=1080.22, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 50=16.67%, 100=8.33%, 250=16.67%, 500=25.00%, 750=16.67% lat (msec) : 2000=16.67% cpu : usr=0.00%, sys=13.87%, ctx=166, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3807KiB/s (3898kB/s), 387KiB/s-2156KiB/s (397kB/s-2208kB/s), io=20.0MiB (20.0MB), run=3325-5380msec WRITE: bw=5329KiB/s (5457kB/s), 1254KiB/s-1937KiB/s (1284kB/s-1984kB/s), io=28.0MiB (29.4MB), run=3325-5380msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=159371: Sun Nov 23 01:04:04 2025 read: IOPS=1, BW=5769KiB/s (5907kB/s)(12.0MiB/2130msec) clat (msec): min=405, max=1254, avg=704.39, stdev=476.70 lat (msec): min=405, max=1254, avg=704.39, stdev=476.70 clat percentiles (msec): | 1.00th=[ 405], 5.00th=[ 405], 10.00th=[ 405], 20.00th=[ 405], | 30.00th=[ 405], 40.00th=[ 456], 50.00th=[ 456], 60.00th=[ 456], | 70.00th=[ 1250], 80.00th=[ 1250], 90.00th=[ 1250], 95.00th=[ 1250], | 99.00th=[ 1250], 99.50th=[ 1250], 99.90th=[ 1250], 99.95th=[ 1250], | 99.99th=[ 1250] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=47.11%, ctx=2945, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159372: Sun Nov 23 01:04:04 2025 read: IOPS=0, BW=1683KiB/s (1723kB/s)(4096KiB/2434msec) clat (nsec): min=468847k, max=468847k, avg=468846730.00, stdev= 0.00 lat (nsec): min=468850k, max=468850k, avg=468850010.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 468], 5.00th=[ 468], 10.00th=[ 468], 20.00th=[ 468], | 30.00th=[ 468], 40.00th=[ 468], 50.00th=[ 468], 60.00th=[ 468], | 70.00th=[ 468], 80.00th=[ 468], 90.00th=[ 468], 95.00th=[ 468], | 99.00th=[ 468], 99.50th=[ 468], 99.90th=[ 468], 99.95th=[ 468], | 99.99th=[ 468] bw ( KiB/s): min= 7380, max= 7380, per=100.00%, avg=7380.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3366KiB/s (3446kB/s)(8192KiB/2434msec); 0 zone resets clat (msec): min=815, max=1102, avg=959.31, stdev=203.09 lat (msec): min=815, max=1103, avg=959.50, stdev=203.00 clat percentiles (msec): | 1.00th=[ 818], 5.00th=[ 818], 10.00th=[ 818], 20.00th=[ 818], | 30.00th=[ 818], 40.00th=[ 818], 50.00th=[ 818], 60.00th=[ 1099], | 70.00th=[ 1099], 80.00th=[ 1099], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 8192, max= 8192, per=67.88%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=20.47%, ctx=1011, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159373: Sun Nov 23 01:04:04 2025 write: IOPS=1, BW=4526KiB/s (4635kB/s)(12.0MiB/2715msec); 0 zone resets clat (msec): min=635, max=1237, avg=892.31, stdev=310.78 lat (msec): min=635, max=1238, avg=892.64, stdev=310.85 clat percentiles (msec): | 1.00th=[ 634], 5.00th=[ 634], 10.00th=[ 634], 20.00th=[ 634], | 30.00th=[ 634], 40.00th=[ 802], 50.00th=[ 802], 60.00th=[ 802], | 70.00th=[ 1234], 80.00th=[ 1234], 90.00th=[ 1234], 95.00th=[ 1234], | 99.00th=[ 1234], 99.50th=[ 1234], 99.90th=[ 1234], 99.95th=[ 1234], | 99.99th=[ 1234] bw ( KiB/s): min= 7937, max= 8192, per=66.82%, avg=8064.50, stdev=180.31, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.04%, sys=17.17%, ctx=170, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159374: Sun Nov 23 01:04:04 2025 write: IOPS=1, BW=4939KiB/s (5057kB/s)(12.0MiB/2488msec); 0 zone resets clat (msec): min=326, max=1229, avg=813.42, stdev=455.30 lat (msec): min=327, max=1229, avg=813.72, stdev=455.22 clat percentiles (msec): | 1.00th=[ 326], 5.00th=[ 326], 10.00th=[ 326], 20.00th=[ 326], | 30.00th=[ 326], 40.00th=[ 885], 50.00th=[ 885], 60.00th=[ 885], | 70.00th=[ 1234], 80.00th=[ 1234], 90.00th=[ 1234], 95.00th=[ 1234], | 99.00th=[ 1234], 99.50th=[ 1234], 99.90th=[ 1234], 99.95th=[ 1234], | 99.99th=[ 1234] bw ( KiB/s): min= 7393, max= 8192, per=64.57%, avg=7792.50, stdev=564.98, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=14.96%, ctx=88, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6731KiB/s (6893kB/s), 1683KiB/s-5769KiB/s (1723kB/s-5907kB/s), io=16.0MiB (16.8MB), run=2130-2434msec WRITE: bw=11.8MiB/s (12.4MB/s), 3366KiB/s-4939KiB/s (3446kB/s-5057kB/s), io=32.0MiB (33.6MB), run=2434-2715msec rand-rw: (groupid=0, jobs=1): err= 0: pid=159375: Sun Nov 23 01:04:04 2025 read: IOPS=1, BW=4230KiB/s (4331kB/s)(12.0MiB/2905msec) clat (msec): min=144, max=2103, avg=963.43, stdev=1018.19 lat (msec): min=144, max=2103, avg=963.44, stdev=1018.19 clat percentiles (msec): | 1.00th=[ 144], 5.00th=[ 144], 10.00th=[ 144], 20.00th=[ 144], | 30.00th=[ 144], 40.00th=[ 642], 50.00th=[ 642], 60.00th=[ 642], | 70.00th=[ 2106], 80.00th=[ 2106], 90.00th=[ 2106], 95.00th=[ 2106], | 99.00th=[ 2106], 99.50th=[ 2106], 99.90th=[ 2106], 99.95th=[ 2106], | 99.99th=[ 2106] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=9.13%, ctx=89, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159376: Sun Nov 23 01:04:04 2025 read: IOPS=0, BW=1417KiB/s (1451kB/s)(4096KiB/2890msec) clat (nsec): min=1778.8M, max=1778.8M, avg=1778797171.00, stdev= 0.00 lat (nsec): min=1778.8M, max=1778.8M, avg=1778798976.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1787], 5.00th=[ 1787], 10.00th=[ 1787], 20.00th=[ 1787], | 30.00th=[ 1787], 40.00th=[ 1787], 50.00th=[ 1787], 60.00th=[ 1787], | 70.00th=[ 1787], 80.00th=[ 1787], 90.00th=[ 1787], 95.00th=[ 1787], | 99.00th=[ 1787], 99.50th=[ 1787], 99.90th=[ 1787], 99.95th=[ 1787], | 99.99th=[ 1787] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2835KiB/s (2903kB/s)(8192KiB/2890msec); 0 zone resets clat (msec): min=421, max=675, avg=548.24, stdev=179.47 lat (msec): min=421, max=675, avg=548.51, stdev=179.45 clat percentiles (msec): | 1.00th=[ 422], 5.00th=[ 422], 10.00th=[ 422], 20.00th=[ 422], | 30.00th=[ 422], 40.00th=[ 422], 50.00th=[ 422], 60.00th=[ 676], | 70.00th=[ 676], 80.00th=[ 676], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 8159, max= 8159, per=71.96%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=11.70%, ctx=78, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159377: Sun Nov 23 01:04:04 2025 write: IOPS=1, BW=4392KiB/s (4497kB/s)(12.0MiB/2798msec); 0 zone resets clat (msec): min=162, max=1541, avg=929.24, stdev=702.61 lat (msec): min=162, max=1542, avg=929.50, stdev=702.61 clat percentiles (msec): | 1.00th=[ 163], 5.00th=[ 163], 10.00th=[ 163], 20.00th=[ 163], | 30.00th=[ 163], 40.00th=[ 1083], 50.00th=[ 1083], 60.00th=[ 1083], | 70.00th=[ 1536], 80.00th=[ 1536], 90.00th=[ 1536], 95.00th=[ 1536], | 99.00th=[ 1536], 99.50th=[ 1536], 99.90th=[ 1536], 99.95th=[ 1536], | 99.99th=[ 1536] bw ( KiB/s): min= 8192, max= 8192, per=72.25%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=66.67% cpu : usr=0.00%, sys=12.08%, ctx=61, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=159378: Sun Nov 23 01:04:04 2025 write: IOPS=1, BW=4623KiB/s (4734kB/s)(12.0MiB/2658msec); 0 zone resets clat (msec): min=497, max=1099, avg=867.08, stdev=323.52 lat (msec): min=497, max=1099, avg=867.35, stdev=323.53 clat percentiles (msec): | 1.00th=[ 498], 5.00th=[ 498], 10.00th=[ 498], 20.00th=[ 498], | 30.00th=[ 498], 40.00th=[ 1003], 50.00th=[ 1003], 60.00th=[ 1003], | 70.00th=[ 1099], 80.00th=[ 1099], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 8159, max= 8192, per=72.11%, avg=8175.50, stdev=23.33, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.04%, sys=13.32%, ctx=79, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5640KiB/s (5775kB/s), 1417KiB/s-4230KiB/s (1451kB/s-4331kB/s), io=16.0MiB (16.8MB), run=2890-2905msec WRITE: bw=11.1MiB/s (11.6MB/s), 2835KiB/s-4623KiB/s (2903kB/s-4734kB/s), io=32.0MiB (33.6MB), run=2658-2890msec