== sanity test 398b: DIO and buffer IO race ============== 10:30:24 (1758810624) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.90072 s, 17.4 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189334: Thu Sep 25 10:31:31 2025 read: IOPS=26, BW=106KiB/s (109kB/s)(5992KiB/56428msec) clat (usec): min=193, max=749008, avg=20831.43, stdev=22226.88 lat (usec): min=194, max=749011, avg=20832.60, stdev=22226.94 clat percentiles (usec): | 1.00th=[ 338], 5.00th=[ 7832], 10.00th=[ 9110], 20.00th=[ 11076], | 30.00th=[ 13435], 40.00th=[ 15270], 50.00th=[ 18220], 60.00th=[ 20841], | 70.00th=[ 23462], 80.00th=[ 27395], 90.00th=[ 34341], 95.00th=[ 42206], | 99.00th=[ 66323], 99.50th=[ 73925], 99.90th=[ 96994], 99.95th=[750781], | 99.99th=[750781] bw ( KiB/s): min= 13, max= 184, per=25.27%, avg=106.12, stdev=26.86, samples=112 iops : min= 3, max= 46, avg=26.18, stdev= 6.68, samples=112 write: IOPS=27, BW=112KiB/s (114kB/s)(6296KiB/56428msec); 0 zone resets clat (usec): min=869, max=69145, avg=15886.82, stdev=12859.69 lat (usec): min=870, max=69146, avg=15888.40, stdev=12859.75 clat percentiles (usec): | 1.00th=[ 1172], 5.00th=[ 2057], 10.00th=[ 2671], 20.00th=[ 3490], | 30.00th=[ 6194], 40.00th=[10421], 50.00th=[13042], 60.00th=[16188], | 70.00th=[20579], 80.00th=[25822], 90.00th=[33817], 95.00th=[41681], | 99.00th=[55313], 99.50th=[58983], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 31, max= 192, per=25.55%, avg=112.44, stdev=32.42, samples=111 iops : min= 7, max= 48, avg=27.76, stdev= 8.16, samples=111 lat (usec) : 250=0.26%, 500=0.52%, 750=0.16%, 1000=0.29% lat (msec) : 2=2.38%, 4=9.38%, 10=13.70%, 20=36.62%, 50=34.34% lat (msec) : 100=2.31%, 750=0.03% cpu : usr=0.16%, sys=22.48%, ctx=9005, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189335: Thu Sep 25 10:31:31 2025 read: IOPS=26, BW=104KiB/s (107kB/s)(5940KiB/57107msec) clat (usec): min=136, max=728355, avg=20815.80, stdev=21451.37 lat (usec): min=136, max=728355, avg=20818.77, stdev=21451.32 clat percentiles (usec): | 1.00th=[ 363], 5.00th=[ 8029], 10.00th=[ 9634], 20.00th=[ 11600], | 30.00th=[ 13698], 40.00th=[ 16057], 50.00th=[ 18482], 60.00th=[ 20841], | 70.00th=[ 23462], 80.00th=[ 27395], 90.00th=[ 33817], 95.00th=[ 40633], | 99.00th=[ 60031], 99.50th=[ 72877], 99.90th=[ 90702], 99.95th=[725615], | 99.99th=[725615] bw ( KiB/s): min= 16, max= 175, per=24.92%, avg=104.65, stdev=26.11, samples=112 iops : min= 4, max= 43, avg=25.81, stdev= 6.56, samples=112 write: IOPS=27, BW=111KiB/s (114kB/s)(6348KiB/57107msec); 0 zone resets clat (usec): min=843, max=96132, avg=16341.82, stdev=13895.03 lat (usec): min=844, max=96133, avg=16344.07, stdev=13894.53 clat percentiles (usec): | 1.00th=[ 1254], 5.00th=[ 2212], 10.00th=[ 2704], 20.00th=[ 3458], | 30.00th=[ 5604], 40.00th=[ 9503], 50.00th=[12518], 60.00th=[16450], | 70.00th=[20841], 80.00th=[27395], 90.00th=[35914], 95.00th=[43779], | 99.00th=[58459], 99.50th=[65799], 99.90th=[82314], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 39, max= 224, per=25.34%, avg=111.51, stdev=33.56, samples=112 iops : min= 9, max= 56, avg=27.53, stdev= 8.40, samples=112 lat (usec) : 250=0.10%, 500=0.72%, 750=0.16%, 1000=0.16% lat (msec) : 2=1.82%, 4=10.29%, 10=13.93%, 20=35.22%, 50=35.25% lat (msec) : 100=2.31%, 750=0.03% cpu : usr=0.14%, sys=22.44%, ctx=9049, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189336: Thu Sep 25 10:31:31 2025 read: IOPS=26, BW=108KiB/s (111kB/s)(6028KiB/55842msec) clat (usec): min=158, max=97413, avg=19842.25, stdev=11251.15 lat (usec): min=160, max=97414, avg=19844.42, stdev=11250.62 clat percentiles (usec): | 1.00th=[ 338], 5.00th=[ 7635], 10.00th=[ 9110], 20.00th=[11076], | 30.00th=[12911], 40.00th=[15008], 50.00th=[17433], 60.00th=[20579], | 70.00th=[23200], 80.00th=[27132], 90.00th=[34341], 95.00th=[41157], | 99.00th=[57410], 99.50th=[64226], 99.90th=[88605], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 13, max= 176, per=25.65%, avg=107.75, stdev=26.41, samples=111 iops : min= 3, max= 44, avg=26.59, stdev= 6.62, samples=111 write: IOPS=28, BW=112KiB/s (115kB/s)(6260KiB/55842msec); 0 zone resets clat (usec): min=801, max=737129, avg=16440.65, stdev=23127.19 lat (usec): min=803, max=737131, avg=16442.72, stdev=23127.48 clat percentiles (usec): | 1.00th=[ 1221], 5.00th=[ 2147], 10.00th=[ 2573], 20.00th=[ 3359], | 30.00th=[ 5211], 40.00th=[ 9372], 50.00th=[ 12649], 60.00th=[ 15926], | 70.00th=[ 20317], 80.00th=[ 26346], 90.00th=[ 34866], 95.00th=[ 42730], | 99.00th=[ 63701], 99.50th=[ 73925], 99.90th=[160433], 99.95th=[734004], | 99.99th=[734004] bw ( KiB/s): min= 20, max= 232, per=25.42%, avg=111.83, stdev=32.91, samples=111 iops : min= 5, max= 58, avg=27.61, stdev= 8.18, samples=111 lat (usec) : 250=0.23%, 500=0.85%, 750=0.23%, 1000=0.29% lat (msec) : 2=2.05%, 4=10.55%, 10=14.00%, 20=35.64%, 50=33.95% lat (msec) : 100=2.15%, 250=0.03%, 750=0.03% cpu : usr=0.17%, sys=22.33%, ctx=8868, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189337: Thu Sep 25 10:31:31 2025 read: IOPS=27, BW=108KiB/s (111kB/s)(6052KiB/55912msec) clat (usec): min=163, max=109990, avg=19959.05, stdev=11324.13 lat (usec): min=164, max=109990, avg=19962.52, stdev=11327.52 clat percentiles (usec): | 1.00th=[ 297], 5.00th=[ 7635], 10.00th=[ 9241], 20.00th=[ 11207], | 30.00th=[ 13304], 40.00th=[ 15401], 50.00th=[ 17957], 60.00th=[ 20579], | 70.00th=[ 23725], 80.00th=[ 27395], 90.00th=[ 32900], 95.00th=[ 40633], | 99.00th=[ 57934], 99.50th=[ 65799], 99.90th=[ 94897], 99.95th=[109577], | 99.99th=[109577] bw ( KiB/s): min= 32, max= 184, per=25.96%, avg=109.04, stdev=26.15, samples=110 iops : min= 8, max= 46, avg=26.91, stdev= 6.57, samples=110 write: IOPS=27, BW=112KiB/s (114kB/s)(6236KiB/55912msec); 0 zone resets clat (usec): min=684, max=744278, avg=16352.94, stdev=22866.37 lat (usec): min=685, max=744279, avg=16354.64, stdev=22866.45 clat percentiles (usec): | 1.00th=[ 1205], 5.00th=[ 2008], 10.00th=[ 2638], 20.00th=[ 3523], | 30.00th=[ 5604], 40.00th=[ 9765], 50.00th=[ 12911], 60.00th=[ 15926], | 70.00th=[ 20579], 80.00th=[ 26346], 90.00th=[ 34341], 95.00th=[ 42206], | 99.00th=[ 51643], 99.50th=[ 63701], 99.90th=[166724], 99.95th=[742392], | 99.99th=[742392] bw ( KiB/s): min= 32, max= 224, per=25.49%, avg=112.17, stdev=33.03, samples=110 iops : min= 8, max= 56, avg=27.69, stdev= 8.28, samples=110 lat (usec) : 250=0.36%, 500=1.11%, 750=0.20%, 1000=0.07% lat (msec) : 2=2.54%, 4=9.24%, 10=13.67%, 20=36.07%, 50=35.09% lat (msec) : 100=1.56%, 250=0.07%, 750=0.03% cpu : usr=0.20%, sys=22.47%, ctx=8774, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=420KiB/s (431kB/s), 104KiB/s-108KiB/s (107kB/s-111kB/s), io=23.4MiB (24.6MB), run=55842-57107msec WRITE: bw=440KiB/s (451kB/s), 111KiB/s-112KiB/s (114kB/s-115kB/s), io=24.6MiB (25.7MB), run=55842-57107msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189329: Thu Sep 25 10:31:45 2025 read: IOPS=21, BW=84.7KiB/s (86.8kB/s)(5992KiB/70707msec) clat (msec): min=3, max=758, avg=19.80, stdev=24.42 lat (msec): min=3, max=758, avg=19.80, stdev=24.42 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 22], 80.00th=[ 29], 90.00th=[ 42], 95.00th=[ 52], | 99.00th=[ 69], 99.50th=[ 90], 99.90th=[ 110], 99.95th=[ 760], | 99.99th=[ 760] bw ( KiB/s): min= 23, max= 270, per=24.98%, avg=84.69, stdev=49.77, samples=139 iops : min= 5, max= 67, avg=20.73, stdev=12.45, samples=139 write: IOPS=22, BW=89.0KiB/s (91.2kB/s)(6296KiB/70707msec); 0 zone resets clat (msec): min=6, max=111, avg=25.91, stdev=16.22 lat (msec): min=6, max=111, avg=25.91, stdev=16.22 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 26], | 70.00th=[ 31], 80.00th=[ 39], 90.00th=[ 48], 95.00th=[ 58], | 99.00th=[ 80], 99.50th=[ 89], 99.90th=[ 107], 99.95th=[ 111], | 99.99th=[ 111] bw ( KiB/s): min= 7, max= 232, per=25.04%, avg=88.90, stdev=47.51, samples=139 iops : min= 1, max= 58, avg=21.78, stdev=11.93, samples=139 lat (msec) : 4=0.03%, 10=18.62%, 20=38.64%, 50=35.51%, 100=6.90% lat (msec) : 250=0.26%, 1000=0.03% cpu : usr=0.15%, sys=9.69%, ctx=5729, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189330: Thu Sep 25 10:31:45 2025 read: IOPS=21, BW=84.0KiB/s (86.1kB/s)(5940KiB/70678msec) clat (msec): min=3, max=724, avg=19.24, stdev=23.41 lat (msec): min=3, max=724, avg=19.24, stdev=23.41 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 21], 80.00th=[ 29], 90.00th=[ 42], 95.00th=[ 51], | 99.00th=[ 69], 99.50th=[ 77], 99.90th=[ 110], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 15, max= 278, per=24.81%, avg=84.11, stdev=51.74, samples=139 iops : min= 3, max= 69, avg=20.59, stdev=12.98, samples=139 write: IOPS=22, BW=89.8KiB/s (91.0kB/s)(6348KiB/70678msec); 0 zone resets clat (msec): min=5, max=159, avg=26.36, stdev=16.83 lat (msec): min=5, max=159, avg=26.37, stdev=16.83 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 26], | 70.00th=[ 32], 80.00th=[ 39], 90.00th=[ 49], 95.00th=[ 59], | 99.00th=[ 84], 99.50th=[ 97], 99.90th=[ 107], 99.95th=[ 159], | 99.99th=[ 159] bw ( KiB/s): min= 32, max= 231, per=25.27%, avg=89.71, stdev=47.76, samples=139 iops : min= 8, max= 57, avg=21.99, stdev=11.96, samples=139 lat (msec) : 4=0.03%, 10=19.99%, 20=37.40%, 50=35.58%, 100=6.74% lat (msec) : 250=0.23%, 750=0.03% cpu : usr=0.14%, sys=9.57%, ctx=5703, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189331: Thu Sep 25 10:31:45 2025 read: IOPS=21, BW=85.7KiB/s (87.7kB/s)(6028KiB/70370msec) clat (msec): min=3, max=110, avg=19.21, stdev=15.46 lat (msec): min=3, max=110, avg=19.22, stdev=15.46 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 22], 80.00th=[ 28], 90.00th=[ 42], 95.00th=[ 54], | 99.00th=[ 73], 99.50th=[ 81], 99.90th=[ 102], 99.95th=[ 111], | 99.99th=[ 111] bw ( KiB/s): min= 16, max= 256, per=25.38%, avg=86.02, stdev=49.10, samples=139 iops : min= 4, max= 64, avg=21.06, stdev=12.29, samples=139 write: IOPS=22, BW=88.0KiB/s (91.1kB/s)(6260KiB/70370msec); 0 zone resets clat (msec): min=6, max=729, avg=26.30, stdev=24.35 lat (msec): min=6, max=729, avg=26.30, stdev=24.35 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 31], 80.00th=[ 38], 90.00th=[ 48], 95.00th=[ 59], | 99.00th=[ 83], 99.50th=[ 105], 99.90th=[ 159], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 24, max= 240, per=25.16%, avg=89.30, stdev=47.46, samples=139 iops : min= 6, max= 60, avg=21.88, stdev=11.90, samples=139 lat (msec) : 4=0.03%, 10=19.34%, 20=38.61%, 50=34.57%, 100=7.10% lat (msec) : 250=0.33%, 750=0.03% cpu : usr=0.15%, sys=9.58%, ctx=5728, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189332: Thu Sep 25 10:31:45 2025 read: IOPS=21, BW=85.5KiB/s (87.5kB/s)(6052KiB/70803msec) clat (msec): min=3, max=107, avg=19.07, stdev=15.10 lat (msec): min=3, max=107, avg=19.07, stdev=15.10 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 22], 80.00th=[ 30], 90.00th=[ 41], 95.00th=[ 52], | 99.00th=[ 69], 99.50th=[ 79], 99.90th=[ 97], 99.95th=[ 108], | 99.99th=[ 108] bw ( KiB/s): min= 8, max= 304, per=25.04%, avg=84.88, stdev=51.41, samples=139 iops : min= 2, max= 76, avg=20.78, stdev=12.86, samples=139 write: IOPS=22, BW=88.1KiB/s (90.2kB/s)(6236KiB/70803msec); 0 zone resets clat (msec): min=4, max=736, avg=26.74, stdev=24.77 lat (msec): min=4, max=736, avg=26.74, stdev=24.77 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 26], | 70.00th=[ 31], 80.00th=[ 39], 90.00th=[ 49], 95.00th=[ 59], | 99.00th=[ 91], 99.50th=[ 105], 99.90th=[ 140], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 24, max= 239, per=24.69%, avg=87.63, stdev=47.92, samples=139 iops : min= 6, max= 59, avg=21.47, stdev=12.00, samples=139 lat (msec) : 4=0.03%, 10=20.21%, 20=36.75%, 50=35.61%, 100=7.06% lat (msec) : 250=0.29%, 750=0.03% cpu : usr=0.14%, sys=9.54%, ctx=5699, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=339KiB/s (347kB/s), 84.0KiB/s-85.7KiB/s (86.1kB/s-87.7kB/s), io=23.4MiB (24.6MB), run=70370-70803msec WRITE: bw=355KiB/s (364kB/s), 88.1KiB/s-89.8KiB/s (90.2kB/s-91.0kB/s), io=24.6MiB (25.7MB), run=70370-70803msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189363: Thu Sep 25 10:32:07 2025 read: IOPS=23, BW=380KiB/s (389kB/s)(5920KiB/15598msec) clat (usec): min=152, max=83253, avg=21378.42, stdev=11945.74 lat (usec): min=154, max=83254, avg=21379.86, stdev=11945.74 clat percentiles (usec): | 1.00th=[ 265], 5.00th=[ 453], 10.00th=[ 9110], 20.00th=[13304], | 30.00th=[15008], 40.00th=[17171], 50.00th=[20055], 60.00th=[22414], | 70.00th=[25297], 80.00th=[28705], 90.00th=[36439], 95.00th=[41681], | 99.00th=[62653], 99.50th=[65274], 99.90th=[83362], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 224, max= 540, per=24.74%, avg=371.87, stdev=80.40, samples=30 iops : min= 14, max= 33, avg=22.60, stdev= 5.13, samples=30 write: IOPS=25, BW=408KiB/s (418kB/s)(6368KiB/15598msec); 0 zone resets clat (usec): min=1796, max=76200, avg=18969.55, stdev=13419.08 lat (usec): min=1798, max=76201, avg=18972.59, stdev=13419.35 clat percentiles (usec): | 1.00th=[ 2008], 5.00th=[ 3884], 10.00th=[ 4424], 20.00th=[ 6587], | 30.00th=[10421], 40.00th=[13173], 50.00th=[15926], 60.00th=[19792], | 70.00th=[23725], 80.00th=[28705], 90.00th=[37487], 95.00th=[45351], | 99.00th=[62129], 99.50th=[66323], 99.90th=[76022], 99.95th=[76022], | 99.99th=[76022] bw ( KiB/s): min= 191, max= 570, per=25.67%, avg=406.10, stdev=104.34, samples=30 iops : min= 11, max= 35, avg=24.77, stdev= 6.67, samples=30 lat (usec) : 250=0.39%, 500=2.34%, 750=0.65%, 1000=0.26% lat (msec) : 2=0.52%, 4=2.60%, 10=13.54%, 20=35.29%, 50=41.54% lat (msec) : 100=2.86% cpu : usr=0.15%, sys=24.12%, ctx=2589, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189365: Thu Sep 25 10:32:07 2025 read: IOPS=22, BW=367KiB/s (376kB/s)(5744KiB/15640msec) clat (usec): min=152, max=58236, avg=21349.13, stdev=10657.64 lat (usec): min=153, max=58236, avg=21350.44, stdev=10657.69 clat percentiles (usec): | 1.00th=[ 375], 5.00th=[ 6390], 10.00th=[10945], 20.00th=[13042], | 30.00th=[14746], 40.00th=[17433], 50.00th=[20055], 60.00th=[22152], | 70.00th=[25297], 80.00th=[29754], 90.00th=[34866], 95.00th=[42206], | 99.00th=[55837], 99.50th=[57410], 99.90th=[58459], 99.95th=[58459], | 99.99th=[58459] bw ( KiB/s): min= 141, max= 572, per=24.52%, avg=368.55, stdev=91.70, samples=31 iops : min= 8, max= 35, avg=22.39, stdev= 5.72, samples=31 write: IOPS=26, BW=418KiB/s (428kB/s)(6544KiB/15640msec); 0 zone resets clat (usec): min=1671, max=74087, avg=19165.03, stdev=14065.94 lat (usec): min=1672, max=74090, avg=19168.46, stdev=14066.11 clat percentiles (usec): | 1.00th=[ 1926], 5.00th=[ 3621], 10.00th=[ 4490], 20.00th=[ 5997], | 30.00th=[10159], 40.00th=[12780], 50.00th=[15533], 60.00th=[18744], | 70.00th=[23200], 80.00th=[30540], 90.00th=[39584], 95.00th=[49021], | 99.00th=[61604], 99.50th=[63701], 99.90th=[73925], 99.95th=[73925], | 99.99th=[73925] bw ( KiB/s): min= 191, max= 733, per=26.44%, avg=418.35, stdev=119.50, samples=31 iops : min= 11, max= 45, avg=25.52, stdev= 7.55, samples=31 lat (usec) : 250=0.26%, 500=1.04%, 1000=0.13% lat (msec) : 2=1.30%, 4=3.12%, 10=13.54%, 20=37.24%, 50=39.84% lat (msec) : 100=3.52% cpu : usr=0.15%, sys=24.80%, ctx=2618, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189366: Thu Sep 25 10:32:07 2025 read: IOPS=24, BW=393KiB/s (402kB/s)(6256KiB/15928msec) clat (usec): min=270, max=81658, avg=22325.15, stdev=12836.16 lat (usec): min=271, max=81661, avg=22326.80, stdev=12836.28 clat percentiles (usec): | 1.00th=[ 326], 5.00th=[ 586], 10.00th=[10028], 20.00th=[12911], | 30.00th=[15664], 40.00th=[17957], 50.00th=[20317], 60.00th=[23462], | 70.00th=[26346], 80.00th=[30016], 90.00th=[35914], 95.00th=[45876], | 99.00th=[68682], 99.50th=[73925], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 159, max= 544, per=25.95%, avg=390.06, stdev=97.22, samples=31 iops : min= 9, max= 34, avg=23.74, stdev= 6.16, samples=31 write: IOPS=23, BW=379KiB/s (388kB/s)(6032KiB/15928msec); 0 zone resets clat (usec): min=1344, max=115052, avg=18807.25, stdev=15339.06 lat (usec): min=1346, max=115054, avg=18812.07, stdev=15337.66 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 9], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 31], 90.00th=[ 42], 95.00th=[ 48], | 99.00th=[ 73], 99.50th=[ 84], 99.90th=[ 115], 99.95th=[ 115], | 99.99th=[ 115] bw ( KiB/s): min= 191, max= 576, per=23.34%, avg=369.23, stdev=125.20, samples=31 iops : min= 11, max= 36, avg=22.45, stdev= 7.96, samples=31 lat (usec) : 500=2.34%, 750=0.39%, 1000=0.13% lat (msec) : 2=0.13%, 4=4.56%, 10=13.93%, 20=34.51%, 50=39.58% lat (msec) : 100=4.30%, 250=0.13% cpu : usr=0.17%, sys=23.80%, ctx=2606, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189367: Thu Sep 25 10:32:07 2025 read: IOPS=24, BW=391KiB/s (401kB/s)(6032KiB/15412msec) clat (usec): min=179, max=105040, avg=20849.48, stdev=12441.20 lat (usec): min=180, max=105041, avg=20850.87, stdev=12441.29 clat percentiles (usec): | 1.00th=[ 265], 5.00th=[ 498], 10.00th=[ 10028], 20.00th=[ 12387], | 30.00th=[ 14353], 40.00th=[ 16909], 50.00th=[ 19006], 60.00th=[ 21890], | 70.00th=[ 24511], 80.00th=[ 28443], 90.00th=[ 35914], 95.00th=[ 41681], | 99.00th=[ 58983], 99.50th=[ 87557], 99.90th=[105382], 99.95th=[105382], | 99.99th=[105382] bw ( KiB/s): min= 223, max= 512, per=25.75%, avg=387.03, stdev=91.65, samples=30 iops : min= 13, max= 32, avg=23.57, stdev= 5.85, samples=30 write: IOPS=25, BW=406KiB/s (416kB/s)(6256KiB/15412msec); 0 zone resets clat (usec): min=1376, max=90960, avg=19162.35, stdev=15389.37 lat (usec): min=1379, max=90962, avg=19180.15, stdev=15377.09 clat percentiles (usec): | 1.00th=[ 2114], 5.00th=[ 3326], 10.00th=[ 4424], 20.00th=[ 5735], | 30.00th=[ 8356], 40.00th=[12256], 50.00th=[15533], 60.00th=[18744], | 70.00th=[23462], 80.00th=[30802], 90.00th=[39060], 95.00th=[48497], | 99.00th=[81265], 99.50th=[88605], 99.90th=[90702], 99.95th=[90702], | 99.99th=[90702] bw ( KiB/s): min= 191, max= 670, per=25.50%, avg=403.47, stdev=123.97, samples=30 iops : min= 11, max= 41, avg=24.60, stdev= 7.72, samples=30 lat (usec) : 250=0.39%, 500=2.08%, 750=0.65%, 1000=0.26% lat (msec) : 2=0.39%, 4=3.65%, 10=14.58%, 20=36.85%, 50=37.37% lat (msec) : 100=3.65%, 250=0.13% cpu : usr=0.23%, sys=24.13%, ctx=2590, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1504KiB/s (1540kB/s), 367KiB/s-393KiB/s (376kB/s-402kB/s), io=23.4MiB (24.5MB), run=15412-15928msec WRITE: bw=1582KiB/s (1620kB/s), 379KiB/s-418KiB/s (388kB/s-428kB/s), io=24.6MiB (25.8MB), run=15412-15928msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189360: Thu Sep 25 10:32:10 2025 read: IOPS=19, BW=306KiB/s (313kB/s)(5920KiB/19340msec) clat (usec): min=4903, max=73216, avg=21001.42, stdev=15327.11 lat (usec): min=4905, max=73223, avg=21002.82, stdev=15327.07 clat percentiles (usec): | 1.00th=[ 5473], 5.00th=[ 6783], 10.00th=[ 7504], 20.00th=[ 8848], | 30.00th=[10683], 40.00th=[12780], 50.00th=[14746], 60.00th=[18220], | 70.00th=[23462], 80.00th=[30802], 90.00th=[46924], 95.00th=[55837], | 99.00th=[70779], 99.50th=[71828], 99.90th=[72877], 99.95th=[72877], | 99.99th=[72877] bw ( KiB/s): min= 31, max= 800, per=24.20%, avg=299.63, stdev=155.97, samples=38 iops : min= 1, max= 50, avg=18.18, stdev= 9.93, samples=38 write: IOPS=20, BW=329KiB/s (337kB/s)(6368KiB/19340msec); 0 zone resets clat (msec): min=6, max=121, avg=28.88, stdev=18.10 lat (msec): min=6, max=121, avg=28.88, stdev=18.10 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 13], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 23], 60.00th=[ 29], | 70.00th=[ 35], 80.00th=[ 43], 90.00th=[ 53], 95.00th=[ 63], | 99.00th=[ 100], 99.50th=[ 111], 99.90th=[ 122], 99.95th=[ 122], | 99.99th=[ 122] bw ( KiB/s): min= 127, max= 736, per=24.96%, avg=325.03, stdev=140.67, samples=38 iops : min= 7, max= 46, avg=19.79, stdev= 8.94, samples=38 lat (msec) : 10=13.54%, 20=39.71%, 50=37.24%, 100=9.11%, 250=0.39% cpu : usr=0.14%, sys=9.56%, ctx=1535, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189361: Thu Sep 25 10:32:10 2025 read: IOPS=18, BW=297KiB/s (304kB/s)(5744KiB/19326msec) clat (msec): min=4, max=123, avg=20.64, stdev=16.56 lat (msec): min=4, max=123, avg=20.64, stdev=16.56 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 15], 60.00th=[ 18], | 70.00th=[ 23], 80.00th=[ 29], 90.00th=[ 45], 95.00th=[ 55], | 99.00th=[ 82], 99.50th=[ 95], 99.90th=[ 124], 99.95th=[ 124], | 99.99th=[ 124] bw ( KiB/s): min= 63, max= 736, per=23.83%, avg=295.00, stdev=163.66, samples=38 iops : min= 3, max= 46, avg=17.92, stdev=10.40, samples=38 write: IOPS=21, BW=339KiB/s (347kB/s)(6544KiB/19326msec); 0 zone resets clat (msec): min=7, max=122, avg=28.88, stdev=18.31 lat (msec): min=7, max=122, avg=28.89, stdev=18.31 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 29], | 70.00th=[ 35], 80.00th=[ 44], 90.00th=[ 55], 95.00th=[ 62], | 99.00th=[ 87], 99.50th=[ 110], 99.90th=[ 123], 99.95th=[ 123], | 99.99th=[ 123] bw ( KiB/s): min= 95, max= 862, per=25.52%, avg=332.32, stdev=157.70, samples=38 iops : min= 5, max= 53, avg=20.24, stdev= 9.95, samples=38 lat (msec) : 10=15.23%, 20=38.15%, 50=36.72%, 100=9.24%, 250=0.65% cpu : usr=0.11%, sys=9.42%, ctx=1454, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189362: Thu Sep 25 10:32:10 2025 read: IOPS=20, BW=330KiB/s (338kB/s)(6256KiB/18948msec) clat (msec): min=4, max=120, avg=20.60, stdev=15.43 lat (msec): min=4, max=120, avg=20.60, stdev=15.43 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 12], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 22], 80.00th=[ 28], 90.00th=[ 43], 95.00th=[ 53], | 99.00th=[ 82], 99.50th=[ 91], 99.90th=[ 121], 99.95th=[ 121], | 99.99th=[ 121] bw ( KiB/s): min= 96, max= 768, per=26.63%, avg=329.62, stdev=165.59, samples=37 iops : min= 6, max= 48, avg=20.05, stdev=10.41, samples=37 write: IOPS=19, BW=318KiB/s (326kB/s)(6032KiB/18948msec); 0 zone resets clat (msec): min=8, max=121, avg=28.64, stdev=16.33 lat (msec): min=8, max=121, avg=28.64, stdev=16.33 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 11], 10.00th=[ 13], 20.00th=[ 15], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 26], 60.00th=[ 29], | 70.00th=[ 34], 80.00th=[ 41], 90.00th=[ 51], 95.00th=[ 61], | 99.00th=[ 82], 99.50th=[ 107], 99.90th=[ 122], 99.95th=[ 122], | 99.99th=[ 122] bw ( KiB/s): min= 127, max= 896, per=24.11%, avg=313.86, stdev=148.04, samples=37 iops : min= 7, max= 56, avg=19.08, stdev= 9.37, samples=37 lat (msec) : 10=13.80%, 20=37.11%, 50=40.62%, 100=8.07%, 250=0.39% cpu : usr=0.12%, sys=9.98%, ctx=1493, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189364: Thu Sep 25 10:32:10 2025 read: IOPS=19, BW=313KiB/s (320kB/s)(6032KiB/19301msec) clat (msec): min=4, max=109, avg=21.70, stdev=15.72 lat (msec): min=4, max=109, avg=21.70, stdev=15.72 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 11], | 30.00th=[ 12], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 32], 90.00th=[ 47], 95.00th=[ 56], | 99.00th=[ 73], 99.50th=[ 86], 99.90th=[ 110], 99.95th=[ 110], | 99.99th=[ 110] bw ( KiB/s): min= 63, max= 734, per=24.96%, avg=308.97, stdev=147.31, samples=38 iops : min= 3, max= 45, avg=18.79, stdev= 9.28, samples=38 write: IOPS=20, BW=324KiB/s (332kB/s)(6256KiB/19301msec); 0 zone resets clat (usec): min=7909, max=98176, avg=28354.43, stdev=17521.02 lat (usec): min=7912, max=98179, avg=28357.62, stdev=17521.14 clat percentiles (usec): | 1.00th=[ 8979], 5.00th=[10814], 10.00th=[11600], 20.00th=[13829], | 30.00th=[16450], 40.00th=[19006], 50.00th=[22676], 60.00th=[27132], | 70.00th=[34341], 80.00th=[41681], 90.00th=[52167], 95.00th=[62653], | 99.00th=[84411], 99.50th=[98042], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 63, max= 800, per=24.68%, avg=321.39, stdev=155.72, samples=38 iops : min= 3, max= 50, avg=19.55, stdev= 9.92, samples=38 lat (msec) : 10=11.46%, 20=40.89%, 50=38.15%, 100=9.38%, 250=0.13% cpu : usr=0.13%, sys=9.88%, ctx=1588, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1238KiB/s (1268kB/s), 297KiB/s-330KiB/s (304kB/s-338kB/s), io=23.4MiB (24.5MB), run=18948-19340msec WRITE: bw=1303KiB/s (1334kB/s), 318KiB/s-339KiB/s (326kB/s-347kB/s), io=24.6MiB (25.8MB), run=18948-19340msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189381: Thu Sep 25 10:32:21 2025 read: IOPS=2, BW=2467KiB/s (2526kB/s)(7168KiB/2906msec) clat (usec): min=1095, max=1245.6k, avg=270756.49, stdev=436736.64 lat (usec): min=1097, max=1245.6k, avg=270758.10, stdev=436736.72 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 1090], 10.00th=[ 1090], | 20.00th=[ 1713], 30.00th=[ 132645], 40.00th=[ 132645], | 50.00th=[ 160433], 60.00th=[ 175113], 70.00th=[ 175113], | 80.00th=[ 181404], 90.00th=[1249903], 95.00th=[1249903], | 99.00th=[1249903], 99.50th=[1249903], 99.90th=[1249903], | 99.95th=[1249903], 99.99th=[1249903] bw ( KiB/s): min= 3506, max= 6144, per=99.16%, avg=4825.00, stdev=1865.35, samples=2 iops : min= 3, max= 6, avg= 4.50, stdev= 2.12, samples=2 write: IOPS=1, BW=1762KiB/s (1804kB/s)(5120KiB/2906msec); 0 zone resets clat (msec): min=111, max=302, avg=196.30, stdev=74.32 lat (msec): min=111, max=303, avg=196.43, stdev=74.32 clat percentiles (msec): | 1.00th=[ 111], 5.00th=[ 111], 10.00th=[ 111], 20.00th=[ 111], | 30.00th=[ 144], 40.00th=[ 144], 50.00th=[ 203], 60.00th=[ 203], | 70.00th=[ 222], 80.00th=[ 222], 90.00th=[ 305], 95.00th=[ 305], | 99.00th=[ 305], 99.50th=[ 305], 99.90th=[ 305], 99.95th=[ 305], | 99.99th=[ 305] bw ( KiB/s): min= 2048, max= 4096, per=45.09%, avg=3072.00, stdev=1448.15, samples=2 iops : min= 2, max= 4, avg= 3.00, stdev= 1.41, samples=2 lat (msec) : 2=16.67%, 250=66.67%, 500=8.33%, 2000=8.33% cpu : usr=0.00%, sys=38.52%, ctx=987, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189382: Thu Sep 25 10:32:21 2025 read: IOPS=0, BW=487KiB/s (498kB/s)(2048KiB/4208msec) clat (usec): min=814, max=127925, avg=64370.11, stdev=89881.51 lat (usec): min=815, max=127928, avg=64371.97, stdev=89882.36 clat percentiles (usec): | 1.00th=[ 816], 5.00th=[ 816], 10.00th=[ 816], 20.00th=[ 816], | 30.00th=[ 816], 40.00th=[ 816], 50.00th=[ 816], 60.00th=[127402], | 70.00th=[127402], 80.00th=[127402], 90.00th=[127402], 95.00th=[127402], | 99.00th=[127402], 99.50th=[127402], 99.90th=[127402], 99.95th=[127402], | 99.99th=[127402] bw ( KiB/s): min= 1768, max= 1973, per=38.44%, avg=1870.50, stdev=144.96, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2433KiB/s (2492kB/s)(10.0MiB/4208msec); 0 zone resets clat (msec): min=98, max=1726, avg=396.76, stdev=492.76 lat (msec): min=98, max=1726, avg=396.88, stdev=492.76 clat percentiles (msec): | 1.00th=[ 99], 5.00th=[ 99], 10.00th=[ 99], 20.00th=[ 112], | 30.00th=[ 113], 40.00th=[ 144], 50.00th=[ 167], 60.00th=[ 288], | 70.00th=[ 321], 80.00th=[ 430], 90.00th=[ 575], 95.00th=[ 1720], | 99.00th=[ 1720], 99.50th=[ 1720], 99.90th=[ 1720], 99.95th=[ 1720], | 99.99th=[ 1720] bw ( KiB/s): min= 2048, max= 8192, per=66.96%, avg=4562.25, stdev=2590.08, samples=4 iops : min= 2, max= 8, avg= 4.00, stdev= 2.71, samples=4 lat (usec) : 1000=8.33% lat (msec) : 100=8.33%, 250=41.67%, 500=25.00%, 750=8.33%, 2000=8.33% cpu : usr=0.05%, sys=29.19%, ctx=119, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189383: Thu Sep 25 10:32:21 2025 read: IOPS=1, BW=1485KiB/s (1521kB/s)(6144KiB/4136msec) clat (usec): min=1096, max=262165, avg=164413.89, stdev=92978.74 lat (usec): min=1098, max=262166, avg=164415.35, stdev=92978.55 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 1090], 10.00th=[ 1090], 20.00th=[129500], | 30.00th=[129500], 40.00th=[162530], 50.00th=[162530], 60.00th=[198181], | 70.00th=[231736], 80.00th=[231736], 90.00th=[263193], 95.00th=[263193], | 99.00th=[263193], 99.50th=[263193], 99.90th=[263193], 99.95th=[263193], | 99.99th=[263193] bw ( KiB/s): min= 1973, max= 4096, per=52.14%, avg=2537.00, stdev=1039.83, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 write: IOPS=1, BW=1485KiB/s (1521kB/s)(6144KiB/4136msec); 0 zone resets clat (msec): min=202, max=1708, avg=506.57, stdev=591.35 lat (msec): min=202, max=1708, avg=506.70, stdev=591.34 clat percentiles (msec): | 1.00th=[ 203], 5.00th=[ 203], 10.00th=[ 203], 20.00th=[ 222], | 30.00th=[ 222], 40.00th=[ 251], 50.00th=[ 251], 60.00th=[ 317], | 70.00th=[ 342], 80.00th=[ 342], 90.00th=[ 1703], 95.00th=[ 1703], | 99.00th=[ 1703], 99.50th=[ 1703], 99.90th=[ 1703], 99.95th=[ 1703], | 99.99th=[ 1703] bw ( KiB/s): min= 1747, max= 4063, per=34.87%, avg=2375.80, stdev=951.21, samples=5 iops : min= 1, max= 3, avg= 1.80, stdev= 0.84, samples=5 lat (msec) : 2=8.33%, 250=50.00%, 500=33.33%, 2000=8.33% cpu : usr=0.00%, sys=27.67%, ctx=515, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189384: Thu Sep 25 10:32:21 2025 read: IOPS=1, BW=1365KiB/s (1398kB/s)(5120KiB/3751msec) clat (usec): min=1093, max=190457, avg=115623.00, stdev=77336.59 lat (usec): min=1094, max=190459, avg=115624.82, stdev=77336.99 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 1090], 10.00th=[ 1090], 20.00th=[ 1090], | 30.00th=[ 72877], 40.00th=[ 72877], 50.00th=[154141], 60.00th=[154141], | 70.00th=[160433], 80.00th=[160433], 90.00th=[189793], 95.00th=[189793], | 99.00th=[189793], 99.50th=[189793], 99.90th=[189793], 99.95th=[189793], | 99.99th=[189793] bw ( KiB/s): min= 2048, max= 5919, per=81.86%, avg=3983.50, stdev=2737.21, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 write: IOPS=1, BW=1911KiB/s (1957kB/s)(7168KiB/3751msec); 0 zone resets clat (msec): min=176, max=1550, avg=441.23, stdev=491.86 lat (msec): min=176, max=1551, avg=441.37, stdev=491.89 clat percentiles (msec): | 1.00th=[ 178], 5.00th=[ 178], 10.00th=[ 178], 20.00th=[ 218], | 30.00th=[ 251], 40.00th=[ 251], 50.00th=[ 264], 60.00th=[ 309], | 70.00th=[ 309], 80.00th=[ 326], 90.00th=[ 1552], 95.00th=[ 1552], | 99.00th=[ 1552], 99.50th=[ 1552], 99.90th=[ 1552], 99.95th=[ 1552], | 99.99th=[ 1552] bw ( KiB/s): min= 1771, max= 4096, per=41.05%, avg=2796.80, stdev=1190.32, samples=5 iops : min= 1, max= 4, avg= 2.40, stdev= 1.52, samples=5 lat (msec) : 2=8.33%, 100=8.33%, 250=41.67%, 500=33.33%, 2000=8.33% cpu : usr=0.00%, sys=32.05%, ctx=313, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4867KiB/s (4984kB/s), 487KiB/s-2467KiB/s (498kB/s-2526kB/s), io=20.0MiB (20.0MB), run=2906-4208msec WRITE: bw=6814KiB/s (6977kB/s), 1485KiB/s-2433KiB/s (1521kB/s-2492kB/s), io=28.0MiB (29.4MB), run=2906-4208msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189376: Thu Sep 25 10:32:21 2025 read: IOPS=1, BW=1670KiB/s (1711kB/s)(7168KiB/4291msec) clat (msec): min=67, max=1236, avg=314.93, stdev=415.08 lat (msec): min=67, max=1236, avg=314.93, stdev=415.08 clat percentiles (msec): | 1.00th=[ 68], 5.00th=[ 68], 10.00th=[ 68], 20.00th=[ 95], | 30.00th=[ 115], 40.00th=[ 115], 50.00th=[ 169], 60.00th=[ 197], | 70.00th=[ 197], 80.00th=[ 326], 90.00th=[ 1234], 95.00th=[ 1234], | 99.00th=[ 1234], 99.50th=[ 1234], 99.90th=[ 1234], 99.95th=[ 1234], | 99.99th=[ 1234] bw ( KiB/s): min= 2043, max= 3750, per=62.63%, avg=2831.00, stdev=916.17, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 write: IOPS=1, BW=1193KiB/s (1222kB/s)(5120KiB/4291msec); 0 zone resets clat (msec): min=221, max=586, avg=399.67, stdev=141.94 lat (msec): min=221, max=586, avg=399.83, stdev=141.93 clat percentiles (msec): | 1.00th=[ 222], 5.00th=[ 222], 10.00th=[ 222], 20.00th=[ 222], | 30.00th=[ 305], 40.00th=[ 305], 50.00th=[ 418], 60.00th=[ 418], | 70.00th=[ 468], 80.00th=[ 468], 90.00th=[ 584], 95.00th=[ 584], | 99.00th=[ 584], 99.50th=[ 584], 99.90th=[ 584], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 2043, max= 4087, per=40.37%, avg=2555.25, stdev=1021.17, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 100=16.67%, 250=33.33%, 500=33.33%, 750=8.33%, 2000=8.33% cpu : usr=0.02%, sys=12.26%, ctx=147, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189377: Thu Sep 25 10:32:21 2025 read: IOPS=0, BW=452KiB/s (463kB/s)(2048KiB/4530msec) clat (msec): min=72, max=143, avg=108.00, stdev=50.63 lat (msec): min=72, max=143, avg=108.00, stdev=50.63 clat percentiles (msec): | 1.00th=[ 72], 5.00th=[ 72], 10.00th=[ 72], 20.00th=[ 72], | 30.00th=[ 72], 40.00th=[ 72], 50.00th=[ 72], 60.00th=[ 144], | 70.00th=[ 144], 80.00th=[ 144], 90.00th=[ 144], 95.00th=[ 144], | 99.00th=[ 144], 99.50th=[ 144], 99.90th=[ 144], 99.95th=[ 144], | 99.99th=[ 144] bw ( KiB/s): min= 1738, max= 2048, per=41.88%, avg=1893.00, stdev=219.20, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2260KiB/s (2315kB/s)(10.0MiB/4530msec); 0 zone resets clat (msec): min=49, max=2033, avg=425.03, stdev=579.39 lat (msec): min=49, max=2033, avg=425.19, stdev=579.38 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 50], 20.00th=[ 103], | 30.00th=[ 171], 40.00th=[ 211], 50.00th=[ 234], 60.00th=[ 292], | 70.00th=[ 300], 80.00th=[ 376], 90.00th=[ 485], 95.00th=[ 2039], | 99.00th=[ 2039], 99.50th=[ 2039], 99.90th=[ 2039], 99.95th=[ 2039], | 99.99th=[ 2039] bw ( KiB/s): min= 2043, max= 8175, per=64.60%, avg=4088.25, stdev=2889.47, samples=4 iops : min= 1, max= 7, avg= 3.25, stdev= 2.63, samples=4 lat (msec) : 50=8.33%, 100=8.33%, 250=41.67%, 500=33.33%, >=2000=8.33% cpu : usr=0.02%, sys=18.48%, ctx=149, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189378: Thu Sep 25 10:32:21 2025 read: IOPS=1, BW=1562KiB/s (1599kB/s)(6144KiB/3934msec) clat (msec): min=81, max=425, avg=206.85, stdev=127.95 lat (msec): min=81, max=425, avg=206.86, stdev=127.95 clat percentiles (msec): | 1.00th=[ 82], 5.00th=[ 82], 10.00th=[ 82], 20.00th=[ 104], | 30.00th=[ 104], 40.00th=[ 176], 50.00th=[ 176], 60.00th=[ 176], | 70.00th=[ 279], 80.00th=[ 279], 90.00th=[ 426], 95.00th=[ 426], | 99.00th=[ 426], 99.50th=[ 426], 99.90th=[ 426], 99.95th=[ 426], | 99.99th=[ 426] bw ( KiB/s): min= 2043, max= 3750, per=57.82%, avg=2613.67, stdev=984.10, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 write: IOPS=1, BW=1562KiB/s (1599kB/s)(6144KiB/3934msec); 0 zone resets clat (msec): min=85, max=1615, avg=436.50, stdev=592.10 lat (msec): min=86, max=1615, avg=436.64, stdev=592.15 clat percentiles (msec): | 1.00th=[ 86], 5.00th=[ 86], 10.00th=[ 86], 20.00th=[ 92], | 30.00th=[ 92], 40.00th=[ 136], 50.00th=[ 136], 60.00th=[ 264], | 70.00th=[ 426], 80.00th=[ 426], 90.00th=[ 1620], 95.00th=[ 1620], | 99.00th=[ 1620], 99.50th=[ 1620], 99.90th=[ 1620], 99.95th=[ 1620], | 99.99th=[ 1620] bw ( KiB/s): min= 1756, max= 3750, per=37.91%, avg=2399.25, stdev=910.78, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 100=25.00%, 250=33.33%, 500=33.33%, 2000=8.33% cpu : usr=0.03%, sys=13.96%, ctx=165, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189379: Thu Sep 25 10:32:21 2025 read: IOPS=1, BW=1423KiB/s (1457kB/s)(5120KiB/3598msec) clat (msec): min=102, max=202, avg=155.05, stdev=36.01 lat (msec): min=102, max=202, avg=155.06, stdev=36.01 clat percentiles (msec): | 1.00th=[ 103], 5.00th=[ 103], 10.00th=[ 103], 20.00th=[ 103], | 30.00th=[ 153], 40.00th=[ 153], 50.00th=[ 153], 60.00th=[ 153], | 70.00th=[ 167], 80.00th=[ 167], 90.00th=[ 203], 95.00th=[ 203], | 99.00th=[ 203], 99.50th=[ 203], 99.90th=[ 203], 99.95th=[ 203], | 99.99th=[ 203] bw ( KiB/s): min= 3750, max= 4087, per=86.69%, avg=3918.50, stdev=238.29, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1992KiB/s (2040kB/s)(7168KiB/3598msec); 0 zone resets clat (msec): min=78, max=1156, avg=401.40, stdev=371.46 lat (msec): min=78, max=1157, avg=401.53, stdev=371.49 clat percentiles (msec): | 1.00th=[ 79], 5.00th=[ 79], 10.00th=[ 79], 20.00th=[ 159], | 30.00th=[ 230], 40.00th=[ 230], 50.00th=[ 279], 60.00th=[ 305], | 70.00th=[ 305], 80.00th=[ 600], 90.00th=[ 1150], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min= 2043, max= 3488, per=37.99%, avg=2404.25, stdev=722.50, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 100=8.33%, 250=58.33%, 500=16.67%, 750=8.33%, 2000=8.33% cpu : usr=0.03%, sys=9.70%, ctx=101, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4521KiB/s (4629kB/s), 452KiB/s-1670KiB/s (463kB/s-1711kB/s), io=20.0MiB (20.0MB), run=3598-4530msec WRITE: bw=6329KiB/s (6481kB/s), 1193KiB/s-2260KiB/s (1222kB/s-2315kB/s), io=28.0MiB (29.4MB), run=3598-4530msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189390: Thu Sep 25 10:32:31 2025 read: IOPS=1, BW=6982KiB/s (7149kB/s)(12.0MiB/1760msec) clat (msec): min=526, max=638, avg=565.75, stdev=62.75 lat (msec): min=526, max=638, avg=565.75, stdev=62.75 clat percentiles (msec): | 1.00th=[ 527], 5.00th=[ 527], 10.00th=[ 527], 20.00th=[ 527], | 30.00th=[ 527], 40.00th=[ 531], 50.00th=[ 531], 60.00th=[ 531], | 70.00th=[ 642], 80.00th=[ 642], 90.00th=[ 642], 95.00th=[ 642], | 99.00th=[ 642], 99.50th=[ 642], 99.90th=[ 642], 99.95th=[ 642], | 99.99th=[ 642] bw ( KiB/s): min= 6838, max= 8175, per=100.00%, avg=7506.50, stdev=945.40, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=100.00% cpu : usr=0.00%, sys=46.84%, ctx=1632, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189392: Thu Sep 25 10:32:31 2025 read: IOPS=0, BW=1280KiB/s (1311kB/s)(4096KiB/3200msec) clat (nsec): min=520579k, max=520579k, avg=520578881.00, stdev= 0.00 lat (nsec): min=520584k, max=520584k, avg=520584303.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 523], 5.00th=[ 523], 10.00th=[ 523], 20.00th=[ 523], | 30.00th=[ 523], 40.00th=[ 523], 50.00th=[ 523], 60.00th=[ 523], | 70.00th=[ 523], 80.00th=[ 523], 90.00th=[ 523], 95.00th=[ 523], | 99.00th=[ 523], 99.50th=[ 523], 99.90th=[ 523], 99.95th=[ 523], | 99.99th=[ 523] bw ( KiB/s): min= 6553, max= 6553, per=100.00%, avg=6553.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2560KiB/s (2621kB/s)(8192KiB/3200msec); 0 zone resets clat (msec): min=616, max=1963, avg=1289.74, stdev=952.32 lat (msec): min=616, max=1963, avg=1290.32, stdev=952.50 clat percentiles (msec): | 1.00th=[ 617], 5.00th=[ 617], 10.00th=[ 617], 20.00th=[ 617], | 30.00th=[ 617], 40.00th=[ 617], 50.00th=[ 617], 60.00th=[ 1972], | 70.00th=[ 1972], 80.00th=[ 1972], 90.00th=[ 1972], 95.00th=[ 1972], | 99.00th=[ 1972], 99.50th=[ 1972], 99.90th=[ 1972], 99.95th=[ 1972], | 99.99th=[ 1972] bw ( KiB/s): min= 8192, max= 8192, per=87.78%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.03%, sys=32.01%, ctx=692, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189394: Thu Sep 25 10:32:31 2025 write: IOPS=0, BW=3500KiB/s (3584kB/s)(12.0MiB/3511msec); 0 zone resets clat (msec): min=839, max=1301, avg=1136.84, stdev=258.04 lat (msec): min=840, max=1301, avg=1137.34, stdev=257.86 clat percentiles (msec): | 1.00th=[ 844], 5.00th=[ 844], 10.00th=[ 844], 20.00th=[ 844], | 30.00th=[ 844], 40.00th=[ 1267], 50.00th=[ 1267], 60.00th=[ 1267], | 70.00th=[ 1301], 80.00th=[ 1301], 90.00th=[ 1301], 95.00th=[ 1301], | 99.00th=[ 1301], 99.50th=[ 1301], 99.90th=[ 1301], 99.95th=[ 1301], | 99.99th=[ 1301] bw ( KiB/s): min= 8126, max= 8192, per=87.43%, avg=8159.00, stdev=46.67, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=41.20%, ctx=31, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189396: Thu Sep 25 10:32:31 2025 write: IOPS=1, BW=5285KiB/s (5412kB/s)(12.0MiB/2325msec); 0 zone resets clat (msec): min=576, max=869, avg=746.88, stdev=152.29 lat (msec): min=577, max=870, avg=747.51, stdev=152.20 clat percentiles (msec): | 1.00th=[ 575], 5.00th=[ 575], 10.00th=[ 575], 20.00th=[ 575], | 30.00th=[ 575], 40.00th=[ 793], 50.00th=[ 793], 60.00th=[ 793], | 70.00th=[ 869], 80.00th=[ 869], 90.00th=[ 869], 95.00th=[ 869], | 99.00th=[ 869], 99.50th=[ 869], 99.90th=[ 869], 99.95th=[ 869], | 99.99th=[ 869] bw ( KiB/s): min= 8126, max= 8192, per=87.43%, avg=8159.00, stdev=46.67, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 1000=66.67% cpu : usr=0.04%, sys=69.75%, ctx=91, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5120KiB/s (5243kB/s), 1280KiB/s-6982KiB/s (1311kB/s-7149kB/s), io=16.0MiB (16.8MB), run=1760-3200msec WRITE: bw=9333KiB/s (9557kB/s), 2560KiB/s-5285KiB/s (2621kB/s-5412kB/s), io=32.0MiB (33.6MB), run=2325-3511msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189391: Thu Sep 25 10:32:32 2025 read: IOPS=1, BW=6671KiB/s (6831kB/s)(12.0MiB/1842msec) clat (msec): min=169, max=1370, avg=590.20, stdev=676.82 lat (msec): min=169, max=1370, avg=590.20, stdev=676.82 clat percentiles (msec): | 1.00th=[ 169], 5.00th=[ 169], 10.00th=[ 169], 20.00th=[ 169], | 30.00th=[ 169], 40.00th=[ 230], 50.00th=[ 230], 60.00th=[ 230], | 70.00th=[ 1368], 80.00th=[ 1368], 90.00th=[ 1368], 95.00th=[ 1368], | 99.00th=[ 1368], 99.50th=[ 1368], 99.90th=[ 1368], 99.95th=[ 1368], | 99.99th=[ 1368] bw ( KiB/s): min=13149, max=13149, per=100.00%, avg=13149.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, 2000=33.33% cpu : usr=0.00%, sys=17.00%, ctx=100, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189393: Thu Sep 25 10:32:32 2025 read: IOPS=0, BW=1096KiB/s (1122kB/s)(4096KiB/3737msec) clat (nsec): min=1427.4M, max=1427.4M, avg=1427421597.00, stdev= 0.00 lat (nsec): min=1427.4M, max=1427.4M, avg=1427424936.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1435], 5.00th=[ 1435], 10.00th=[ 1435], 20.00th=[ 1435], | 30.00th=[ 1435], 40.00th=[ 1435], 50.00th=[ 1435], 60.00th=[ 1435], | 70.00th=[ 1435], 80.00th=[ 1435], 90.00th=[ 1435], 95.00th=[ 1435], | 99.00th=[ 1435], 99.50th=[ 1435], 99.90th=[ 1435], 99.95th=[ 1435], | 99.99th=[ 1435] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2192KiB/s (2245kB/s)(8192KiB/3737msec); 0 zone resets clat (msec): min=733, max=1413, avg=1073.92, stdev=480.80 lat (msec): min=734, max=1414, avg=1074.38, stdev=480.72 clat percentiles (msec): | 1.00th=[ 735], 5.00th=[ 735], 10.00th=[ 735], 20.00th=[ 735], | 30.00th=[ 735], 40.00th=[ 735], 50.00th=[ 735], 60.00th=[ 1418], | 70.00th=[ 1418], 80.00th=[ 1418], 90.00th=[ 1418], 95.00th=[ 1418], | 99.00th=[ 1418], 99.50th=[ 1418], 99.90th=[ 1418], 99.95th=[ 1418], | 99.99th=[ 1418] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=9.42%, ctx=291, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189395: Thu Sep 25 10:32:32 2025 write: IOPS=0, BW=3126KiB/s (3201kB/s)(12.0MiB/3931msec); 0 zone resets clat (msec): min=367, max=1863, avg=1275.60, stdev=798.04 lat (msec): min=367, max=1863, avg=1276.26, stdev=798.16 clat percentiles (msec): | 1.00th=[ 368], 5.00th=[ 368], 10.00th=[ 368], 20.00th=[ 368], | 30.00th=[ 368], 40.00th=[ 1603], 50.00th=[ 1603], 60.00th=[ 1603], | 70.00th=[ 1871], 80.00th=[ 1871], 90.00th=[ 1871], 95.00th=[ 1871], | 99.00th=[ 1871], 99.50th=[ 1871], 99.90th=[ 1871], 99.95th=[ 1871], | 99.99th=[ 1871] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.05%, sys=21.78%, ctx=110, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189397: Thu Sep 25 10:32:32 2025 write: IOPS=0, BW=3071KiB/s (3145kB/s)(12.0MiB/4001msec); 0 zone resets clat (msec): min=190, max=2622, avg=1285.06, stdev=1233.78 lat (msec): min=190, max=2622, avg=1285.60, stdev=1233.87 clat percentiles (msec): | 1.00th=[ 190], 5.00th=[ 190], 10.00th=[ 190], 20.00th=[ 190], | 30.00th=[ 190], 40.00th=[ 1045], 50.00th=[ 1045], 60.00th=[ 1045], | 70.00th=[ 2635], 80.00th=[ 2635], 90.00th=[ 2635], 95.00th=[ 2635], | 99.00th=[ 2635], 99.50th=[ 2635], 99.90th=[ 2635], 99.95th=[ 2635], | 99.99th=[ 2635] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=16.85%, ctx=97, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4384KiB/s (4489kB/s), 1096KiB/s-6671KiB/s (1122kB/s-6831kB/s), io=16.0MiB (16.8MB), run=1842-3737msec WRITE: bw=8190KiB/s (8387kB/s), 2192KiB/s-3126KiB/s (2245kB/s-3201kB/s), io=32.0MiB (33.6MB), run=3737-4001msec