== sanity test 398b: DIO and buffer IO race ============== 03:37:22 (1753429042) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.31548 s, 21.7 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=193372: Fri Jul 25 03:38:34 2025 read: IOPS=24, BW=96.5KiB/s (98.9kB/s)(5992KiB/62066msec) clat (usec): min=72, max=1202.9k, avg=23177.98, stdev=32586.71 lat (usec): min=72, max=1202.9k, avg=23180.73, stdev=32587.21 clat percentiles (usec): | 1.00th=[ 506], 5.00th=[ 8586], 10.00th=[ 10421], | 20.00th=[ 13173], 30.00th=[ 15664], 40.00th=[ 17957], | 50.00th=[ 20317], 60.00th=[ 23200], 70.00th=[ 26346], | 80.00th=[ 30278], 90.00th=[ 36963], 95.00th=[ 43779], | 99.00th=[ 60556], 99.50th=[ 67634], 99.90th=[ 100140], | 99.95th=[1199571], 99.99th=[1199571] bw ( KiB/s): min= 39, max= 168, per=25.86%, avg=97.77, stdev=28.71, samples=121 iops : min= 9, max= 42, avg=24.06, stdev= 7.23, samples=121 write: IOPS=25, BW=101KiB/s (104kB/s)(6296KiB/62066msec); 0 zone resets clat (usec): min=576, max=81984, avg=17245.34, stdev=12967.88 lat (usec): min=577, max=81985, avg=17246.21, stdev=12967.93 clat percentiles (usec): | 1.00th=[ 1237], 5.00th=[ 2671], 10.00th=[ 3884], 20.00th=[ 5604], | 30.00th=[ 8356], 40.00th=[11338], 50.00th=[14091], 60.00th=[17171], | 70.00th=[21365], 80.00th=[27395], 90.00th=[35914], 95.00th=[42730], | 99.00th=[56361], 99.50th=[63701], 99.90th=[79168], 99.95th=[82314], | 99.99th=[82314] bw ( KiB/s): min= 23, max= 239, per=25.99%, avg=102.92, stdev=37.42, samples=121 iops : min= 5, max= 59, avg=25.35, stdev= 9.37, samples=121 lat (usec) : 100=0.03%, 250=0.03%, 500=0.39%, 750=0.39%, 1000=0.39% lat (msec) : 2=0.81%, 4=4.46%, 10=15.92%, 20=35.61%, 50=39.71% lat (msec) : 100=2.18%, 250=0.03%, 2000=0.03% cpu : usr=0.11%, sys=27.34%, ctx=9118, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193373: Fri Jul 25 03:38:34 2025 read: IOPS=23, BW=94.5KiB/s (96.7kB/s)(5940KiB/62885msec) clat (usec): min=76, max=1230.1k, avg=23242.74, stdev=33363.01 lat (usec): min=76, max=1230.1k, avg=23243.44, stdev=33363.03 clat percentiles (usec): | 1.00th=[ 791], 5.00th=[ 8455], 10.00th=[ 10814], | 20.00th=[ 12911], 30.00th=[ 15401], 40.00th=[ 17695], | 50.00th=[ 20579], 60.00th=[ 23462], 70.00th=[ 26608], | 80.00th=[ 30016], 90.00th=[ 36963], 95.00th=[ 43254], | 99.00th=[ 60556], 99.50th=[ 69731], 99.90th=[ 87557], | 99.95th=[1233126], 99.99th=[1233126] bw ( KiB/s): min= 16, max= 208, per=25.35%, avg=95.82, stdev=29.95, samples=123 iops : min= 4, max= 52, avg=23.57, stdev= 7.52, samples=123 write: IOPS=25, BW=101KiB/s (103kB/s)(6348KiB/62885msec); 0 zone resets clat (usec): min=540, max=94594, avg=17745.50, stdev=13492.38 lat (usec): min=540, max=94594, avg=17746.38, stdev=13492.43 clat percentiles (usec): | 1.00th=[ 955], 5.00th=[ 2704], 10.00th=[ 3589], 20.00th=[ 5538], | 30.00th=[ 8455], 40.00th=[11600], 50.00th=[14746], 60.00th=[17957], | 70.00th=[22152], 80.00th=[27919], 90.00th=[36963], 95.00th=[43779], | 99.00th=[60031], 99.50th=[66847], 99.90th=[79168], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 40, max= 199, per=25.84%, avg=102.34, stdev=33.59, samples=123 iops : min= 10, max= 49, avg=25.20, stdev= 8.42, samples=123 lat (usec) : 100=0.03%, 250=0.10%, 500=0.07%, 750=0.46%, 1000=0.52% lat (msec) : 2=0.68%, 4=5.14%, 10=14.84%, 20=34.93%, 50=40.30% lat (msec) : 100=2.90%, 2000=0.03% cpu : usr=0.10%, sys=27.07%, ctx=9213, majf=0, minf=19 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193374: Fri Jul 25 03:38:34 2025 read: IOPS=23, BW=95.8KiB/s (98.1kB/s)(6028KiB/62953msec) clat (usec): min=154, max=105680, avg=22176.86, stdev=11338.75 lat (usec): min=155, max=105682, avg=22177.66, stdev=11338.80 clat percentiles (usec): | 1.00th=[ 619], 5.00th=[ 8029], 10.00th=[ 10159], 20.00th=[ 13173], | 30.00th=[ 15533], 40.00th=[ 17695], 50.00th=[ 20055], 60.00th=[ 22938], | 70.00th=[ 26084], 80.00th=[ 30540], 90.00th=[ 36439], 95.00th=[ 41681], | 99.00th=[ 57410], 99.50th=[ 68682], 99.90th=[ 83362], 99.95th=[105382], | 99.99th=[105382] bw ( KiB/s): min= 32, max= 199, per=25.71%, avg=97.20, stdev=31.57, samples=123 iops : min= 8, max= 49, avg=23.91, stdev= 7.92, samples=123 write: IOPS=24, BW=99.4KiB/s (102kB/s)(6260KiB/62953msec); 0 zone resets clat (usec): min=554, max=1257.1k, avg=18743.99, stdev=34162.26 lat (usec): min=555, max=1257.1k, avg=18745.36, stdev=34162.17 clat percentiles (usec): | 1.00th=[ 1074], 5.00th=[ 2933], 10.00th=[ 4113], | 20.00th=[ 5866], 30.00th=[ 8717], 40.00th=[ 11994], | 50.00th=[ 14877], 60.00th=[ 18220], 70.00th=[ 22414], | 80.00th=[ 28181], 90.00th=[ 35914], 95.00th=[ 44303], | 99.00th=[ 60031], 99.50th=[ 69731], 99.90th=[ 125305], | 99.95th=[1249903], 99.99th=[1249903] bw ( KiB/s): min= 23, max= 198, per=25.48%, avg=100.89, stdev=32.98, samples=123 iops : min= 5, max= 49, avg=24.84, stdev= 8.23, samples=123 lat (usec) : 250=0.10%, 500=0.26%, 750=0.62%, 1000=0.23% lat (msec) : 2=0.62%, 4=3.91%, 10=16.05%, 20=35.35%, 50=40.17% lat (msec) : 100=2.60%, 250=0.07%, 2000=0.03% cpu : usr=0.11%, sys=27.03%, ctx=9200, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193375: Fri Jul 25 03:38:34 2025 read: IOPS=23, BW=95.5KiB/s (97.8kB/s)(6052KiB/63389msec) clat (usec): min=255, max=87361, avg=22694.66, stdev=11738.33 lat (usec): min=255, max=87362, avg=22695.40, stdev=11738.37 clat percentiles (usec): | 1.00th=[ 685], 5.00th=[ 8160], 10.00th=[10159], 20.00th=[13042], | 30.00th=[15795], 40.00th=[18220], 50.00th=[20579], 60.00th=[23462], | 70.00th=[26870], 80.00th=[31327], 90.00th=[38011], 95.00th=[43779], | 99.00th=[57410], 99.50th=[72877], 99.90th=[86508], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 32, max= 184, per=25.59%, avg=96.73, stdev=29.77, samples=124 iops : min= 8, max= 46, avg=23.80, stdev= 7.48, samples=124 write: IOPS=24, BW=98.4KiB/s (101kB/s)(6236KiB/63389msec); 0 zone resets clat (usec): min=582, max=1272.5k, avg=18510.77, stdev=34358.88 lat (usec): min=583, max=1272.5k, avg=18512.13, stdev=34358.74 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 2769], 10.00th=[ 3851], | 20.00th=[ 5800], 30.00th=[ 8979], 40.00th=[ 12125], | 50.00th=[ 15008], 60.00th=[ 18220], 70.00th=[ 22152], | 80.00th=[ 27395], 90.00th=[ 35390], 95.00th=[ 43779], | 99.00th=[ 59507], 99.50th=[ 66847], 99.90th=[ 101188], | 99.95th=[1266680], 99.99th=[1266680] bw ( KiB/s): min= 39, max= 208, per=25.17%, avg=99.66, stdev=32.81, samples=124 iops : min= 9, max= 52, avg=24.52, stdev= 8.23, samples=124 lat (usec) : 500=0.13%, 750=0.65%, 1000=0.29% lat (msec) : 2=1.01%, 4=4.20%, 10=15.30%, 20=35.06%, 50=40.85% lat (msec) : 100=2.44%, 250=0.03%, 2000=0.03% cpu : usr=0.09%, sys=26.94%, ctx=9024, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=379KiB/s (388kB/s), 94.5KiB/s-96.5KiB/s (96.7kB/s-98.9kB/s), io=23.4MiB (24.6MB), run=62066-63389msec WRITE: bw=397KiB/s (406kB/s), 98.4KiB/s-101KiB/s (101kB/s-104kB/s), io=24.6MiB (25.7MB), run=62066-63389msec rand-rw: (groupid=0, jobs=1): err= 0: pid=193377: Fri Jul 25 03:38:41 2025 read: IOPS=21, BW=85.1KiB/s (87.1kB/s)(5992KiB/70439msec) clat (msec): min=2, max=1206, avg=20.85, stdev=34.82 lat (msec): min=2, max=1206, avg=20.85, stdev=34.82 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 32], 90.00th=[ 44], 95.00th=[ 54], | 99.00th=[ 79], 99.50th=[ 88], 99.90th=[ 126], 99.95th=[ 1200], | 99.99th=[ 1200] bw ( KiB/s): min= 23, max= 344, per=25.20%, avg=85.67, stdev=65.91, samples=138 iops : min= 5, max= 86, avg=21.02, stdev=16.61, samples=138 write: IOPS=22, BW=89.4KiB/s (91.5kB/s)(6296KiB/70439msec); 0 zone resets clat (msec): min=3, max=143, avg=24.83, stdev=18.34 lat (msec): min=3, max=143, avg=24.84, stdev=18.34 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 16], 50.00th=[ 21], 60.00th=[ 26], | 70.00th=[ 32], 80.00th=[ 40], 90.00th=[ 50], 95.00th=[ 58], | 99.00th=[ 86], 99.50th=[ 99], 99.90th=[ 114], 99.95th=[ 144], | 99.99th=[ 144] bw ( KiB/s): min= 32, max= 344, per=25.30%, avg=90.08, stdev=67.11, samples=138 iops : min= 8, max= 86, avg=22.12, stdev=16.91, samples=138 lat (msec) : 4=1.14%, 10=30.50%, 20=24.80%, 50=35.22%, 100=8.04% lat (msec) : 250=0.26%, 2000=0.03% cpu : usr=0.09%, sys=13.86%, ctx=4557, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193378: Fri Jul 25 03:38:41 2025 read: IOPS=21, BW=84.5KiB/s (86.5kB/s)(5940KiB/70290msec) clat (msec): min=3, max=1233, avg=21.02, stdev=35.32 lat (msec): min=3, max=1233, avg=21.02, stdev=35.32 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 32], 90.00th=[ 44], 95.00th=[ 53], | 99.00th=[ 78], 99.50th=[ 87], 99.90th=[ 106], 99.95th=[ 1234], | 99.99th=[ 1234] bw ( KiB/s): min= 15, max= 344, per=24.85%, avg=84.49, stdev=63.81, samples=137 iops : min= 3, max= 86, avg=20.72, stdev=16.06, samples=137 write: IOPS=22, BW=90.3KiB/s (92.5kB/s)(6348KiB/70290msec); 0 zone resets clat (msec): min=4, max=143, avg=24.55, stdev=18.03 lat (msec): min=4, max=143, avg=24.55, stdev=18.03 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 26], | 70.00th=[ 31], 80.00th=[ 40], 90.00th=[ 50], 95.00th=[ 58], | 99.00th=[ 79], 99.50th=[ 97], 99.90th=[ 142], 99.95th=[ 144], | 99.99th=[ 144] bw ( KiB/s): min= 23, max= 328, per=25.16%, avg=89.57, stdev=63.93, samples=137 iops : min= 5, max= 82, avg=21.99, stdev=16.12, samples=137 lat (msec) : 4=0.52%, 10=30.44%, 20=25.42%, 50=35.94%, 100=7.45% lat (msec) : 250=0.20%, 2000=0.03% cpu : usr=0.09%, sys=13.85%, ctx=4424, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193379: Fri Jul 25 03:38:41 2025 read: IOPS=21, BW=85.0KiB/s (88.0kB/s)(6028KiB/70133msec) clat (usec): min=2769, max=98023, avg=19785.35, stdev=16013.26 lat (usec): min=2770, max=98024, avg=19786.16, stdev=16013.29 clat percentiles (usec): | 1.00th=[ 3818], 5.00th=[ 4621], 10.00th=[ 5473], 20.00th=[ 6849], | 30.00th=[ 8586], 40.00th=[11600], 50.00th=[15008], 60.00th=[18220], | 70.00th=[23200], 80.00th=[30802], 90.00th=[41157], 95.00th=[51643], | 99.00th=[77071], 99.50th=[85459], 99.90th=[92799], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 23, max= 328, per=25.36%, avg=86.24, stdev=64.55, samples=137 iops : min= 5, max= 82, avg=21.16, stdev=16.28, samples=137 write: IOPS=22, BW=89.3KiB/s (91.4kB/s)(6260KiB/70133msec); 0 zone resets clat (msec): min=4, max=1241, avg=25.69, stdev=35.66 lat (msec): min=4, max=1241, avg=25.69, stdev=35.66 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 12], 40.00th=[ 17], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 32], 80.00th=[ 39], 90.00th=[ 51], 95.00th=[ 61], | 99.00th=[ 82], 99.50th=[ 95], 99.90th=[ 142], 99.95th=[ 1250], | 99.99th=[ 1250] bw ( KiB/s): min= 23, max= 320, per=25.06%, avg=89.22, stdev=62.51, samples=137 iops : min= 5, max= 80, avg=21.91, stdev=15.76, samples=137 lat (msec) : 4=0.81%, 10=29.43%, 20=26.30%, 50=35.61%, 100=7.65% lat (msec) : 250=0.16%, 2000=0.03% cpu : usr=0.09%, sys=14.10%, ctx=4644, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193380: Fri Jul 25 03:38:41 2025 read: IOPS=21, BW=85.0KiB/s (88.0kB/s)(6052KiB/70405msec) clat (msec): min=2, max=107, avg=20.40, stdev=16.90 lat (msec): min=2, max=107, avg=20.40, stdev=16.90 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 33], 90.00th=[ 47], 95.00th=[ 56], | 99.00th=[ 74], 99.50th=[ 85], 99.90th=[ 100], 99.95th=[ 108], | 99.99th=[ 108] bw ( KiB/s): min= 16, max= 376, per=25.56%, avg=86.89, stdev=68.99, samples=138 iops : min= 4, max= 94, avg=21.33, stdev=17.37, samples=138 write: IOPS=22, BW=88.6KiB/s (90.7kB/s)(6236KiB/70405msec); 0 zone resets clat (msec): min=3, max=1242, avg=25.29, stdev=35.79 lat (msec): min=3, max=1242, avg=25.29, stdev=35.79 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 25], | 70.00th=[ 32], 80.00th=[ 40], 90.00th=[ 50], 95.00th=[ 59], | 99.00th=[ 84], 99.50th=[ 101], 99.90th=[ 126], 99.95th=[ 1250], | 99.99th=[ 1250] bw ( KiB/s): min= 24, max= 344, per=25.09%, avg=89.33, stdev=67.89, samples=138 iops : min= 6, max= 86, avg=21.93, stdev=17.11, samples=138 lat (msec) : 4=1.43%, 10=30.08%, 20=25.88%, 50=34.31%, 100=8.04% lat (msec) : 250=0.23%, 2000=0.03% cpu : usr=0.08%, sys=13.85%, ctx=4540, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=341KiB/s (349kB/s), 84.5KiB/s-85.0KiB/s (86.5kB/s-88.0kB/s), io=23.4MiB (24.6MB), run=70133-70439msec WRITE: bw=357KiB/s (365kB/s), 88.6KiB/s-90.3KiB/s (90.7kB/s-92.5kB/s), io=24.6MiB (25.7MB), run=70133-70439msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=193406: Fri Jul 25 03:39:05 2025 read: IOPS=20, BW=332KiB/s (340kB/s)(5920KiB/17821msec) clat (usec): min=126, max=145087, avg=25742.11, stdev=15848.39 lat (usec): min=127, max=145088, avg=25743.18, stdev=15848.45 clat percentiles (usec): | 1.00th=[ 147], 5.00th=[ 693], 10.00th=[ 10159], 20.00th=[ 15270], | 30.00th=[ 17695], 40.00th=[ 20841], 50.00th=[ 23200], 60.00th=[ 26346], | 70.00th=[ 30540], 80.00th=[ 34866], 90.00th=[ 43779], 95.00th=[ 49546], | 99.00th=[ 80217], 99.50th=[104334], 99.90th=[145753], 99.95th=[145753], | 99.99th=[145753] bw ( KiB/s): min= 160, max= 542, per=24.30%, avg=326.53, stdev=80.66, samples=34 iops : min= 10, max= 33, avg=19.82, stdev= 5.13, samples=34 write: IOPS=22, BW=357KiB/s (366kB/s)(6368KiB/17821msec); 0 zone resets clat (usec): min=1189, max=124292, avg=20549.37, stdev=16091.61 lat (usec): min=1190, max=124294, avg=20552.97, stdev=16090.21 clat percentiles (usec): | 1.00th=[ 1450], 5.00th=[ 2868], 10.00th=[ 4359], 20.00th=[ 6521], | 30.00th=[ 9503], 40.00th=[ 12780], 50.00th=[ 16188], 60.00th=[ 20841], | 70.00th=[ 26346], 80.00th=[ 33162], 90.00th=[ 41681], 95.00th=[ 52691], | 99.00th=[ 72877], 99.50th=[ 78119], 99.90th=[124257], 99.95th=[124257], | 99.99th=[124257] bw ( KiB/s): min= 127, max= 480, per=25.20%, avg=356.32, stdev=99.81, samples=34 iops : min= 7, max= 30, avg=21.68, stdev= 6.29, samples=34 lat (usec) : 250=1.04%, 500=0.78%, 750=0.78%, 1000=0.13% lat (msec) : 2=2.47%, 4=2.47%, 10=13.28%, 20=26.17%, 50=47.66% lat (msec) : 100=4.82%, 250=0.39% cpu : usr=0.10%, sys=22.73%, ctx=2436, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193407: Fri Jul 25 03:39:05 2025 read: IOPS=20, BW=324KiB/s (332kB/s)(5744KiB/17722msec) clat (usec): min=123, max=92195, avg=25110.74, stdev=13464.23 lat (usec): min=124, max=92196, avg=25111.62, stdev=13464.27 clat percentiles (usec): | 1.00th=[ 157], 5.00th=[ 490], 10.00th=[11076], 20.00th=[15401], | 30.00th=[18220], 40.00th=[21103], 50.00th=[23725], 60.00th=[26608], | 70.00th=[30278], 80.00th=[34341], 90.00th=[40633], 95.00th=[47449], | 99.00th=[70779], 99.50th=[77071], 99.90th=[91751], 99.95th=[91751], | 99.99th=[91751] bw ( KiB/s): min= 189, max= 508, per=24.29%, avg=326.50, stdev=78.87, samples=34 iops : min= 11, max= 31, avg=19.82, stdev= 4.96, samples=34 write: IOPS=23, BW=369KiB/s (378kB/s)(6544KiB/17722msec); 0 zone resets clat (usec): min=1247, max=100658, avg=21013.13, stdev=17339.22 lat (usec): min=1248, max=100659, avg=21029.80, stdev=17331.96 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 1795], 10.00th=[ 3720], 20.00th=[ 6456], | 30.00th=[ 9896], 40.00th=[ 13173], 50.00th=[ 16581], 60.00th=[ 20579], | 70.00th=[ 25297], 80.00th=[ 33162], 90.00th=[ 44827], 95.00th=[ 56886], | 99.00th=[ 73925], 99.50th=[ 92799], 99.90th=[100140], 99.95th=[100140], | 99.99th=[100140] bw ( KiB/s): min= 224, max= 608, per=26.05%, avg=368.41, stdev=94.45, samples=34 iops : min= 14, max= 38, avg=22.44, stdev= 6.01, samples=34 lat (usec) : 250=1.30%, 500=1.04%, 750=0.26%, 1000=0.26% lat (msec) : 2=3.12%, 4=3.26%, 10=10.94%, 20=28.13%, 50=45.96% lat (msec) : 100=5.60%, 250=0.13% cpu : usr=0.12%, sys=22.96%, ctx=2301, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193409: Fri Jul 25 03:39:05 2025 read: IOPS=22, BW=353KiB/s (361kB/s)(6256KiB/17723msec) clat (usec): min=113, max=151208, avg=24902.11, stdev=15038.05 lat (usec): min=114, max=151209, avg=24903.11, stdev=15038.08 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 494], 10.00th=[ 11863], 20.00th=[ 14615], | 30.00th=[ 17433], 40.00th=[ 20055], 50.00th=[ 23200], 60.00th=[ 26346], | 70.00th=[ 29492], 80.00th=[ 34341], 90.00th=[ 41157], 95.00th=[ 50070], | 99.00th=[ 63701], 99.50th=[108528], 99.90th=[152044], 99.95th=[152044], | 99.99th=[152044] bw ( KiB/s): min= 192, max= 512, per=26.49%, avg=356.06, stdev=84.45, samples=34 iops : min= 12, max= 32, avg=21.68, stdev= 5.34, samples=34 write: IOPS=21, BW=340KiB/s (349kB/s)(6032KiB/17723msec); 0 zone resets clat (usec): min=1527, max=103979, avg=21095.73, stdev=16349.98 lat (usec): min=1528, max=103981, avg=21097.89, stdev=16349.86 clat percentiles (usec): | 1.00th=[ 1598], 5.00th=[ 3097], 10.00th=[ 4359], 20.00th=[ 7046], | 30.00th=[ 10159], 40.00th=[ 13698], 50.00th=[ 17171], 60.00th=[ 20841], | 70.00th=[ 26608], 80.00th=[ 32900], 90.00th=[ 44827], 95.00th=[ 52167], | 99.00th=[ 81265], 99.50th=[ 95945], 99.90th=[104334], 99.95th=[104334], | 99.99th=[104334] bw ( KiB/s): min= 192, max= 508, per=23.74%, avg=335.65, stdev=83.97, samples=34 iops : min= 12, max= 31, avg=20.38, stdev= 5.08, samples=34 lat (usec) : 250=1.17%, 500=1.43%, 750=0.65%, 1000=0.39% lat (msec) : 2=1.43%, 4=3.12%, 10=10.29%, 20=30.21%, 50=45.57% lat (msec) : 100=5.34%, 250=0.39% cpu : usr=0.08%, sys=22.77%, ctx=2527, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193411: Fri Jul 25 03:39:05 2025 read: IOPS=21, BW=351KiB/s (359kB/s)(6032KiB/17196msec) clat (usec): min=123, max=106396, avg=23700.12, stdev=13943.91 lat (usec): min=124, max=106397, avg=23701.03, stdev=13943.90 clat percentiles (usec): | 1.00th=[ 133], 5.00th=[ 247], 10.00th=[ 807], 20.00th=[ 13435], | 30.00th=[ 17695], 40.00th=[ 20317], 50.00th=[ 22676], 60.00th=[ 26084], | 70.00th=[ 28705], 80.00th=[ 32900], 90.00th=[ 40633], 95.00th=[ 46400], | 99.00th=[ 56886], 99.50th=[ 82314], 99.90th=[106431], 99.95th=[106431], | 99.99th=[106431] bw ( KiB/s): min= 220, max= 536, per=25.99%, avg=349.30, stdev=82.74, samples=33 iops : min= 13, max= 33, avg=21.27, stdev= 5.10, samples=33 write: IOPS=22, BW=364KiB/s (373kB/s)(6256KiB/17196msec); 0 zone resets clat (usec): min=1376, max=84899, avg=20855.49, stdev=16355.30 lat (usec): min=1378, max=84901, avg=20857.37, stdev=16355.44 clat percentiles (usec): | 1.00th=[ 1500], 5.00th=[ 2769], 10.00th=[ 4178], 20.00th=[ 6521], | 30.00th=[ 9241], 40.00th=[13304], 50.00th=[16581], 60.00th=[19792], | 70.00th=[26608], 80.00th=[34866], 90.00th=[43779], 95.00th=[52167], | 99.00th=[72877], 99.50th=[82314], 99.90th=[84411], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 158, max= 544, per=25.69%, avg=363.27, stdev=97.98, samples=33 iops : min= 9, max= 34, avg=22.12, stdev= 6.29, samples=33 lat (usec) : 250=2.47%, 500=1.56%, 750=0.65%, 1000=0.39% lat (msec) : 2=2.21%, 4=3.12%, 10=12.24%, 20=27.08%, 50=45.57% lat (msec) : 100=4.56%, 250=0.13% cpu : usr=0.12%, sys=23.20%, ctx=2377, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1344KiB/s (1376kB/s), 324KiB/s-353KiB/s (332kB/s-361kB/s), io=23.4MiB (24.5MB), run=17196-17821msec WRITE: bw=1414KiB/s (1448kB/s), 340KiB/s-369KiB/s (349kB/s-378kB/s), io=24.6MiB (25.8MB), run=17196-17821msec rand-rw: (groupid=0, jobs=1): err= 0: pid=193408: Fri Jul 25 03:39:11 2025 read: IOPS=16, BW=258KiB/s (264kB/s)(5920KiB/22987msec) clat (msec): min=5, max=118, avg=23.88, stdev=17.58 lat (msec): min=5, max=118, avg=23.88, stdev=17.58 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 16], 50.00th=[ 18], 60.00th=[ 22], | 70.00th=[ 26], 80.00th=[ 33], 90.00th=[ 51], 95.00th=[ 62], | 99.00th=[ 85], 99.50th=[ 93], 99.90th=[ 118], 99.95th=[ 118], | 99.99th=[ 118] bw ( KiB/s): min= 62, max= 576, per=24.85%, avg=255.73, stdev=133.43, samples=45 iops : min= 3, max= 36, avg=15.47, stdev= 8.48, samples=45 write: IOPS=17, BW=277KiB/s (284kB/s)(6368KiB/22987msec); 0 zone resets clat (msec): min=7, max=152, avg=35.20, stdev=21.86 lat (msec): min=7, max=152, avg=35.21, stdev=21.86 clat percentiles (msec): | 1.00th=[ 11], 5.00th=[ 14], 10.00th=[ 16], 20.00th=[ 19], | 30.00th=[ 21], 40.00th=[ 24], 50.00th=[ 28], 60.00th=[ 35], | 70.00th=[ 41], 80.00th=[ 51], 90.00th=[ 65], 95.00th=[ 80], | 99.00th=[ 123], 99.50th=[ 133], 99.90th=[ 153], 99.95th=[ 153], | 99.99th=[ 153] bw ( KiB/s): min= 124, max= 576, per=25.58%, avg=276.98, stdev=115.01, samples=45 iops : min= 7, max= 36, avg=16.80, stdev= 7.29, samples=45 lat (msec) : 10=7.55%, 20=33.85%, 50=42.84%, 100=14.97%, 250=0.78% cpu : usr=0.09%, sys=8.47%, ctx=1170, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193410: Fri Jul 25 03:39:11 2025 read: IOPS=15, BW=247KiB/s (253kB/s)(5744KiB/23256msec) clat (msec): min=5, max=147, avg=26.41, stdev=21.84 lat (msec): min=5, max=147, avg=26.41, stdev=21.84 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 27], 80.00th=[ 39], 90.00th=[ 55], 95.00th=[ 70], | 99.00th=[ 117], 99.50th=[ 136], 99.90th=[ 148], 99.95th=[ 148], | 99.99th=[ 148] bw ( KiB/s): min= 31, max= 573, per=23.92%, avg=246.18, stdev=118.96, samples=45 iops : min= 1, max= 35, avg=14.87, stdev= 7.53, samples=45 write: IOPS=17, BW=281KiB/s (288kB/s)(6544KiB/23256msec); 0 zone resets clat (msec): min=8, max=135, avg=33.34, stdev=20.10 lat (msec): min=8, max=135, avg=33.34, stdev=20.10 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 13], 10.00th=[ 15], 20.00th=[ 19], | 30.00th=[ 21], 40.00th=[ 24], 50.00th=[ 28], 60.00th=[ 32], | 70.00th=[ 38], 80.00th=[ 48], 90.00th=[ 64], 95.00th=[ 73], | 99.00th=[ 99], 99.50th=[ 121], 99.90th=[ 136], 99.95th=[ 136], | 99.99th=[ 136] bw ( KiB/s): min= 93, max= 704, per=25.34%, avg=274.40, stdev=124.51, samples=45 iops : min= 5, max= 44, avg=16.62, stdev= 7.87, samples=45 lat (msec) : 10=6.51%, 20=31.51%, 50=47.14%, 100=13.54%, 250=1.30% cpu : usr=0.08%, sys=8.72%, ctx=1153, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193412: Fri Jul 25 03:39:11 2025 read: IOPS=16, BW=270KiB/s (276kB/s)(6256KiB/23192msec) clat (msec): min=4, max=140, avg=25.87, stdev=20.96 lat (msec): min=4, max=140, avg=25.87, stdev=20.96 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 16], 50.00th=[ 19], 60.00th=[ 22], | 70.00th=[ 27], 80.00th=[ 37], 90.00th=[ 57], 95.00th=[ 71], | 99.00th=[ 108], 99.50th=[ 122], 99.90th=[ 142], 99.95th=[ 142], | 99.99th=[ 142] bw ( KiB/s): min= 95, max= 606, per=26.15%, avg=269.13, stdev=129.03, samples=45 iops : min= 5, max= 37, avg=16.31, stdev= 8.11, samples=45 write: IOPS=16, BW=260KiB/s (266kB/s)(6032KiB/23192msec); 0 zone resets clat (msec): min=8, max=118, avg=34.35, stdev=19.85 lat (msec): min=8, max=118, avg=34.36, stdev=19.85 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 14], 10.00th=[ 16], 20.00th=[ 18], | 30.00th=[ 22], 40.00th=[ 25], 50.00th=[ 28], 60.00th=[ 33], | 70.00th=[ 41], 80.00th=[ 50], 90.00th=[ 63], 95.00th=[ 75], | 99.00th=[ 102], 99.50th=[ 114], 99.90th=[ 118], 99.95th=[ 118], | 99.99th=[ 118] bw ( KiB/s): min= 31, max= 672, per=23.57%, avg=255.31, stdev=120.17, samples=45 iops : min= 1, max= 42, avg=15.42, stdev= 7.68, samples=45 lat (msec) : 10=7.16%, 20=33.46%, 50=42.58%, 100=15.76%, 250=1.04% cpu : usr=0.07%, sys=8.50%, ctx=1096, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193413: Fri Jul 25 03:39:11 2025 read: IOPS=16, BW=264KiB/s (270kB/s)(6032KiB/22869msec) clat (msec): min=4, max=113, avg=24.44, stdev=17.89 lat (msec): min=4, max=113, avg=24.44, stdev=17.89 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 27], 80.00th=[ 33], 90.00th=[ 48], 95.00th=[ 59], | 99.00th=[ 103], 99.50th=[ 112], 99.90th=[ 114], 99.95th=[ 114], | 99.99th=[ 114] bw ( KiB/s): min= 63, max= 640, per=25.71%, avg=264.51, stdev=138.40, samples=45 iops : min= 3, max= 40, avg=16.00, stdev= 8.73, samples=45 write: IOPS=17, BW=274KiB/s (280kB/s)(6256KiB/22869msec); 0 zone resets clat (msec): min=9, max=154, avg=34.81, stdev=21.39 lat (msec): min=9, max=154, avg=34.81, stdev=21.39 clat percentiles (msec): | 1.00th=[ 11], 5.00th=[ 13], 10.00th=[ 16], 20.00th=[ 19], | 30.00th=[ 22], 40.00th=[ 25], 50.00th=[ 29], 60.00th=[ 34], | 70.00th=[ 40], 80.00th=[ 49], 90.00th=[ 65], 95.00th=[ 78], | 99.00th=[ 114], 99.50th=[ 136], 99.90th=[ 155], 99.95th=[ 155], | 99.99th=[ 155] bw ( KiB/s): min= 93, max= 608, per=25.34%, avg=274.44, stdev=118.66, samples=45 iops : min= 5, max= 38, avg=16.62, stdev= 7.56, samples=45 lat (msec) : 10=6.77%, 20=32.16%, 50=47.14%, 100=12.63%, 250=1.30% cpu : usr=0.07%, sys=8.77%, ctx=1134, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1030KiB/s (1055kB/s), 247KiB/s-270KiB/s (253kB/s-276kB/s), io=23.4MiB (24.5MB), run=22869-23256msec WRITE: bw=1084KiB/s (1110kB/s), 260KiB/s-281KiB/s (266kB/s-288kB/s), io=24.6MiB (25.8MB), run=22869-23256msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=193424: Fri Jul 25 03:39:22 2025 read: IOPS=1, BW=1145KiB/s (1172kB/s)(7168KiB/6262msec) clat (msec): min=37, max=1938, avg=727.90, stdev=802.99 lat (msec): min=37, max=1938, avg=727.90, stdev=803.00 clat percentiles (msec): | 1.00th=[ 38], 5.00th=[ 38], 10.00th=[ 38], 20.00th=[ 40], | 30.00th=[ 78], 40.00th=[ 78], 50.00th=[ 284], 60.00th=[ 1234], | 70.00th=[ 1234], 80.00th=[ 1485], 90.00th=[ 1938], 95.00th=[ 1938], | 99.00th=[ 1938], 99.50th=[ 1938], 99.90th=[ 1938], 99.95th=[ 1938], | 99.99th=[ 1938] bw ( KiB/s): min= 2000, max= 2048, per=62.07%, avg=2029.80, stdev=17.85, samples=5 iops : min= 1, max= 2, avg= 1.20, stdev= 0.45, samples=5 write: IOPS=0, BW=818KiB/s (837kB/s)(5120KiB/6262msec); 0 zone resets clat (msec): min=41, max=407, avg=191.96, stdev=146.44 lat (msec): min=41, max=408, avg=192.02, stdev=146.44 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 42], 10.00th=[ 42], 20.00th=[ 42], | 30.00th=[ 81], 40.00th=[ 81], 50.00th=[ 176], 60.00th=[ 176], | 70.00th=[ 255], 80.00th=[ 255], 90.00th=[ 409], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 2031, max= 2035, per=44.41%, avg=2033.00, stdev= 2.83, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 50=25.00%, 100=16.67%, 250=8.33%, 500=25.00%, 2000=25.00% cpu : usr=0.02%, sys=4.18%, ctx=63, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193426: Fri Jul 25 03:39:22 2025 read: IOPS=0, BW=356KiB/s (365kB/s)(2048KiB/5750msec) clat (msec): min=60, max=327, avg=193.73, stdev=188.64 lat (msec): min=60, max=327, avg=193.73, stdev=188.64 clat percentiles (msec): | 1.00th=[ 61], 5.00th=[ 61], 10.00th=[ 61], 20.00th=[ 61], | 30.00th=[ 61], 40.00th=[ 61], 50.00th=[ 61], 60.00th=[ 326], | 70.00th=[ 326], 80.00th=[ 326], 90.00th=[ 326], 95.00th=[ 326], | 99.00th=[ 326], 99.50th=[ 326], 99.90th=[ 326], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 2035, max= 2048, per=62.43%, avg=2041.50, stdev= 9.19, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1781KiB/s (1824kB/s)(10.0MiB/5750msec); 0 zone resets clat (msec): min=41, max=2060, avg=514.18, stdev=628.54 lat (msec): min=41, max=2060, avg=514.26, stdev=628.53 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 42], 10.00th=[ 42], 20.00th=[ 108], | 30.00th=[ 203], 40.00th=[ 266], 50.00th=[ 279], 60.00th=[ 326], | 70.00th=[ 334], 80.00th=[ 334], 90.00th=[ 1200], 95.00th=[ 2056], | 99.00th=[ 2056], 99.50th=[ 2056], 99.90th=[ 2056], 99.95th=[ 2056], | 99.99th=[ 2056] bw ( KiB/s): min= 2035, max= 6144, per=80.34%, avg=3677.80, stdev=1718.69, samples=5 iops : min= 1, max= 6, avg= 3.00, stdev= 2.12, samples=5 lat (msec) : 50=8.33%, 100=8.33%, 250=16.67%, 500=50.00%, 2000=8.33% lat (msec) : >=2000=8.33% cpu : usr=0.02%, sys=6.51%, ctx=111, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193428: Fri Jul 25 03:39:22 2025 read: IOPS=1, BW=1148KiB/s (1176kB/s)(6144KiB/5352msec) clat (msec): min=64, max=122, avg=91.65, stdev=25.12 lat (msec): min=64, max=122, avg=91.65, stdev=25.12 clat percentiles (msec): | 1.00th=[ 65], 5.00th=[ 65], 10.00th=[ 65], 20.00th=[ 73], | 30.00th=[ 73], 40.00th=[ 79], 50.00th=[ 79], 60.00th=[ 90], | 70.00th=[ 123], 80.00th=[ 123], 90.00th=[ 123], 95.00th=[ 123], | 99.00th=[ 123], 99.50th=[ 123], 99.90th=[ 123], 99.95th=[ 123], | 99.99th=[ 123] bw ( KiB/s): min= 2035, max= 6144, per=100.00%, avg=3404.67, stdev=2372.33, samples=3 iops : min= 1, max= 6, avg= 2.67, stdev= 2.89, samples=3 write: IOPS=1, BW=1148KiB/s (1176kB/s)(6144KiB/5352msec); 0 zone resets clat (msec): min=83, max=2415, avg=763.66, stdev=936.58 lat (msec): min=83, max=2415, avg=763.73, stdev=936.57 clat percentiles (msec): | 1.00th=[ 84], 5.00th=[ 84], 10.00th=[ 84], 20.00th=[ 182], | 30.00th=[ 182], 40.00th=[ 197], 50.00th=[ 197], 60.00th=[ 347], | 70.00th=[ 1368], 80.00th=[ 1368], 90.00th=[ 2400], 95.00th=[ 2400], | 99.00th=[ 2400], 99.50th=[ 2400], 99.90th=[ 2400], 99.95th=[ 2400], | 99.99th=[ 2400] bw ( KiB/s): min= 2035, max= 4096, per=74.46%, avg=3409.00, stdev=1189.92, samples=3 iops : min= 1, max= 4, avg= 3.00, stdev= 1.73, samples=3 lat (msec) : 100=41.67%, 250=33.33%, 500=8.33%, 2000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=5.40%, ctx=72, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193429: Fri Jul 25 03:39:22 2025 read: IOPS=0, BW=842KiB/s (862kB/s)(5120KiB/6082msec) clat (msec): min=74, max=268, avg=148.62, stdev=72.90 lat (msec): min=74, max=268, avg=148.63, stdev=72.90 clat percentiles (msec): | 1.00th=[ 75], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 75], | 30.00th=[ 111], 40.00th=[ 111], 50.00th=[ 144], 60.00th=[ 144], | 70.00th=[ 146], 80.00th=[ 146], 90.00th=[ 268], 95.00th=[ 268], | 99.00th=[ 268], 99.50th=[ 268], 99.90th=[ 268], 99.95th=[ 268], | 99.99th=[ 268] bw ( KiB/s): min= 2035, max= 6144, per=100.00%, avg=4089.50, stdev=2905.50, samples=2 iops : min= 1, max= 6, avg= 3.50, stdev= 3.54, samples=2 write: IOPS=1, BW=1179KiB/s (1207kB/s)(7168KiB/6082msec); 0 zone resets clat (msec): min=188, max=2096, avg=756.16, stdev=732.15 lat (msec): min=188, max=2096, avg=756.23, stdev=732.15 clat percentiles (msec): | 1.00th=[ 190], 5.00th=[ 190], 10.00th=[ 190], 20.00th=[ 207], | 30.00th=[ 284], 40.00th=[ 284], 50.00th=[ 527], 60.00th=[ 550], | 70.00th=[ 550], 80.00th=[ 1452], 90.00th=[ 2089], 95.00th=[ 2089], | 99.00th=[ 2089], 99.50th=[ 2089], 99.90th=[ 2089], 99.95th=[ 2089], | 99.99th=[ 2089] bw ( KiB/s): min= 2035, max= 4079, per=62.35%, avg=2854.60, stdev=1110.44, samples=5 iops : min= 1, max= 3, avg= 2.20, stdev= 0.84, samples=5 lat (msec) : 100=8.33%, 250=41.67%, 500=16.67%, 750=16.67%, 2000=8.33% lat (msec) : >=2000=8.33% cpu : usr=0.02%, sys=4.32%, ctx=75, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3271KiB/s (3349kB/s), 356KiB/s-1148KiB/s (365kB/s-1176kB/s), io=20.0MiB (20.0MB), run=5352-6262msec WRITE: bw=4579KiB/s (4689kB/s), 818KiB/s-1781KiB/s (837kB/s-1824kB/s), io=28.0MiB (29.4MB), run=5352-6262msec rand-rw: (groupid=0, jobs=1): err= 0: pid=193422: Fri Jul 25 03:39:22 2025 read: IOPS=1, BW=1217KiB/s (1246kB/s)(7168KiB/5890msec) clat (msec): min=84, max=2173, avg=679.23, stdev=976.32 lat (msec): min=84, max=2173, avg=679.23, stdev=976.32 clat percentiles (msec): | 1.00th=[ 85], 5.00th=[ 85], 10.00th=[ 85], 20.00th=[ 89], | 30.00th=[ 114], 40.00th=[ 114], 50.00th=[ 126], 60.00th=[ 129], | 70.00th=[ 129], 80.00th=[ 2039], 90.00th=[ 2165], 95.00th=[ 2165], | 99.00th=[ 2165], 99.50th=[ 2165], 99.90th=[ 2165], 99.95th=[ 2165], | 99.99th=[ 2165] bw ( KiB/s): min= 2048, max= 4096, per=92.31%, avg=3063.75, stdev=1172.96, samples=4 iops : min= 2, max= 4, avg= 2.75, stdev= 0.96, samples=4 write: IOPS=0, BW=869KiB/s (890kB/s)(5120KiB/5890msec); 0 zone resets clat (msec): min=82, max=387, avg=208.75, stdev=129.38 lat (msec): min=82, max=387, avg=208.82, stdev=129.38 clat percentiles (msec): | 1.00th=[ 83], 5.00th=[ 83], 10.00th=[ 83], 20.00th=[ 83], | 30.00th=[ 103], 40.00th=[ 103], 50.00th=[ 180], 60.00th=[ 180], | 70.00th=[ 292], 80.00th=[ 292], 90.00th=[ 388], 95.00th=[ 388], | 99.00th=[ 388], 99.50th=[ 388], 99.90th=[ 388], 99.95th=[ 388], | 99.99th=[ 388] bw ( KiB/s): min= 2048, max= 4096, per=58.76%, avg=2730.67, stdev=1182.41, samples=3 iops : min= 2, max= 4, avg= 2.67, stdev= 1.15, samples=3 lat (msec) : 100=25.00%, 250=41.67%, 500=16.67%, >=2000=16.67% cpu : usr=0.00%, sys=15.54%, ctx=1008, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193423: Fri Jul 25 03:39:22 2025 read: IOPS=0, BW=376KiB/s (385kB/s)(2048KiB/5446msec) clat (msec): min=135, max=146, avg=141.04, stdev= 7.97 lat (msec): min=135, max=146, avg=141.05, stdev= 7.98 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 136], | 30.00th=[ 136], 40.00th=[ 136], 50.00th=[ 136], 60.00th=[ 146], | 70.00th=[ 146], 80.00th=[ 146], 90.00th=[ 146], 95.00th=[ 146], | 99.00th=[ 146], 99.50th=[ 146], 99.90th=[ 146], 99.95th=[ 146], | 99.99th=[ 146] bw ( KiB/s): min= 1861, max= 2039, per=58.75%, avg=1950.00, stdev=125.87, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1880KiB/s (1925kB/s)(10.0MiB/5446msec); 0 zone resets clat (msec): min=79, max=2052, avg=494.39, stdev=639.44 lat (msec): min=79, max=2052, avg=495.03, stdev=639.14 clat percentiles (msec): | 1.00th=[ 81], 5.00th=[ 81], 10.00th=[ 81], 20.00th=[ 146], | 30.00th=[ 174], 40.00th=[ 176], 50.00th=[ 180], 60.00th=[ 190], | 70.00th=[ 232], 80.00th=[ 523], 90.00th=[ 1200], 95.00th=[ 2056], | 99.00th=[ 2056], 99.50th=[ 2056], 99.90th=[ 2056], 99.95th=[ 2056], | 99.99th=[ 2056] bw ( KiB/s): min= 2031, max= 6119, per=87.37%, avg=4060.00, stdev=2336.15, samples=4 iops : min= 1, max= 5, avg= 3.00, stdev= 2.31, samples=4 lat (msec) : 100=8.33%, 250=66.67%, 750=8.33%, 2000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=14.66%, ctx=233, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193425: Fri Jul 25 03:39:22 2025 read: IOPS=1, BW=1285KiB/s (1316kB/s)(6144KiB/4781msec) clat (msec): min=71, max=174, avg=126.69, stdev=36.38 lat (msec): min=71, max=174, avg=126.69, stdev=36.38 clat percentiles (msec): | 1.00th=[ 72], 5.00th=[ 72], 10.00th=[ 72], 20.00th=[ 114], | 30.00th=[ 114], 40.00th=[ 120], 50.00th=[ 120], 60.00th=[ 123], | 70.00th=[ 159], 80.00th=[ 159], 90.00th=[ 176], 95.00th=[ 176], | 99.00th=[ 176], 99.50th=[ 176], 99.90th=[ 176], 99.95th=[ 176], | 99.99th=[ 176] bw ( KiB/s): min= 2043, max= 6047, per=100.00%, avg=4045.00, stdev=2831.26, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=1, BW=1285KiB/s (1316kB/s)(6144KiB/4781msec); 0 zone resets clat (msec): min=78, max=2051, avg=667.73, stdev=841.09 lat (msec): min=78, max=2051, avg=667.80, stdev=841.08 clat percentiles (msec): | 1.00th=[ 79], 5.00th=[ 79], 10.00th=[ 79], 20.00th=[ 116], | 30.00th=[ 116], 40.00th=[ 176], 50.00th=[ 176], 60.00th=[ 205], | 70.00th=[ 1385], 80.00th=[ 1385], 90.00th=[ 2056], 95.00th=[ 2056], | 99.00th=[ 2056], 99.50th=[ 2056], 99.90th=[ 2056], 99.95th=[ 2056], | 99.99th=[ 2056] bw ( KiB/s): min= 2015, max= 3710, per=52.74%, avg=2451.00, stdev=839.44, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 100=16.67%, 250=66.67%, 2000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=12.07%, ctx=386, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193427: Fri Jul 25 03:39:22 2025 read: IOPS=0, BW=830KiB/s (850kB/s)(5120KiB/6169msec) clat (msec): min=119, max=218, avg=150.49, stdev=39.37 lat (msec): min=119, max=218, avg=150.49, stdev=39.37 clat percentiles (msec): | 1.00th=[ 121], 5.00th=[ 121], 10.00th=[ 121], 20.00th=[ 121], | 30.00th=[ 128], 40.00th=[ 128], 50.00th=[ 144], 60.00th=[ 144], | 70.00th=[ 144], 80.00th=[ 144], 90.00th=[ 220], 95.00th=[ 220], | 99.00th=[ 220], 99.50th=[ 220], 99.90th=[ 220], 99.95th=[ 220], | 99.99th=[ 220] bw ( KiB/s): min= 4079, max= 4087, per=100.00%, avg=4083.00, stdev= 5.66, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1162KiB/s (1190kB/s)(7168KiB/6169msec); 0 zone resets clat (msec): min=116, max=2131, avg=755.17, stdev=748.00 lat (msec): min=116, max=2131, avg=755.25, stdev=748.00 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 148], | 30.00th=[ 317], 40.00th=[ 317], 50.00th=[ 550], 60.00th=[ 609], | 70.00th=[ 609], 80.00th=[ 1418], 90.00th=[ 2140], 95.00th=[ 2140], | 99.00th=[ 2140], 99.50th=[ 2140], 99.90th=[ 2140], 99.95th=[ 2140], | 99.99th=[ 2140] bw ( KiB/s): min= 2015, max= 4096, per=61.50%, avg=2857.80, stdev=1126.28, samples=5 iops : min= 1, max= 4, avg= 2.20, stdev= 1.30, samples=5 lat (msec) : 250=58.33%, 500=8.33%, 750=16.67%, 2000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=11.20%, ctx=331, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3320KiB/s (3400kB/s), 376KiB/s-1285KiB/s (385kB/s-1316kB/s), io=20.0MiB (20.0MB), run=4781-6169msec WRITE: bw=4648KiB/s (4759kB/s), 869KiB/s-1880KiB/s (890kB/s-1925kB/s), io=28.0MiB (29.4MB), run=4781-6169msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=193441: Fri Jul 25 03:39:32 2025 read: IOPS=2, BW=9191KiB/s (9411kB/s)(12.0MiB/1337msec) clat (msec): min=371, max=481, avg=434.63, stdev=56.79 lat (msec): min=371, max=481, avg=434.63, stdev=56.79 clat percentiles (msec): | 1.00th=[ 372], 5.00th=[ 372], 10.00th=[ 372], 20.00th=[ 372], | 30.00th=[ 372], 40.00th=[ 451], 50.00th=[ 451], 60.00th=[ 451], | 70.00th=[ 481], 80.00th=[ 481], 90.00th=[ 481], 95.00th=[ 481], | 99.00th=[ 481], 99.50th=[ 481], 99.90th=[ 481], 99.95th=[ 481], | 99.99th=[ 481] bw ( KiB/s): min= 5587, max= 8159, per=100.00%, avg=6873.00, stdev=1818.68, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=100.00% cpu : usr=0.00%, sys=41.69%, ctx=1254, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193442: Fri Jul 25 03:39:32 2025 read: IOPS=0, BW=1495KiB/s (1531kB/s)(4096KiB/2740msec) clat (nsec): min=419071k, max=419071k, avg=419070644.00, stdev= 0.00 lat (nsec): min=419072k, max=419072k, avg=419072119.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 418], 5.00th=[ 418], 10.00th=[ 418], 20.00th=[ 418], | 30.00th=[ 418], 40.00th=[ 418], 50.00th=[ 418], 60.00th=[ 418], | 70.00th=[ 418], 80.00th=[ 418], 90.00th=[ 418], 95.00th=[ 418], | 99.00th=[ 418], 99.50th=[ 418], 99.90th=[ 418], 99.95th=[ 418], | 99.99th=[ 418] bw ( KiB/s): min= 5673, max= 5673, per=94.88%, avg=5673.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2990KiB/s (3062kB/s)(8192KiB/2740msec); 0 zone resets clat (msec): min=752, max=1479, avg=1116.30, stdev=513.88 lat (msec): min=753, max=1480, avg=1116.58, stdev=514.02 clat percentiles (msec): | 1.00th=[ 751], 5.00th=[ 751], 10.00th=[ 751], 20.00th=[ 751], | 30.00th=[ 751], 40.00th=[ 751], 50.00th=[ 751], 60.00th=[ 1485], | 70.00th=[ 1485], 80.00th=[ 1485], 90.00th=[ 1485], 95.00th=[ 1485], | 99.00th=[ 1485], 99.50th=[ 1485], 99.90th=[ 1485], 99.95th=[ 1485], | 99.99th=[ 1485] bw ( KiB/s): min= 8175, max= 8175, per=82.61%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=39.21%, ctx=527, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193443: Fri Jul 25 03:39:32 2025 write: IOPS=0, BW=3711KiB/s (3800kB/s)(12.0MiB/3311msec); 0 zone resets clat (msec): min=869, max=1374, avg=1076.55, stdev=264.51 lat (msec): min=869, max=1374, avg=1076.80, stdev=264.43 clat percentiles (msec): | 1.00th=[ 869], 5.00th=[ 869], 10.00th=[ 869], 20.00th=[ 869], | 30.00th=[ 869], 40.00th=[ 986], 50.00th=[ 986], 60.00th=[ 986], | 70.00th=[ 1368], 80.00th=[ 1368], 90.00th=[ 1368], 95.00th=[ 1368], | 99.00th=[ 1368], 99.50th=[ 1368], 99.90th=[ 1368], 99.95th=[ 1368], | 99.99th=[ 1368] bw ( KiB/s): min= 8175, max= 8175, per=82.61%, avg=8175.00, stdev= 0.00, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=66.67%, 2000=33.33% cpu : usr=0.00%, sys=42.57%, ctx=78, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193444: Fri Jul 25 03:39:32 2025 write: IOPS=1, BW=4955KiB/s (5074kB/s)(12.0MiB/2480msec); 0 zone resets clat (msec): min=588, max=922, avg=801.88, stdev=185.43 lat (msec): min=588, max=923, avg=802.16, stdev=185.45 clat percentiles (msec): | 1.00th=[ 592], 5.00th=[ 592], 10.00th=[ 592], 20.00th=[ 592], | 30.00th=[ 592], 40.00th=[ 894], 50.00th=[ 894], 60.00th=[ 894], | 70.00th=[ 927], 80.00th=[ 927], 90.00th=[ 927], 95.00th=[ 927], | 99.00th=[ 927], 99.50th=[ 927], 99.90th=[ 927], 99.95th=[ 927], | 99.99th=[ 927] bw ( KiB/s): min= 8159, max= 8175, per=82.53%, avg=8167.00, stdev=11.31, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=66.67% cpu : usr=0.00%, sys=54.86%, ctx=98, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5980KiB/s (6123kB/s), 1495KiB/s-9191KiB/s (1531kB/s-9411kB/s), io=16.0MiB (16.8MB), run=1337-2740msec WRITE: bw=9897KiB/s (10.1MB/s), 2990KiB/s-4955KiB/s (3062kB/s-5074kB/s), io=32.0MiB (33.6MB), run=2480-3311msec rand-rw: (groupid=0, jobs=1): err= 0: pid=193437: Fri Jul 25 03:39:33 2025 read: IOPS=1, BW=4249KiB/s (4351kB/s)(12.0MiB/2892msec) clat (msec): min=609, max=1618, avg=951.46, stdev=577.83 lat (msec): min=609, max=1618, avg=951.46, stdev=577.83 clat percentiles (msec): | 1.00th=[ 609], 5.00th=[ 609], 10.00th=[ 609], 20.00th=[ 609], | 30.00th=[ 609], 40.00th=[ 625], 50.00th=[ 625], 60.00th=[ 625], | 70.00th=[ 1620], 80.00th=[ 1620], 90.00th=[ 1620], 95.00th=[ 1620], | 99.00th=[ 1620], 99.50th=[ 1620], 99.90th=[ 1620], 99.95th=[ 1620], | 99.99th=[ 1620] bw ( KiB/s): min= 8094, max= 8192, per=100.00%, avg=8143.00, stdev=69.30, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=11.17%, ctx=53, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193438: Fri Jul 25 03:39:33 2025 read: IOPS=0, BW=1184KiB/s (1213kB/s)(4096KiB/3458msec) clat (nsec): min=781810k, max=781810k, avg=781810429.00, stdev= 0.00 lat (nsec): min=781813k, max=781813k, avg=781813157.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 785], 5.00th=[ 785], 10.00th=[ 785], 20.00th=[ 785], | 30.00th=[ 785], 40.00th=[ 785], 50.00th=[ 785], 60.00th=[ 785], | 70.00th=[ 785], 80.00th=[ 785], 90.00th=[ 785], 95.00th=[ 785], | 99.00th=[ 785], 99.50th=[ 785], 99.90th=[ 785], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2369KiB/s (2426kB/s)(8192KiB/3458msec); 0 zone resets clat (msec): min=709, max=1925, avg=1317.46, stdev=859.69 lat (msec): min=709, max=1925, avg=1317.80, stdev=859.68 clat percentiles (msec): | 1.00th=[ 709], 5.00th=[ 709], 10.00th=[ 709], 20.00th=[ 709], | 30.00th=[ 709], 40.00th=[ 709], 50.00th=[ 709], 60.00th=[ 1921], | 70.00th=[ 1921], 80.00th=[ 1921], 90.00th=[ 1921], 95.00th=[ 1921], | 99.00th=[ 1921], 99.50th=[ 1921], 99.90th=[ 1921], 99.95th=[ 1921], | 99.99th=[ 1921] bw ( KiB/s): min= 8175, max= 8175, per=89.94%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=15.16%, ctx=67, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193439: Fri Jul 25 03:39:33 2025 write: IOPS=0, BW=3409KiB/s (3490kB/s)(12.0MiB/3605msec); 0 zone resets clat (msec): min=252, max=2998, avg=1191.26, stdev=1565.19 lat (msec): min=252, max=2998, avg=1191.54, stdev=1565.09 clat percentiles (msec): | 1.00th=[ 253], 5.00th=[ 253], 10.00th=[ 253], 20.00th=[ 253], | 30.00th=[ 253], 40.00th=[ 326], 50.00th=[ 326], 60.00th=[ 326], | 70.00th=[ 3004], 80.00th=[ 3004], 90.00th=[ 3004], 95.00th=[ 3004], | 99.00th=[ 3004], 99.50th=[ 3004], 99.90th=[ 3004], 99.95th=[ 3004], | 99.99th=[ 3004] bw ( KiB/s): min= 7742, max= 8192, per=87.66%, avg=7967.00, stdev=318.20, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=19.42%, ctx=121, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=193440: Fri Jul 25 03:39:33 2025 write: IOPS=0, BW=3434KiB/s (3517kB/s)(12.0MiB/3578msec); 0 zone resets clat (msec): min=246, max=2766, avg=1161.74, stdev=1393.97 lat (msec): min=246, max=2766, avg=1162.17, stdev=1393.86 clat percentiles (msec): | 1.00th=[ 247], 5.00th=[ 247], 10.00th=[ 247], 20.00th=[ 247], | 30.00th=[ 247], 40.00th=[ 472], 50.00th=[ 472], 60.00th=[ 472], | 70.00th=[ 2769], 80.00th=[ 2769], 90.00th=[ 2769], 95.00th=[ 2769], | 99.00th=[ 2769], 99.50th=[ 2769], 99.90th=[ 2769], 99.95th=[ 2769], | 99.99th=[ 2769] bw ( KiB/s): min= 8192, max= 8192, per=90.13%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=15.38%, ctx=65, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4738KiB/s (4852kB/s), 1184KiB/s-4249KiB/s (1213kB/s-4351kB/s), io=16.0MiB (16.8MB), run=2892-3458msec WRITE: bw=9090KiB/s (9308kB/s), 2369KiB/s-3434KiB/s (2426kB/s-3517kB/s), io=32.0MiB (33.6MB), run=3458-3605msec