== sanity test 398b: DIO and buffer IO race ============== 14:43:55 (1784227435) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.97117 s, 16.9 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=151733: Thu Jul 16 14:44:58 2026 read: IOPS=28, BW=114KiB/s (117kB/s)(5992KiB/52421msec) clat (usec): min=19, max=721110, avg=18250.80, stdev=21346.75 lat (usec): min=19, max=721111, avg=18252.17, stdev=21346.74 clat percentiles (usec): | 1.00th=[ 53], 5.00th=[ 128], 10.00th=[ 7832], 20.00th=[ 10028], | 30.00th=[ 11731], 40.00th=[ 13566], 50.00th=[ 15795], 60.00th=[ 18482], | 70.00th=[ 21103], 80.00th=[ 25035], 90.00th=[ 30278], 95.00th=[ 35914], | 99.00th=[ 59507], 99.50th=[ 74974], 99.90th=[130548], 99.95th=[717226], | 99.99th=[717226] bw ( KiB/s): min= 7, max= 192, per=25.41%, avg=113.82, stdev=34.06, samples=104 iops : min= 1, max= 48, avg=28.06, stdev= 8.51, samples=104 write: IOPS=30, BW=120KiB/s (123kB/s)(6296KiB/52421msec); 0 zone resets clat (usec): min=702, max=125127, avg=15794.74, stdev=13951.41 lat (usec): min=702, max=125128, avg=15796.15, stdev=13951.31 clat percentiles (usec): | 1.00th=[ 1123], 5.00th=[ 1942], 10.00th=[ 2573], 20.00th=[ 3556], | 30.00th=[ 5342], 40.00th=[ 9634], 50.00th=[ 12911], 60.00th=[ 16188], | 70.00th=[ 20317], 80.00th=[ 25035], 90.00th=[ 32900], 95.00th=[ 40633], | 99.00th=[ 61080], 99.50th=[ 78119], 99.90th=[116917], 99.95th=[125305], | 99.99th=[125305] bw ( KiB/s): min= 40, max= 200, per=25.80%, avg=120.99, stdev=31.90, samples=103 iops : min= 10, max= 50, avg=29.85, stdev= 7.97, samples=103 lat (usec) : 20=0.03%, 50=0.33%, 100=1.40%, 250=1.11%, 750=0.03% lat (usec) : 1000=0.23% lat (msec) : 2=2.57%, 4=9.28%, 10=15.63%, 20=36.91%, 50=30.73% lat (msec) : 100=1.60%, 250=0.13%, 750=0.03% cpu : usr=0.11%, sys=20.40%, ctx=8174, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151734: Thu Jul 16 14:44:58 2026 read: IOPS=28, BW=112KiB/s (115kB/s)(5940KiB/52803msec) clat (usec): min=38, max=96140, avg=17786.37, stdev=10821.45 lat (usec): min=38, max=96143, avg=17787.15, stdev=10821.47 clat percentiles (usec): | 1.00th=[ 70], 5.00th=[ 196], 10.00th=[ 7832], 20.00th=[10028], | 30.00th=[11863], 40.00th=[13698], 50.00th=[15795], 60.00th=[18482], | 70.00th=[21103], 80.00th=[24511], 90.00th=[30016], 95.00th=[35914], | 99.00th=[59507], 99.50th=[66847], 99.90th=[84411], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 7, max= 215, per=25.08%, avg=112.37, stdev=34.06, samples=105 iops : min= 1, max= 53, avg=27.69, stdev= 8.52, samples=105 write: IOPS=30, BW=120KiB/s (123kB/s)(6348KiB/52803msec); 0 zone resets clat (usec): min=771, max=711925, avg=16478.90, stdev=22577.94 lat (usec): min=771, max=711926, avg=16480.34, stdev=22578.00 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 1713], 10.00th=[ 2507], 20.00th=[ 3654], | 30.00th=[ 5997], 40.00th=[ 9634], 50.00th=[ 12780], 60.00th=[ 15926], | 70.00th=[ 20317], 80.00th=[ 25560], 90.00th=[ 33817], 95.00th=[ 41157], | 99.00th=[ 64750], 99.50th=[ 80217], 99.90th=[126354], 99.95th=[708838], | 99.99th=[708838] bw ( KiB/s): min= 40, max= 247, per=25.87%, avg=121.32, stdev=32.19, samples=104 iops : min= 10, max= 61, avg=29.92, stdev= 8.05, samples=104 lat (usec) : 50=0.07%, 100=1.33%, 250=1.04%, 500=0.10%, 750=0.07% lat (usec) : 1000=0.39% lat (msec) : 2=3.12%, 4=8.11%, 10=16.31%, 20=37.24%, 50=30.11% lat (msec) : 100=2.02%, 250=0.07%, 750=0.03% cpu : usr=0.12%, sys=20.50%, ctx=8432, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151735: Thu Jul 16 14:44:58 2026 read: IOPS=28, BW=113KiB/s (115kB/s)(6028KiB/53492msec) clat (usec): min=31, max=710002, avg=18492.38, stdev=21283.73 lat (usec): min=32, max=710003, avg=18493.95, stdev=21284.13 clat percentiles (usec): | 1.00th=[ 54], 5.00th=[ 143], 10.00th=[ 7701], 20.00th=[ 10028], | 30.00th=[ 11994], 40.00th=[ 13435], 50.00th=[ 15795], 60.00th=[ 18482], | 70.00th=[ 21365], 80.00th=[ 25035], 90.00th=[ 30802], 95.00th=[ 36963], | 99.00th=[ 61080], 99.50th=[ 82314], 99.90th=[129500], 99.95th=[708838], | 99.99th=[708838] bw ( KiB/s): min= 7, max= 214, per=25.01%, avg=112.04, stdev=33.75, samples=106 iops : min= 1, max= 53, avg=27.60, stdev= 8.46, samples=106 write: IOPS=29, BW=117KiB/s (120kB/s)(6260KiB/53492msec); 0 zone resets clat (usec): min=840, max=165954, avg=16238.61, stdev=14675.15 lat (usec): min=841, max=165956, avg=16239.96, stdev=14675.02 clat percentiles (usec): | 1.00th=[ 1106], 5.00th=[ 1745], 10.00th=[ 2311], 20.00th=[ 3425], | 30.00th=[ 5604], 40.00th=[ 10028], 50.00th=[ 13304], 60.00th=[ 16581], | 70.00th=[ 20579], 80.00th=[ 27132], 90.00th=[ 33424], 95.00th=[ 41681], | 99.00th=[ 61604], 99.50th=[ 84411], 99.90th=[121111], 99.95th=[166724], | 99.99th=[166724] bw ( KiB/s): min= 21, max= 230, per=24.74%, avg=116.01, stdev=30.43, samples=106 iops : min= 5, max= 57, avg=28.60, stdev= 7.55, samples=106 lat (usec) : 50=0.33%, 100=1.30%, 250=1.17%, 500=0.03%, 750=0.03% lat (usec) : 1000=0.13% lat (msec) : 2=3.45%, 4=8.40%, 10=15.01%, 20=36.95%, 50=31.15% lat (msec) : 100=1.79%, 250=0.23%, 750=0.03% cpu : usr=0.11%, sys=19.77%, ctx=8224, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151736: Thu Jul 16 14:44:58 2026 read: IOPS=28, BW=114KiB/s (116kB/s)(6052KiB/53248msec) clat (usec): min=34, max=118742, avg=17770.74, stdev=11661.08 lat (usec): min=35, max=118743, avg=17771.78, stdev=11661.08 clat percentiles (usec): | 1.00th=[ 57], 5.00th=[ 131], 10.00th=[ 7242], 20.00th=[ 9765], | 30.00th=[ 11338], 40.00th=[ 13566], 50.00th=[ 15926], 60.00th=[ 18220], | 70.00th=[ 21365], 80.00th=[ 25035], 90.00th=[ 30278], 95.00th=[ 36439], | 99.00th=[ 55837], 99.50th=[ 77071], 99.90th=[109577], 99.95th=[119014], | 99.99th=[119014] bw ( KiB/s): min= 47, max= 222, per=25.82%, avg=115.65, stdev=32.71, samples=104 iops : min= 11, max= 55, avg=28.51, stdev= 8.19, samples=104 write: IOPS=29, BW=117KiB/s (120kB/s)(6236KiB/53248msec); 0 zone resets clat (usec): min=828, max=751109, avg=16765.79, stdev=23594.47 lat (usec): min=828, max=751110, avg=16766.88, stdev=23594.48 clat percentiles (usec): | 1.00th=[ 1123], 5.00th=[ 1663], 10.00th=[ 2540], 20.00th=[ 3752], | 30.00th=[ 6128], 40.00th=[ 9765], 50.00th=[ 13173], 60.00th=[ 16909], | 70.00th=[ 21103], 80.00th=[ 26084], 90.00th=[ 34866], 95.00th=[ 42206], | 99.00th=[ 57934], 99.50th=[ 67634], 99.90th=[208667], 99.95th=[750781], | 99.99th=[750781] bw ( KiB/s): min= 24, max= 230, per=25.17%, avg=118.07, stdev=32.47, samples=105 iops : min= 6, max= 57, avg=29.11, stdev= 8.05, samples=105 lat (usec) : 50=0.23%, 100=1.46%, 250=1.27%, 500=0.10%, 1000=0.33% lat (msec) : 2=3.22%, 4=7.39%, 10=16.86%, 20=35.90%, 50=31.38% lat (msec) : 100=1.69%, 250=0.13%, 1000=0.03% cpu : usr=0.14%, sys=19.72%, ctx=8285, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=449KiB/s (460kB/s), 112KiB/s-114KiB/s (115kB/s-117kB/s), io=23.4MiB (24.6MB), run=52421-53492msec WRITE: bw=470KiB/s (481kB/s), 117KiB/s-120KiB/s (120kB/s-123kB/s), io=24.6MiB (25.7MB), run=52421-53492msec fio: pid=151730, err=5/file:backend.c:481, func=full resid, error=Input/output error rand-rw: (groupid=0, jobs=1): err= 0: pid=151729: Thu Jul 16 14:45:30 2026 read: IOPS=17, BW=69.7KiB/s (71.3kB/s)(5992KiB/86026msec) clat (msec): min=2, max=718, avg=15.23, stdev=23.44 lat (msec): min=2, max=718, avg=15.24, stdev=23.44 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 9], 60.00th=[ 12], | 70.00th=[ 16], 80.00th=[ 22], 90.00th=[ 33], 95.00th=[ 46], | 99.00th=[ 79], 99.50th=[ 92], 99.90th=[ 116], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 15, max= 192, per=25.86%, avg=69.56, stdev=42.04, samples=170 iops : min= 3, max= 48, avg=17.02, stdev=10.58, samples=170 write: IOPS=18, BW=73.2KiB/s (74.9kB/s)(6296KiB/86026msec); 0 zone resets clat (msec): min=14, max=223, avg=40.02, stdev=21.29 lat (msec): min=14, max=223, avg=40.02, stdev=21.29 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 19], 10.00th=[ 21], 20.00th=[ 24], | 30.00th=[ 26], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 40], | 70.00th=[ 47], 80.00th=[ 56], 90.00th=[ 70], 95.00th=[ 82], | 99.00th=[ 108], 99.50th=[ 128], 99.90th=[ 176], 99.95th=[ 224], | 99.99th=[ 224] bw ( KiB/s): min= 8, max= 168, per=25.96%, avg=73.19, stdev=36.67, samples=170 iops : min= 2, max= 42, avg=17.93, stdev= 9.27, samples=170 lat (msec) : 4=2.38%, 10=24.22%, 20=16.67%, 50=41.73%, 100=14.00% lat (msec) : 250=0.98%, 750=0.03% cpu : usr=0.08%, sys=6.45%, ctx=5894, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 5 (file:backend.c:481, func=full resid, error=Input/output error): pid=151730: Thu Jul 16 14:45:30 2026 read: IOPS=16, BW=65.4KiB/s (66.0kB/s)(5152KiB/78759msec) clat (msec): min=3, max=698, avg=16.24, stdev=24.25 lat (msec): min=3, max=698, avg=16.24, stdev=24.25 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 10], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 25], 90.00th=[ 36], 95.00th=[ 48], | 99.00th=[ 75], 99.50th=[ 85], 99.90th=[ 121], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 7, max= 192, per=24.34%, avg=65.48, stdev=41.79, samples=155 iops : min= 1, max= 48, avg=15.99, stdev=10.54, samples=155 write: IOPS=17, BW=69.6KiB/s (71.2kB/s)(5480KiB/78759msec); 0 zone resets clat (msec): min=14, max=165, avg=42.06, stdev=21.27 lat (msec): min=14, max=165, avg=42.07, stdev=21.27 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 32], 50.00th=[ 36], 60.00th=[ 43], | 70.00th=[ 50], 80.00th=[ 58], 90.00th=[ 71], 95.00th=[ 84], | 99.00th=[ 110], 99.50th=[ 138], 99.90th=[ 163], 99.95th=[ 167], | 99.99th=[ 167] bw ( KiB/s): min= 16, max= 144, per=24.54%, avg=69.20, stdev=33.31, samples=155 iops : min= 4, max= 36, avg=16.92, stdev= 8.42, samples=155 lat (msec) : 4=1.54%, 10=23.92%, 20=14.14%, 50=43.14%, 100=16.32% lat (msec) : 250=0.86%, 750=0.04% cpu : usr=0.09%, sys=6.34%, ctx=4868, majf=0, minf=20 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1289,1370,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151731: Thu Jul 16 14:45:30 2026 read: IOPS=17, BW=70.1KiB/s (71.8kB/s)(6028KiB/85960msec) clat (msec): min=2, max=136, avg=14.17, stdev=13.41 lat (msec): min=2, max=136, avg=14.17, stdev=13.41 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 10], 60.00th=[ 12], | 70.00th=[ 15], 80.00th=[ 21], 90.00th=[ 32], 95.00th=[ 42], | 99.00th=[ 64], 99.50th=[ 80], 99.90th=[ 115], 99.95th=[ 138], | 99.99th=[ 138] bw ( KiB/s): min= 15, max= 232, per=26.09%, avg=70.18, stdev=42.09, samples=170 iops : min= 3, max= 58, avg=17.18, stdev=10.61, samples=170 write: IOPS=18, BW=72.8KiB/s (74.6kB/s)(6260KiB/85960msec); 0 zone resets clat (msec): min=14, max=724, avg=41.15, stdev=28.40 lat (msec): min=14, max=724, avg=41.15, stdev=28.40 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 19], 10.00th=[ 21], 20.00th=[ 24], | 30.00th=[ 27], 40.00th=[ 30], 50.00th=[ 34], 60.00th=[ 40], | 70.00th=[ 47], 80.00th=[ 56], 90.00th=[ 72], 95.00th=[ 83], | 99.00th=[ 124], 99.50th=[ 140], 99.90th=[ 215], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 15, max= 152, per=25.83%, avg=72.85, stdev=36.30, samples=170 iops : min= 3, max= 38, avg=17.85, stdev= 9.18, samples=170 lat (msec) : 4=2.15%, 10=24.15%, 20=17.42%, 50=41.70%, 100=13.38% lat (msec) : 250=1.17%, 750=0.03% cpu : usr=0.09%, sys=6.41%, ctx=5860, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151732: Thu Jul 16 14:45:30 2026 read: IOPS=17, BW=71.1KiB/s (72.8kB/s)(6052KiB/85164msec) clat (msec): min=2, max=134, avg=14.46, stdev=14.25 lat (msec): min=2, max=134, avg=14.46, stdev=14.25 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 9], 60.00th=[ 12], | 70.00th=[ 15], 80.00th=[ 22], 90.00th=[ 33], 95.00th=[ 43], | 99.00th=[ 69], 99.50th=[ 89], 99.90th=[ 114], 99.95th=[ 134], | 99.99th=[ 134] bw ( KiB/s): min= 7, max= 216, per=26.56%, avg=71.44, stdev=41.61, samples=167 iops : min= 1, max= 54, avg=17.49, stdev=10.48, samples=167 write: IOPS=18, BW=73.2KiB/s (74.0kB/s)(6236KiB/85164msec); 0 zone resets clat (msec): min=12, max=735, avg=40.46, stdev=27.56 lat (msec): min=12, max=735, avg=40.46, stdev=27.56 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 19], 10.00th=[ 21], 20.00th=[ 23], | 30.00th=[ 26], 40.00th=[ 30], 50.00th=[ 33], 60.00th=[ 39], | 70.00th=[ 47], 80.00th=[ 56], 90.00th=[ 69], 95.00th=[ 82], | 99.00th=[ 109], 99.50th=[ 124], 99.90th=[ 197], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 15, max= 168, per=25.94%, avg=73.15, stdev=36.63, samples=168 iops : min= 3, max= 42, avg=17.92, stdev= 9.27, samples=168 lat (msec) : 4=2.21%, 10=25.03%, 20=15.79%, 50=42.35%, 100=13.61% lat (msec) : 250=0.98%, 750=0.03% cpu : usr=0.08%, sys=6.57%, ctx=5853, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=270KiB/s (276kB/s), 65.4KiB/s-71.1KiB/s (66.0kB/s-72.8kB/s), io=22.7MiB (23.8MB), run=78759-86026msec WRITE: bw=282KiB/s (289kB/s), 69.6KiB/s-73.2KiB/s (71.2kB/s-74.0kB/s), io=23.7MiB (24.9MB), run=78759-86026msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=151766: Thu Jul 16 14:45:49 2026 read: IOPS=27, BW=439KiB/s (450kB/s)(5920KiB/13477msec) clat (usec): min=36, max=107062, avg=19230.07, stdev=13638.37 lat (usec): min=37, max=107064, avg=19230.87, stdev=13638.39 clat percentiles (usec): | 1.00th=[ 50], 5.00th=[ 82], 10.00th=[ 133], 20.00th=[ 10421], | 30.00th=[ 13304], 40.00th=[ 16057], 50.00th=[ 17957], 60.00th=[ 20317], | 70.00th=[ 22938], 80.00th=[ 25822], 90.00th=[ 35914], 95.00th=[ 41157], | 99.00th=[ 57934], 99.50th=[104334], 99.90th=[107480], 99.95th=[107480], | 99.99th=[107480] bw ( KiB/s): min= 287, max= 672, per=25.50%, avg=431.65, stdev=102.62, samples=26 iops : min= 17, max= 42, avg=26.42, stdev= 6.43, samples=26 write: IOPS=29, BW=473KiB/s (484kB/s)(6368KiB/13477msec); 0 zone resets clat (usec): min=1345, max=64977, avg=15899.55, stdev=12786.46 lat (usec): min=1346, max=64978, avg=15901.17, stdev=12786.58 clat percentiles (usec): | 1.00th=[ 1483], 5.00th=[ 2212], 10.00th=[ 3097], 20.00th=[ 4686], | 30.00th=[ 6128], 40.00th=[ 9110], 50.00th=[11863], 60.00th=[16319], | 70.00th=[20841], 80.00th=[26084], 90.00th=[34866], 95.00th=[41157], | 99.00th=[54789], 99.50th=[57934], 99.90th=[64750], 99.95th=[64750], | 99.99th=[64750] bw ( KiB/s): min= 254, max= 768, per=26.42%, avg=470.62, stdev=113.68, samples=26 iops : min= 15, max= 48, avg=28.88, stdev= 7.32, samples=26 lat (usec) : 50=0.52%, 100=2.47%, 250=2.99%, 500=0.13% lat (msec) : 2=1.69%, 4=6.38%, 10=16.67%, 20=32.68%, 50=34.24% lat (msec) : 100=1.95%, 250=0.26% cpu : usr=0.13%, sys=22.12%, ctx=2183, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151767: Thu Jul 16 14:45:49 2026 read: IOPS=26, BW=420KiB/s (430kB/s)(5744KiB/13666msec) clat (usec): min=45, max=124000, avg=19354.93, stdev=12927.75 lat (usec): min=45, max=124001, avg=19355.82, stdev=12927.71 clat percentiles (usec): | 1.00th=[ 56], 5.00th=[ 109], 10.00th=[ 219], 20.00th=[ 11338], | 30.00th=[ 13435], 40.00th=[ 15270], 50.00th=[ 17171], 60.00th=[ 20841], | 70.00th=[ 24249], 80.00th=[ 27657], 90.00th=[ 32637], 95.00th=[ 39060], | 99.00th=[ 57410], 99.50th=[ 74974], 99.90th=[124257], 99.95th=[124257], | 99.99th=[124257] bw ( KiB/s): min= 223, max= 604, per=24.81%, avg=420.07, stdev=96.58, samples=27 iops : min= 13, max= 37, avg=25.70, stdev= 6.02, samples=27 write: IOPS=29, BW=479KiB/s (490kB/s)(6544KiB/13666msec); 0 zone resets clat (usec): min=1295, max=111900, avg=16225.90, stdev=14442.23 lat (usec): min=1296, max=111902, avg=16227.49, stdev=14442.19 clat percentiles (usec): | 1.00th=[ 1614], 5.00th=[ 2507], 10.00th=[ 3326], 20.00th=[ 4359], | 30.00th=[ 6063], 40.00th=[ 8586], 50.00th=[ 12125], 60.00th=[ 16909], | 70.00th=[ 20055], 80.00th=[ 25035], 90.00th=[ 36963], 95.00th=[ 44303], | 99.00th=[ 65274], 99.50th=[ 73925], 99.90th=[111674], 99.95th=[111674], | 99.99th=[111674] bw ( KiB/s): min= 254, max= 796, per=26.74%, avg=476.30, stdev=141.77, samples=27 iops : min= 15, max= 49, avg=29.22, stdev= 8.89, samples=27 lat (usec) : 50=0.13%, 100=1.95%, 250=2.99%, 1000=0.13% lat (msec) : 2=1.43%, 4=6.77%, 10=17.84%, 20=32.16%, 50=34.24% lat (msec) : 100=2.08%, 250=0.26% cpu : usr=0.19%, sys=22.47%, ctx=2236, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151768: Thu Jul 16 14:45:49 2026 read: IOPS=27, BW=442KiB/s (453kB/s)(6256KiB/14146msec) clat (usec): min=48, max=117783, avg=19417.29, stdev=13860.70 lat (usec): min=48, max=117784, avg=19418.51, stdev=13860.76 clat percentiles (usec): | 1.00th=[ 60], 5.00th=[ 98], 10.00th=[ 143], 20.00th=[ 10159], | 30.00th=[ 13304], 40.00th=[ 15533], 50.00th=[ 17957], 60.00th=[ 20317], | 70.00th=[ 23462], 80.00th=[ 27657], 90.00th=[ 33817], 95.00th=[ 42730], | 99.00th=[ 76022], 99.50th=[ 80217], 99.90th=[117965], 99.95th=[117965], | 99.99th=[117965] bw ( KiB/s): min= 256, max= 672, per=25.86%, avg=437.81, stdev=104.87, samples=27 iops : min= 16, max= 42, avg=26.81, stdev= 6.66, samples=27 write: IOPS=26, BW=426KiB/s (437kB/s)(6032KiB/14146msec); 0 zone resets clat (usec): min=962, max=101544, avg=17089.14, stdev=13879.68 lat (usec): min=963, max=101545, avg=17090.75, stdev=13879.69 clat percentiles (usec): | 1.00th=[ 1172], 5.00th=[ 2704], 10.00th=[ 3720], 20.00th=[ 4948], | 30.00th=[ 7701], 40.00th=[ 10421], 50.00th=[ 13173], 60.00th=[ 16909], | 70.00th=[ 22414], 80.00th=[ 26608], 90.00th=[ 35390], 95.00th=[ 43254], | 99.00th=[ 66847], 99.50th=[ 72877], 99.90th=[101188], 99.95th=[101188], | 99.99th=[101188] bw ( KiB/s): min= 192, max= 605, per=23.01%, avg=409.74, stdev=100.46, samples=27 iops : min= 12, max= 37, avg=25.04, stdev= 6.24, samples=27 lat (usec) : 50=0.13%, 100=2.86%, 250=3.12%, 500=0.13%, 1000=0.13% lat (msec) : 2=1.43%, 4=4.69%, 10=15.89%, 20=33.85%, 50=35.29% lat (msec) : 100=2.21%, 250=0.26% cpu : usr=0.11%, sys=21.36%, ctx=2229, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151769: Thu Jul 16 14:45:49 2026 read: IOPS=28, BW=459KiB/s (470kB/s)(6032KiB/13154msec) clat (usec): min=41, max=102590, avg=18662.38, stdev=12905.44 lat (usec): min=42, max=102591, avg=18663.18, stdev=12905.49 clat percentiles (usec): | 1.00th=[ 52], 5.00th=[ 74], 10.00th=[ 104], 20.00th=[ 9634], | 30.00th=[ 12780], 40.00th=[ 15926], 50.00th=[ 18482], 60.00th=[ 21103], | 70.00th=[ 24249], 80.00th=[ 27132], 90.00th=[ 31851], 95.00th=[ 38011], | 99.00th=[ 57410], 99.50th=[ 83362], 99.90th=[102237], 99.95th=[102237], | 99.99th=[102237] bw ( KiB/s): min= 286, max= 608, per=26.80%, avg=453.72, stdev=106.43, samples=25 iops : min= 17, max= 38, avg=27.80, stdev= 6.75, samples=25 write: IOPS=29, BW=476KiB/s (487kB/s)(6256KiB/13154msec); 0 zone resets clat (usec): min=1282, max=96372, avg=15303.59, stdev=14911.64 lat (usec): min=1283, max=96374, avg=15308.06, stdev=14910.58 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 2409], 10.00th=[ 3228], 20.00th=[ 4424], | 30.00th=[ 5342], 40.00th=[ 7242], 50.00th=[10028], 60.00th=[13829], | 70.00th=[17957], 80.00th=[25035], 90.00th=[33817], 95.00th=[42730], | 99.00th=[80217], 99.50th=[86508], 99.90th=[95945], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 222, max= 702, per=26.68%, avg=475.12, stdev=145.47, samples=25 iops : min= 13, max= 43, avg=29.16, stdev= 9.10, samples=25 lat (usec) : 50=0.39%, 100=4.30%, 250=3.39% lat (msec) : 2=1.95%, 4=6.38%, 10=19.40%, 20=28.78%, 50=32.81% lat (msec) : 100=2.47%, 250=0.13% cpu : usr=0.16%, sys=22.32%, ctx=2132, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1693KiB/s (1734kB/s), 420KiB/s-459KiB/s (430kB/s-470kB/s), io=23.4MiB (24.5MB), run=13154-14146msec WRITE: bw=1781KiB/s (1824kB/s), 426KiB/s-479KiB/s (437kB/s-490kB/s), io=24.6MiB (25.8MB), run=13154-14146msec rand-rw: (groupid=0, jobs=1): err= 0: pid=151761: Thu Jul 16 14:45:58 2026 read: IOPS=16, BW=257KiB/s (263kB/s)(5920KiB/23026msec) clat (msec): min=3, max=136, avg=16.85, stdev=18.21 lat (msec): min=3, max=136, avg=16.85, stdev=18.21 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 22], 90.00th=[ 35], 95.00th=[ 58], | 99.00th=[ 95], 99.50th=[ 112], 99.90th=[ 138], 99.95th=[ 138], | 99.99th=[ 138] bw ( KiB/s): min= 63, max= 608, per=24.18%, avg=249.80, stdev=139.62, samples=45 iops : min= 3, max= 38, avg=15.11, stdev= 8.84, samples=45 write: IOPS=17, BW=277KiB/s (283kB/s)(6368KiB/23026msec); 0 zone resets clat (msec): min=14, max=178, avg=42.12, stdev=23.51 lat (msec): min=14, max=178, avg=42.13, stdev=23.50 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 24], | 30.00th=[ 28], 40.00th=[ 30], 50.00th=[ 35], 60.00th=[ 40], | 70.00th=[ 48], 80.00th=[ 59], 90.00th=[ 78], 95.00th=[ 90], | 99.00th=[ 126], 99.50th=[ 142], 99.90th=[ 180], 99.95th=[ 180], | 99.99th=[ 180] bw ( KiB/s): min= 95, max= 544, per=25.14%, avg=273.00, stdev=140.76, samples=45 iops : min= 5, max= 34, avg=16.56, stdev= 9.01, samples=45 lat (msec) : 4=0.26%, 10=23.44%, 20=17.19%, 50=42.32%, 100=15.10% lat (msec) : 250=1.69% cpu : usr=0.05%, sys=6.93%, ctx=1564, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151763: Thu Jul 16 14:45:58 2026 read: IOPS=15, BW=248KiB/s (254kB/s)(5744KiB/23184msec) clat (msec): min=2, max=108, avg=15.16, stdev=15.47 lat (msec): min=2, max=108, avg=15.16, stdev=15.47 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 10], 60.00th=[ 12], | 70.00th=[ 15], 80.00th=[ 20], 90.00th=[ 34], 95.00th=[ 51], | 99.00th=[ 77], 99.50th=[ 87], 99.90th=[ 109], 99.95th=[ 109], | 99.99th=[ 109] bw ( KiB/s): min= 32, max= 608, per=24.09%, avg=248.83, stdev=160.46, samples=46 iops : min= 2, max= 38, avg=15.07, stdev=10.23, samples=46 write: IOPS=17, BW=282KiB/s (289kB/s)(6544KiB/23184msec); 0 zone resets clat (msec): min=11, max=163, avg=43.06, stdev=26.16 lat (msec): min=11, max=163, avg=43.06, stdev=26.16 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 19], 10.00th=[ 22], 20.00th=[ 25], | 30.00th=[ 27], 40.00th=[ 30], 50.00th=[ 35], 60.00th=[ 40], | 70.00th=[ 46], 80.00th=[ 60], 90.00th=[ 83], 95.00th=[ 99], | 99.00th=[ 142], 99.50th=[ 142], 99.90th=[ 165], 99.95th=[ 165], | 99.99th=[ 165] bw ( KiB/s): min= 126, max= 736, per=25.97%, avg=282.00, stdev=146.62, samples=46 iops : min= 7, max= 46, avg=17.13, stdev= 9.37, samples=46 lat (msec) : 4=1.56%, 10=23.05%, 20=17.19%, 50=42.06%, 100=13.41% lat (msec) : 250=2.73% cpu : usr=0.09%, sys=6.66%, ctx=1597, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151764: Thu Jul 16 14:45:58 2026 read: IOPS=17, BW=274KiB/s (280kB/s)(6256KiB/22866msec) clat (msec): min=3, max=142, avg=16.45, stdev=17.18 lat (msec): min=3, max=142, avg=16.46, stdev=17.18 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 10], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 23], 90.00th=[ 41], 95.00th=[ 52], | 99.00th=[ 85], 99.50th=[ 90], 99.90th=[ 142], 99.95th=[ 142], | 99.99th=[ 142] bw ( KiB/s): min= 63, max= 832, per=26.63%, avg=275.04, stdev=181.79, samples=45 iops : min= 3, max= 52, avg=16.69, stdev=11.54, samples=45 write: IOPS=16, BW=264KiB/s (270kB/s)(6032KiB/22866msec); 0 zone resets clat (msec): min=15, max=148, avg=43.26, stdev=25.54 lat (msec): min=15, max=148, avg=43.26, stdev=25.54 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 25], | 30.00th=[ 28], 40.00th=[ 30], 50.00th=[ 35], 60.00th=[ 41], | 70.00th=[ 47], 80.00th=[ 59], 90.00th=[ 81], 95.00th=[ 103], | 99.00th=[ 134], 99.50th=[ 142], 99.90th=[ 150], 99.95th=[ 150], | 99.99th=[ 150] bw ( KiB/s): min= 63, max= 512, per=24.01%, avg=260.71, stdev=139.47, samples=45 iops : min= 3, max= 32, avg=15.80, stdev= 8.93, samples=45 lat (msec) : 4=0.26%, 10=25.91%, 20=15.63%, 50=42.71%, 100=12.63% lat (msec) : 250=2.86% cpu : usr=0.08%, sys=6.94%, ctx=1627, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151765: Thu Jul 16 14:45:58 2026 read: IOPS=16, BW=262KiB/s (268kB/s)(6032KiB/23018msec) clat (msec): min=3, max=100, avg=15.59, stdev=15.05 lat (msec): min=3, max=100, avg=15.59, stdev=15.05 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 10], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 23], 90.00th=[ 34], 95.00th=[ 47], | 99.00th=[ 75], 99.50th=[ 99], 99.90th=[ 102], 99.95th=[ 102], | 99.99th=[ 102] bw ( KiB/s): min= 31, max= 638, per=24.72%, avg=255.38, stdev=150.14, samples=45 iops : min= 1, max= 39, avg=15.47, stdev= 9.49, samples=45 write: IOPS=16, BW=272KiB/s (278kB/s)(6256KiB/23018msec); 0 zone resets clat (msec): min=14, max=164, avg=43.60, stdev=25.36 lat (msec): min=14, max=164, avg=43.60, stdev=25.36 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 21], 10.00th=[ 22], 20.00th=[ 26], | 30.00th=[ 29], 40.00th=[ 32], 50.00th=[ 35], 60.00th=[ 39], | 70.00th=[ 49], 80.00th=[ 61], 90.00th=[ 74], 95.00th=[ 104], | 99.00th=[ 130], 99.50th=[ 155], 99.90th=[ 165], 99.95th=[ 165], | 99.99th=[ 165] bw ( KiB/s): min= 95, max= 544, per=24.74%, avg=268.67, stdev=137.26, samples=45 iops : min= 5, max= 34, avg=16.29, stdev= 8.83, samples=45 lat (msec) : 4=0.78%, 10=25.00%, 20=14.45%, 50=42.71%, 100=14.32% lat (msec) : 250=2.73% cpu : usr=0.11%, sys=6.74%, ctx=1585, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1033KiB/s (1058kB/s), 248KiB/s-274KiB/s (254kB/s-280kB/s), io=23.4MiB (24.5MB), run=22866-23184msec WRITE: bw=1087KiB/s (1113kB/s), 264KiB/s-282KiB/s (270kB/s-289kB/s), io=24.6MiB (25.8MB), run=22866-23184msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=151781: Thu Jul 16 14:46:08 2026 read: IOPS=2, BW=2145KiB/s (2196kB/s)(7168KiB/3342msec) clat (msec): min=50, max=1105, avg=270.50, stdev=374.28 lat (msec): min=50, max=1105, avg=270.50, stdev=374.28 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 51], 10.00th=[ 51], 20.00th=[ 72], | 30.00th=[ 96], 40.00th=[ 96], 50.00th=[ 136], 60.00th=[ 201], | 70.00th=[ 201], 80.00th=[ 236], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 1957, max= 5180, per=76.15%, avg=3730.67, stdev=1635.80, samples=3 iops : min= 1, max= 5, avg= 3.00, stdev= 2.00, samples=3 write: IOPS=1, BW=1532KiB/s (1569kB/s)(5120KiB/3342msec); 0 zone resets clat (msec): min=115, max=604, avg=286.21, stdev=210.65 lat (msec): min=115, max=604, avg=286.28, stdev=210.65 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 116], | 30.00th=[ 153], 40.00th=[ 153], 50.00th=[ 159], 60.00th=[ 159], | 70.00th=[ 401], 80.00th=[ 401], 90.00th=[ 609], 95.00th=[ 609], | 99.00th=[ 609], 99.50th=[ 609], 99.90th=[ 609], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 1726, max= 2027, per=28.19%, avg=1933.25, stdev=141.85, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 lat (msec) : 100=25.00%, 250=50.00%, 500=8.33%, 750=8.33%, 2000=8.33% cpu : usr=0.03%, sys=8.44%, ctx=72, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151782: Thu Jul 16 14:46:08 2026 read: IOPS=0, BW=506KiB/s (518kB/s)(2048KiB/4050msec) clat (msec): min=70, max=258, avg=164.39, stdev=133.15 lat (msec): min=70, max=258, avg=164.39, stdev=133.15 clat percentiles (msec): | 1.00th=[ 70], 5.00th=[ 70], 10.00th=[ 70], 20.00th=[ 70], | 30.00th=[ 70], 40.00th=[ 70], 50.00th=[ 70], 60.00th=[ 259], | 70.00th=[ 259], 80.00th=[ 259], 90.00th=[ 259], 95.00th=[ 259], | 99.00th=[ 259], 99.50th=[ 259], 99.90th=[ 259], 99.95th=[ 259], | 99.99th=[ 259] bw ( KiB/s): min= 1726, max= 1957, per=37.59%, avg=1841.50, stdev=163.34, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2528KiB/s (2589kB/s)(10.0MiB/4050msec); 0 zone resets clat (msec): min=87, max=1567, avg=365.80, stdev=436.68 lat (msec): min=87, max=1567, avg=365.87, stdev=436.68 clat percentiles (msec): | 1.00th=[ 88], 5.00th=[ 88], 10.00th=[ 88], 20.00th=[ 106], | 30.00th=[ 116], 40.00th=[ 184], 50.00th=[ 201], 60.00th=[ 309], | 70.00th=[ 338], 80.00th=[ 372], 90.00th=[ 376], 95.00th=[ 1569], | 99.00th=[ 1569], 99.50th=[ 1569], 99.90th=[ 1569], 99.95th=[ 1569], | 99.99th=[ 1569] bw ( KiB/s): min= 1957, max= 4055, per=44.13%, avg=3026.75, stdev=1183.29, samples=4 iops : min= 1, max= 3, avg= 2.25, stdev= 0.96, samples=4 lat (msec) : 100=16.67%, 250=33.33%, 500=41.67%, 2000=8.33% cpu : usr=0.00%, sys=9.21%, ctx=103, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151784: Thu Jul 16 14:46:08 2026 read: IOPS=1, BW=1470KiB/s (1505kB/s)(6144KiB/4180msec) clat (msec): min=65, max=429, avg=152.19, stdev=140.04 lat (msec): min=65, max=429, avg=152.20, stdev=140.04 clat percentiles (msec): | 1.00th=[ 66], 5.00th=[ 66], 10.00th=[ 66], 20.00th=[ 72], | 30.00th=[ 72], 40.00th=[ 75], 50.00th=[ 75], 60.00th=[ 116], | 70.00th=[ 155], 80.00th=[ 155], 90.00th=[ 430], 95.00th=[ 430], | 99.00th=[ 430], 99.50th=[ 430], 99.90th=[ 430], 99.95th=[ 430], | 99.99th=[ 430] bw ( KiB/s): min= 2048, max= 4047, per=68.11%, avg=3336.67, stdev=1117.97, samples=3 iops : min= 2, max= 3, avg= 2.67, stdev= 0.58, samples=3 write: IOPS=1, BW=1470KiB/s (1505kB/s)(6144KiB/4180msec); 0 zone resets clat (msec): min=116, max=1552, avg=537.29, stdev=516.20 lat (msec): min=116, max=1552, avg=537.38, stdev=516.20 clat percentiles (msec): | 1.00th=[ 117], 5.00th=[ 117], 10.00th=[ 117], 20.00th=[ 275], | 30.00th=[ 275], 40.00th=[ 359], 50.00th=[ 359], 60.00th=[ 384], | 70.00th=[ 535], 80.00th=[ 535], 90.00th=[ 1552], 95.00th=[ 1552], | 99.00th=[ 1552], 99.50th=[ 1552], 99.90th=[ 1552], 99.95th=[ 1552], | 99.99th=[ 1552] bw ( KiB/s): min= 2023, max= 4096, per=42.30%, avg=2901.25, stdev=1044.69, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.50, samples=4 lat (msec) : 100=25.00%, 250=25.00%, 500=33.33%, 750=8.33%, 2000=8.33% cpu : usr=0.02%, sys=8.73%, ctx=89, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151785: Thu Jul 16 14:46:08 2026 read: IOPS=1, BW=1253KiB/s (1283kB/s)(5120KiB/4086msec) clat (msec): min=28, max=406, avg=181.49, stdev=181.08 lat (msec): min=28, max=406, avg=181.49, stdev=181.08 clat percentiles (msec): | 1.00th=[ 29], 5.00th=[ 29], 10.00th=[ 29], 20.00th=[ 29], | 30.00th=[ 51], 40.00th=[ 51], 50.00th=[ 73], 60.00th=[ 73], | 70.00th=[ 351], 80.00th=[ 351], 90.00th=[ 409], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 6144, max= 6144, per=100.00%, avg=6144.00, stdev= 0.00, samples=1 iops : min= 6, max= 6, avg= 6.00, stdev= 0.00, samples=1 write: IOPS=1, BW=1754KiB/s (1796kB/s)(7168KiB/4086msec); 0 zone resets clat (msec): min=78, max=1588, avg=446.55, stdev=549.59 lat (msec): min=78, max=1588, avg=446.62, stdev=549.59 clat percentiles (msec): | 1.00th=[ 80], 5.00th=[ 80], 10.00th=[ 80], 20.00th=[ 89], | 30.00th=[ 109], 40.00th=[ 109], 50.00th=[ 146], 60.00th=[ 472], | 70.00th=[ 472], 80.00th=[ 642], 90.00th=[ 1586], 95.00th=[ 1586], | 99.00th=[ 1586], 99.50th=[ 1586], 99.90th=[ 1586], 99.95th=[ 1586], | 99.99th=[ 1586] bw ( KiB/s): min= 1729, max= 2027, per=28.20%, avg=1934.00, stdev=140.39, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 lat (msec) : 50=8.33%, 100=33.33%, 250=16.67%, 500=25.00%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=11.16%, ctx=160, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4900KiB/s (5017kB/s), 506KiB/s-2145KiB/s (518kB/s-2196kB/s), io=20.0MiB (20.0MB), run=3342-4180msec WRITE: bw=6859KiB/s (7024kB/s), 1470KiB/s-2528KiB/s (1505kB/s-2589kB/s), io=28.0MiB (29.4MB), run=3342-4180msec rand-rw: (groupid=0, jobs=1): err= 0: pid=151786: Thu Jul 16 14:46:08 2026 read: IOPS=2, BW=2148KiB/s (2200kB/s)(7168KiB/3337msec) clat (usec): min=459, max=1301.8k, avg=289567.08, stdev=451154.02 lat (usec): min=460, max=1301.8k, avg=289569.84, stdev=451153.32 clat percentiles (usec): | 1.00th=[ 461], 5.00th=[ 461], 10.00th=[ 461], | 20.00th=[ 105382], 30.00th=[ 114820], 40.00th=[ 114820], | 50.00th=[ 124257], 60.00th=[ 160433], 70.00th=[ 160433], | 80.00th=[ 217056], 90.00th=[1300235], 95.00th=[1300235], | 99.00th=[1300235], 99.50th=[1300235], 99.90th=[1300235], | 99.95th=[1300235], 99.99th=[1300235] bw ( KiB/s): min= 1777, max= 6144, per=60.75%, avg=3004.25, stdev=2097.06, samples=4 iops : min= 1, max= 6, avg= 2.75, stdev= 2.22, samples=4 write: IOPS=1, BW=1534KiB/s (1571kB/s)(5120KiB/3337msec); 0 zone resets clat (msec): min=162, max=315, avg=247.73, stdev=64.30 lat (msec): min=162, max=315, avg=247.79, stdev=64.29 clat percentiles (msec): | 1.00th=[ 163], 5.00th=[ 163], 10.00th=[ 163], 20.00th=[ 163], | 30.00th=[ 207], 40.00th=[ 207], 50.00th=[ 251], 60.00th=[ 251], | 70.00th=[ 305], 80.00th=[ 305], 90.00th=[ 317], 95.00th=[ 317], | 99.00th=[ 317], 99.50th=[ 317], 99.90th=[ 317], 99.95th=[ 317], | 99.99th=[ 317] bw ( KiB/s): min= 2003, max= 4096, per=39.23%, avg=2715.67, stdev=1195.62, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 lat (usec) : 500=8.33% lat (msec) : 250=58.33%, 500=25.00%, 2000=8.33% cpu : usr=0.00%, sys=35.58%, ctx=882, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151787: Thu Jul 16 14:46:08 2026 read: IOPS=0, BW=495KiB/s (506kB/s)(2048KiB/4141msec) clat (msec): min=126, max=141, avg=133.86, stdev=11.10 lat (msec): min=126, max=141, avg=133.86, stdev=11.10 clat percentiles (msec): | 1.00th=[ 127], 5.00th=[ 127], 10.00th=[ 127], 20.00th=[ 127], | 30.00th=[ 127], 40.00th=[ 127], 50.00th=[ 127], 60.00th=[ 142], | 70.00th=[ 142], 80.00th=[ 142], 90.00th=[ 142], 95.00th=[ 142], | 99.00th=[ 142], 99.50th=[ 142], 99.90th=[ 142], 99.95th=[ 142], | 99.99th=[ 142] bw ( KiB/s): min= 1768, max= 2011, per=38.21%, avg=1889.50, stdev=171.83, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2473KiB/s (2532kB/s)(10.0MiB/4141msec); 0 zone resets clat (msec): min=85, max=1556, avg=377.37, stdev=434.90 lat (msec): min=85, max=1556, avg=377.44, stdev=434.89 clat percentiles (msec): | 1.00th=[ 86], 5.00th=[ 86], 10.00th=[ 86], 20.00th=[ 89], | 30.00th=[ 107], 40.00th=[ 140], 50.00th=[ 288], 60.00th=[ 342], | 70.00th=[ 363], 80.00th=[ 372], 90.00th=[ 439], 95.00th=[ 1552], | 99.00th=[ 1552], 99.50th=[ 1552], 99.90th=[ 1552], 99.95th=[ 1552], | 99.99th=[ 1552] bw ( KiB/s): min= 2011, max= 6144, per=52.88%, avg=3661.20, stdev=1717.45, samples=5 iops : min= 1, max= 6, avg= 3.20, stdev= 1.92, samples=5 lat (msec) : 100=16.67%, 250=33.33%, 500=41.67%, 2000=8.33% cpu : usr=0.02%, sys=29.23%, ctx=444, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151788: Thu Jul 16 14:46:08 2026 read: IOPS=1, BW=1607KiB/s (1646kB/s)(6144KiB/3823msec) clat (usec): min=480, max=139264, avg=63734.67, stdev=70261.52 lat (usec): min=482, max=139265, avg=63735.60, stdev=70261.34 clat percentiles (usec): | 1.00th=[ 482], 5.00th=[ 482], 10.00th=[ 482], 20.00th=[ 537], | 30.00th=[ 537], 40.00th=[ 676], 50.00th=[ 676], 60.00th=[104334], | 70.00th=[137364], 80.00th=[137364], 90.00th=[139461], 95.00th=[139461], | 99.00th=[139461], 99.50th=[139461], 99.90th=[139461], 99.95th=[139461], | 99.99th=[139461] bw ( KiB/s): min= 2011, max= 4096, per=68.18%, avg=3371.33, stdev=1178.92, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=1, BW=1607KiB/s (1646kB/s)(6144KiB/3823msec); 0 zone resets clat (msec): min=245, max=1681, avg=569.58, stdev=556.90 lat (msec): min=245, max=1681, avg=569.62, stdev=556.90 clat percentiles (msec): | 1.00th=[ 245], 5.00th=[ 245], 10.00th=[ 245], 20.00th=[ 259], | 30.00th=[ 259], 40.00th=[ 296], 50.00th=[ 296], 60.00th=[ 372], | 70.00th=[ 567], 80.00th=[ 567], 90.00th=[ 1687], 95.00th=[ 1687], | 99.00th=[ 1687], 99.50th=[ 1687], 99.90th=[ 1687], 99.95th=[ 1687], | 99.99th=[ 1687] bw ( KiB/s): min= 2003, max= 3543, per=34.69%, avg=2401.25, stdev=761.42, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (usec) : 500=8.33%, 750=16.67% lat (msec) : 250=33.33%, 500=25.00%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=20.49%, ctx=383, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=151789: Thu Jul 16 14:46:08 2026 read: IOPS=1, BW=1497KiB/s (1533kB/s)(5120KiB/3420msec) clat (usec): min=294, max=160533, avg=62986.93, stdev=85668.54 lat (usec): min=295, max=160535, avg=62987.97, stdev=85668.85 clat percentiles (usec): | 1.00th=[ 293], 5.00th=[ 293], 10.00th=[ 293], 20.00th=[ 293], | 30.00th=[ 502], 40.00th=[ 502], 50.00th=[ 562], 60.00th=[ 562], | 70.00th=[152044], 80.00th=[152044], 90.00th=[160433], 95.00th=[160433], | 99.00th=[160433], 99.50th=[160433], 99.90th=[160433], 99.95th=[160433], | 99.99th=[160433] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=2, BW=2096KiB/s (2146kB/s)(7168KiB/3420msec); 0 zone resets clat (msec): min=115, max=1556, avg=435.60, stdev=503.29 lat (msec): min=115, max=1556, avg=435.67, stdev=503.28 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 157], | 30.00th=[ 218], 40.00th=[ 218], 50.00th=[ 300], 60.00th=[ 309], | 70.00th=[ 309], 80.00th=[ 393], 90.00th=[ 1552], 95.00th=[ 1552], | 99.00th=[ 1552], 99.50th=[ 1552], 99.90th=[ 1552], 99.95th=[ 1552], | 99.99th=[ 1552] bw ( KiB/s): min= 1762, max= 4007, per=37.64%, avg=2605.67, stdev=1221.99, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 lat (usec) : 500=8.33%, 750=16.67% lat (msec) : 250=41.67%, 500=25.00%, 2000=8.33% cpu : usr=0.00%, sys=27.11%, ctx=106, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4946KiB/s (5064kB/s), 495KiB/s-2148KiB/s (506kB/s-2200kB/s), io=20.0MiB (20.0MB), run=3337-4141msec WRITE: bw=6924KiB/s (7090kB/s), 1534KiB/s-2473KiB/s (1571kB/s-2532kB/s), io=28.0MiB (29.4MB), run=3337-4141msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes