== sanity test 398b: DIO and buffer IO race ============== 05:11:45 (1788513105) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 5.86012 s, 8.6 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=145331: Fri Sep 4 05:14:25 2026 read: IOPS=10, BW=42.4KiB/s (43.4kB/s)(5992KiB/141250msec) clat (usec): min=32, max=628938, avg=46712.98, stdev=38225.65 lat (usec): min=33, max=628939, avg=46731.14, stdev=38231.20 clat percentiles (usec): | 1.00th=[ 50], 5.00th=[ 60], 10.00th=[ 2606], 20.00th=[ 21627], | 30.00th=[ 27395], 40.00th=[ 33162], 50.00th=[ 39060], 60.00th=[ 46400], | 70.00th=[ 56361], 80.00th=[ 69731], 90.00th=[ 88605], 95.00th=[107480], | 99.00th=[175113], 99.50th=[208667], 99.90th=[295699], 99.95th=[624952], | 99.99th=[624952] bw ( KiB/s): min= 7, max= 110, per=25.43%, avg=42.48, stdev=20.23, samples=273 iops : min= 1, max= 27, avg= 9.99, stdev= 5.10, samples=273 write: IOPS=11, BW=44.6KiB/s (45.6kB/s)(6296KiB/141250msec); 0 zone resets clat (usec): min=1093, max=441451, avg=44998.59, stdev=49475.47 lat (usec): min=1094, max=441452, avg=45005.21, stdev=49475.55 clat percentiles (usec): | 1.00th=[ 1205], 5.00th=[ 1401], 10.00th=[ 2278], 20.00th=[ 5276], | 30.00th=[ 9765], 40.00th=[ 19792], 50.00th=[ 31851], 60.00th=[ 42730], | 70.00th=[ 57410], 80.00th=[ 77071], 90.00th=[104334], 95.00th=[137364], | 99.00th=[223347], 99.50th=[256902], 99.90th=[421528], 99.95th=[442500], | 99.99th=[442500] bw ( KiB/s): min= 7, max= 104, per=25.30%, avg=44.27, stdev=18.97, samples=276 iops : min= 1, max= 26, avg=10.43, stdev= 4.80, samples=276 lat (usec) : 50=0.49%, 100=3.52%, 250=0.49%, 500=0.13%, 750=0.03% lat (usec) : 1000=0.13% lat (msec) : 2=4.62%, 4=3.97%, 10=7.23%, 20=8.56%, 50=34.86% lat (msec) : 100=27.21%, 250=8.37%, 500=0.36%, 750=0.03% cpu : usr=0.06%, sys=4.79%, ctx=6679, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=145332: Fri Sep 4 05:14:25 2026 read: IOPS=10, BW=41.8KiB/s (42.8kB/s)(5940KiB/142015msec) clat (usec): min=25, max=624559, avg=47637.74, stdev=38174.25 lat (usec): min=25, max=624560, avg=47659.59, stdev=38179.91 clat percentiles (usec): | 1.00th=[ 50], 5.00th=[ 65], 10.00th=[ 12911], 20.00th=[ 22676], | 30.00th=[ 27919], 40.00th=[ 33817], 50.00th=[ 39060], 60.00th=[ 47449], | 70.00th=[ 56886], 80.00th=[ 68682], 90.00th=[ 90702], 95.00th=[110625], | 99.00th=[179307], 99.50th=[204473], 99.90th=[320865], 99.95th=[624952], | 99.99th=[624952] bw ( KiB/s): min= 7, max= 95, per=24.98%, avg=41.72, stdev=19.73, samples=277 iops : min= 1, max= 23, avg= 9.79, stdev= 4.98, samples=277 write: IOPS=11, BW=44.7KiB/s (45.8kB/s)(6348KiB/142015msec); 0 zone resets clat (usec): min=1170, max=513684, avg=44666.00, stdev=47681.15 lat (usec): min=1171, max=513686, avg=44668.69, stdev=47680.35 clat percentiles (usec): | 1.00th=[ 1254], 5.00th=[ 1450], 10.00th=[ 2057], 20.00th=[ 5276], | 30.00th=[ 10421], 40.00th=[ 19530], 50.00th=[ 31327], 60.00th=[ 43779], | 70.00th=[ 58459], 80.00th=[ 76022], 90.00th=[102237], 95.00th=[137364], | 99.00th=[212861], 99.50th=[229639], 99.90th=[333448], 99.95th=[513803], | 99.99th=[513803] bw ( KiB/s): min= 7, max= 119, per=25.29%, avg=44.27, stdev=20.90, samples=279 iops : min= 1, max= 29, avg=10.44, stdev= 5.28, samples=279 lat (usec) : 50=0.62%, 100=2.99%, 250=0.10%, 500=0.10%, 750=0.03% lat (usec) : 1000=0.03% lat (msec) : 2=5.18%, 4=3.68%, 10=6.54%, 20=9.41%, 50=35.22% lat (msec) : 100=26.63%, 250=9.08%, 500=0.33%, 750=0.07% cpu : usr=0.07%, sys=4.96%, ctx=6746, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=145333: Fri Sep 4 05:14:25 2026 read: IOPS=10, BW=42.1KiB/s (43.1kB/s)(6028KiB/143161msec) clat (usec): min=30, max=606143, avg=45952.34, stdev=36949.57 lat (usec): min=30, max=606145, avg=45956.80, stdev=36952.79 clat percentiles (usec): | 1.00th=[ 48], 5.00th=[ 59], 10.00th=[ 6652], 20.00th=[ 21627], | 30.00th=[ 27132], 40.00th=[ 32637], 50.00th=[ 38536], 60.00th=[ 45351], | 70.00th=[ 54264], 80.00th=[ 66847], 90.00th=[ 86508], 95.00th=[111674], | 99.00th=[166724], 99.50th=[200279], 99.90th=[248513], 99.95th=[608175], | 99.99th=[608175] bw ( KiB/s): min= 7, max= 104, per=25.13%, avg=41.97, stdev=19.53, samples=279 iops : min= 1, max= 26, avg= 9.85, stdev= 4.93, samples=279 write: IOPS=10, BW=43.7KiB/s (44.8kB/s)(6260KiB/143161msec); 0 zone resets clat (usec): min=1116, max=674751, avg=46940.96, stdev=49379.70 lat (usec): min=1117, max=674753, avg=46943.31, stdev=49379.68 clat percentiles (usec): | 1.00th=[ 1237], 5.00th=[ 1401], 10.00th=[ 2474], 20.00th=[ 5866], | 30.00th=[ 11600], 40.00th=[ 23200], 50.00th=[ 35390], 60.00th=[ 47449], | 70.00th=[ 60031], 80.00th=[ 80217], 90.00th=[105382], 95.00th=[133694], | 99.00th=[204473], 99.50th=[261096], 99.90th=[375391], 99.95th=[675283], | 99.99th=[675283] bw ( KiB/s): min= 7, max= 120, per=24.80%, avg=43.40, stdev=20.25, samples=280 iops : min= 1, max= 30, avg=10.21, stdev= 5.10, samples=280 lat (usec) : 50=0.81%, 100=3.19%, 250=0.23%, 500=0.20%, 750=0.07% lat (usec) : 1000=0.03% lat (msec) : 2=4.39%, 4=3.42%, 10=6.80%, 20=8.50%, 50=36.20% lat (msec) : 100=26.73%, 250=9.11%, 500=0.26%, 750=0.07% cpu : usr=0.07%, sys=4.85%, ctx=6779, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=145334: Fri Sep 4 05:14:25 2026 read: IOPS=10, BW=42.2KiB/s (43.2kB/s)(6052KiB/143306msec) clat (usec): min=35, max=269173, avg=44924.72, stdev=32939.97 lat (usec): min=35, max=269173, avg=44926.48, stdev=32939.96 clat percentiles (usec): | 1.00th=[ 48], 5.00th=[ 60], 10.00th=[ 8848], 20.00th=[ 21627], | 30.00th=[ 27395], 40.00th=[ 33162], 50.00th=[ 38536], 60.00th=[ 44827], | 70.00th=[ 54264], 80.00th=[ 65799], 90.00th=[ 84411], 95.00th=[106431], | 99.00th=[154141], 99.50th=[187696], 99.90th=[254804], 99.95th=[270533], | 99.99th=[270533] bw ( KiB/s): min= 7, max= 104, per=25.17%, avg=42.03, stdev=20.06, samples=278 iops : min= 1, max= 26, avg= 9.87, stdev= 5.07, samples=278 write: IOPS=10, BW=43.5KiB/s (44.6kB/s)(6236KiB/143306msec); 0 zone resets clat (usec): min=1117, max=613306, avg=48063.91, stdev=52091.62 lat (usec): min=1119, max=613308, avg=48066.93, stdev=52093.12 clat percentiles (usec): | 1.00th=[ 1237], 5.00th=[ 1418], 10.00th=[ 2474], 20.00th=[ 5407], | 30.00th=[ 10028], 40.00th=[ 22676], 50.00th=[ 33817], 60.00th=[ 46924], | 70.00th=[ 61080], 80.00th=[ 81265], 90.00th=[113771], 95.00th=[147850], | 99.00th=[223347], 99.50th=[248513], 99.90th=[438305], 99.95th=[616563], | 99.99th=[616563] bw ( KiB/s): min= 7, max= 143, per=24.71%, avg=43.24, stdev=21.06, samples=279 iops : min= 1, max= 35, avg=10.18, stdev= 5.31, samples=279 lat (usec) : 50=0.85%, 100=3.52%, 250=0.26%, 500=0.03%, 1000=0.10% lat (msec) : 2=4.23%, 4=3.65%, 10=7.58%, 20=7.42%, 50=36.85% lat (msec) : 100=25.72%, 250=9.51%, 500=0.26%, 750=0.03% cpu : usr=0.07%, sys=4.67%, ctx=6809, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=168KiB/s (172kB/s), 41.8KiB/s-42.4KiB/s (42.8kB/s-43.4kB/s), io=23.4MiB (24.6MB), run=141250-143306msec WRITE: bw=175KiB/s (180kB/s), 43.5KiB/s-44.7KiB/s (44.6kB/s-45.8kB/s), io=24.6MiB (25.7MB), run=141250-143306msec rand-rw: (groupid=0, jobs=1): err= 0: pid=145336: Fri Sep 4 05:16:57 2026 read: IOPS=5, BW=20.4KiB/s (20.9kB/s)(5992KiB/293778msec) clat (msec): min=6, max=653, avg=38.83, stdev=46.75 lat (msec): min=6, max=653, avg=38.84, stdev=46.75 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 25], 60.00th=[ 30], | 70.00th=[ 37], 80.00th=[ 51], 90.00th=[ 80], 95.00th=[ 115], | 99.00th=[ 232], 99.50th=[ 284], 99.90th=[ 584], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 5, max= 87, per=29.65%, avg=24.02, stdev=16.61, samples=481 iops : min= 1, max= 21, avg= 5.36, stdev= 4.20, samples=481 write: IOPS=5, BW=21.4KiB/s (21.9kB/s)(6296KiB/293778msec); 0 zone resets clat (msec): min=30, max=729, avg=149.60, stdev=79.75 lat (msec): min=30, max=729, avg=149.60, stdev=79.75 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 62], 10.00th=[ 72], 20.00th=[ 85], | 30.00th=[ 100], 40.00th=[ 113], 50.00th=[ 129], 60.00th=[ 150], | 70.00th=[ 174], 80.00th=[ 203], 90.00th=[ 257], 95.00th=[ 305], | 99.00th=[ 418], 99.50th=[ 481], 99.90th=[ 558], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 7, max= 56, per=25.10%, avg=21.34, stdev=10.64, samples=566 iops : min= 1, max= 14, avg= 4.68, stdev= 2.71, samples=566 lat (msec) : 10=2.28%, 20=15.92%, 50=20.96%, 100=22.01%, 250=32.81% lat (msec) : 500=5.76%, 750=0.26% cpu : usr=0.04%, sys=1.29%, ctx=5233, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=145337: Fri Sep 4 05:16:57 2026 read: IOPS=5, BW=20.2KiB/s (20.6kB/s)(5940KiB/294556msec) clat (msec): min=6, max=779, avg=39.61, stdev=49.85 lat (msec): min=6, max=779, avg=39.61, stdev=49.85 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 29], | 70.00th=[ 38], 80.00th=[ 54], 90.00th=[ 85], 95.00th=[ 122], | 99.00th=[ 215], 99.50th=[ 279], 99.90th=[ 558], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 7, max= 88, per=29.03%, avg=23.51, stdev=16.52, samples=486 iops : min= 1, max= 22, avg= 5.23, stdev= 4.17, samples=486 write: IOPS=5, BW=21.6KiB/s (22.1kB/s)(6348KiB/294556msec); 0 zone resets clat (msec): min=32, max=588, avg=148.45, stdev=76.98 lat (msec): min=32, max=588, avg=148.45, stdev=76.98 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 63], 10.00th=[ 72], 20.00th=[ 86], | 30.00th=[ 102], 40.00th=[ 115], 50.00th=[ 131], 60.00th=[ 148], | 70.00th=[ 167], 80.00th=[ 203], 90.00th=[ 249], 95.00th=[ 300], | 99.00th=[ 422], 99.50th=[ 498], 99.90th=[ 567], 99.95th=[ 592], | 99.99th=[ 592] bw ( KiB/s): min= 7, max= 63, per=25.01%, avg=21.26, stdev=10.54, samples=571 iops : min= 1, max= 15, avg= 4.66, stdev= 2.70, samples=571 lat (msec) : 10=2.70%, 20=17.12%, 50=19.14%, 100=21.03%, 250=34.54% lat (msec) : 500=5.14%, 750=0.29%, 1000=0.03% cpu : usr=0.04%, sys=1.26%, ctx=5212, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=145338: Fri Sep 4 05:16:57 2026 read: IOPS=5, BW=20.5KiB/s (20.0kB/s)(6028KiB/293942msec) clat (msec): min=5, max=343, avg=38.60, stdev=36.89 lat (msec): min=5, max=343, avg=38.61, stdev=36.89 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 18], 40.00th=[ 22], 50.00th=[ 26], 60.00th=[ 32], | 70.00th=[ 40], 80.00th=[ 54], 90.00th=[ 85], 95.00th=[ 116], | 99.00th=[ 182], 99.50th=[ 199], 99.90th=[ 284], 99.95th=[ 342], | 99.99th=[ 342] bw ( KiB/s): min= 7, max= 88, per=28.92%, avg=23.42, stdev=15.94, samples=495 iops : min= 1, max= 22, avg= 5.21, stdev= 4.03, samples=495 write: IOPS=5, BW=21.3KiB/s (21.8kB/s)(6260KiB/293942msec); 0 zone resets clat (msec): min=37, max=887, avg=150.54, stdev=80.24 lat (msec): min=37, max=887, avg=150.54, stdev=80.24 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 63], 10.00th=[ 70], 20.00th=[ 86], | 30.00th=[ 102], 40.00th=[ 116], 50.00th=[ 132], 60.00th=[ 150], | 70.00th=[ 176], 80.00th=[ 207], 90.00th=[ 251], 95.00th=[ 296], | 99.00th=[ 418], 99.50th=[ 514], 99.90th=[ 600], 99.95th=[ 885], | 99.99th=[ 885] bw ( KiB/s): min= 7, max= 56, per=25.02%, avg=21.26, stdev=10.54, samples=564 iops : min= 1, max= 14, avg= 4.66, stdev= 2.70, samples=564 lat (msec) : 10=2.05%, 20=16.37%, 50=20.44%, 100=21.39%, 250=34.47% lat (msec) : 500=5.01%, 750=0.23%, 1000=0.03% cpu : usr=0.05%, sys=1.21%, ctx=5127, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=145339: Fri Sep 4 05:16:57 2026 read: IOPS=5, BW=20.7KiB/s (21.2kB/s)(6052KiB/292571msec) clat (msec): min=6, max=561, avg=39.36, stdev=43.39 lat (msec): min=6, max=561, avg=39.36, stdev=43.39 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 25], 60.00th=[ 30], | 70.00th=[ 38], 80.00th=[ 54], 90.00th=[ 88], 95.00th=[ 121], | 99.00th=[ 209], 99.50th=[ 284], 99.90th=[ 384], 99.95th=[ 558], | 99.99th=[ 558] bw ( KiB/s): min= 7, max= 117, per=29.55%, avg=23.94, stdev=17.24, samples=486 iops : min= 1, max= 29, avg= 5.34, stdev= 4.37, samples=486 write: IOPS=5, BW=21.3KiB/s (21.8kB/s)(6236KiB/292571msec); 0 zone resets clat (msec): min=33, max=891, avg=149.36, stdev=79.22 lat (msec): min=33, max=891, avg=149.36, stdev=79.22 clat percentiles (msec): | 1.00th=[ 49], 5.00th=[ 62], 10.00th=[ 72], 20.00th=[ 86], | 30.00th=[ 101], 40.00th=[ 115], 50.00th=[ 131], 60.00th=[ 148], | 70.00th=[ 171], 80.00th=[ 203], 90.00th=[ 253], 95.00th=[ 296], | 99.00th=[ 397], 99.50th=[ 456], 99.90th=[ 735], 99.95th=[ 894], | 99.99th=[ 894] bw ( KiB/s): min= 7, max= 55, per=24.98%, avg=21.24, stdev=10.58, samples=562 iops : min= 1, max= 13, avg= 4.66, stdev= 2.71, samples=562 lat (msec) : 10=2.47%, 20=16.44%, 50=20.05%, 100=21.32%, 250=34.08% lat (msec) : 500=5.40%, 750=0.20%, 1000=0.03% cpu : usr=0.04%, sys=1.30%, ctx=5274, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=81.5KiB/s (83.5kB/s), 20.2KiB/s-20.7KiB/s (20.6kB/s-21.2kB/s), io=23.4MiB (24.6MB), run=292571-294556msec WRITE: bw=85.3KiB/s (87.4kB/s), 21.3KiB/s-21.6KiB/s (21.8kB/s-22.1kB/s), io=24.6MiB (25.7MB), run=292571-294556msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 fio: pid=145401, err=2/file:filesetup.c:713, func=open(/mnt/lustre/f398b.sanity), error=No such file or directory fio: pid=145403, err=2/file:filesetup.c:713, func=open(/mnt/lustre/f398b.sanity), error=No such file or directory fio: pid=145402, err=2/file:filesetup.c:713, func=open(/mnt/lustre/f398b.sanity), error=No such file or directory Run status group 0 (all jobs): rand-rw: (groupid=0, jobs=1): err= 0: pid=145396: Fri Sep 4 05:18:00 2026 read: IOPS=6, BW=112KiB/s (114kB/s)(5920KiB/53070msec) clat (msec): min=7, max=165, avg=29.07, stdev=16.11 lat (msec): min=7, max=165, avg=29.07, stdev=16.11 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 13], 10.00th=[ 14], 20.00th=[ 17], | 30.00th=[ 21], 40.00th=[ 23], 50.00th=[ 26], 60.00th=[ 29], | 70.00th=[ 34], 80.00th=[ 39], 90.00th=[ 47], 95.00th=[ 59], | 99.00th=[ 85], 99.50th=[ 104], 99.90th=[ 167], 99.95th=[ 167], | 99.99th=[ 167] bw ( KiB/s): min= 31, max= 415, per=26.25%, avg=116.57, stdev=62.04, samples=100 iops : min= 1, max= 25, avg= 6.52, stdev= 3.91, samples=100 write: IOPS=7, BW=120KiB/s (123kB/s)(6368KiB/53070msec); 0 zone resets clat (msec): min=42, max=232, avg=106.17, stdev=30.28 lat (msec): min=42, max=232, avg=106.17, stdev=30.28 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 65], 10.00th=[ 70], 20.00th=[ 82], | 30.00th=[ 89], 40.00th=[ 95], 50.00th=[ 103], 60.00th=[ 112], | 70.00th=[ 120], 80.00th=[ 130], 90.00th=[ 146], 95.00th=[ 159], | 99.00th=[ 197], 99.50th=[ 218], 99.90th=[ 232], 99.95th=[ 232], | 99.99th=[ 232] bw ( KiB/s): min= 31, max= 191, per=25.60%, avg=119.55, stdev=31.02, samples=105 iops : min= 1, max= 11, avg= 6.70, stdev= 1.94, samples=105 lat (msec) : 10=0.52%, 20=14.32%, 50=30.21%, 100=27.21%, 250=27.73% cpu : usr=0.06%, sys=1.83%, ctx=1550, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=145397: Fri Sep 4 05:18:00 2026 read: IOPS=6, BW=107KiB/s (109kB/s)(5744KiB/53874msec) clat (usec): min=6625, max=90494, avg=27556.83, stdev=13327.74 lat (usec): min=6626, max=90496, avg=27558.57, stdev=13327.71 clat percentiles (usec): | 1.00th=[ 7504], 5.00th=[10552], 10.00th=[12780], 20.00th=[15926], | 30.00th=[18744], 40.00th=[21890], 50.00th=[24773], 60.00th=[28967], | 70.00th=[33817], 80.00th=[38011], 90.00th=[45351], 95.00th=[51119], | 99.00th=[66323], 99.50th=[71828], 99.90th=[90702], 99.95th=[90702], | 99.99th=[90702] bw ( KiB/s): min= 31, max= 415, per=25.63%, avg=113.80, stdev=71.14, samples=100 iops : min= 1, max= 25, avg= 6.34, stdev= 4.53, samples=100 write: IOPS=7, BW=121KiB/s (124kB/s)(6544KiB/53874msec); 0 zone resets clat (msec): min=55, max=254, avg=107.09, stdev=31.95 lat (msec): min=55, max=254, avg=107.09, stdev=31.94 clat percentiles (msec): | 1.00th=[ 61], 5.00th=[ 65], 10.00th=[ 71], 20.00th=[ 81], | 30.00th=[ 88], 40.00th=[ 95], 50.00th=[ 101], 60.00th=[ 110], | 70.00th=[ 117], 80.00th=[ 133], 90.00th=[ 148], 95.00th=[ 169], | 99.00th=[ 201], 99.50th=[ 209], 99.90th=[ 255], 99.95th=[ 255], | 99.99th=[ 255] bw ( KiB/s): min= 63, max= 190, per=25.87%, avg=120.83, stdev=30.59, samples=107 iops : min= 3, max= 11, avg= 6.79, stdev= 1.89, samples=107 lat (msec) : 10=1.82%, 20=13.93%, 50=28.39%, 100=28.39%, 250=27.34% lat (msec) : 500=0.13% cpu : usr=0.07%, sys=1.88%, ctx=1509, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=145398: Fri Sep 4 05:18:00 2026 read: IOPS=7, BW=121KiB/s (124kB/s)(6256KiB/51799msec) clat (msec): min=7, max=148, avg=29.35, stdev=16.58 lat (msec): min=7, max=148, avg=29.35, stdev=16.58 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 14], 20.00th=[ 17], | 30.00th=[ 20], 40.00th=[ 23], 50.00th=[ 26], 60.00th=[ 30], | 70.00th=[ 35], 80.00th=[ 42], 90.00th=[ 49], 95.00th=[ 57], | 99.00th=[ 88], 99.50th=[ 121], 99.90th=[ 148], 99.95th=[ 148], | 99.99th=[ 148] bw ( KiB/s): min= 31, max= 319, per=28.49%, avg=126.51, stdev=69.93, samples=98 iops : min= 1, max= 19, avg= 7.15, stdev= 4.42, samples=98 write: IOPS=7, BW=116KiB/s (119kB/s)(6032KiB/51799msec); 0 zone resets clat (msec): min=51, max=263, avg=106.36, stdev=33.14 lat (msec): min=51, max=263, avg=106.36, stdev=33.14 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 64], 10.00th=[ 72], 20.00th=[ 80], | 30.00th=[ 87], 40.00th=[ 92], 50.00th=[ 100], 60.00th=[ 107], | 70.00th=[ 121], 80.00th=[ 133], 90.00th=[ 150], 95.00th=[ 167], | 99.00th=[ 222], 99.50th=[ 245], 99.90th=[ 264], 99.95th=[ 264], | 99.99th=[ 264] bw ( KiB/s): min= 62, max= 191, per=24.82%, avg=115.91, stdev=30.85, samples=102 iops : min= 3, max= 11, avg= 6.48, stdev= 1.92, samples=102 lat (msec) : 10=1.17%, 20=16.67%, 50=28.26%, 100=29.43%, 250=24.35% lat (msec) : 500=0.13% cpu : usr=0.07%, sys=2.00%, ctx=1517, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=145399: Fri Sep 4 05:18:00 2026 read: IOPS=7, BW=114KiB/s (117kB/s)(6032KiB/52928msec) clat (msec): min=8, max=115, avg=28.61, stdev=14.97 lat (msec): min=8, max=115, avg=28.62, stdev=14.97 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 18], | 30.00th=[ 20], 40.00th=[ 22], 50.00th=[ 26], 60.00th=[ 29], | 70.00th=[ 34], 80.00th=[ 39], 90.00th=[ 48], 95.00th=[ 58], | 99.00th=[ 82], 99.50th=[ 90], 99.90th=[ 116], 99.95th=[ 116], | 99.99th=[ 116] bw ( KiB/s): min= 31, max= 320, per=27.45%, avg=121.90, stdev=64.33, samples=97 iops : min= 1, max= 20, avg= 6.85, stdev= 4.00, samples=97 write: IOPS=7, BW=118KiB/s (121kB/s)(6256KiB/52928msec); 0 zone resets clat (msec): min=42, max=255, avg=107.28, stdev=33.60 lat (msec): min=42, max=255, avg=107.29, stdev=33.60 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 62], 10.00th=[ 70], 20.00th=[ 79], | 30.00th=[ 87], 40.00th=[ 95], 50.00th=[ 104], 60.00th=[ 113], | 70.00th=[ 122], 80.00th=[ 132], 90.00th=[ 148], 95.00th=[ 163], | 99.00th=[ 215], 99.50th=[ 224], 99.90th=[ 255], 99.95th=[ 255], | 99.99th=[ 255] bw ( KiB/s): min= 63, max= 221, per=25.27%, avg=118.01, stdev=30.98, samples=105 iops : min= 3, max= 13, avg= 6.61, stdev= 1.93, samples=105 lat (msec) : 10=0.52%, 20=16.15%, 50=28.52%, 100=26.82%, 250=27.86% lat (msec) : 500=0.13% cpu : usr=0.06%, sys=1.91%, ctx=1581, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=445KiB/s (455kB/s), 107KiB/s-121KiB/s (109kB/s-124kB/s), io=23.4MiB (24.5MB), run=51799-53874msec WRITE: bw=468KiB/s (479kB/s), 116KiB/s-121KiB/s (119kB/s-124kB/s), io=24.6MiB (25.8MB), run=51799-53874msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 Run status group 0 (all jobs): rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 Run status group 0 (all jobs): mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 rand-rw: Laying out IO file (1 file / 12MiB) fio: file creation disallowed by allow_file_create=0 Run status group 0 (all jobs): Run status group 0 (all jobs):