== sanity test 398b: DIO and buffer IO race ============== 19:45:53 (1755042353) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.75205 s, 13.4 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189170: Tue Aug 12 19:47:20 2025 read: IOPS=20, BW=80.9KiB/s (82.8kB/s)(5992KiB/74108msec) clat (usec): min=133, max=138838, avg=27335.74, stdev=16130.16 lat (usec): min=134, max=138839, avg=27339.13, stdev=16129.24 clat percentiles (usec): | 1.00th=[ 176], 5.00th=[ 9110], 10.00th=[ 12125], 20.00th=[ 15270], | 30.00th=[ 18220], 40.00th=[ 20841], 50.00th=[ 23987], 60.00th=[ 27919], | 70.00th=[ 32637], 80.00th=[ 38011], 90.00th=[ 46400], 95.00th=[ 55837], | 99.00th=[ 88605], 99.50th=[ 92799], 99.90th=[121111], 99.95th=[139461], | 99.99th=[139461] bw ( KiB/s): min= 22, max= 143, per=25.04%, avg=80.87, stdev=22.61, samples=146 iops : min= 5, max= 35, avg=19.62, stdev= 5.66, samples=146 write: IOPS=21, BW=84.0KiB/s (86.0kB/s)(6296KiB/74108msec); 0 zone resets clat (usec): min=807, max=737912, avg=20896.20, stdev=27006.97 lat (usec): min=808, max=737913, avg=20897.68, stdev=27006.95 clat percentiles (usec): | 1.00th=[ 996], 5.00th=[ 1156], 10.00th=[ 1467], 20.00th=[ 2606], | 30.00th=[ 4883], 40.00th=[ 11207], 50.00th=[ 15664], 60.00th=[ 20055], | 70.00th=[ 26346], 80.00th=[ 34866], 90.00th=[ 49021], 95.00th=[ 60556], | 99.00th=[ 84411], 99.50th=[106431], 99.90th=[141558], 99.95th=[734004], | 99.99th=[734004] bw ( KiB/s): min= 8, max= 151, per=25.05%, avg=84.93, stdev=28.97, samples=146 iops : min= 2, max= 37, avg=20.64, stdev= 7.27, samples=146 lat (usec) : 250=0.81%, 500=0.75%, 750=0.10%, 1000=0.59% lat (msec) : 2=7.00%, 4=6.48%, 10=7.06%, 20=25.88%, 50=42.55% lat (msec) : 100=8.27%, 250=0.49%, 750=0.03% cpu : usr=0.11%, sys=10.63%, ctx=6224, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189172: Tue Aug 12 19:47:20 2025 read: IOPS=20, BW=80.7KiB/s (82.6kB/s)(5940KiB/73597msec) clat (usec): min=103, max=137642, avg=27710.22, stdev=16214.21 lat (usec): min=104, max=137643, avg=27725.42, stdev=16233.39 clat percentiles (usec): | 1.00th=[ 155], 5.00th=[ 9110], 10.00th=[ 12387], 20.00th=[ 15533], | 30.00th=[ 17957], 40.00th=[ 20841], 50.00th=[ 24511], 60.00th=[ 28443], | 70.00th=[ 32900], 80.00th=[ 38536], 90.00th=[ 47449], 95.00th=[ 56361], | 99.00th=[ 85459], 99.50th=[ 99091], 99.90th=[127402], 99.95th=[137364], | 99.99th=[137364] bw ( KiB/s): min= 7, max= 136, per=25.00%, avg=80.76, stdev=23.52, samples=145 iops : min= 1, max= 34, avg=19.59, stdev= 5.90, samples=145 write: IOPS=21, BW=86.3KiB/s (88.3kB/s)(6348KiB/73597msec); 0 zone resets clat (usec): min=839, max=765312, avg=20259.50, stdev=27466.23 lat (usec): min=841, max=765314, avg=20260.85, stdev=27466.28 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1139], 10.00th=[ 1500], 20.00th=[ 2704], | 30.00th=[ 4752], 40.00th=[ 10290], 50.00th=[ 15008], 60.00th=[ 19530], | 70.00th=[ 24511], 80.00th=[ 32900], 90.00th=[ 46924], 95.00th=[ 57410], | 99.00th=[ 88605], 99.50th=[106431], 99.90th=[185598], 99.95th=[767558], | 99.99th=[767558] bw ( KiB/s): min= 7, max= 159, per=25.33%, avg=85.86, stdev=28.60, samples=146 iops : min= 1, max= 39, avg=20.87, stdev= 7.13, samples=146 lat (usec) : 250=0.91%, 500=0.72%, 750=0.03%, 1000=0.29% lat (msec) : 2=7.62%, 4=5.86%, 10=8.01%, 20=25.42%, 50=42.87% lat (msec) : 100=7.71%, 250=0.52%, 1000=0.03% cpu : usr=0.13%, sys=10.61%, ctx=6078, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189175: Tue Aug 12 19:47:20 2025 read: IOPS=20, BW=81.3KiB/s (83.2kB/s)(6028KiB/74158msec) clat (usec): min=113, max=124925, avg=27688.97, stdev=16277.30 lat (usec): min=114, max=124926, avg=27690.36, stdev=16277.30 clat percentiles (usec): | 1.00th=[ 153], 5.00th=[ 9503], 10.00th=[ 12256], 20.00th=[ 15139], | 30.00th=[ 18220], 40.00th=[ 21365], 50.00th=[ 24773], 60.00th=[ 28443], | 70.00th=[ 32637], 80.00th=[ 38011], 90.00th=[ 46400], 95.00th=[ 57410], | 99.00th=[ 84411], 99.50th=[101188], 99.90th=[113771], 99.95th=[125305], | 99.99th=[125305] bw ( KiB/s): min= 32, max= 133, per=25.39%, avg=82.01, stdev=22.19, samples=145 iops : min= 8, max= 33, avg=19.92, stdev= 5.57, samples=145 write: IOPS=21, BW=84.4KiB/s (86.4kB/s)(6260KiB/74158msec); 0 zone resets clat (usec): min=930, max=802691, avg=20537.94, stdev=28447.03 lat (usec): min=931, max=802693, avg=20539.37, stdev=28447.09 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1188], 10.00th=[ 1483], 20.00th=[ 2474], | 30.00th=[ 4555], 40.00th=[ 9503], 50.00th=[ 14353], 60.00th=[ 19268], | 70.00th=[ 25297], 80.00th=[ 34866], 90.00th=[ 47449], 95.00th=[ 58983], | 99.00th=[ 89654], 99.50th=[104334], 99.90th=[156238], 99.95th=[801113], | 99.99th=[801113] bw ( KiB/s): min= 16, max= 182, per=24.95%, avg=84.59, stdev=31.88, samples=146 iops : min= 4, max= 45, avg=20.56, stdev= 8.01, samples=146 lat (usec) : 250=0.98%, 500=0.65%, 1000=0.26% lat (msec) : 2=7.81%, 4=6.67%, 10=7.23%, 20=24.67%, 50=43.16% lat (msec) : 100=7.98%, 250=0.55%, 1000=0.03% cpu : usr=0.12%, sys=10.58%, ctx=6093, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189176: Tue Aug 12 19:47:20 2025 read: IOPS=20, BW=81.7KiB/s (83.7kB/s)(6052KiB/74069msec) clat (usec): min=111, max=135883, avg=26656.15, stdev=15410.57 lat (usec): min=112, max=135883, avg=26657.56, stdev=15410.47 clat percentiles (usec): | 1.00th=[ 153], 5.00th=[ 8160], 10.00th=[ 11076], 20.00th=[ 14746], | 30.00th=[ 17433], 40.00th=[ 20579], 50.00th=[ 23987], 60.00th=[ 27657], | 70.00th=[ 31589], 80.00th=[ 37487], 90.00th=[ 46400], 95.00th=[ 54264], | 99.00th=[ 74974], 99.50th=[ 85459], 99.90th=[113771], 99.95th=[135267], | 99.99th=[135267] bw ( KiB/s): min= 39, max= 144, per=25.50%, avg=82.37, stdev=21.60, samples=145 iops : min= 9, max= 36, avg=20.00, stdev= 5.40, samples=145 write: IOPS=21, BW=84.2KiB/s (86.2kB/s)(6236KiB/74069msec); 0 zone resets clat (usec): min=798, max=813611, avg=21455.51, stdev=28922.46 lat (usec): min=798, max=813613, avg=21457.01, stdev=28922.45 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 1172], 10.00th=[ 1565], 20.00th=[ 2671], | 30.00th=[ 4948], 40.00th=[ 10421], 50.00th=[ 15795], 60.00th=[ 20055], | 70.00th=[ 26346], 80.00th=[ 36439], 90.00th=[ 49546], 95.00th=[ 62653], | 99.00th=[ 88605], 99.50th=[105382], 99.90th=[166724], 99.95th=[809501], | 99.99th=[809501] bw ( KiB/s): min= 8, max= 192, per=24.84%, avg=84.21, stdev=27.13, samples=146 iops : min= 2, max= 48, avg=20.46, stdev= 6.82, samples=146 lat (usec) : 250=1.17%, 500=0.46%, 750=0.07%, 1000=0.23% lat (msec) : 2=7.26%, 4=5.99%, 10=8.56%, 20=25.36%, 50=42.45% lat (msec) : 100=8.07%, 250=0.36%, 1000=0.03% cpu : usr=0.11%, sys=10.79%, ctx=6018, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=324KiB/s (332kB/s), 80.7KiB/s-81.7KiB/s (82.6kB/s-83.7kB/s), io=23.4MiB (24.6MB), run=73597-74158msec WRITE: bw=339KiB/s (347kB/s), 84.2KiB/s-86.3KiB/s (86.2kB/s-88.3kB/s), io=24.6MiB (25.7MB), run=73597-74158msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189169: Tue Aug 12 19:47:50 2025 read: IOPS=14, BW=57.8KiB/s (59.2kB/s)(5992KiB/103692msec) clat (msec): min=4, max=885, avg=26.70, stdev=30.55 lat (msec): min=4, max=885, avg=26.70, stdev=30.55 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 16], 50.00th=[ 19], 60.00th=[ 22], | 70.00th=[ 28], 80.00th=[ 39], 90.00th=[ 56], 95.00th=[ 71], | 99.00th=[ 103], 99.50th=[ 120], 99.90th=[ 186], 99.95th=[ 885], | 99.99th=[ 885] bw ( KiB/s): min= 7, max= 158, per=25.08%, avg=57.69, stdev=30.66, samples=203 iops : min= 1, max= 39, avg=13.83, stdev= 7.72, samples=203 write: IOPS=15, BW=60.7KiB/s (62.2kB/s)(6296KiB/103692msec); 0 zone resets clat (msec): min=9, max=175, avg=40.30, stdev=23.07 lat (msec): min=9, max=175, avg=40.30, stdev=23.07 clat percentiles (msec): | 1.00th=[ 13], 5.00th=[ 16], 10.00th=[ 19], 20.00th=[ 23], | 30.00th=[ 26], 40.00th=[ 30], 50.00th=[ 34], 60.00th=[ 40], | 70.00th=[ 47], 80.00th=[ 57], 90.00th=[ 70], 95.00th=[ 84], | 99.00th=[ 126], 99.50th=[ 140], 99.90th=[ 167], 99.95th=[ 176], | 99.99th=[ 176] bw ( KiB/s): min= 15, max= 151, per=25.17%, avg=60.66, stdev=28.00, samples=203 iops : min= 3, max= 37, avg=14.57, stdev= 7.03, samples=203 lat (msec) : 10=5.70%, 20=28.39%, 50=46.03%, 100=18.36%, 250=1.50% lat (msec) : 1000=0.03% cpu : usr=0.08%, sys=3.85%, ctx=3604, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189171: Tue Aug 12 19:47:50 2025 read: IOPS=14, BW=57.3KiB/s (58.6kB/s)(5940KiB/103733msec) clat (msec): min=4, max=785, avg=26.45, stdev=29.27 lat (msec): min=4, max=785, avg=26.45, stdev=29.27 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 16], 50.00th=[ 18], 60.00th=[ 22], | 70.00th=[ 28], 80.00th=[ 36], 90.00th=[ 55], 95.00th=[ 72], | 99.00th=[ 111], 99.50th=[ 132], 99.90th=[ 201], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 7, max= 167, per=24.89%, avg=57.25, stdev=32.11, samples=203 iops : min= 1, max= 41, avg=13.72, stdev= 8.06, samples=203 write: IOPS=15, BW=61.2KiB/s (62.7kB/s)(6348KiB/103733msec); 0 zone resets clat (msec): min=7, max=191, avg=40.45, stdev=24.20 lat (msec): min=7, max=191, avg=40.45, stdev=24.20 clat percentiles (msec): | 1.00th=[ 13], 5.00th=[ 17], 10.00th=[ 19], 20.00th=[ 23], | 30.00th=[ 26], 40.00th=[ 29], 50.00th=[ 33], 60.00th=[ 39], | 70.00th=[ 45], 80.00th=[ 57], 90.00th=[ 73], 95.00th=[ 90], | 99.00th=[ 128], 99.50th=[ 144], 99.90th=[ 176], 99.95th=[ 192], | 99.99th=[ 192] bw ( KiB/s): min= 8, max= 159, per=25.38%, avg=61.17, stdev=29.63, samples=203 iops : min= 2, max= 39, avg=14.70, stdev= 7.46, samples=203 lat (msec) : 10=5.79%, 20=28.29%, 50=47.20%, 100=16.34%, 250=2.34% lat (msec) : 1000=0.03% cpu : usr=0.08%, sys=3.70%, ctx=3607, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189173: Tue Aug 12 19:47:50 2025 read: IOPS=14, BW=58.1KiB/s (59.5kB/s)(6028KiB/103732msec) clat (msec): min=4, max=198, avg=26.12, stdev=22.54 lat (msec): min=4, max=198, avg=26.12, stdev=22.54 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 16], 50.00th=[ 18], 60.00th=[ 22], | 70.00th=[ 27], 80.00th=[ 36], 90.00th=[ 55], 95.00th=[ 71], | 99.00th=[ 112], 99.50th=[ 142], 99.90th=[ 186], 99.95th=[ 199], | 99.99th=[ 199] bw ( KiB/s): min= 7, max= 159, per=25.51%, avg=58.67, stdev=30.84, samples=201 iops : min= 1, max= 39, avg=14.08, stdev= 7.75, samples=201 write: IOPS=15, BW=60.3KiB/s (61.8kB/s)(6260KiB/103732msec); 0 zone resets clat (msec): min=7, max=794, avg=40.96, stdev=30.75 lat (msec): min=7, max=794, avg=40.96, stdev=30.75 clat percentiles (msec): | 1.00th=[ 13], 5.00th=[ 17], 10.00th=[ 19], 20.00th=[ 23], | 30.00th=[ 26], 40.00th=[ 29], 50.00th=[ 33], 60.00th=[ 39], | 70.00th=[ 46], 80.00th=[ 56], 90.00th=[ 74], 95.00th=[ 89], | 99.00th=[ 133], 99.50th=[ 144], 99.90th=[ 194], 99.95th=[ 793], | 99.99th=[ 793] bw ( KiB/s): min= 15, max= 143, per=24.87%, avg=59.94, stdev=28.93, samples=204 iops : min= 3, max= 35, avg=14.40, stdev= 7.27, samples=204 lat (msec) : 10=6.35%, 20=28.03%, 50=47.14%, 100=16.21%, 250=2.25% lat (msec) : 1000=0.03% cpu : usr=0.10%, sys=3.78%, ctx=3590, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189174: Tue Aug 12 19:47:50 2025 read: IOPS=14, BW=58.2KiB/s (59.6kB/s)(6052KiB/104013msec) clat (msec): min=4, max=170, avg=26.21, stdev=21.75 lat (msec): min=4, max=170, avg=26.21, stdev=21.75 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 16], 50.00th=[ 19], 60.00th=[ 22], | 70.00th=[ 27], 80.00th=[ 37], 90.00th=[ 55], 95.00th=[ 73], | 99.00th=[ 108], 99.50th=[ 124], 99.90th=[ 169], 99.95th=[ 171], | 99.99th=[ 171] bw ( KiB/s): min= 7, max= 174, per=25.38%, avg=58.36, stdev=30.68, samples=203 iops : min= 1, max= 43, avg=14.00, stdev= 7.69, samples=203 write: IOPS=14, BW=59.0KiB/s (61.4kB/s)(6236KiB/104013msec); 0 zone resets clat (msec): min=7, max=798, avg=41.11, stdev=30.82 lat (msec): min=7, max=798, avg=41.12, stdev=30.82 clat percentiles (msec): | 1.00th=[ 13], 5.00th=[ 17], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 26], 40.00th=[ 29], 50.00th=[ 33], 60.00th=[ 40], | 70.00th=[ 47], 80.00th=[ 57], 90.00th=[ 72], 95.00th=[ 86], | 99.00th=[ 136], 99.50th=[ 153], 99.90th=[ 205], 99.95th=[ 802], | 99.99th=[ 802] bw ( KiB/s): min= 7, max= 136, per=24.69%, avg=59.50, stdev=28.21, samples=205 iops : min= 1, max= 34, avg=14.28, stdev= 7.11, samples=205 lat (msec) : 10=6.22%, 20=27.60%, 50=47.01%, 100=17.12%, 250=2.02% lat (msec) : 1000=0.03% cpu : usr=0.09%, sys=3.68%, ctx=3592, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=231KiB/s (236kB/s), 57.3KiB/s-58.2KiB/s (58.6kB/s-59.6kB/s), io=23.4MiB (24.6MB), run=103692-104013msec WRITE: bw=242KiB/s (248kB/s), 59.0KiB/s-61.2KiB/s (61.4kB/s-62.7kB/s), io=24.6MiB (25.7MB), run=103692-104013msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189217: Tue Aug 12 19:48:14 2025 read: IOPS=20, BW=326KiB/s (334kB/s)(5920KiB/18133msec) clat (usec): min=126, max=96828, avg=25740.47, stdev=16075.96 lat (usec): min=126, max=96829, avg=25741.63, stdev=16075.95 clat percentiles (usec): | 1.00th=[ 159], 5.00th=[ 306], 10.00th=[10814], 20.00th=[14353], | 30.00th=[16909], 40.00th=[19792], 50.00th=[23200], 60.00th=[25822], | 70.00th=[30540], 80.00th=[35914], 90.00th=[46924], 95.00th=[54789], | 99.00th=[90702], 99.50th=[95945], 99.90th=[96994], 99.95th=[96994], | 99.99th=[96994] bw ( KiB/s): min= 220, max= 505, per=24.49%, avg=318.57, stdev=72.84, samples=35 iops : min= 13, max= 31, avg=19.26, stdev= 4.51, samples=35 write: IOPS=21, BW=351KiB/s (360kB/s)(6368KiB/18133msec); 0 zone resets clat (usec): min=1517, max=109527, avg=21284.14, stdev=19728.92 lat (usec): min=1520, max=109530, avg=21290.02, stdev=19740.50 clat percentiles (usec): | 1.00th=[ 1713], 5.00th=[ 2540], 10.00th=[ 3163], 20.00th=[ 4359], | 30.00th=[ 6521], 40.00th=[ 10290], 50.00th=[ 15401], 60.00th=[ 20841], | 70.00th=[ 27132], 80.00th=[ 36439], 90.00th=[ 49021], 95.00th=[ 60031], | 99.00th=[ 94897], 99.50th=[104334], 99.90th=[109577], 99.95th=[109577], | 99.99th=[109577] bw ( KiB/s): min= 94, max= 542, per=25.55%, avg=349.83, stdev=109.57, samples=35 iops : min= 5, max= 33, avg=21.20, stdev= 6.86, samples=35 lat (usec) : 250=1.95%, 500=1.30%, 1000=0.13% lat (msec) : 2=1.30%, 4=7.55%, 10=12.76%, 20=25.00%, 50=41.67% lat (msec) : 100=8.07%, 250=0.26% cpu : usr=0.10%, sys=14.82%, ctx=1779, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189218: Tue Aug 12 19:48:14 2025 read: IOPS=19, BW=320KiB/s (327kB/s)(5744KiB/17970msec) clat (usec): min=147, max=75889, avg=26147.42, stdev=14611.85 lat (usec): min=148, max=75891, avg=26167.92, stdev=14619.60 clat percentiles (usec): | 1.00th=[ 227], 5.00th=[ 371], 10.00th=[11600], 20.00th=[14877], | 30.00th=[17695], 40.00th=[20841], 50.00th=[24511], 60.00th=[26870], | 70.00th=[31589], 80.00th=[36963], 90.00th=[44303], 95.00th=[52167], | 99.00th=[70779], 99.50th=[74974], 99.90th=[76022], 99.95th=[76022], | 99.99th=[76022] bw ( KiB/s): min= 159, max= 542, per=24.38%, avg=317.23, stdev=89.54, samples=35 iops : min= 9, max= 33, avg=19.14, stdev= 5.56, samples=35 write: IOPS=22, BW=364KiB/s (373kB/s)(6544KiB/17970msec); 0 zone resets clat (usec): min=1344, max=106581, avg=20790.77, stdev=19334.40 lat (usec): min=1345, max=106583, avg=20793.43, stdev=19334.40 clat percentiles (usec): | 1.00th=[ 1500], 5.00th=[ 2040], 10.00th=[ 2606], 20.00th=[ 4228], | 30.00th=[ 6915], 40.00th=[ 11469], 50.00th=[ 16319], 60.00th=[ 20055], | 70.00th=[ 24511], 80.00th=[ 31589], 90.00th=[ 46400], 95.00th=[ 65799], | 99.00th=[ 84411], 99.50th=[ 92799], 99.90th=[106431], 99.95th=[106431], | 99.99th=[106431] bw ( KiB/s): min= 158, max= 539, per=26.09%, avg=357.23, stdev=87.29, samples=35 iops : min= 9, max= 33, avg=21.69, stdev= 5.48, samples=35 lat (usec) : 250=0.78%, 500=2.08%, 750=0.26% lat (msec) : 2=2.47%, 4=7.16%, 10=10.42%, 20=26.04%, 50=43.49% lat (msec) : 100=7.03%, 250=0.26% cpu : usr=0.13%, sys=15.35%, ctx=1815, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189219: Tue Aug 12 19:48:14 2025 read: IOPS=21, BW=340KiB/s (348kB/s)(6256KiB/18398msec) clat (usec): min=142, max=107560, avg=26562.16, stdev=16888.99 lat (usec): min=144, max=107562, avg=26563.48, stdev=16888.95 clat percentiles (usec): | 1.00th=[ 153], 5.00th=[ 355], 10.00th=[ 8455], 20.00th=[ 14615], | 30.00th=[ 17695], 40.00th=[ 20317], 50.00th=[ 24773], 60.00th=[ 27657], | 70.00th=[ 31851], 80.00th=[ 36963], 90.00th=[ 47449], 95.00th=[ 56886], | 99.00th=[ 88605], 99.50th=[ 92799], 99.90th=[107480], 99.95th=[107480], | 99.99th=[107480] bw ( KiB/s): min= 159, max= 572, per=26.04%, avg=338.78, stdev=88.00, samples=36 iops : min= 9, max= 35, avg=20.50, stdev= 5.45, samples=36 write: IOPS=20, BW=328KiB/s (336kB/s)(6032KiB/18398msec); 0 zone resets clat (usec): min=1564, max=94580, avg=20946.10, stdev=17279.55 lat (usec): min=1565, max=94583, avg=20952.00, stdev=17278.45 clat percentiles (usec): | 1.00th=[ 1696], 5.00th=[ 2737], 10.00th=[ 3982], 20.00th=[ 5407], | 30.00th=[ 8979], 40.00th=[12911], 50.00th=[16319], 60.00th=[20579], | 70.00th=[25560], 80.00th=[34341], 90.00th=[47449], 95.00th=[55837], | 99.00th=[74974], 99.50th=[87557], 99.90th=[94897], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 125, max= 480, per=23.46%, avg=321.19, stdev=99.82, samples=36 iops : min= 7, max= 30, avg=19.39, stdev= 6.29, samples=36 lat (usec) : 250=1.17%, 500=2.60%, 750=0.39%, 1000=0.39% lat (msec) : 2=1.17%, 4=4.30%, 10=11.59%, 20=26.82%, 50=43.49% lat (msec) : 100=7.94%, 250=0.13% cpu : usr=0.11%, sys=14.94%, ctx=1920, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189220: Tue Aug 12 19:48:14 2025 read: IOPS=20, BW=336KiB/s (344kB/s)(6032KiB/17977msec) clat (usec): min=118, max=108020, avg=26122.30, stdev=15885.27 lat (usec): min=119, max=108021, avg=26123.50, stdev=15885.22 clat percentiles (usec): | 1.00th=[ 155], 5.00th=[ 231], 10.00th=[ 8717], 20.00th=[ 14484], | 30.00th=[ 18482], 40.00th=[ 21627], 50.00th=[ 24249], 60.00th=[ 27919], | 70.00th=[ 31851], 80.00th=[ 37487], 90.00th=[ 45876], 95.00th=[ 53216], | 99.00th=[ 72877], 99.50th=[ 76022], 99.90th=[108528], 99.95th=[108528], | 99.99th=[108528] bw ( KiB/s): min= 126, max= 479, per=25.49%, avg=331.60, stdev=95.85, samples=35 iops : min= 7, max= 29, avg=20.06, stdev= 6.02, samples=35 write: IOPS=21, BW=348KiB/s (356kB/s)(6256KiB/17977msec); 0 zone resets clat (usec): min=1310, max=114594, avg=20491.92, stdev=17944.11 lat (usec): min=1312, max=114596, avg=20494.34, stdev=17943.87 clat percentiles (usec): | 1.00th=[ 1565], 5.00th=[ 1942], 10.00th=[ 3261], 20.00th=[ 4752], | 30.00th=[ 7111], 40.00th=[ 11469], 50.00th=[ 15401], 60.00th=[ 20317], | 70.00th=[ 26346], 80.00th=[ 33424], 90.00th=[ 45876], 95.00th=[ 54264], | 99.00th=[ 71828], 99.50th=[ 99091], 99.90th=[114820], 99.95th=[114820], | 99.99th=[114820] bw ( KiB/s): min= 157, max= 570, per=25.27%, avg=346.00, stdev=93.93, samples=35 iops : min= 9, max= 35, avg=20.94, stdev= 5.81, samples=35 lat (usec) : 250=2.47%, 500=1.56%, 750=0.39%, 1000=0.13% lat (msec) : 2=2.86%, 4=4.95%, 10=11.59%, 20=23.96%, 50=44.66% lat (msec) : 100=7.16%, 250=0.26% cpu : usr=0.16%, sys=15.02%, ctx=1872, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1302KiB/s (1333kB/s), 320KiB/s-340KiB/s (327kB/s-348kB/s), io=23.4MiB (24.5MB), run=17970-18398msec WRITE: bw=1370KiB/s (1403kB/s), 328KiB/s-364KiB/s (336kB/s-373kB/s), io=24.6MiB (25.8MB), run=17970-18398msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189222: Tue Aug 12 19:48:20 2025 read: IOPS=15, BW=248KiB/s (254kB/s)(5920KiB/23843msec) clat (msec): min=5, max=164, avg=25.38, stdev=22.09 lat (msec): min=5, max=164, avg=25.38, stdev=22.09 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 12], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 22], | 70.00th=[ 28], 80.00th=[ 38], 90.00th=[ 59], 95.00th=[ 74], | 99.00th=[ 110], 99.50th=[ 132], 99.90th=[ 165], 99.95th=[ 165], | 99.99th=[ 165] bw ( KiB/s): min= 31, max= 701, per=24.70%, avg=246.02, stdev=155.38, samples=47 iops : min= 1, max= 43, avg=14.70, stdev= 9.76, samples=47 write: IOPS=16, BW=267KiB/s (273kB/s)(6368KiB/23843msec); 0 zone resets clat (msec): min=7, max=162, avg=35.89, stdev=22.99 lat (msec): min=7, max=162, avg=35.89, stdev=22.99 clat percentiles (msec): | 1.00th=[ 11], 5.00th=[ 14], 10.00th=[ 16], 20.00th=[ 19], | 30.00th=[ 21], 40.00th=[ 24], 50.00th=[ 28], 60.00th=[ 34], | 70.00th=[ 43], 80.00th=[ 52], 90.00th=[ 71], 95.00th=[ 80], | 99.00th=[ 106], 99.50th=[ 148], 99.90th=[ 163], 99.95th=[ 163], | 99.99th=[ 163] bw ( KiB/s): min= 63, max= 573, per=25.42%, avg=266.40, stdev=142.92, samples=47 iops : min= 3, max= 35, avg=15.98, stdev= 9.07, samples=47 lat (msec) : 10=9.38%, 20=32.68%, 50=40.23%, 100=16.54%, 250=1.17% cpu : usr=0.07%, sys=5.66%, ctx=972, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189223: Tue Aug 12 19:48:20 2025 read: IOPS=14, BW=239KiB/s (245kB/s)(5744KiB/24033msec) clat (msec): min=4, max=109, avg=25.63, stdev=20.96 lat (msec): min=4, max=109, avg=25.63, stdev=20.96 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 19], 60.00th=[ 22], | 70.00th=[ 28], 80.00th=[ 40], 90.00th=[ 59], 95.00th=[ 77], | 99.00th=[ 94], 99.50th=[ 97], 99.90th=[ 110], 99.95th=[ 110], | 99.99th=[ 110] bw ( KiB/s): min= 55, max= 764, per=23.91%, avg=238.15, stdev=144.23, samples=47 iops : min= 3, max= 47, avg=14.21, stdev= 9.13, samples=47 write: IOPS=17, BW=272KiB/s (279kB/s)(6544KiB/24033msec); 0 zone resets clat (msec): min=8, max=162, avg=36.14, stdev=23.99 lat (msec): min=8, max=162, avg=36.14, stdev=23.99 clat percentiles (msec): | 1.00th=[ 11], 5.00th=[ 14], 10.00th=[ 16], 20.00th=[ 19], | 30.00th=[ 22], 40.00th=[ 24], 50.00th=[ 29], 60.00th=[ 33], | 70.00th=[ 41], 80.00th=[ 53], 90.00th=[ 66], 95.00th=[ 90], | 99.00th=[ 116], 99.50th=[ 140], 99.90th=[ 163], 99.95th=[ 163], | 99.99th=[ 163] bw ( KiB/s): min= 95, max= 669, per=25.73%, avg=269.64, stdev=140.02, samples=47 iops : min= 5, max= 41, avg=16.19, stdev= 8.81, samples=47 lat (msec) : 10=10.42%, 20=28.91%, 50=42.71%, 100=16.41%, 250=1.56% cpu : usr=0.11%, sys=5.71%, ctx=975, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189224: Tue Aug 12 19:48:20 2025 read: IOPS=16, BW=265KiB/s (272kB/s)(6256KiB/23583msec) clat (msec): min=5, max=139, avg=24.96, stdev=22.05 lat (msec): min=5, max=139, avg=24.97, stdev=22.05 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 26], 80.00th=[ 34], 90.00th=[ 58], 95.00th=[ 75], | 99.00th=[ 100], 99.50th=[ 133], 99.90th=[ 140], 99.95th=[ 140], | 99.99th=[ 140] bw ( KiB/s): min= 63, max= 768, per=26.72%, avg=266.09, stdev=170.90, samples=46 iops : min= 3, max= 48, avg=15.98, stdev=10.81, samples=46 write: IOPS=15, BW=256KiB/s (262kB/s)(6032KiB/23583msec); 0 zone resets clat (msec): min=8, max=143, avg=36.33, stdev=23.28 lat (msec): min=8, max=143, avg=36.33, stdev=23.28 clat percentiles (msec): | 1.00th=[ 11], 5.00th=[ 14], 10.00th=[ 16], 20.00th=[ 19], | 30.00th=[ 22], 40.00th=[ 25], 50.00th=[ 29], 60.00th=[ 34], | 70.00th=[ 42], 80.00th=[ 51], 90.00th=[ 70], 95.00th=[ 87], | 99.00th=[ 118], 99.50th=[ 142], 99.90th=[ 144], 99.95th=[ 144], | 99.99th=[ 144] bw ( KiB/s): min= 63, max= 571, per=24.06%, avg=252.13, stdev=124.99, samples=46 iops : min= 3, max= 35, avg=15.09, stdev= 7.92, samples=46 lat (msec) : 10=9.38%, 20=35.03%, 50=38.41%, 100=15.89%, 250=1.30% cpu : usr=0.09%, sys=5.81%, ctx=999, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189225: Tue Aug 12 19:48:20 2025 read: IOPS=16, BW=256KiB/s (262kB/s)(6032KiB/23545msec) clat (msec): min=4, max=164, avg=25.06, stdev=22.40 lat (msec): min=4, max=164, avg=25.06, stdev=22.40 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 25], 80.00th=[ 37], 90.00th=[ 58], 95.00th=[ 77], | 99.00th=[ 107], 99.50th=[ 118], 99.90th=[ 165], 99.95th=[ 165], | 99.99th=[ 165] bw ( KiB/s): min= 96, max= 640, per=25.58%, avg=254.80, stdev=145.74, samples=46 iops : min= 6, max= 40, avg=15.26, stdev= 9.20, samples=46 write: IOPS=16, BW=266KiB/s (272kB/s)(6256KiB/23545msec); 0 zone resets clat (msec): min=9, max=164, avg=35.64, stdev=22.33 lat (msec): min=9, max=164, avg=35.64, stdev=22.33 clat percentiles (msec): | 1.00th=[ 13], 5.00th=[ 14], 10.00th=[ 17], 20.00th=[ 19], | 30.00th=[ 22], 40.00th=[ 24], 50.00th=[ 28], 60.00th=[ 33], | 70.00th=[ 43], 80.00th=[ 52], 90.00th=[ 68], 95.00th=[ 82], | 99.00th=[ 117], 99.50th=[ 131], 99.90th=[ 165], 99.95th=[ 165], | 99.99th=[ 165] bw ( KiB/s): min= 63, max= 669, per=25.22%, avg=264.30, stdev=143.26, samples=46 iops : min= 3, max= 41, avg=15.87, stdev= 9.08, samples=46 lat (msec) : 10=9.51%, 20=33.20%, 50=39.84%, 100=16.28%, 250=1.17% cpu : usr=0.08%, sys=5.83%, ctx=980, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=997KiB/s (1021kB/s), 239KiB/s-265KiB/s (245kB/s-272kB/s), io=23.4MiB (24.5MB), run=23545-24033msec WRITE: bw=1049KiB/s (1074kB/s), 256KiB/s-272KiB/s (262kB/s-279kB/s), io=24.6MiB (25.8MB), run=23545-24033msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189239: Tue Aug 12 19:48:31 2025 read: IOPS=1, BW=1613KiB/s (1651kB/s)(7168KiB/4445msec) clat (usec): min=890, max=2665.5k, avg=459370.37, stdev=974971.96 lat (usec): min=891, max=2665.5k, avg=459371.89, stdev=974972.06 clat percentiles (usec): | 1.00th=[ 889], 5.00th=[ 889], 10.00th=[ 889], | 20.00th=[ 2212], 30.00th=[ 108528], 40.00th=[ 108528], | 50.00th=[ 139461], 60.00th=[ 147850], 70.00th=[ 147850], | 80.00th=[ 149947], 90.00th=[2667578], 95.00th=[2667578], | 99.00th=[2667578], 99.50th=[2667578], 99.90th=[2667578], | 99.95th=[2667578], 99.99th=[2667578] bw ( KiB/s): min= 4039, max= 5446, per=100.00%, avg=4742.50, stdev=994.90, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=1, BW=1152KiB/s (1180kB/s)(5120KiB/4445msec); 0 zone resets clat (msec): min=106, max=429, avg=242.42, stdev=134.30 lat (msec): min=106, max=429, avg=242.51, stdev=134.33 clat percentiles (msec): | 1.00th=[ 107], 5.00th=[ 107], 10.00th=[ 107], 20.00th=[ 107], | 30.00th=[ 159], 40.00th=[ 159], 50.00th=[ 184], 60.00th=[ 184], | 70.00th=[ 334], 80.00th=[ 334], 90.00th=[ 430], 95.00th=[ 430], | 99.00th=[ 430], 99.50th=[ 430], 99.90th=[ 430], 99.95th=[ 430], | 99.99th=[ 430] bw ( KiB/s): min= 1815, max= 4063, per=49.64%, avg=2939.00, stdev=1589.58, samples=2 iops : min= 1, max= 3, avg= 2.00, stdev= 1.41, samples=2 lat (usec) : 1000=8.33% lat (msec) : 4=8.33%, 250=58.33%, 500=16.67%, >=2000=8.33% cpu : usr=0.02%, sys=21.87%, ctx=529, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189240: Tue Aug 12 19:48:31 2025 read: IOPS=0, BW=423KiB/s (433kB/s)(2048KiB/4842msec) clat (msec): min=113, max=168, avg=141.20, stdev=38.74 lat (msec): min=113, max=168, avg=141.20, stdev=38.74 clat percentiles (msec): | 1.00th=[ 114], 5.00th=[ 114], 10.00th=[ 114], 20.00th=[ 114], | 30.00th=[ 114], 40.00th=[ 114], 50.00th=[ 114], 60.00th=[ 169], | 70.00th=[ 169], 80.00th=[ 169], 90.00th=[ 169], 95.00th=[ 169], | 99.00th=[ 169], 99.50th=[ 169], 99.90th=[ 169], 99.95th=[ 169], | 99.99th=[ 169] bw ( KiB/s): min= 1855, max= 2043, per=46.09%, avg=1949.00, stdev=132.94, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2115KiB/s (2166kB/s)(10.0MiB/4842msec); 0 zone resets clat (msec): min=99, max=2312, avg=449.83, stdev=670.25 lat (msec): min=99, max=2312, avg=449.94, stdev=670.25 clat percentiles (msec): | 1.00th=[ 100], 5.00th=[ 100], 10.00th=[ 100], 20.00th=[ 104], | 30.00th=[ 118], 40.00th=[ 125], 50.00th=[ 234], 60.00th=[ 259], | 70.00th=[ 300], 80.00th=[ 409], 90.00th=[ 542], 95.00th=[ 2299], | 99.00th=[ 2299], 99.50th=[ 2299], 99.90th=[ 2299], 99.95th=[ 2299], | 99.99th=[ 2299] bw ( KiB/s): min= 2031, max= 5919, per=75.19%, avg=4451.75, stdev=1864.55, samples=4 iops : min= 1, max= 5, avg= 3.50, stdev= 1.91, samples=4 lat (msec) : 100=8.33%, 250=50.00%, 500=25.00%, 750=8.33%, >=2000=8.33% cpu : usr=0.02%, sys=23.36%, ctx=206, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189241: Tue Aug 12 19:48:31 2025 read: IOPS=1, BW=1371KiB/s (1404kB/s)(6144KiB/4482msec) clat (usec): min=1041, max=227180, avg=153323.20, stdev=79161.91 lat (usec): min=1042, max=227181, avg=153324.55, stdev=79161.88 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 1045], 10.00th=[ 1045], 20.00th=[156238], | 30.00th=[156238], 40.00th=[162530], 50.00th=[162530], 60.00th=[170918], | 70.00th=[200279], 80.00th=[200279], 90.00th=[227541], 95.00th=[227541], | 99.00th=[227541], 99.50th=[227541], 99.90th=[227541], 99.95th=[227541], | 99.99th=[227541] bw ( KiB/s): min= 2031, max= 4039, per=78.95%, avg=3338.67, stdev=1133.43, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=1, BW=1371KiB/s (1404kB/s)(6144KiB/4482msec); 0 zone resets clat (msec): min=101, max=2060, avg=584.88, stdev=748.54 lat (msec): min=101, max=2060, avg=584.97, stdev=748.52 clat percentiles (msec): | 1.00th=[ 103], 5.00th=[ 103], 10.00th=[ 103], 20.00th=[ 159], | 30.00th=[ 159], 40.00th=[ 184], 50.00th=[ 184], 60.00th=[ 372], | 70.00th=[ 634], 80.00th=[ 634], 90.00th=[ 2056], 95.00th=[ 2056], | 99.00th=[ 2056], 99.50th=[ 2056], 99.90th=[ 2056], 99.95th=[ 2056], | 99.99th=[ 2056] bw ( KiB/s): min= 1969, max= 3696, per=41.02%, avg=2428.75, stdev=845.26, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 2=8.33%, 250=66.67%, 500=8.33%, 750=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=20.60%, ctx=217, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189242: Tue Aug 12 19:48:31 2025 read: IOPS=1, BW=1195KiB/s (1224kB/s)(5120KiB/4284msec) clat (usec): min=867, max=170694, avg=62579.37, stdev=85104.31 lat (usec): min=868, max=170695, avg=62580.63, stdev=85104.02 clat percentiles (usec): | 1.00th=[ 865], 5.00th=[ 865], 10.00th=[ 865], 20.00th=[ 865], | 30.00th=[ 930], 40.00th=[ 930], 50.00th=[ 1074], 60.00th=[ 1074], | 70.00th=[139461], 80.00th=[139461], 90.00th=[170918], 95.00th=[170918], | 99.00th=[170918], 99.50th=[170918], 99.90th=[170918], 99.95th=[170918], | 99.99th=[170918] bw ( KiB/s): min= 4039, max= 4063, per=95.79%, avg=4051.00, stdev=16.97, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1673KiB/s (1713kB/s)(7168KiB/4284msec); 0 zone resets clat (msec): min=180, max=2180, avg=563.70, stdev=728.74 lat (msec): min=180, max=2180, avg=564.14, stdev=728.70 clat percentiles (msec): | 1.00th=[ 180], 5.00th=[ 180], 10.00th=[ 180], 20.00th=[ 182], | 30.00th=[ 190], 40.00th=[ 190], 50.00th=[ 268], 60.00th=[ 342], | 70.00th=[ 342], 80.00th=[ 609], 90.00th=[ 2165], 95.00th=[ 2165], | 99.00th=[ 2165], 99.50th=[ 2165], 99.90th=[ 2165], 99.95th=[ 2165], | 99.99th=[ 2165] bw ( KiB/s): min= 1868, max= 4063, per=40.17%, avg=2378.40, stdev=943.33, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 lat (usec) : 1000=16.67% lat (msec) : 2=8.33%, 250=41.67%, 500=16.67%, 750=8.33%, >=2000=8.33% cpu : usr=0.09%, sys=23.09%, ctx=227, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4230KiB/s (4331kB/s), 423KiB/s-1613KiB/s (433kB/s-1651kB/s), io=20.0MiB (20.0MB), run=4284-4842msec WRITE: bw=5922KiB/s (6064kB/s), 1152KiB/s-2115KiB/s (1180kB/s-2166kB/s), io=28.0MiB (29.4MB), run=4284-4842msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189244: Tue Aug 12 19:48:32 2025 read: IOPS=1, BW=1574KiB/s (1612kB/s)(7168KiB/4553msec) clat (msec): min=74, max=2003, avg=436.55, stdev=717.28 lat (msec): min=74, max=2003, avg=436.55, stdev=717.28 clat percentiles (msec): | 1.00th=[ 75], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 80], | 30.00th=[ 88], 40.00th=[ 88], 50.00th=[ 96], 60.00th=[ 110], | 70.00th=[ 110], 80.00th=[ 609], 90.00th=[ 2005], 95.00th=[ 2005], | 99.00th=[ 2005], 99.50th=[ 2005], 99.90th=[ 2005], 99.95th=[ 2005], | 99.99th=[ 2005] bw ( KiB/s): min= 1903, max= 4079, per=77.92%, avg=3340.33, stdev=1244.93, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 write: IOPS=1, BW=1125KiB/s (1152kB/s)(5120KiB/4553msec); 0 zone resets clat (msec): min=98, max=571, avg=285.09, stdev=186.63 lat (msec): min=98, max=571, avg=285.29, stdev=186.59 clat percentiles (msec): | 1.00th=[ 100], 5.00th=[ 100], 10.00th=[ 100], 20.00th=[ 100], | 30.00th=[ 174], 40.00th=[ 174], 50.00th=[ 222], 60.00th=[ 222], | 70.00th=[ 363], 80.00th=[ 363], 90.00th=[ 575], 95.00th=[ 575], | 99.00th=[ 575], 99.50th=[ 575], 99.90th=[ 575], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 2019, max= 2048, per=33.88%, avg=2033.50, stdev=20.51, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 100=41.67%, 250=25.00%, 500=8.33%, 750=16.67%, >=2000=8.33% cpu : usr=0.00%, sys=11.18%, ctx=112, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189245: Tue Aug 12 19:48:32 2025 read: IOPS=0, BW=433KiB/s (444kB/s)(2048KiB/4728msec) clat (msec): min=39, max=361, avg=200.92, stdev=227.62 lat (msec): min=39, max=361, avg=200.92, stdev=227.62 clat percentiles (msec): | 1.00th=[ 41], 5.00th=[ 41], 10.00th=[ 41], 20.00th=[ 41], | 30.00th=[ 41], 40.00th=[ 41], 50.00th=[ 41], 60.00th=[ 363], | 70.00th=[ 363], 80.00th=[ 363], 90.00th=[ 363], 95.00th=[ 363], | 99.00th=[ 363], 99.50th=[ 363], 99.90th=[ 363], 99.95th=[ 363], | 99.99th=[ 363] bw ( KiB/s): min= 1404, max= 2048, per=40.26%, avg=1726.00, stdev=455.38, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2166KiB/s (2218kB/s)(10.0MiB/4728msec); 0 zone resets clat (msec): min=71, max=2371, avg=431.07, stdev=689.38 lat (msec): min=71, max=2371, avg=431.17, stdev=689.37 clat percentiles (msec): | 1.00th=[ 72], 5.00th=[ 72], 10.00th=[ 72], 20.00th=[ 86], | 30.00th=[ 138], 40.00th=[ 201], 50.00th=[ 205], 60.00th=[ 234], | 70.00th=[ 275], 80.00th=[ 359], 90.00th=[ 376], 95.00th=[ 2366], | 99.00th=[ 2366], 99.50th=[ 2366], 99.90th=[ 2366], 99.95th=[ 2366], | 99.99th=[ 2366] bw ( KiB/s): min= 1914, max= 6131, per=59.06%, avg=3545.00, stdev=1993.25, samples=4 iops : min= 1, max= 5, avg= 2.75, stdev= 2.06, samples=4 lat (msec) : 50=8.33%, 100=16.67%, 250=33.33%, 500=33.33%, >=2000=8.33% cpu : usr=0.06%, sys=9.63%, ctx=164, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189246: Tue Aug 12 19:48:32 2025 read: IOPS=1, BW=1286KiB/s (1317kB/s)(6144KiB/4777msec) clat (msec): min=60, max=524, avg=154.63, stdev=183.56 lat (msec): min=60, max=524, avg=154.63, stdev=183.56 clat percentiles (msec): | 1.00th=[ 61], 5.00th=[ 61], 10.00th=[ 61], 20.00th=[ 61], | 30.00th=[ 61], 40.00th=[ 67], 50.00th=[ 67], 60.00th=[ 82], | 70.00th=[ 136], 80.00th=[ 136], 90.00th=[ 527], 95.00th=[ 527], | 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527], | 99.99th=[ 527] bw ( KiB/s): min= 4096, max= 6119, per=100.00%, avg=5107.50, stdev=1430.48, samples=2 iops : min= 4, max= 5, avg= 4.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1286KiB/s (1317kB/s)(6144KiB/4777msec); 0 zone resets clat (msec): min=95, max=2357, avg=628.64, stdev=858.67 lat (msec): min=95, max=2357, avg=628.74, stdev=858.67 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 96], 10.00th=[ 96], 20.00th=[ 153], | 30.00th=[ 153], 40.00th=[ 296], 50.00th=[ 296], 60.00th=[ 430], | 70.00th=[ 443], 80.00th=[ 443], 90.00th=[ 2366], 95.00th=[ 2366], | 99.00th=[ 2366], 99.50th=[ 2366], 99.90th=[ 2366], 99.95th=[ 2366], | 99.99th=[ 2366] bw ( KiB/s): min= 1872, max= 2048, per=33.03%, avg=1982.40, stdev=83.03, samples=5 iops : min= 1, max= 2, avg= 1.20, stdev= 0.45, samples=5 lat (msec) : 100=41.67%, 250=16.67%, 500=25.00%, 750=8.33%, >=2000=8.33% cpu : usr=0.02%, sys=9.13%, ctx=100, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189247: Tue Aug 12 19:48:32 2025 read: IOPS=1, BW=1083KiB/s (1109kB/s)(5120KiB/4726msec) clat (msec): min=49, max=698, avg=221.56, stdev=271.31 lat (msec): min=49, max=698, avg=221.56, stdev=271.31 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 50], 20.00th=[ 50], | 30.00th=[ 84], 40.00th=[ 84], 50.00th=[ 92], 60.00th=[ 92], | 70.00th=[ 184], 80.00th=[ 184], 90.00th=[ 701], 95.00th=[ 701], | 99.00th=[ 701], 99.50th=[ 701], 99.90th=[ 701], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 1914, max= 4039, per=62.14%, avg=2664.00, stdev=1192.42, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=1, BW=1517KiB/s (1553kB/s)(7168KiB/4726msec); 0 zone resets clat (msec): min=56, max=1676, avg=464.33, stdev=565.36 lat (msec): min=56, max=1676, avg=464.45, stdev=565.34 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 56], 10.00th=[ 56], 20.00th=[ 144], | 30.00th=[ 182], 40.00th=[ 182], 50.00th=[ 230], 60.00th=[ 338], | 70.00th=[ 338], 80.00th=[ 625], 90.00th=[ 1670], 95.00th=[ 1670], | 99.00th=[ 1670], 99.50th=[ 1670], 99.90th=[ 1670], 99.95th=[ 1670], | 99.99th=[ 1670] bw ( KiB/s): min= 2019, max= 4096, per=50.95%, avg=3058.25, stdev=1188.52, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.50, samples=4 lat (msec) : 50=8.33%, 100=25.00%, 250=33.33%, 500=8.33%, 750=16.67% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=11.43%, ctx=132, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4287KiB/s (4390kB/s), 433KiB/s-1574KiB/s (444kB/s-1612kB/s), io=20.0MiB (20.0MB), run=4553-4777msec WRITE: bw=6002KiB/s (6146kB/s), 1125KiB/s-2166KiB/s (1152kB/s-2218kB/s), io=28.0MiB (29.4MB), run=4553-4777msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189253: Tue Aug 12 19:48:41 2025 read: IOPS=1, BW=6289KiB/s (6440kB/s)(12.0MiB/1954msec) clat (msec): min=384, max=942, avg=624.41, stdev=287.11 lat (msec): min=384, max=942, avg=624.41, stdev=287.11 clat percentiles (msec): | 1.00th=[ 384], 5.00th=[ 384], 10.00th=[ 384], 20.00th=[ 384], | 30.00th=[ 384], 40.00th=[ 550], 50.00th=[ 550], 60.00th=[ 550], | 70.00th=[ 944], 80.00th=[ 944], 90.00th=[ 944], 95.00th=[ 944], | 99.00th=[ 944], 99.50th=[ 944], 99.90th=[ 944], 99.95th=[ 944], | 99.99th=[ 944] bw ( KiB/s): min= 7627, max= 8192, per=100.00%, avg=7909.50, stdev=399.52, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=29.34%, ctx=896, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189254: Tue Aug 12 19:48:41 2025 read: IOPS=0, BW=1181KiB/s (1209kB/s)(4096KiB/3468msec) clat (nsec): min=445967k, max=445967k, avg=445967131.00, stdev= 0.00 lat (nsec): min=445971k, max=445971k, avg=445971268.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 447], 5.00th=[ 447], 10.00th=[ 447], 20.00th=[ 447], | 30.00th=[ 447], 40.00th=[ 447], 50.00th=[ 447], 60.00th=[ 447], | 70.00th=[ 447], 80.00th=[ 447], 90.00th=[ 447], 95.00th=[ 447], | 99.00th=[ 447], 99.50th=[ 447], 99.90th=[ 447], 99.95th=[ 447], | 99.99th=[ 447] bw ( KiB/s): min= 7515, max= 7515, per=100.00%, avg=7515.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2362KiB/s (2419kB/s)(8192KiB/3468msec); 0 zone resets clat (msec): min=686, max=2309, avg=1497.55, stdev=1147.63 lat (msec): min=686, max=2309, avg=1497.95, stdev=1147.79 clat percentiles (msec): | 1.00th=[ 684], 5.00th=[ 684], 10.00th=[ 684], 20.00th=[ 684], | 30.00th=[ 684], 40.00th=[ 684], 50.00th=[ 684], 60.00th=[ 2299], | 70.00th=[ 2299], 80.00th=[ 2299], 90.00th=[ 2299], 95.00th=[ 2299], | 99.00th=[ 2299], 99.50th=[ 2299], 99.90th=[ 2299], 99.95th=[ 2299], | 99.99th=[ 2299] bw ( KiB/s): min= 8094, max= 8094, per=91.27%, avg=8094.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, >=2000=33.33% cpu : usr=0.03%, sys=36.86%, ctx=482, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189255: Tue Aug 12 19:48:41 2025 write: IOPS=0, BW=3326KiB/s (3405kB/s)(12.0MiB/3695msec); 0 zone resets clat (msec): min=822, max=1603, avg=1220.69, stdev=390.61 lat (msec): min=822, max=1603, avg=1221.17, stdev=390.54 clat percentiles (msec): | 1.00th=[ 827], 5.00th=[ 827], 10.00th=[ 827], 20.00th=[ 827], | 30.00th=[ 827], 40.00th=[ 1234], 50.00th=[ 1234], 60.00th=[ 1234], | 70.00th=[ 1603], 80.00th=[ 1603], 90.00th=[ 1603], 95.00th=[ 1603], | 99.00th=[ 1603], 99.50th=[ 1603], 99.90th=[ 1603], 99.95th=[ 1603], | 99.99th=[ 1603] bw ( KiB/s): min= 7135, max= 8192, per=86.42%, avg=7663.50, stdev=747.41, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.05%, sys=36.90%, ctx=56, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189256: Tue Aug 12 19:48:41 2025 write: IOPS=1, BW=4649KiB/s (4761kB/s)(12.0MiB/2643msec); 0 zone resets clat (msec): min=574, max=1342, avg=839.00, stdev=436.09 lat (msec): min=574, max=1342, avg=839.42, stdev=436.24 clat percentiles (msec): | 1.00th=[ 575], 5.00th=[ 575], 10.00th=[ 575], 20.00th=[ 575], | 30.00th=[ 575], 40.00th=[ 600], 50.00th=[ 600], 60.00th=[ 600], | 70.00th=[ 1351], 80.00th=[ 1351], 90.00th=[ 1351], 95.00th=[ 1351], | 99.00th=[ 1351], 99.50th=[ 1351], 99.90th=[ 1351], 99.95th=[ 1351], | 99.99th=[ 1351] bw ( KiB/s): min= 8094, max= 8192, per=91.82%, avg=8143.00, stdev=69.30, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=57.57%, ctx=135, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4724KiB/s (4838kB/s), 1181KiB/s-6289KiB/s (1209kB/s-6440kB/s), io=16.0MiB (16.8MB), run=1954-3468msec WRITE: bw=8868KiB/s (9081kB/s), 2362KiB/s-4649KiB/s (2419kB/s-4761kB/s), io=32.0MiB (33.6MB), run=2643-3695msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189258: Tue Aug 12 19:48:42 2025 read: IOPS=0, BW=3052KiB/s (3125kB/s)(12.0MiB/4026msec) clat (msec): min=779, max=1797, avg=1312.59, stdev=510.48 lat (msec): min=779, max=1797, avg=1312.59, stdev=510.48 clat percentiles (msec): | 1.00th=[ 776], 5.00th=[ 776], 10.00th=[ 776], 20.00th=[ 776], | 30.00th=[ 776], 40.00th=[ 1368], 50.00th=[ 1368], 60.00th=[ 1368], | 70.00th=[ 1804], 80.00th=[ 1804], 90.00th=[ 1804], 95.00th=[ 1804], | 99.00th=[ 1804], 99.50th=[ 1804], 99.90th=[ 1804], 99.95th=[ 1804], | 99.99th=[ 1804] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=11.23%, ctx=86, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189259: Tue Aug 12 19:48:42 2025 read: IOPS=0, BW=1014KiB/s (1038kB/s)(4096KiB/4039msec) clat (nsec): min=1626.2M, max=1626.2M, avg=1626166605.00, stdev= 0.00 lat (nsec): min=1626.2M, max=1626.2M, avg=1626169485.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1620], 5.00th=[ 1620], 10.00th=[ 1620], 20.00th=[ 1620], | 30.00th=[ 1620], 40.00th=[ 1620], 50.00th=[ 1620], 60.00th=[ 1620], | 70.00th=[ 1620], 80.00th=[ 1620], 90.00th=[ 1620], 95.00th=[ 1620], | 99.00th=[ 1620], 99.50th=[ 1620], 99.90th=[ 1620], 99.95th=[ 1620], | 99.99th=[ 1620] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2028KiB/s (2077kB/s)(8192KiB/4039msec); 0 zone resets clat (msec): min=897, max=1438, avg=1167.97, stdev=382.38 lat (msec): min=897, max=1438, avg=1168.44, stdev=382.55 clat percentiles (msec): | 1.00th=[ 902], 5.00th=[ 902], 10.00th=[ 902], 20.00th=[ 902], | 30.00th=[ 902], 40.00th=[ 902], 50.00th=[ 902], 60.00th=[ 1435], | 70.00th=[ 1435], 80.00th=[ 1435], 90.00th=[ 1435], 95.00th=[ 1435], | 99.00th=[ 1435], 99.50th=[ 1435], 99.90th=[ 1435], 99.95th=[ 1435], | 99.99th=[ 1435] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.02%, sys=10.18%, ctx=60, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189260: Tue Aug 12 19:48:42 2025 write: IOPS=0, BW=2910KiB/s (2980kB/s)(12.0MiB/4223msec); 0 zone resets clat (msec): min=115, max=2823, avg=1387.12, stdev=1361.54 lat (msec): min=116, max=2824, avg=1387.76, stdev=1361.26 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 116], | 30.00th=[ 116], 40.00th=[ 1217], 50.00th=[ 1217], 60.00th=[ 1217], | 70.00th=[ 2836], 80.00th=[ 2836], 90.00th=[ 2836], 95.00th=[ 2836], | 99.00th=[ 2836], 99.50th=[ 2836], 99.90th=[ 2836], 99.95th=[ 2836], | 99.99th=[ 2836] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.05%, sys=20.20%, ctx=119, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189261: Tue Aug 12 19:48:42 2025 write: IOPS=0, BW=3711KiB/s (3800kB/s)(12.0MiB/3311msec); 0 zone resets clat (msec): min=462, max=1935, avg=1086.73, stdev=761.98 lat (msec): min=462, max=1936, avg=1087.31, stdev=762.21 clat percentiles (msec): | 1.00th=[ 464], 5.00th=[ 464], 10.00th=[ 464], 20.00th=[ 464], | 30.00th=[ 464], 40.00th=[ 860], 50.00th=[ 860], 60.00th=[ 860], | 70.00th=[ 1938], 80.00th=[ 1938], 90.00th=[ 1938], 95.00th=[ 1938], | 99.00th=[ 1938], 99.50th=[ 1938], 99.90th=[ 1938], 99.95th=[ 1938], | 99.99th=[ 1938] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.06%, sys=18.55%, ctx=150, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4056KiB/s (4154kB/s), 1014KiB/s-3052KiB/s (1038kB/s-3125kB/s), io=16.0MiB (16.8MB), run=4026-4039msec WRITE: bw=7759KiB/s (7946kB/s), 2028KiB/s-3711KiB/s (2077kB/s-3800kB/s), io=32.0MiB (33.6MB), run=3311-4223msec