== sanity test 398b: DIO and buffer IO race ============== 18:26:32 (1773613592) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 8.52271 s, 5.9 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146513: Sun Mar 15 18:30:18 2026 read: IOPS=7, BW=30.3KiB/s (30.0kB/s)(5992KiB/198005msec) clat (usec): min=35, max=1185.1k, avg=64931.33, stdev=54742.11 lat (usec): min=35, max=1185.1k, avg=64945.54, stdev=54753.17 clat percentiles (usec): | 1.00th=[ 53], 5.00th=[ 68], 10.00th=[ 18220], | 20.00th=[ 31589], 30.00th=[ 40109], 40.00th=[ 47973], | 50.00th=[ 55837], 60.00th=[ 63701], 70.00th=[ 77071], | 80.00th=[ 93848], 90.00th=[ 120062], 95.00th=[ 143655], | 99.00th=[ 252707], 99.50th=[ 295699], 99.90th=[ 434111], | 99.95th=[1182794], 99.99th=[1182794] bw ( KiB/s): min= 7, max= 79, per=25.26%, avg=30.31, stdev=14.72, samples=382 iops : min= 1, max= 19, avg= 6.90, stdev= 3.71, samples=382 write: IOPS=7, BW=31.8KiB/s (32.6kB/s)(6296KiB/198005msec); 0 zone resets clat (usec): min=1283, max=790867, avg=63828.04, stdev=65648.50 lat (usec): min=1284, max=790869, avg=63832.26, stdev=65647.12 clat percentiles (usec): | 1.00th=[ 1434], 5.00th=[ 1811], 10.00th=[ 3032], 20.00th=[ 8455], | 30.00th=[ 15926], 40.00th=[ 32375], 50.00th=[ 49021], 60.00th=[ 62653], | 70.00th=[ 82314], 80.00th=[109577], 90.00th=[145753], 95.00th=[183501], | 99.00th=[270533], 99.50th=[337642], 99.90th=[633340], 99.95th=[792724], | 99.99th=[792724] bw ( KiB/s): min= 7, max= 102, per=25.49%, avg=31.86, stdev=16.24, samples=382 iops : min= 1, max= 25, avg= 7.30, stdev= 4.10, samples=382 lat (usec) : 50=0.26%, 100=3.12%, 250=0.39%, 500=0.10%, 750=0.10% lat (msec) : 2=3.45%, 4=2.67%, 10=5.76%, 20=6.48%, 50=24.61% lat (msec) : 100=32.49%, 250=19.30%, 500=1.17%, 750=0.03%, 1000=0.03% lat (msec) : 2000=0.03% cpu : usr=0.06%, sys=4.30%, ctx=6801, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146514: Sun Mar 15 18:30:18 2026 read: IOPS=7, BW=30.3KiB/s (31.0kB/s)(5940KiB/196005msec) clat (usec): min=33, max=1247.5k, avg=63715.13, stdev=54865.04 lat (usec): min=33, max=1247.5k, avg=63730.99, stdev=54866.80 clat percentiles (usec): | 1.00th=[ 55], 5.00th=[ 69], 10.00th=[ 16909], | 20.00th=[ 30540], 30.00th=[ 39584], 40.00th=[ 46400], | 50.00th=[ 54264], 60.00th=[ 64226], 70.00th=[ 74974], | 80.00th=[ 91751], 90.00th=[ 120062], 95.00th=[ 143655], | 99.00th=[ 214959], 99.50th=[ 278922], 99.90th=[ 438305], | 99.95th=[1249903], 99.99th=[1249903] bw ( KiB/s): min= 7, max= 79, per=25.77%, avg=30.93, stdev=15.47, samples=371 iops : min= 1, max= 19, avg= 7.06, stdev= 3.91, samples=371 write: IOPS=8, BW=32.4KiB/s (33.2kB/s)(6348KiB/196005msec); 0 zone resets clat (usec): min=1203, max=606881, avg=63696.58, stdev=63344.40 lat (usec): min=1203, max=606882, avg=63700.76, stdev=63344.60 clat percentiles (usec): | 1.00th=[ 1401], 5.00th=[ 1778], 10.00th=[ 3458], 20.00th=[ 8848], | 30.00th=[ 16450], 40.00th=[ 32637], 50.00th=[ 49546], 60.00th=[ 64226], | 70.00th=[ 82314], 80.00th=[108528], 90.00th=[147850], 95.00th=[181404], | 99.00th=[265290], 99.50th=[329253], 99.90th=[450888], 99.95th=[608175], | 99.99th=[608175] bw ( KiB/s): min= 7, max= 94, per=25.76%, avg=32.20, stdev=16.14, samples=381 iops : min= 1, max= 23, avg= 7.39, stdev= 4.08, samples=381 lat (usec) : 50=0.20%, 100=3.06%, 250=0.33%, 500=0.23%, 750=0.10% lat (usec) : 1000=0.03% lat (msec) : 2=3.68%, 4=2.18%, 10=5.83%, 20=7.23%, 50=24.80% lat (msec) : 100=32.75%, 250=18.55%, 500=0.98%, 750=0.03%, 2000=0.03% cpu : usr=0.07%, sys=4.29%, ctx=6747, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146515: Sun Mar 15 18:30:18 2026 read: IOPS=7, BW=31.0KiB/s (31.8kB/s)(6028KiB/194167msec) clat (usec): min=38, max=818144, avg=62834.24, stdev=49703.09 lat (usec): min=38, max=818146, avg=62841.81, stdev=49698.20 clat percentiles (usec): | 1.00th=[ 55], 5.00th=[ 67], 10.00th=[ 13960], 20.00th=[ 29492], | 30.00th=[ 38011], 40.00th=[ 45876], 50.00th=[ 54789], 60.00th=[ 63177], | 70.00th=[ 76022], 80.00th=[ 92799], 90.00th=[117965], 95.00th=[141558], | 99.00th=[214959], 99.50th=[246416], 99.90th=[574620], 99.95th=[817890], | 99.99th=[817890] bw ( KiB/s): min= 7, max= 86, per=26.38%, avg=31.66, stdev=15.98, samples=368 iops : min= 1, max= 21, avg= 7.24, stdev= 4.01, samples=368 write: IOPS=8, BW=32.2KiB/s (33.0kB/s)(6260KiB/194167msec); 0 zone resets clat (usec): min=1287, max=1477.9k, avg=63372.21, stdev=73023.00 lat (usec): min=1289, max=1477.9k, avg=63377.57, stdev=73022.67 clat percentiles (usec): | 1.00th=[ 1483], 5.00th=[ 1876], 10.00th=[ 3556], | 20.00th=[ 8455], 30.00th=[ 15139], 40.00th=[ 28967], | 50.00th=[ 45351], 60.00th=[ 61604], 70.00th=[ 82314], | 80.00th=[ 108528], 90.00th=[ 143655], 95.00th=[ 183501], | 99.00th=[ 258999], 99.50th=[ 333448], 99.90th=[ 692061], | 99.95th=[1484784], 99.99th=[1484784] bw ( KiB/s): min= 7, max= 94, per=25.79%, avg=32.24, stdev=16.71, samples=375 iops : min= 1, max= 23, avg= 7.40, stdev= 4.22, samples=375 lat (usec) : 50=0.29%, 100=3.48%, 250=0.29%, 500=0.16%, 750=0.07% lat (usec) : 1000=0.03% lat (msec) : 2=3.45%, 4=2.44%, 10=6.35%, 20=6.71%, 50=25.94% lat (msec) : 100=30.96%, 250=19.08%, 500=0.62%, 750=0.07%, 1000=0.03% lat (msec) : 2000=0.03% cpu : usr=0.07%, sys=4.45%, ctx=6766, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146516: Sun Mar 15 18:30:18 2026 read: IOPS=7, BW=30.3KiB/s (31.0kB/s)(6052KiB/199672msec) clat (usec): min=35, max=580831, avg=64995.66, stdev=49756.54 lat (usec): min=35, max=580834, avg=65011.97, stdev=49802.26 clat percentiles (usec): | 1.00th=[ 54], 5.00th=[ 67], 10.00th=[ 1811], 20.00th=[ 30278], | 30.00th=[ 38536], 40.00th=[ 46924], 50.00th=[ 55837], 60.00th=[ 66323], | 70.00th=[ 79168], 80.00th=[ 94897], 90.00th=[124257], 95.00th=[147850], | 99.00th=[212861], 99.50th=[274727], 99.90th=[446694], 99.95th=[583009], | 99.99th=[583009] bw ( KiB/s): min= 7, max= 95, per=25.96%, avg=31.16, stdev=15.06, samples=375 iops : min= 1, max= 23, avg= 7.11, stdev= 3.80, samples=375 write: IOPS=7, BW=31.2KiB/s (31.0kB/s)(6236KiB/199672msec); 0 zone resets clat (usec): min=1246, max=1366.7k, avg=64827.67, stdev=74511.60 lat (usec): min=1247, max=1366.7k, avg=64831.26, stdev=74511.50 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 1827], 10.00th=[ 2868], | 20.00th=[ 7635], 30.00th=[ 14484], 40.00th=[ 30016], | 50.00th=[ 45876], 60.00th=[ 62129], 70.00th=[ 84411], | 80.00th=[ 109577], 90.00th=[ 154141], 95.00th=[ 189793], | 99.00th=[ 308282], 99.50th=[ 358613], 99.90th=[ 517997], | 99.95th=[1367344], 99.99th=[1367344] bw ( KiB/s): min= 7, max= 94, per=24.77%, avg=30.97, stdev=16.18, samples=388 iops : min= 1, max= 23, avg= 7.07, stdev= 4.09, samples=388 lat (usec) : 50=0.33%, 100=3.87%, 250=0.26%, 500=0.26%, 750=0.07% lat (msec) : 2=3.48%, 4=2.99%, 10=5.86%, 20=6.84%, 50=24.06% lat (msec) : 100=31.25%, 250=19.27%, 500=1.37%, 750=0.07%, 2000=0.03% cpu : usr=0.07%, sys=4.10%, ctx=6655, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=120KiB/s (123kB/s), 30.3KiB/s-31.0KiB/s (30.0kB/s-31.8kB/s), io=23.4MiB (24.6MB), run=194167-199672msec WRITE: bw=126KiB/s (129kB/s), 31.2KiB/s-32.4KiB/s (31.0kB/s-33.2kB/s), io=24.6MiB (25.7MB), run=194167-199672msec fio: pid=146508, err=5/file:backend.c:481, func=full resid, error=Input/output error rand-rw: (groupid=0, jobs=1): err= 5 (file:backend.c:481, func=full resid, error=Input/output error): pid=146508: Sun Mar 15 18:34:29 2026 read: IOPS=2, BW=11.9KiB/s (12.2kB/s)(4236KiB/355189msec) clat (msec): min=9, max=1285, avg=66.45, stdev=78.26 lat (msec): min=9, max=1285, avg=66.45, stdev=78.26 clat percentiles (msec): | 1.00th=[ 12], 5.00th=[ 16], 10.00th=[ 20], 20.00th=[ 24], | 30.00th=[ 31], 40.00th=[ 37], 50.00th=[ 44], 60.00th=[ 54], | 70.00th=[ 69], 80.00th=[ 92], 90.00th=[ 133], 95.00th=[ 184], | 99.00th=[ 334], 99.50th=[ 493], 99.90th=[ 1062], 99.95th=[ 1284], | 99.99th=[ 1284] bw ( KiB/s): min= 7, max= 71, per=33.97%, avg=16.64, stdev=11.30, samples=482 iops : min= 1, max= 17, avg= 3.49, stdev= 2.86, samples=482 write: IOPS=3, BW=12.6KiB/s (12.9kB/s)(4460KiB/355189msec); 0 zone resets clat (msec): min=56, max=858, avg=254.94, stdev=110.12 lat (msec): min=56, max=858, avg=254.95, stdev=110.13 clat percentiles (msec): | 1.00th=[ 87], 5.00th=[ 121], 10.00th=[ 138], 20.00th=[ 163], | 30.00th=[ 188], 40.00th=[ 209], 50.00th=[ 234], 60.00th=[ 259], | 70.00th=[ 296], 80.00th=[ 338], 90.00th=[ 401], 95.00th=[ 468], | 99.00th=[ 584], 99.50th=[ 659], 99.90th=[ 785], 99.95th=[ 860], | 99.99th=[ 860] bw ( KiB/s): min= 6, max= 39, per=25.02%, avg=12.76, stdev= 5.94, samples=652 iops : min= 1, max= 9, avg= 2.51, stdev= 1.52, samples=652 lat (msec) : 10=0.23%, 20=5.93%, 50=21.75%, 100=13.15%, 250=35.36% lat (msec) : 500=21.61%, 750=1.75%, 1000=0.09%, 2000=0.09% cpu : usr=0.03%, sys=1.01%, ctx=3721, majf=0, minf=19 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1060,1115,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146509: Sun Mar 15 18:34:29 2026 read: IOPS=3, BW=13.2KiB/s (13.5kB/s)(5940KiB/450700msec) clat (msec): min=7, max=1290, avg=57.12, stdev=59.58 lat (msec): min=7, max=1290, avg=57.12, stdev=59.58 clat percentiles (msec): | 1.00th=[ 11], 5.00th=[ 17], 10.00th=[ 19], 20.00th=[ 24], | 30.00th=[ 29], 40.00th=[ 35], 50.00th=[ 42], 60.00th=[ 51], | 70.00th=[ 63], 80.00th=[ 74], 90.00th=[ 111], 95.00th=[ 157], | 99.00th=[ 259], 99.50th=[ 321], 99.90th=[ 600], 99.95th=[ 1284], | 99.99th=[ 1284] bw ( KiB/s): min= 7, max= 87, per=36.36%, avg=17.82, stdev=11.99, samples=634 iops : min= 1, max= 21, avg= 3.78, stdev= 3.02, samples=634 write: IOPS=3, BW=14.1KiB/s (14.4kB/s)(6348KiB/450700msec); 0 zone resets clat (msec): min=58, max=861, avg=230.19, stdev=104.77 lat (msec): min=58, max=861, avg=230.20, stdev=104.78 clat percentiles (msec): | 1.00th=[ 85], 5.00th=[ 109], 10.00th=[ 123], 20.00th=[ 146], | 30.00th=[ 167], 40.00th=[ 186], 50.00th=[ 207], 60.00th=[ 230], | 70.00th=[ 259], 80.00th=[ 305], 90.00th=[ 376], 95.00th=[ 430], | 99.00th=[ 558], 99.50th=[ 634], 99.90th=[ 860], 99.95th=[ 860], | 99.99th=[ 860] bw ( KiB/s): min= 6, max= 39, per=27.57%, avg=14.06, stdev= 6.32, samples=847 iops : min= 1, max= 9, avg= 2.83, stdev= 1.62, samples=847 lat (msec) : 10=0.20%, 20=5.66%, 50=23.14%, 100=15.46%, 250=37.70% lat (msec) : 500=16.54%, 750=1.11%, 1000=0.16%, 2000=0.03% cpu : usr=0.03%, sys=1.13%, ctx=5765, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146510: Sun Mar 15 18:34:29 2026 read: IOPS=3, BW=13.4KiB/s (13.7kB/s)(6028KiB/450191msec) clat (msec): min=8, max=609, avg=59.89, stdev=57.92 lat (msec): min=8, max=609, avg=59.89, stdev=57.92 clat percentiles (msec): | 1.00th=[ 11], 5.00th=[ 16], 10.00th=[ 19], 20.00th=[ 24], | 30.00th=[ 30], 40.00th=[ 36], 50.00th=[ 43], 60.00th=[ 52], | 70.00th=[ 63], 80.00th=[ 82], 90.00th=[ 120], 95.00th=[ 163], | 99.00th=[ 292], 99.50th=[ 372], 99.90th=[ 550], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 7, max= 79, per=36.25%, avg=17.76, stdev=12.31, samples=645 iops : min= 1, max= 19, avg= 3.76, stdev= 3.11, samples=645 write: IOPS=3, BW=13.9KiB/s (14.2kB/s)(6260KiB/450191msec); 0 zone resets clat (msec): min=49, max=1532, avg=229.65, stdev=108.65 lat (msec): min=49, max=1532, avg=229.66, stdev=108.65 clat percentiles (msec): | 1.00th=[ 79], 5.00th=[ 103], 10.00th=[ 118], 20.00th=[ 146], | 30.00th=[ 169], 40.00th=[ 186], 50.00th=[ 207], 60.00th=[ 230], | 70.00th=[ 259], 80.00th=[ 305], 90.00th=[ 372], 95.00th=[ 435], | 99.00th=[ 550], 99.50th=[ 651], 99.90th=[ 835], 99.95th=[ 1536], | 99.99th=[ 1536] bw ( KiB/s): min= 6, max= 40, per=27.51%, avg=14.03, stdev= 6.54, samples=837 iops : min= 1, max= 10, avg= 2.83, stdev= 1.68, samples=837 lat (msec) : 10=0.29%, 20=6.58%, 50=21.81%, 100=15.85%, 250=37.89% lat (msec) : 500=16.41%, 750=1.07%, 1000=0.07%, 2000=0.03% cpu : usr=0.03%, sys=1.08%, ctx=5671, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146511: Sun Mar 15 18:34:29 2026 read: IOPS=3, BW=13.4KiB/s (13.7kB/s)(6052KiB/450764msec) clat (msec): min=9, max=1065, avg=58.22, stdev=59.88 lat (msec): min=9, max=1065, avg=58.23, stdev=59.88 clat percentiles (msec): | 1.00th=[ 12], 5.00th=[ 16], 10.00th=[ 18], 20.00th=[ 24], | 30.00th=[ 29], 40.00th=[ 35], 50.00th=[ 43], 60.00th=[ 51], | 70.00th=[ 61], 80.00th=[ 79], 90.00th=[ 111], 95.00th=[ 157], | 99.00th=[ 296], 99.50th=[ 384], 99.90th=[ 558], 99.95th=[ 1062], | 99.99th=[ 1062] bw ( KiB/s): min= 6, max= 72, per=36.54%, avg=17.90, stdev=12.64, samples=642 iops : min= 1, max= 18, avg= 3.80, stdev= 3.20, samples=642 write: IOPS=3, BW=13.8KiB/s (14.2kB/s)(6236KiB/450764msec); 0 zone resets clat (msec): min=52, max=1407, avg=232.28, stdev=103.79 lat (msec): min=52, max=1407, avg=232.28, stdev=103.79 clat percentiles (msec): | 1.00th=[ 80], 5.00th=[ 108], 10.00th=[ 125], 20.00th=[ 146], | 30.00th=[ 169], 40.00th=[ 190], 50.00th=[ 213], 60.00th=[ 239], | 70.00th=[ 271], 80.00th=[ 309], 90.00th=[ 363], 95.00th=[ 422], | 99.00th=[ 542], 99.50th=[ 600], 99.90th=[ 852], 99.95th=[ 1401], | 99.99th=[ 1401] bw ( KiB/s): min= 6, max= 47, per=27.23%, avg=13.89, stdev= 6.31, samples=841 iops : min= 1, max= 11, avg= 2.79, stdev= 1.61, samples=841 lat (msec) : 10=0.13%, 20=6.41%, 50=22.59%, 100=15.66%, 250=36.30% lat (msec) : 500=17.94%, 750=0.88%, 1000=0.03%, 2000=0.07% cpu : usr=0.03%, sys=1.14%, ctx=5764, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=49.4KiB/s (50.6kB/s), 11.9KiB/s-13.4KiB/s (12.2kB/s-13.7kB/s), io=21.7MiB (22.8MB), run=355189-450764msec WRITE: bw=51.7KiB/s (52.9kB/s), 12.6KiB/s-14.1KiB/s (12.9kB/s-14.4kB/s), io=22.8MiB (23.9MB), run=355189-450764msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146598: Sun Mar 15 18:35:40 2026 read: IOPS=7, BW=117KiB/s (120kB/s)(5920KiB/50683msec) clat (usec): min=42, max=453768, avg=68452.48, stdev=57743.41 lat (usec): min=43, max=453770, avg=68470.21, stdev=57740.32 clat percentiles (usec): | 1.00th=[ 45], 5.00th=[ 63], 10.00th=[ 70], 20.00th=[ 127], | 30.00th=[ 36439], 40.00th=[ 50070], 50.00th=[ 66323], 60.00th=[ 78119], | 70.00th=[ 94897], 80.00th=[111674], 90.00th=[135267], 95.00th=[166724], | 99.00th=[238027], 99.50th=[278922], 99.90th=[455082], 99.95th=[455082], | 99.99th=[455082] bw ( KiB/s): min= 30, max= 284, per=26.38%, avg=120.29, stdev=54.79, samples=96 iops : min= 1, max= 17, avg= 6.71, stdev= 3.49, samples=96 write: IOPS=7, BW=126KiB/s (129kB/s)(6368KiB/50683msec); 0 zone resets clat (usec): min=1749, max=394543, avg=62946.39, stdev=67972.11 lat (usec): min=1752, max=394545, avg=62972.30, stdev=67957.54 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 6], 20.00th=[ 10], | 30.00th=[ 15], 40.00th=[ 23], 50.00th=[ 34], 60.00th=[ 56], | 70.00th=[ 83], 80.00th=[ 115], 90.00th=[ 165], 95.00th=[ 199], | 99.00th=[ 300], 99.50th=[ 368], 99.90th=[ 397], 99.95th=[ 397], | 99.99th=[ 397] bw ( KiB/s): min= 31, max= 349, per=26.29%, avg=126.19, stdev=67.59, samples=98 iops : min= 1, max= 21, avg= 7.09, stdev= 4.31, samples=98 lat (usec) : 50=0.78%, 100=8.20%, 250=1.43%, 500=0.26%, 750=0.26% lat (usec) : 1000=0.13% lat (msec) : 2=0.65%, 4=4.17%, 10=6.77%, 20=9.38%, 50=17.84% lat (msec) : 100=24.09%, 250=24.74%, 500=1.30% cpu : usr=0.05%, sys=4.41%, ctx=1517, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146599: Sun Mar 15 18:35:40 2026 read: IOPS=7, BW=113KiB/s (116kB/s)(5744KiB/50615msec) clat (usec): min=37, max=597701, avg=72503.78, stdev=66335.22 lat (usec): min=37, max=597702, avg=72515.14, stdev=66339.83 clat percentiles (usec): | 1.00th=[ 44], 5.00th=[ 64], 10.00th=[ 71], 20.00th=[ 243], | 30.00th=[ 41157], 40.00th=[ 53216], 50.00th=[ 67634], 60.00th=[ 80217], | 70.00th=[ 93848], 80.00th=[111674], 90.00th=[149947], 95.00th=[170918], | 99.00th=[283116], 99.50th=[425722], 99.90th=[599786], 99.95th=[599786], | 99.99th=[599786] bw ( KiB/s): min= 30, max= 281, per=25.60%, avg=116.74, stdev=57.85, samples=96 iops : min= 1, max= 17, avg= 6.49, stdev= 3.68, samples=96 write: IOPS=8, BW=129KiB/s (132kB/s)(6544KiB/50615msec); 0 zone resets clat (usec): min=1756, max=445739, avg=59745.83, stdev=61159.45 lat (usec): min=1758, max=445745, avg=59762.01, stdev=61157.03 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 7], 20.00th=[ 12], | 30.00th=[ 16], 40.00th=[ 23], 50.00th=[ 37], 60.00th=[ 61], | 70.00th=[ 80], 80.00th=[ 106], 90.00th=[ 150], 95.00th=[ 171], | 99.00th=[ 255], 99.50th=[ 309], 99.90th=[ 447], 99.95th=[ 447], | 99.99th=[ 447] bw ( KiB/s): min= 28, max= 319, per=27.38%, avg=131.44, stdev=71.58, samples=97 iops : min= 1, max= 19, avg= 7.42, stdev= 4.58, samples=97 lat (usec) : 50=1.04%, 100=7.29%, 250=1.04%, 500=0.39%, 750=0.39% lat (msec) : 2=0.52%, 4=2.73%, 10=7.03%, 20=10.55%, 50=16.02% lat (msec) : 100=28.26%, 250=23.44%, 500=1.17%, 750=0.13% cpu : usr=0.06%, sys=4.71%, ctx=1486, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146601: Sun Mar 15 18:35:40 2026 read: IOPS=7, BW=120KiB/s (123kB/s)(6256KiB/52207msec) clat (usec): min=33, max=504841, avg=72278.90, stdev=59284.11 lat (usec): min=33, max=504842, avg=72280.21, stdev=59284.14 clat percentiles (usec): | 1.00th=[ 42], 5.00th=[ 60], 10.00th=[ 70], 20.00th=[ 668], | 30.00th=[ 44303], 40.00th=[ 56886], 50.00th=[ 68682], 60.00th=[ 79168], | 70.00th=[ 91751], 80.00th=[114820], 90.00th=[135267], 95.00th=[158335], | 99.00th=[274727], 99.50th=[371196], 99.90th=[505414], 99.95th=[505414], | 99.99th=[505414] bw ( KiB/s): min= 31, max= 284, per=26.77%, avg=122.08, stdev=58.67, samples=100 iops : min= 1, max= 17, avg= 6.84, stdev= 3.75, samples=100 write: IOPS=7, BW=116KiB/s (118kB/s)(6032KiB/52207msec); 0 zone resets clat (usec): min=1679, max=404783, avg=62865.80, stdev=68716.23 lat (usec): min=1681, max=404785, avg=62871.00, stdev=68716.75 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 6], 20.00th=[ 12], | 30.00th=[ 17], 40.00th=[ 23], 50.00th=[ 40], 60.00th=[ 58], | 70.00th=[ 77], 80.00th=[ 108], 90.00th=[ 150], 95.00th=[ 192], | 99.00th=[ 334], 99.50th=[ 372], 99.90th=[ 405], 99.95th=[ 405], | 99.99th=[ 405] bw ( KiB/s): min= 30, max= 307, per=24.72%, avg=118.64, stdev=61.72, samples=99 iops : min= 1, max= 19, avg= 6.61, stdev= 3.92, samples=99 lat (usec) : 50=1.04%, 100=7.29%, 250=0.65%, 500=0.91%, 750=0.39% lat (msec) : 2=0.39%, 4=3.52%, 10=5.47%, 20=8.85%, 50=15.76% lat (msec) : 100=31.51%, 250=22.27%, 500=1.82%, 750=0.13% cpu : usr=0.06%, sys=4.21%, ctx=1570, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146602: Sun Mar 15 18:35:40 2026 read: IOPS=7, BW=115KiB/s (118kB/s)(6032KiB/52461msec) clat (usec): min=39, max=445663, avg=70591.51, stdev=61712.38 lat (usec): min=40, max=445663, avg=70619.69, stdev=61698.01 clat percentiles (usec): | 1.00th=[ 43], 5.00th=[ 59], 10.00th=[ 67], 20.00th=[ 94], | 30.00th=[ 37487], 40.00th=[ 52691], 50.00th=[ 63701], 60.00th=[ 78119], | 70.00th=[ 91751], 80.00th=[110625], 90.00th=[152044], 95.00th=[181404], | 99.00th=[250610], 99.50th=[265290], 99.90th=[446694], 99.95th=[446694], | 99.99th=[446694] bw ( KiB/s): min= 30, max= 340, per=25.47%, avg=116.15, stdev=55.14, samples=100 iops : min= 1, max= 21, avg= 6.46, stdev= 3.53, samples=100 write: IOPS=7, BW=119KiB/s (122kB/s)(6256KiB/52461msec); 0 zone resets clat (usec): min=1713, max=632990, avg=65407.77, stdev=71211.99 lat (usec): min=1715, max=632992, avg=65426.48, stdev=71206.92 clat percentiles (usec): | 1.00th=[ 1893], 5.00th=[ 2573], 10.00th=[ 4146], 20.00th=[ 10945], | 30.00th=[ 16057], 40.00th=[ 22676], 50.00th=[ 40109], 60.00th=[ 61080], | 70.00th=[ 86508], 80.00th=[116917], 90.00th=[156238], 95.00th=[191890], | 99.00th=[304088], 99.50th=[333448], 99.90th=[633340], 99.95th=[633340], | 99.99th=[633340] bw ( KiB/s): min= 29, max= 350, per=24.85%, avg=119.26, stdev=60.53, samples=102 iops : min= 1, max= 21, avg= 6.65, stdev= 3.84, samples=102 lat (usec) : 50=1.43%, 100=8.85%, 250=0.78%, 500=0.39% lat (msec) : 2=1.30%, 4=4.30%, 10=4.43%, 20=9.51%, 50=15.23% lat (msec) : 100=28.65%, 250=23.70%, 500=1.30%, 750=0.13% cpu : usr=0.06%, sys=4.37%, ctx=1553, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=457KiB/s (468kB/s), 113KiB/s-120KiB/s (116kB/s-123kB/s), io=23.4MiB (24.5MB), run=50615-52461msec WRITE: bw=480KiB/s (492kB/s), 116KiB/s-129KiB/s (118kB/s-132kB/s), io=24.6MiB (25.8MB), run=50615-52461msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146603: Sun Mar 15 18:37:01 2026 read: IOPS=2, BW=45.5KiB/s (46.6kB/s)(5920KiB/130203msec) clat (msec): min=12, max=510, avg=75.60, stdev=70.51 lat (msec): min=12, max=510, avg=75.60, stdev=70.51 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 22], 10.00th=[ 26], 20.00th=[ 32], | 30.00th=[ 39], 40.00th=[ 46], 50.00th=[ 52], 60.00th=[ 61], | 70.00th=[ 73], 80.00th=[ 97], 90.00th=[ 163], 95.00th=[ 211], | 99.00th=[ 388], 99.50th=[ 439], 99.90th=[ 510], 99.95th=[ 510], | 99.99th=[ 510] bw ( KiB/s): min= 29, max= 192, per=35.02%, avg=62.68, stdev=37.76, samples=184 iops : min= 1, max= 12, avg= 3.12, stdev= 2.44, samples=184 write: IOPS=3, BW=48.9KiB/s (50.1kB/s)(6368KiB/130203msec); 0 zone resets clat (msec): min=69, max=810, avg=256.52, stdev=117.30 lat (msec): min=69, max=810, avg=256.53, stdev=117.30 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 121], 10.00th=[ 140], 20.00th=[ 176], | 30.00th=[ 192], 40.00th=[ 211], 50.00th=[ 228], 60.00th=[ 251], | 70.00th=[ 284], 80.00th=[ 321], 90.00th=[ 414], 95.00th=[ 510], | 99.00th=[ 659], 99.50th=[ 768], 99.90th=[ 810], 99.95th=[ 810], | 99.99th=[ 810] bw ( KiB/s): min= 29, max= 128, per=27.69%, avg=52.32, stdev=21.83, samples=237 iops : min= 1, max= 8, avg= 2.46, stdev= 1.45, samples=237 lat (msec) : 20=1.30%, 50=21.61%, 100=17.06%, 250=37.63%, 500=19.40% lat (msec) : 750=2.73%, 1000=0.26% cpu : usr=0.03%, sys=1.03%, ctx=1418, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146604: Sun Mar 15 18:37:01 2026 read: IOPS=2, BW=43.1KiB/s (44.2kB/s)(5744KiB/133158msec) clat (msec): min=11, max=384, avg=76.93, stdev=67.32 lat (msec): min=11, max=384, avg=76.93, stdev=67.32 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 23], 10.00th=[ 27], 20.00th=[ 33], | 30.00th=[ 41], 40.00th=[ 47], 50.00th=[ 54], 60.00th=[ 65], | 70.00th=[ 81], 80.00th=[ 97], 90.00th=[ 148], 95.00th=[ 239], | 99.00th=[ 363], 99.50th=[ 380], 99.90th=[ 384], 99.95th=[ 384], | 99.99th=[ 384] bw ( KiB/s): min= 28, max= 216, per=35.46%, avg=63.47, stdev=37.83, samples=175 iops : min= 1, max= 13, avg= 3.15, stdev= 2.41, samples=175 write: IOPS=3, BW=49.1KiB/s (50.3kB/s)(6544KiB/133158msec); 0 zone resets clat (msec): min=82, max=818, avg=257.79, stdev=119.31 lat (msec): min=82, max=818, avg=257.79, stdev=119.31 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 122], 10.00th=[ 142], 20.00th=[ 167], | 30.00th=[ 188], 40.00th=[ 205], 50.00th=[ 232], 60.00th=[ 255], | 70.00th=[ 288], 80.00th=[ 321], 90.00th=[ 418], 95.00th=[ 493], | 99.00th=[ 651], 99.50th=[ 735], 99.90th=[ 818], 99.95th=[ 818], | 99.99th=[ 818] bw ( KiB/s): min= 28, max= 128, per=28.01%, avg=52.93, stdev=22.19, samples=241 iops : min= 1, max= 8, avg= 2.50, stdev= 1.47, samples=241 lat (msec) : 20=1.04%, 50=19.92%, 100=17.97%, 250=36.46%, 500=22.14% lat (msec) : 750=2.21%, 1000=0.26% cpu : usr=0.04%, sys=1.03%, ctx=1464, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146605: Sun Mar 15 18:37:01 2026 read: IOPS=3, BW=49.7KiB/s (50.9kB/s)(6256KiB/125807msec) clat (msec): min=10, max=661, avg=74.13, stdev=79.98 lat (msec): min=10, max=661, avg=74.13, stdev=79.98 clat percentiles (msec): | 1.00th=[ 13], 5.00th=[ 20], 10.00th=[ 23], 20.00th=[ 32], | 30.00th=[ 37], 40.00th=[ 44], 50.00th=[ 51], 60.00th=[ 62], | 70.00th=[ 74], 80.00th=[ 89], 90.00th=[ 136], 95.00th=[ 209], | 99.00th=[ 447], 99.50th=[ 575], 99.90th=[ 659], 99.95th=[ 659], | 99.99th=[ 659] bw ( KiB/s): min= 29, max= 256, per=37.90%, avg=67.83, stdev=45.78, samples=181 iops : min= 1, max= 16, avg= 3.44, stdev= 2.93, samples=181 write: IOPS=2, BW=47.9KiB/s (49.1kB/s)(6032KiB/125807msec); 0 zone resets clat (msec): min=99, max=751, avg=256.48, stdev=110.90 lat (msec): min=99, max=751, avg=256.49, stdev=110.90 clat percentiles (msec): | 1.00th=[ 112], 5.00th=[ 130], 10.00th=[ 140], 20.00th=[ 167], | 30.00th=[ 184], 40.00th=[ 201], 50.00th=[ 226], 60.00th=[ 264], | 70.00th=[ 292], 80.00th=[ 338], 90.00th=[ 414], 95.00th=[ 468], | 99.00th=[ 634], 99.50th=[ 651], 99.90th=[ 751], 99.95th=[ 751], | 99.99th=[ 751] bw ( KiB/s): min= 27, max= 127, per=27.39%, avg=51.77, stdev=21.40, samples=226 iops : min= 1, max= 7, avg= 2.42, stdev= 1.42, samples=226 lat (msec) : 20=2.73%, 50=22.66%, 100=17.97%, 250=33.46%, 500=21.35% lat (msec) : 750=1.69%, 1000=0.13% cpu : usr=0.03%, sys=1.02%, ctx=1411, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146606: Sun Mar 15 18:37:01 2026 read: IOPS=2, BW=45.9KiB/s (46.0kB/s)(6032KiB/131431msec) clat (msec): min=11, max=908, avg=78.05, stdev=87.93 lat (msec): min=11, max=908, avg=78.05, stdev=87.93 clat percentiles (msec): | 1.00th=[ 14], 5.00th=[ 21], 10.00th=[ 26], 20.00th=[ 34], | 30.00th=[ 39], 40.00th=[ 47], 50.00th=[ 55], 60.00th=[ 64], | 70.00th=[ 78], 80.00th=[ 93], 90.00th=[ 148], 95.00th=[ 215], | 99.00th=[ 376], 99.50th=[ 844], 99.90th=[ 911], 99.95th=[ 911], | 99.99th=[ 911] bw ( KiB/s): min= 29, max= 224, per=36.11%, avg=64.64, stdev=43.20, samples=181 iops : min= 1, max= 14, avg= 3.25, stdev= 2.73, samples=181 write: IOPS=2, BW=47.6KiB/s (48.7kB/s)(6256KiB/131431msec); 0 zone resets clat (msec): min=91, max=737, avg=260.54, stdev=113.61 lat (msec): min=91, max=737, avg=260.55, stdev=113.61 clat percentiles (msec): | 1.00th=[ 100], 5.00th=[ 133], 10.00th=[ 150], 20.00th=[ 178], | 30.00th=[ 194], 40.00th=[ 211], 50.00th=[ 234], 60.00th=[ 257], | 70.00th=[ 284], 80.00th=[ 326], 90.00th=[ 418], 95.00th=[ 523], | 99.00th=[ 651], 99.50th=[ 701], 99.90th=[ 735], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 27, max= 96, per=27.54%, avg=52.04, stdev=20.47, samples=235 iops : min= 1, max= 6, avg= 2.45, stdev= 1.37, samples=235 lat (msec) : 20=2.34%, 50=20.05%, 100=18.23%, 250=35.29%, 500=20.96% lat (msec) : 750=2.86%, 1000=0.26% cpu : usr=0.03%, sys=1.00%, ctx=1419, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=180KiB/s (184kB/s), 43.1KiB/s-49.7KiB/s (44.2kB/s-50.9kB/s), io=23.4MiB (24.5MB), run=125807-133158msec WRITE: bw=189KiB/s (194kB/s), 47.6KiB/s-49.1KiB/s (48.7kB/s-50.3kB/s), io=24.6MiB (25.8MB), run=125807-133158msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146640: Sun Mar 15 18:37:28 2026 read: IOPS=1, BW=1030KiB/s (1054kB/s)(7168KiB/6962msec) clat (msec): min=15, max=1780, avg=498.48, stdev=587.70 lat (msec): min=15, max=1780, avg=498.48, stdev=587.70 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 16], 10.00th=[ 16], 20.00th=[ 245], | 30.00th=[ 257], 40.00th=[ 257], 50.00th=[ 292], 60.00th=[ 338], | 70.00th=[ 338], 80.00th=[ 558], 90.00th=[ 1787], 95.00th=[ 1787], | 99.00th=[ 1787], 99.50th=[ 1787], 99.90th=[ 1787], 99.95th=[ 1787], | 99.99th=[ 1787] bw ( KiB/s): min= 2007, max= 4055, per=100.00%, avg=3024.00, stdev=1162.98, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 write: IOPS=0, BW=735KiB/s (753kB/s)(5120KiB/6962msec); 0 zone resets clat (msec): min=196, max=1120, avg=628.67, stdev=369.36 lat (msec): min=196, max=1120, avg=629.28, stdev=369.19 clat percentiles (msec): | 1.00th=[ 197], 5.00th=[ 197], 10.00th=[ 197], 20.00th=[ 197], | 30.00th=[ 384], 40.00th=[ 384], 50.00th=[ 584], 60.00th=[ 584], | 70.00th=[ 860], 80.00th=[ 860], 90.00th=[ 1116], 95.00th=[ 1116], | 99.00th=[ 1116], 99.50th=[ 1116], 99.90th=[ 1116], 99.95th=[ 1116], | 99.99th=[ 1116] bw ( KiB/s): min= 1946, max= 2015, per=65.07%, avg=1995.75, stdev=33.38, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 lat (msec) : 20=8.33%, 250=16.67%, 500=33.33%, 750=16.67%, 1000=8.33% lat (msec) : 2000=16.67% cpu : usr=0.00%, sys=7.51%, ctx=298, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146641: Sun Mar 15 18:37:28 2026 read: IOPS=0, BW=219KiB/s (224kB/s)(2048KiB/9348msec) clat (msec): min=330, max=577, avg=453.88, stdev=175.18 lat (msec): min=330, max=577, avg=453.88, stdev=175.18 clat percentiles (msec): | 1.00th=[ 330], 5.00th=[ 330], 10.00th=[ 330], 20.00th=[ 330], | 30.00th=[ 330], 40.00th=[ 330], 50.00th=[ 330], 60.00th=[ 575], | 70.00th=[ 575], 80.00th=[ 575], 90.00th=[ 575], 95.00th=[ 575], | 99.00th=[ 575], 99.50th=[ 575], 99.90th=[ 575], 99.95th=[ 575], | 99.99th=[ 575] bw ( KiB/s): min= 1370, max= 2023, per=77.47%, avg=1696.50, stdev=461.74, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1095KiB/s (1122kB/s)(10.0MiB/9348msec); 0 zone resets clat (msec): min=212, max=2833, avg=804.47, stdev=811.66 lat (msec): min=212, max=2833, avg=804.57, stdev=811.66 clat percentiles (msec): | 1.00th=[ 213], 5.00th=[ 213], 10.00th=[ 213], 20.00th=[ 247], | 30.00th=[ 271], 40.00th=[ 359], 50.00th=[ 401], 60.00th=[ 405], | 70.00th=[ 919], 80.00th=[ 1099], 90.00th=[ 1301], 95.00th=[ 2836], | 99.00th=[ 2836], 99.50th=[ 2836], 99.90th=[ 2836], 99.95th=[ 2836], | 99.99th=[ 2836] bw ( KiB/s): min= 1954, max= 4063, per=83.33%, avg=2555.86, stdev=980.17, samples=7 iops : min= 1, max= 3, avg= 1.57, stdev= 0.98, samples=7 lat (msec) : 250=16.67%, 500=41.67%, 750=8.33%, 1000=8.33%, 2000=16.67% lat (msec) : >=2000=8.33% cpu : usr=0.00%, sys=6.72%, ctx=176, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146642: Sun Mar 15 18:37:28 2026 read: IOPS=0, BW=738KiB/s (756kB/s)(6144KiB/8323msec) clat (usec): min=901, max=1159.4k, avg=364502.04, stdev=424949.71 lat (usec): min=903, max=1159.4k, avg=364503.43, stdev=424949.65 clat percentiles (usec): | 1.00th=[ 906], 5.00th=[ 906], 10.00th=[ 906], | 20.00th=[ 3458], 30.00th=[ 3458], 40.00th=[ 299893], | 50.00th=[ 299893], 60.00th=[ 316670], 70.00th=[ 408945], | 80.00th=[ 408945], 90.00th=[1166017], 95.00th=[1166017], | 99.00th=[1166017], 99.50th=[1166017], 99.90th=[1166017], | 99.95th=[1166017], 99.99th=[1166017] bw ( KiB/s): min= 1943, max= 2027, per=90.83%, avg=1989.20, stdev=41.11, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 write: IOPS=0, BW=738KiB/s (756kB/s)(6144KiB/8323msec); 0 zone resets clat (msec): min=407, max=2851, avg=1001.73, stdev=963.75 lat (msec): min=407, max=2851, avg=1001.86, stdev=963.76 clat percentiles (msec): | 1.00th=[ 409], 5.00th=[ 409], 10.00th=[ 409], 20.00th=[ 435], | 30.00th=[ 435], 40.00th=[ 506], 50.00th=[ 506], 60.00th=[ 527], | 70.00th=[ 1284], 80.00th=[ 1284], 90.00th=[ 2836], 95.00th=[ 2836], | 99.00th=[ 2836], 99.50th=[ 2836], 99.90th=[ 2836], 99.95th=[ 2836], | 99.99th=[ 2836] bw ( KiB/s): min= 1943, max= 2015, per=64.57%, avg=1980.50, stdev=32.02, samples=6 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=6 lat (usec) : 1000=8.33% lat (msec) : 4=8.33%, 500=41.67%, 750=16.67%, 2000=16.67%, >=2000=8.33% cpu : usr=0.08%, sys=6.15%, ctx=164, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146643: Sun Mar 15 18:37:28 2026 read: IOPS=0, BW=662KiB/s (678kB/s)(5120KiB/7738msec) clat (usec): min=733, max=600810, avg=280797.21, stdev=271494.30 lat (usec): min=734, max=600811, avg=280798.37, stdev=271493.99 clat percentiles (usec): | 1.00th=[ 734], 5.00th=[ 734], 10.00th=[ 734], 20.00th=[ 734], | 30.00th=[ 799], 40.00th=[ 799], 50.00th=[341836], 60.00th=[341836], | 70.00th=[459277], 80.00th=[459277], 90.00th=[599786], 95.00th=[599786], | 99.00th=[599786], 99.50th=[599786], 99.90th=[599786], 99.95th=[599786], | 99.99th=[599786] bw ( KiB/s): min= 2015, max= 3886, per=100.00%, avg=2642.67, stdev=1076.77, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=0, BW=926KiB/s (949kB/s)(7168KiB/7738msec); 0 zone resets clat (msec): min=289, max=2538, avg=848.49, stdev=801.17 lat (msec): min=289, max=2538, avg=848.62, stdev=801.25 clat percentiles (msec): | 1.00th=[ 292], 5.00th=[ 292], 10.00th=[ 292], 20.00th=[ 347], | 30.00th=[ 368], 40.00th=[ 368], 50.00th=[ 567], 60.00th=[ 684], | 70.00th=[ 684], 80.00th=[ 1150], 90.00th=[ 2534], 95.00th=[ 2534], | 99.00th=[ 2534], 99.50th=[ 2534], 99.90th=[ 2534], 99.95th=[ 2534], | 99.99th=[ 2534] bw ( KiB/s): min= 1943, max= 2015, per=64.83%, avg=1988.33, stdev=34.75, samples=6 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=6 lat (usec) : 750=8.33%, 1000=8.33% lat (msec) : 500=41.67%, 750=25.00%, 2000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=5.78%, ctx=157, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2191KiB/s (2243kB/s), 219KiB/s-1030KiB/s (224kB/s-1054kB/s), io=20.0MiB (20.0MB), run=6962-9348msec WRITE: bw=3067KiB/s (3141kB/s), 735KiB/s-1095KiB/s (753kB/s-1122kB/s), io=28.0MiB (29.4MB), run=6962-9348msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146636: Sun Mar 15 18:37:28 2026 read: IOPS=1, BW=1083KiB/s (1109kB/s)(7168KiB/6616msec) clat (msec): min=185, max=1415, avg=461.08, stdev=427.52 lat (msec): min=185, max=1415, avg=461.08, stdev=427.52 clat percentiles (msec): | 1.00th=[ 186], 5.00th=[ 186], 10.00th=[ 186], 20.00th=[ 249], | 30.00th=[ 268], 40.00th=[ 268], 50.00th=[ 330], 60.00th=[ 384], | 70.00th=[ 384], 80.00th=[ 397], 90.00th=[ 1418], 95.00th=[ 1418], | 99.00th=[ 1418], 99.50th=[ 1418], 99.90th=[ 1418], 99.95th=[ 1418], | 99.99th=[ 1418] bw ( KiB/s): min= 1935, max= 4007, per=100.00%, avg=2387.40, stdev=905.82, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 write: IOPS=0, BW=774KiB/s (792kB/s)(5120KiB/6616msec); 0 zone resets clat (msec): min=356, max=873, avg=588.16, stdev=213.70 lat (msec): min=356, max=873, avg=588.26, stdev=213.68 clat percentiles (msec): | 1.00th=[ 355], 5.00th=[ 355], 10.00th=[ 355], 20.00th=[ 355], | 30.00th=[ 422], 40.00th=[ 422], 50.00th=[ 567], 60.00th=[ 567], | 70.00th=[ 726], 80.00th=[ 726], 90.00th=[ 877], 95.00th=[ 877], | 99.00th=[ 877], 99.50th=[ 877], 99.90th=[ 877], 99.95th=[ 877], | 99.99th=[ 877] bw ( KiB/s): min= 1973, max= 2019, per=67.30%, avg=2003.40, stdev=18.13, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (msec) : 250=16.67%, 500=50.00%, 750=16.67%, 1000=8.33%, 2000=8.33% cpu : usr=0.00%, sys=2.62%, ctx=86, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146637: Sun Mar 15 18:37:28 2026 read: IOPS=0, BW=218KiB/s (223kB/s)(2048KiB/9393msec) clat (msec): min=235, max=1128, avg=682.01, stdev=631.49 lat (msec): min=235, max=1128, avg=682.01, stdev=631.49 clat percentiles (msec): | 1.00th=[ 236], 5.00th=[ 236], 10.00th=[ 236], 20.00th=[ 236], | 30.00th=[ 236], 40.00th=[ 236], 50.00th=[ 236], 60.00th=[ 1133], | 70.00th=[ 1133], 80.00th=[ 1133], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 1973, max= 2003, per=93.51%, avg=1988.00, stdev=21.21, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1090KiB/s (1116kB/s)(10.0MiB/9393msec); 0 zone resets clat (msec): min=267, max=2440, avg=756.10, stdev=657.79 lat (msec): min=267, max=2440, avg=760.77, stdev=658.85 clat percentiles (msec): | 1.00th=[ 268], 5.00th=[ 268], 10.00th=[ 268], 20.00th=[ 288], | 30.00th=[ 351], 40.00th=[ 456], 50.00th=[ 489], 60.00th=[ 575], | 70.00th=[ 617], 80.00th=[ 877], 90.00th=[ 1217], 95.00th=[ 2433], | 99.00th=[ 2433], 99.50th=[ 2433], 99.90th=[ 2433], 99.95th=[ 2433], | 99.99th=[ 2433] bw ( KiB/s): min= 1943, max= 4071, per=86.77%, avg=2583.14, stdev=984.82, samples=7 iops : min= 1, max= 3, avg= 1.57, stdev= 0.98, samples=7 lat (msec) : 250=8.33%, 500=41.67%, 750=16.67%, 1000=8.33%, 2000=16.67% lat (msec) : >=2000=8.33% cpu : usr=0.01%, sys=1.83%, ctx=79, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146638: Sun Mar 15 18:37:28 2026 read: IOPS=0, BW=835KiB/s (855kB/s)(6144KiB/7358msec) clat (msec): min=193, max=645, avg=315.30, stdev=166.16 lat (msec): min=193, max=645, avg=315.30, stdev=166.16 clat percentiles (msec): | 1.00th=[ 194], 5.00th=[ 194], 10.00th=[ 194], 20.00th=[ 220], | 30.00th=[ 220], 40.00th=[ 259], 50.00th=[ 259], 60.00th=[ 279], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 642], 95.00th=[ 642], | 99.00th=[ 642], 99.50th=[ 642], 99.90th=[ 642], 99.95th=[ 642], | 99.99th=[ 642] bw ( KiB/s): min= 2000, max= 4031, per=100.00%, avg=2521.25, stdev=1006.60, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=0, BW=835KiB/s (855kB/s)(6144KiB/7358msec); 0 zone resets clat (msec): min=415, max=2158, avg=893.38, stdev=676.35 lat (msec): min=415, max=2159, avg=893.48, stdev=676.35 clat percentiles (msec): | 1.00th=[ 414], 5.00th=[ 414], 10.00th=[ 414], 20.00th=[ 439], | 30.00th=[ 439], 40.00th=[ 542], 50.00th=[ 542], 60.00th=[ 651], | 70.00th=[ 1150], 80.00th=[ 1150], 90.00th=[ 2165], 95.00th=[ 2165], | 99.00th=[ 2165], 99.50th=[ 2165], 99.90th=[ 2165], 99.95th=[ 2165], | 99.99th=[ 2165] bw ( KiB/s): min= 1585, max= 2019, per=64.76%, avg=1928.00, stdev=168.74, samples=6 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=6 lat (msec) : 250=16.67%, 500=41.67%, 750=25.00%, 2000=8.33%, >=2000=8.33% cpu : usr=0.07%, sys=2.11%, ctx=63, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146639: Sun Mar 15 18:37:28 2026 read: IOPS=0, BW=532KiB/s (544kB/s)(5120KiB/9629msec) clat (msec): min=87, max=1238, avg=590.36, stdev=566.08 lat (msec): min=87, max=1238, avg=590.36, stdev=566.08 clat percentiles (msec): | 1.00th=[ 88], 5.00th=[ 88], 10.00th=[ 88], 20.00th=[ 88], | 30.00th=[ 199], 40.00th=[ 199], 50.00th=[ 253], 60.00th=[ 253], | 70.00th=[ 1183], 80.00th=[ 1183], 90.00th=[ 1234], 95.00th=[ 1234], | 99.00th=[ 1234], 99.50th=[ 1234], 99.90th=[ 1234], 99.95th=[ 1234], | 99.99th=[ 1234] bw ( KiB/s): min= 1943, max= 4087, per=100.00%, avg=2688.33, stdev=1212.15, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=0, BW=744KiB/s (762kB/s)(7168KiB/9629msec); 0 zone resets clat (msec): min=269, max=2639, avg=878.74, stdev=830.33 lat (msec): min=269, max=2639, avg=879.12, stdev=830.38 clat percentiles (msec): | 1.00th=[ 271], 5.00th=[ 271], 10.00th=[ 271], 20.00th=[ 351], | 30.00th=[ 464], 40.00th=[ 464], 50.00th=[ 493], 60.00th=[ 810], | 70.00th=[ 810], 80.00th=[ 1133], 90.00th=[ 2635], 95.00th=[ 2635], | 99.00th=[ 2635], 99.50th=[ 2635], 99.90th=[ 2635], 99.95th=[ 2635], | 99.99th=[ 2635] bw ( KiB/s): min= 2000, max= 2035, per=67.67%, avg=2014.50, stdev=12.52, samples=6 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=6 lat (msec) : 100=8.33%, 250=8.33%, 500=41.67%, 1000=8.33%, 2000=25.00% lat (msec) : >=2000=8.33% cpu : usr=0.00%, sys=2.16%, ctx=100, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2127KiB/s (2178kB/s), 218KiB/s-1083KiB/s (223kB/s-1109kB/s), io=20.0MiB (20.0MB), run=6616-9629msec WRITE: bw=2978KiB/s (3049kB/s), 744KiB/s-1090KiB/s (762kB/s-1116kB/s), io=28.0MiB (29.4MB), run=6616-9629msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146651: Sun Mar 15 18:37:53 2026 read: IOPS=0, BW=1501KiB/s (1537kB/s)(12.0MiB/8188msec) clat (msec): min=751, max=5184, avg=2676.32, stdev=2273.40 lat (msec): min=751, max=5184, avg=2676.32, stdev=2273.40 clat percentiles (msec): | 1.00th=[ 751], 5.00th=[ 751], 10.00th=[ 751], 20.00th=[ 751], | 30.00th=[ 751], 40.00th=[ 2089], 50.00th=[ 2089], 60.00th=[ 2089], | 70.00th=[ 5201], 80.00th=[ 5201], 90.00th=[ 5201], 95.00th=[ 5201], | 99.00th=[ 5201], 99.50th=[ 5201], 99.90th=[ 5201], 99.95th=[ 5201], | 99.99th=[ 5201] bw ( KiB/s): min= 8143, max= 8143, per=100.00%, avg=8143.00, stdev= 0.00, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=33.33%, >=2000=66.67% cpu : usr=0.00%, sys=1.45%, ctx=56, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146652: Sun Mar 15 18:37:53 2026 read: IOPS=0, BW=486KiB/s (497kB/s)(4096KiB/8435msec) clat (nsec): min=5524.9M, max=5524.9M, avg=5524887828.00, stdev= 0.00 lat (nsec): min=5524.9M, max=5524.9M, avg=5524890607.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 5537], 5.00th=[ 5537], 10.00th=[ 5537], 20.00th=[ 5537], | 30.00th=[ 5537], 40.00th=[ 5537], 50.00th=[ 5537], 60.00th=[ 5537], | 70.00th=[ 5537], 80.00th=[ 5537], 90.00th=[ 5537], 95.00th=[ 5537], | 99.00th=[ 5537], 99.50th=[ 5537], 99.90th=[ 5537], 99.95th=[ 5537], | 99.99th=[ 5537] bw ( KiB/s): min= 8143, max= 8143, per=100.00%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=971KiB/s (995kB/s)(8192KiB/8435msec); 0 zone resets clat (msec): min=647, max=1994, avg=1321.06, stdev=952.85 lat (msec): min=647, max=1995, avg=1321.51, stdev=952.80 clat percentiles (msec): | 1.00th=[ 651], 5.00th=[ 651], 10.00th=[ 651], 20.00th=[ 651], | 30.00th=[ 651], 40.00th=[ 651], 50.00th=[ 651], 60.00th=[ 1989], | 70.00th=[ 1989], 80.00th=[ 1989], 90.00th=[ 1989], 95.00th=[ 1989], | 99.00th=[ 1989], 99.50th=[ 1989], 99.90th=[ 1989], 99.95th=[ 1989], | 99.99th=[ 1989] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=1.20%, ctx=57, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146655: Sun Mar 15 18:37:53 2026 write: IOPS=0, BW=1519KiB/s (1556kB/s)(12.0MiB/8088msec); 0 zone resets clat (msec): min=1434, max=3922, avg=2593.38, stdev=1253.13 lat (msec): min=1437, max=3923, avg=2594.78, stdev=1252.11 clat percentiles (msec): | 1.00th=[ 1435], 5.00th=[ 1435], 10.00th=[ 1435], 20.00th=[ 1435], | 30.00th=[ 1435], 40.00th=[ 2433], 50.00th=[ 2433], 60.00th=[ 2433], | 70.00th=[ 3910], 80.00th=[ 3910], 90.00th=[ 3910], 95.00th=[ 3910], | 99.00th=[ 3910], 99.50th=[ 3910], 99.90th=[ 3910], 99.95th=[ 3910], | 99.99th=[ 3910] bw ( KiB/s): min= 8015, max= 8159, per=100.00%, avg=8087.00, stdev=101.82, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 2000=33.33%, >=2000=66.67% cpu : usr=0.00%, sys=3.38%, ctx=75, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146658: Sun Mar 15 18:37:53 2026 write: IOPS=0, BW=1902KiB/s (1948kB/s)(12.0MiB/6459msec); 0 zone resets clat (msec): min=585, max=3316, avg=2110.48, stdev=1392.91 lat (msec): min=586, max=3316, avg=2112.89, stdev=1393.43 clat percentiles (msec): | 1.00th=[ 584], 5.00th=[ 584], 10.00th=[ 584], 20.00th=[ 584], | 30.00th=[ 584], 40.00th=[ 2433], 50.00th=[ 2433], 60.00th=[ 2433], | 70.00th=[ 3306], 80.00th=[ 3306], 90.00th=[ 3306], 95.00th=[ 3306], | 99.00th=[ 3306], 99.50th=[ 3306], 99.90th=[ 3306], 99.95th=[ 3306], | 99.99th=[ 3306] bw ( KiB/s): min= 7937, max= 8110, per=100.00%, avg=8023.50, stdev=122.33, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, >=2000=66.67% cpu : usr=0.00%, sys=2.35%, ctx=61, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1942KiB/s (1989kB/s), 486KiB/s-1501KiB/s (497kB/s-1537kB/s), io=16.0MiB (16.8MB), run=8188-8435msec WRITE: bw=3885KiB/s (3978kB/s), 971KiB/s-1902KiB/s (995kB/s-1948kB/s), io=32.0MiB (33.6MB), run=6459-8435msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146653: Sun Mar 15 18:37:54 2026 read: IOPS=0, BW=1692KiB/s (1733kB/s)(12.0MiB/7262msec) clat (msec): min=1191, max=4449, avg=2387.82, stdev=1793.50 lat (msec): min=1191, max=4449, avg=2387.82, stdev=1793.50 clat percentiles (msec): | 1.00th=[ 1183], 5.00th=[ 1183], 10.00th=[ 1183], 20.00th=[ 1183], | 30.00th=[ 1183], 40.00th=[ 1519], 50.00th=[ 1519], 60.00th=[ 1519], | 70.00th=[ 4463], 80.00th=[ 4463], 90.00th=[ 4463], 95.00th=[ 4463], | 99.00th=[ 4463], 99.50th=[ 4463], 99.90th=[ 4463], 99.95th=[ 4463], | 99.99th=[ 4463] bw ( KiB/s): min= 7529, max= 7984, per=100.00%, avg=7756.50, stdev=321.73, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 2000=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=4.72%, ctx=215, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146654: Sun Mar 15 18:37:54 2026 read: IOPS=0, BW=507KiB/s (519kB/s)(4096KiB/8085msec) clat (nsec): min=4501.6M, max=4501.6M, avg=4501636968.00, stdev= 0.00 lat (nsec): min=4501.6M, max=4501.6M, avg=4501639282.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 4530], 5.00th=[ 4530], 10.00th=[ 4530], 20.00th=[ 4530], | 30.00th=[ 4530], 40.00th=[ 4530], 50.00th=[ 4530], 60.00th=[ 4530], | 70.00th=[ 4530], 80.00th=[ 4530], 90.00th=[ 4530], 95.00th=[ 4530], | 99.00th=[ 4530], 99.50th=[ 4530], 99.90th=[ 4530], 99.95th=[ 4530], | 99.99th=[ 4530] bw ( KiB/s): min= 7699, max= 7699, per=100.00%, avg=7699.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1013KiB/s (1038kB/s)(8192KiB/8085msec); 0 zone resets clat (msec): min=1040, max=2398, avg=1719.87, stdev=960.15 lat (msec): min=1041, max=2399, avg=1720.46, stdev=960.20 clat percentiles (msec): | 1.00th=[ 1045], 5.00th=[ 1045], 10.00th=[ 1045], 20.00th=[ 1045], | 30.00th=[ 1045], 40.00th=[ 1045], 50.00th=[ 1045], 60.00th=[ 2400], | 70.00th=[ 2400], 80.00th=[ 2400], 90.00th=[ 2400], 95.00th=[ 2400], | 99.00th=[ 2400], 99.50th=[ 2400], 99.90th=[ 2400], 99.95th=[ 2400], | 99.99th=[ 2400] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 2000=33.33%, >=2000=66.67% cpu : usr=0.01%, sys=4.16%, ctx=440, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146656: Sun Mar 15 18:37:54 2026 write: IOPS=0, BW=1328KiB/s (1360kB/s)(12.0MiB/9250msec); 0 zone resets clat (msec): min=424, max=7067, avg=3050.90, stdev=3533.03 lat (msec): min=426, max=7068, avg=3051.83, stdev=3532.51 clat percentiles (msec): | 1.00th=[ 426], 5.00th=[ 426], 10.00th=[ 426], 20.00th=[ 426], | 30.00th=[ 426], 40.00th=[ 1653], 50.00th=[ 1653], 60.00th=[ 1653], | 70.00th=[ 7080], 80.00th=[ 7080], 90.00th=[ 7080], 95.00th=[ 7080], | 99.00th=[ 7080], 99.50th=[ 7080], 99.90th=[ 7080], 99.95th=[ 7080], | 99.99th=[ 7080] bw ( KiB/s): min= 7447, max= 7846, per=100.00%, avg=7646.50, stdev=282.14, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.02%, sys=1.94%, ctx=62, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146657: Sun Mar 15 18:37:54 2026 write: IOPS=0, BW=1583KiB/s (1621kB/s)(12.0MiB/7761msec); 0 zone resets clat (msec): min=1591, max=3769, avg=2532.37, stdev=1118.75 lat (msec): min=1592, max=3769, avg=2533.12, stdev=1118.65 clat percentiles (msec): | 1.00th=[ 1586], 5.00th=[ 1586], 10.00th=[ 1586], 20.00th=[ 1586], | 30.00th=[ 1586], 40.00th=[ 2232], 50.00th=[ 2232], 60.00th=[ 2232], | 70.00th=[ 3775], 80.00th=[ 3775], 90.00th=[ 3775], 95.00th=[ 3775], | 99.00th=[ 3775], 99.50th=[ 3775], 99.90th=[ 3775], 99.95th=[ 3775], | 99.99th=[ 3775] bw ( KiB/s): min= 7907, max= 8110, per=100.00%, avg=8008.50, stdev=143.54, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 2000=33.33%, >=2000=66.67% cpu : usr=0.00%, sys=1.12%, ctx=43, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2026KiB/s (2075kB/s), 507KiB/s-1692KiB/s (519kB/s-1733kB/s), io=16.0MiB (16.8MB), run=7262-8085msec WRITE: bw=3542KiB/s (3628kB/s), 1013KiB/s-1583KiB/s (1038kB/s-1621kB/s), io=32.0MiB (33.6MB), run=7761-9250msec