== sanity test 398b: DIO and buffer IO race ============== 13:03:39 (1761239019) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 4.59517 s, 11.0 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147529: Thu Oct 23 13:05:34 2025 read: IOPS=15, BW=60.3KiB/s (61.7kB/s)(5992KiB/99442msec) clat (usec): min=102, max=712362, avg=32732.08, stdev=31882.43 lat (usec): min=102, max=712364, avg=32733.28, stdev=31882.47 clat percentiles (usec): | 1.00th=[ 129], 5.00th=[ 161], 10.00th=[ 289], 20.00th=[ 15533], | 30.00th=[ 19006], 40.00th=[ 23200], 50.00th=[ 27395], 60.00th=[ 31851], | 70.00th=[ 39060], 80.00th=[ 47973], 90.00th=[ 60556], 95.00th=[ 72877], | 99.00th=[135267], 99.50th=[179307], 99.90th=[291505], 99.95th=[708838], | 99.99th=[708838] bw ( KiB/s): min= 7, max= 134, per=25.17%, avg=60.15, stdev=23.74, samples=195 iops : min= 1, max= 33, avg=14.43, stdev= 5.98, samples=195 write: IOPS=15, BW=63.3KiB/s (64.8kB/s)(6296KiB/99442msec); 0 zone resets clat (usec): min=872, max=371959, avg=31827.63, stdev=33231.15 lat (usec): min=873, max=371960, avg=31829.97, stdev=33230.63 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 1074], 10.00th=[ 1156], 20.00th=[ 1926], | 30.00th=[ 7373], 40.00th=[ 15401], 50.00th=[ 23987], 60.00th=[ 32113], | 70.00th=[ 41157], 80.00th=[ 54789], 90.00th=[ 74974], 95.00th=[ 92799], | 99.00th=[145753], 99.50th=[160433], 99.90th=[254804], 99.95th=[371196], | 99.99th=[371196] bw ( KiB/s): min= 7, max= 134, per=25.25%, avg=63.38, stdev=25.80, samples=194 iops : min= 1, max= 33, avg=15.25, stdev= 6.51, samples=194 lat (usec) : 250=4.52%, 500=0.65%, 750=0.10%, 1000=0.52% lat (msec) : 2=10.12%, 4=2.41%, 10=5.40%, 20=15.23%, 50=40.17% lat (msec) : 100=18.03%, 250=2.64%, 500=0.16%, 750=0.03% cpu : usr=0.07%, sys=6.13%, ctx=6148, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147530: Thu Oct 23 13:05:34 2025 read: IOPS=14, BW=59.3KiB/s (60.7kB/s)(5940KiB/100157msec) clat (usec): min=104, max=871561, avg=32437.50, stdev=35135.36 lat (usec): min=105, max=871563, avg=32441.01, stdev=35136.13 clat percentiles (usec): | 1.00th=[ 126], 5.00th=[ 161], 10.00th=[ 693], 20.00th=[ 15401], | 30.00th=[ 19268], 40.00th=[ 22938], 50.00th=[ 27132], 60.00th=[ 32375], | 70.00th=[ 38536], 80.00th=[ 47449], 90.00th=[ 60556], 95.00th=[ 72877], | 99.00th=[ 98042], 99.50th=[119014], 99.90th=[608175], 99.95th=[868221], | 99.99th=[868221] bw ( KiB/s): min= 7, max= 144, per=24.84%, avg=59.36, stdev=23.85, samples=194 iops : min= 1, max= 36, avg=14.23, stdev= 6.02, samples=194 write: IOPS=15, BW=63.4KiB/s (64.9kB/s)(6348KiB/100157msec); 0 zone resets clat (usec): min=871, max=334430, avg=32553.39, stdev=33434.42 lat (usec): min=872, max=334431, avg=32557.38, stdev=33433.64 clat percentiles (usec): | 1.00th=[ 988], 5.00th=[ 1090], 10.00th=[ 1172], 20.00th=[ 2638], | 30.00th=[ 7504], 40.00th=[ 16712], 50.00th=[ 25035], 60.00th=[ 32900], | 70.00th=[ 42730], 80.00th=[ 56361], 90.00th=[ 73925], 95.00th=[ 94897], | 99.00th=[152044], 99.50th=[170918], 99.90th=[225444], 99.95th=[333448], | 99.99th=[333448] bw ( KiB/s): min= 15, max= 151, per=25.18%, avg=63.19, stdev=23.52, samples=194 iops : min= 3, max= 37, avg=15.19, stdev= 5.91, samples=194 lat (usec) : 250=4.10%, 500=0.72%, 750=0.07%, 1000=0.62% lat (msec) : 2=8.92%, 4=2.90%, 10=5.44%, 20=15.66%, 50=40.07% lat (msec) : 100=19.01%, 250=2.38%, 500=0.07%, 750=0.03%, 1000=0.03% cpu : usr=0.06%, sys=6.40%, ctx=6373, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147531: Thu Oct 23 13:05:34 2025 read: IOPS=15, BW=60.9KiB/s (62.4kB/s)(6028KiB/98911msec) clat (usec): min=81, max=660649, avg=31913.73, stdev=30055.48 lat (usec): min=82, max=660650, avg=31918.76, stdev=30053.63 clat percentiles (usec): | 1.00th=[ 125], 5.00th=[ 163], 10.00th=[ 289], 20.00th=[ 14484], | 30.00th=[ 19268], 40.00th=[ 22938], 50.00th=[ 26870], 60.00th=[ 31327], | 70.00th=[ 38011], 80.00th=[ 47449], 90.00th=[ 60031], 95.00th=[ 72877], | 99.00th=[127402], 99.50th=[156238], 99.90th=[261096], 99.95th=[658506], | 99.99th=[658506] bw ( KiB/s): min= 7, max= 152, per=25.58%, avg=61.14, stdev=26.99, samples=193 iops : min= 1, max= 38, avg=14.67, stdev= 6.81, samples=193 write: IOPS=15, BW=63.3KiB/s (64.8kB/s)(6260KiB/98911msec); 0 zone resets clat (usec): min=908, max=976487, avg=32305.42, stdev=39847.16 lat (usec): min=909, max=976489, avg=32306.99, stdev=39847.16 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 1090], 10.00th=[ 1172], 20.00th=[ 2573], | 30.00th=[ 7832], 40.00th=[ 16450], 50.00th=[ 25035], 60.00th=[ 32375], | 70.00th=[ 42730], 80.00th=[ 53740], 90.00th=[ 71828], 95.00th=[ 87557], | 99.00th=[137364], 99.50th=[181404], 99.90th=[256902], 99.95th=[977273], | 99.99th=[977273] bw ( KiB/s): min= 7, max= 135, per=25.41%, avg=63.78, stdev=24.90, samples=192 iops : min= 1, max= 33, avg=15.34, stdev= 6.24, samples=192 lat (usec) : 100=0.13%, 250=4.33%, 500=0.98%, 750=0.20%, 1000=0.62% lat (msec) : 2=9.05%, 4=2.99%, 10=5.24%, 20=14.39%, 50=41.41% lat (msec) : 100=18.23%, 250=2.28%, 500=0.10%, 750=0.03%, 1000=0.03% cpu : usr=0.09%, sys=6.22%, ctx=6220, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147532: Thu Oct 23 13:05:34 2025 read: IOPS=15, BW=60.6KiB/s (62.1kB/s)(6052KiB/99846msec) clat (usec): min=104, max=347585, avg=32113.88, stdev=23560.02 lat (usec): min=104, max=347586, avg=32115.10, stdev=23560.03 clat percentiles (usec): | 1.00th=[ 122], 5.00th=[ 163], 10.00th=[ 363], 20.00th=[ 15926], | 30.00th=[ 20055], 40.00th=[ 23725], 50.00th=[ 27395], 60.00th=[ 33162], | 70.00th=[ 39060], 80.00th=[ 48497], 90.00th=[ 60556], 95.00th=[ 70779], | 99.00th=[104334], 99.50th=[122160], 99.90th=[191890], 99.95th=[346031], | 99.99th=[346031] bw ( KiB/s): min= 15, max= 135, per=25.56%, avg=61.09, stdev=22.68, samples=194 iops : min= 3, max= 33, avg=14.65, stdev= 5.69, samples=194 write: IOPS=15, BW=62.5KiB/s (63.0kB/s)(6236KiB/99846msec); 0 zone resets clat (usec): min=892, max=916627, avg=32693.17, stdev=42666.64 lat (usec): min=892, max=916630, avg=32699.07, stdev=42667.49 clat percentiles (usec): | 1.00th=[ 979], 5.00th=[ 1074], 10.00th=[ 1156], 20.00th=[ 2114], | 30.00th=[ 6325], 40.00th=[ 14877], 50.00th=[ 24249], 60.00th=[ 32375], | 70.00th=[ 42730], 80.00th=[ 53216], 90.00th=[ 71828], 95.00th=[ 94897], | 99.00th=[156238], 99.50th=[229639], 99.90th=[417334], 99.95th=[918553], | 99.99th=[918553] bw ( KiB/s): min= 7, max= 152, per=24.93%, avg=62.58, stdev=25.60, samples=195 iops : min= 1, max= 38, avg=15.04, stdev= 6.43, samples=195 lat (usec) : 250=4.62%, 500=0.33%, 750=0.03%, 1000=0.85% lat (msec) : 2=9.41%, 4=2.64%, 10=6.02%, 20=13.96%, 50=41.60% lat (msec) : 100=17.74%, 250=2.57%, 500=0.20%, 1000=0.03% cpu : usr=0.09%, sys=6.20%, ctx=6239, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=240KiB/s (245kB/s), 59.3KiB/s-60.9KiB/s (60.7kB/s-62.4kB/s), io=23.4MiB (24.6MB), run=98911-100157msec WRITE: bw=251KiB/s (257kB/s), 62.5KiB/s-63.4KiB/s (63.0kB/s-64.9kB/s), io=24.6MiB (25.7MB), run=98911-100157msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147524: Thu Oct 23 13:07:57 2025 read: IOPS=6, BW=24.6KiB/s (25.2kB/s)(5992KiB/243586msec) clat (msec): min=5, max=747, avg=34.97, stdev=34.15 lat (msec): min=5, max=747, avg=34.97, stdev=34.15 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 18], 40.00th=[ 22], 50.00th=[ 28], 60.00th=[ 34], | 70.00th=[ 41], 80.00th=[ 51], 90.00th=[ 65], 95.00th=[ 78], | 99.00th=[ 138], 99.50th=[ 199], 99.90th=[ 443], 99.95th=[ 751], | 99.99th=[ 751] bw ( KiB/s): min= 7, max= 95, per=27.25%, avg=26.70, stdev=16.14, samples=434 iops : min= 1, max= 23, avg= 6.06, stdev= 4.07, samples=434 write: IOPS=6, BW=25.8KiB/s (26.5kB/s)(6296KiB/243586msec); 0 zone resets clat (msec): min=42, max=622, avg=121.26, stdev=51.07 lat (msec): min=42, max=622, avg=121.26, stdev=51.07 clat percentiles (msec): | 1.00th=[ 55], 5.00th=[ 64], 10.00th=[ 71], 20.00th=[ 83], | 30.00th=[ 92], 40.00th=[ 101], 50.00th=[ 110], 60.00th=[ 121], | 70.00th=[ 133], 80.00th=[ 153], 90.00th=[ 188], 95.00th=[ 220], | 99.00th=[ 284], 99.50th=[ 326], 99.90th=[ 422], 99.95th=[ 625], | 99.99th=[ 625] bw ( KiB/s): min= 7, max= 48, per=24.69%, avg=25.43, stdev= 9.53, samples=478 iops : min= 1, max= 12, avg= 5.73, stdev= 2.45, samples=478 lat (msec) : 10=2.41%, 20=15.43%, 50=21.35%, 100=28.78%, 250=30.73% lat (msec) : 500=1.24%, 750=0.07% cpu : usr=0.05%, sys=1.43%, ctx=5689, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147525: Thu Oct 23 13:07:57 2025 read: IOPS=6, BW=24.4KiB/s (24.9kB/s)(5940KiB/243867msec) clat (msec): min=5, max=915, avg=33.47, stdev=36.50 lat (msec): min=5, max=915, avg=33.47, stdev=36.50 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 21], 50.00th=[ 25], 60.00th=[ 32], | 70.00th=[ 40], 80.00th=[ 47], 90.00th=[ 60], 95.00th=[ 77], | 99.00th=[ 132], 99.50th=[ 178], 99.90th=[ 439], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 7, max= 95, per=26.97%, avg=26.43, stdev=16.75, samples=435 iops : min= 1, max= 23, avg= 5.99, stdev= 4.24, samples=435 write: IOPS=6, BW=26.0KiB/s (26.7kB/s)(6348KiB/243867msec); 0 zone resets clat (msec): min=41, max=421, avg=122.14, stdev=51.96 lat (msec): min=41, max=421, avg=122.15, stdev=51.96 clat percentiles (msec): | 1.00th=[ 53], 5.00th=[ 65], 10.00th=[ 73], 20.00th=[ 84], | 30.00th=[ 92], 40.00th=[ 101], 50.00th=[ 110], 60.00th=[ 121], | 70.00th=[ 134], 80.00th=[ 153], 90.00th=[ 190], 95.00th=[ 226], | 99.00th=[ 309], 99.50th=[ 351], 99.90th=[ 405], 99.95th=[ 422], | 99.99th=[ 422] bw ( KiB/s): min= 7, max= 48, per=24.77%, avg=25.51, stdev= 9.55, samples=481 iops : min= 1, max= 12, avg= 5.75, stdev= 2.46, samples=481 lat (msec) : 10=2.70%, 20=16.44%, 50=20.96%, 100=27.47%, 250=30.66% lat (msec) : 500=1.73%, 1000=0.03% cpu : usr=0.04%, sys=1.45%, ctx=5767, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147526: Thu Oct 23 13:07:57 2025 read: IOPS=6, BW=24.7KiB/s (25.3kB/s)(6028KiB/243794msec) clat (msec): min=4, max=439, avg=34.89, stdev=30.59 lat (msec): min=4, max=439, avg=34.89, stdev=30.59 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 22], 50.00th=[ 27], 60.00th=[ 34], | 70.00th=[ 42], 80.00th=[ 52], 90.00th=[ 64], 95.00th=[ 79], | 99.00th=[ 138], 99.50th=[ 174], 99.90th=[ 439], 99.95th=[ 439], | 99.99th=[ 439] bw ( KiB/s): min= 7, max= 104, per=26.94%, avg=26.40, stdev=16.96, samples=442 iops : min= 1, max= 26, avg= 5.98, stdev= 4.28, samples=442 write: IOPS=6, BW=25.7KiB/s (26.3kB/s)(6260KiB/243794msec); 0 zone resets clat (msec): min=41, max=935, avg=121.97, stdev=55.55 lat (msec): min=41, max=935, avg=121.97, stdev=55.55 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 66], 10.00th=[ 72], 20.00th=[ 81], | 30.00th=[ 90], 40.00th=[ 100], 50.00th=[ 109], 60.00th=[ 121], | 70.00th=[ 136], 80.00th=[ 155], 90.00th=[ 190], 95.00th=[ 226], | 99.00th=[ 296], 99.50th=[ 334], 99.90th=[ 625], 99.95th=[ 936], | 99.99th=[ 936] bw ( KiB/s): min= 7, max= 48, per=24.57%, avg=25.31, stdev= 9.62, samples=478 iops : min= 1, max= 12, avg= 5.70, stdev= 2.48, samples=478 lat (msec) : 10=2.51%, 20=16.18%, 50=20.35%, 100=29.49%, 250=30.11% lat (msec) : 500=1.30%, 750=0.03%, 1000=0.03% cpu : usr=0.04%, sys=1.45%, ctx=5854, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147527: Thu Oct 23 13:07:57 2025 read: IOPS=6, BW=24.0KiB/s (25.6kB/s)(6052KiB/242487msec) clat (msec): min=4, max=252, avg=32.97, stdev=25.59 lat (msec): min=4, max=252, avg=32.97, stdev=25.59 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 21], 50.00th=[ 25], 60.00th=[ 32], | 70.00th=[ 40], 80.00th=[ 49], 90.00th=[ 62], 95.00th=[ 77], | 99.00th=[ 131], 99.50th=[ 165], 99.90th=[ 228], 99.95th=[ 253], | 99.99th=[ 253] bw ( KiB/s): min= 7, max= 112, per=27.05%, avg=26.51, stdev=16.64, samples=442 iops : min= 1, max= 28, avg= 6.01, stdev= 4.22, samples=442 write: IOPS=6, BW=25.7KiB/s (26.3kB/s)(6236KiB/242487msec); 0 zone resets clat (msec): min=36, max=935, avg=123.34, stdev=57.73 lat (msec): min=36, max=935, avg=123.34, stdev=57.73 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 63], 10.00th=[ 70], 20.00th=[ 82], | 30.00th=[ 91], 40.00th=[ 101], 50.00th=[ 112], 60.00th=[ 123], | 70.00th=[ 136], 80.00th=[ 157], 90.00th=[ 194], 95.00th=[ 226], | 99.00th=[ 305], 99.50th=[ 334], 99.90th=[ 735], 99.95th=[ 936], | 99.99th=[ 936] bw ( KiB/s): min= 7, max= 55, per=24.62%, avg=25.36, stdev= 9.62, samples=475 iops : min= 1, max= 13, avg= 5.71, stdev= 2.46, samples=475 lat (msec) : 10=2.99%, 20=16.50%, 50=20.83%, 100=27.64%, 250=30.34% lat (msec) : 500=1.63%, 750=0.03%, 1000=0.03% cpu : usr=0.04%, sys=1.44%, ctx=5704, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=98.5KiB/s (101kB/s), 24.4KiB/s-24.0KiB/s (24.9kB/s-25.6kB/s), io=23.4MiB (24.6MB), run=242487-243867msec WRITE: bw=103KiB/s (106kB/s), 25.7KiB/s-26.0KiB/s (26.3kB/s-26.7kB/s), io=24.6MiB (25.7MB), run=242487-243867msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147582: Thu Oct 23 13:08:28 2025 read: IOPS=17, BW=274KiB/s (281kB/s)(5920KiB/21567msec) clat (usec): min=109, max=160986, avg=29103.00, stdev=25398.35 lat (usec): min=109, max=160986, avg=29104.38, stdev=25398.30 clat percentiles (usec): | 1.00th=[ 116], 5.00th=[ 147], 10.00th=[ 167], 20.00th=[ 281], | 30.00th=[ 15008], 40.00th=[ 22676], 50.00th=[ 27395], 60.00th=[ 31065], | 70.00th=[ 38011], 80.00th=[ 45351], 90.00th=[ 61604], 95.00th=[ 76022], | 99.00th=[104334], 99.50th=[139461], 99.90th=[160433], 99.95th=[160433], | 99.99th=[160433] bw ( KiB/s): min= 61, max= 479, per=25.27%, avg=272.90, stdev=97.92, samples=42 iops : min= 3, max= 29, avg=16.33, stdev= 6.18, samples=42 write: IOPS=18, BW=295KiB/s (302kB/s)(6368KiB/21567msec); 0 zone resets clat (usec): min=1304, max=231512, avg=26664.04, stdev=31732.60 lat (usec): min=1305, max=231513, avg=26666.24, stdev=31732.57 clat percentiles (usec): | 1.00th=[ 1483], 5.00th=[ 1647], 10.00th=[ 1811], 20.00th=[ 2966], | 30.00th=[ 5407], 40.00th=[ 9241], 50.00th=[ 15139], 60.00th=[ 22414], | 70.00th=[ 32375], 80.00th=[ 44827], 90.00th=[ 66847], 95.00th=[ 88605], | 99.00th=[158335], 99.50th=[202376], 99.90th=[231736], 99.95th=[231736], | 99.99th=[231736] bw ( KiB/s): min= 92, max= 574, per=25.89%, avg=294.33, stdev=118.25, samples=42 iops : min= 5, max= 35, avg=17.71, stdev= 7.48, samples=42 lat (usec) : 250=8.46%, 500=3.26%, 750=0.13%, 1000=0.13% lat (msec) : 2=7.16%, 4=5.86%, 10=9.24%, 20=12.24%, 50=36.72% lat (msec) : 100=14.45%, 250=2.34% cpu : usr=0.12%, sys=8.77%, ctx=1442, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147583: Thu Oct 23 13:08:28 2025 read: IOPS=16, BW=262KiB/s (269kB/s)(5744KiB/21902msec) clat (usec): min=107, max=412862, avg=30871.83, stdev=33286.19 lat (usec): min=107, max=412863, avg=30884.32, stdev=33283.78 clat percentiles (usec): | 1.00th=[ 118], 5.00th=[ 155], 10.00th=[ 172], 20.00th=[ 281], | 30.00th=[ 15926], 40.00th=[ 20841], 50.00th=[ 25560], 60.00th=[ 30278], | 70.00th=[ 39060], 80.00th=[ 49546], 90.00th=[ 65799], 95.00th=[ 86508], | 99.00th=[117965], 99.50th=[137364], 99.90th=[413139], 99.95th=[413139], | 99.99th=[413139] bw ( KiB/s): min= 31, max= 508, per=24.32%, avg=262.63, stdev=107.76, samples=43 iops : min= 1, max= 31, avg=15.70, stdev= 6.81, samples=43 write: IOPS=18, BW=299KiB/s (306kB/s)(6544KiB/21902msec); 0 zone resets clat (usec): min=1307, max=151303, avg=26235.41, stdev=29609.92 lat (usec): min=1308, max=151304, avg=26241.16, stdev=29607.55 clat percentiles (usec): | 1.00th=[ 1401], 5.00th=[ 1565], 10.00th=[ 1729], 20.00th=[ 2999], | 30.00th=[ 6194], 40.00th=[ 9503], 50.00th=[ 15664], 60.00th=[ 21890], | 70.00th=[ 31327], 80.00th=[ 44303], 90.00th=[ 69731], 95.00th=[ 89654], | 99.00th=[137364], 99.50th=[143655], 99.90th=[152044], 99.95th=[152044], | 99.99th=[152044] bw ( KiB/s): min= 61, max= 474, per=26.22%, avg=298.07, stdev=78.41, samples=43 iops : min= 3, max= 29, avg=17.93, stdev= 4.94, samples=43 lat (usec) : 250=8.72%, 500=1.95%, 750=0.26% lat (msec) : 2=8.20%, 4=4.43%, 10=9.77%, 20=14.84%, 50=34.24% lat (msec) : 100=15.23%, 250=2.21%, 500=0.13% cpu : usr=0.09%, sys=8.68%, ctx=1416, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147584: Thu Oct 23 13:08:28 2025 read: IOPS=17, BW=282KiB/s (289kB/s)(6256KiB/22161msec) clat (usec): min=96, max=429814, avg=30022.18, stdev=31131.72 lat (usec): min=97, max=429815, avg=30023.58, stdev=31131.72 clat percentiles (usec): | 1.00th=[ 115], 5.00th=[ 149], 10.00th=[ 167], 20.00th=[ 260], | 30.00th=[ 15533], 40.00th=[ 21890], 50.00th=[ 27132], 60.00th=[ 33424], | 70.00th=[ 39060], 80.00th=[ 46400], 90.00th=[ 62129], 95.00th=[ 77071], | 99.00th=[ 99091], 99.50th=[105382], 99.90th=[429917], 99.95th=[429917], | 99.99th=[429917] bw ( KiB/s): min= 123, max= 508, per=26.28%, avg=283.81, stdev=98.67, samples=43 iops : min= 7, max= 31, avg=17.02, stdev= 6.23, samples=43 write: IOPS=17, BW=272KiB/s (279kB/s)(6032KiB/22161msec); 0 zone resets clat (usec): min=1335, max=201643, avg=27135.06, stdev=30076.61 lat (usec): min=1336, max=201646, avg=27139.25, stdev=30075.48 clat percentiles (usec): | 1.00th=[ 1467], 5.00th=[ 1647], 10.00th=[ 1795], 20.00th=[ 2671], | 30.00th=[ 6063], 40.00th=[ 10159], 50.00th=[ 15533], 60.00th=[ 23462], | 70.00th=[ 34341], 80.00th=[ 47973], 90.00th=[ 65274], 95.00th=[ 94897], | 99.00th=[119014], 99.50th=[166724], 99.90th=[202376], 99.95th=[202376], | 99.99th=[202376] bw ( KiB/s): min= 94, max= 510, per=24.22%, avg=275.43, stdev=103.28, samples=42 iops : min= 5, max= 31, avg=16.48, stdev= 6.47, samples=42 lat (usec) : 100=0.26%, 250=9.51%, 500=1.43%, 1000=0.39% lat (msec) : 2=7.55%, 4=4.95%, 10=8.59%, 20=12.89%, 50=36.72% lat (msec) : 100=15.49%, 250=2.08%, 500=0.13% cpu : usr=0.08%, sys=8.37%, ctx=1449, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147585: Thu Oct 23 13:08:28 2025 read: IOPS=17, BW=280KiB/s (287kB/s)(6032KiB/21516msec) clat (usec): min=93, max=494338, avg=29384.83, stdev=34072.93 lat (usec): min=94, max=494340, avg=29386.12, stdev=34072.95 clat percentiles (usec): | 1.00th=[ 122], 5.00th=[ 151], 10.00th=[ 169], 20.00th=[ 277], | 30.00th=[ 14091], 40.00th=[ 19530], 50.00th=[ 25560], 60.00th=[ 32900], | 70.00th=[ 39584], 80.00th=[ 48497], 90.00th=[ 61080], 95.00th=[ 73925], | 99.00th=[106431], 99.50th=[119014], 99.90th=[492831], 99.95th=[492831], | 99.99th=[492831] bw ( KiB/s): min= 61, max= 537, per=25.74%, avg=277.95, stdev=100.60, samples=42 iops : min= 3, max= 33, avg=16.67, stdev= 6.37, samples=42 write: IOPS=18, BW=291KiB/s (298kB/s)(6256KiB/21516msec); 0 zone resets clat (usec): min=1308, max=259649, avg=26430.06, stdev=31271.19 lat (usec): min=1310, max=259651, avg=26432.47, stdev=31271.26 clat percentiles (usec): | 1.00th=[ 1369], 5.00th=[ 1582], 10.00th=[ 1778], 20.00th=[ 2900], | 30.00th=[ 5211], 40.00th=[ 9110], 50.00th=[ 15008], 60.00th=[ 22938], | 70.00th=[ 33424], 80.00th=[ 46400], 90.00th=[ 65274], 95.00th=[ 81265], | 99.00th=[156238], 99.50th=[187696], 99.90th=[258999], 99.95th=[258999], | 99.99th=[258999] bw ( KiB/s): min= 30, max= 627, per=25.56%, avg=290.64, stdev=123.01, samples=42 iops : min= 1, max= 39, avg=17.45, stdev= 7.74, samples=42 lat (usec) : 100=0.13%, 250=9.38%, 500=2.47%, 750=0.26%, 1000=0.52% lat (msec) : 2=7.03%, 4=7.03%, 10=8.07%, 20=14.45%, 50=32.03% lat (msec) : 100=17.19%, 250=1.17%, 500=0.26% cpu : usr=0.12%, sys=8.61%, ctx=1401, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1081KiB/s (1107kB/s), 262KiB/s-282KiB/s (269kB/s-289kB/s), io=23.4MiB (24.5MB), run=21516-22161msec WRITE: bw=1137KiB/s (1164kB/s), 272KiB/s-299KiB/s (279kB/s-306kB/s), io=24.6MiB (25.8MB), run=21516-22161msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147587: Thu Oct 23 13:09:08 2025 read: IOPS=5, BW=95.9KiB/s (98.2kB/s)(5920KiB/61751msec) clat (msec): min=6, max=170, avg=29.40, stdev=26.07 lat (msec): min=6, max=170, avg=29.40, stdev=26.07 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 29], 80.00th=[ 38], 90.00th=[ 61], 95.00th=[ 88], | 99.00th=[ 127], 99.50th=[ 169], 99.90th=[ 171], 99.95th=[ 171], | 99.99th=[ 171] bw ( KiB/s): min= 28, max= 384, per=26.99%, avg=103.92, stdev=64.24, samples=110 iops : min= 1, max= 24, avg= 5.72, stdev= 4.08, samples=110 write: IOPS=6, BW=103KiB/s (106kB/s)(6368KiB/61751msec); 0 zone resets clat (msec): min=30, max=499, avg=127.67, stdev=56.00 lat (msec): min=30, max=499, avg=127.68, stdev=56.00 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 63], 10.00th=[ 74], 20.00th=[ 85], | 30.00th=[ 94], 40.00th=[ 107], 50.00th=[ 117], 60.00th=[ 131], | 70.00th=[ 142], 80.00th=[ 159], 90.00th=[ 190], 95.00th=[ 236], | 99.00th=[ 347], 99.50th=[ 397], 99.90th=[ 502], 99.95th=[ 502], | 99.99th=[ 502] bw ( KiB/s): min= 31, max= 192, per=25.42%, avg=102.97, stdev=38.61, samples=120 iops : min= 1, max= 12, avg= 5.67, stdev= 2.49, samples=120 lat (msec) : 10=2.60%, 20=22.66%, 50=16.93%, 100=22.27%, 250=34.11% lat (msec) : 500=1.43% cpu : usr=0.04%, sys=1.57%, ctx=1387, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147588: Thu Oct 23 13:09:08 2025 read: IOPS=5, BW=92.5KiB/s (94.7kB/s)(5744KiB/62081msec) clat (msec): min=6, max=250, avg=29.60, stdev=29.35 lat (msec): min=6, max=250, avg=29.60, stdev=29.35 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 36], 90.00th=[ 64], 95.00th=[ 93], | 99.00th=[ 138], 99.50th=[ 224], 99.90th=[ 251], 99.95th=[ 251], | 99.99th=[ 251] bw ( KiB/s): min= 25, max= 256, per=26.23%, avg=101.00, stdev=61.68, samples=111 iops : min= 1, max= 16, avg= 5.52, stdev= 3.91, samples=111 write: IOPS=6, BW=105KiB/s (108kB/s)(6544KiB/62081msec); 0 zone resets clat (msec): min=43, max=326, avg=125.68, stdev=49.60 lat (msec): min=43, max=326, avg=125.69, stdev=49.60 clat percentiles (msec): | 1.00th=[ 57], 5.00th=[ 67], 10.00th=[ 71], 20.00th=[ 86], | 30.00th=[ 96], 40.00th=[ 107], 50.00th=[ 115], 60.00th=[ 129], | 70.00th=[ 142], 80.00th=[ 155], 90.00th=[ 192], 95.00th=[ 232], | 99.00th=[ 284], 99.50th=[ 305], 99.90th=[ 326], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 31, max= 192, per=25.90%, avg=104.89, stdev=40.87, samples=122 iops : min= 1, max= 12, avg= 5.79, stdev= 2.61, samples=122 lat (msec) : 10=3.91%, 20=20.05%, 50=16.54%, 100=22.27%, 250=35.42% lat (msec) : 500=1.82% cpu : usr=0.04%, sys=1.44%, ctx=1403, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147589: Thu Oct 23 13:09:08 2025 read: IOPS=6, BW=104KiB/s (107kB/s)(6256KiB/60151msec) clat (msec): min=7, max=181, avg=27.71, stdev=25.03 lat (msec): min=7, max=181, avg=27.71, stdev=25.03 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 20], 60.00th=[ 22], | 70.00th=[ 27], 80.00th=[ 35], 90.00th=[ 57], 95.00th=[ 80], | 99.00th=[ 129], 99.50th=[ 165], 99.90th=[ 182], 99.95th=[ 182], | 99.99th=[ 182] bw ( KiB/s): min= 28, max= 351, per=29.14%, avg=112.17, stdev=72.96, samples=110 iops : min= 1, max= 21, avg= 6.25, stdev= 4.62, samples=110 write: IOPS=6, BW=100KiB/s (103kB/s)(6032KiB/60151msec); 0 zone resets clat (msec): min=36, max=504, avg=130.70, stdev=55.60 lat (msec): min=36, max=504, avg=130.71, stdev=55.60 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 69], 10.00th=[ 79], 20.00th=[ 90], | 30.00th=[ 99], 40.00th=[ 108], 50.00th=[ 120], 60.00th=[ 132], | 70.00th=[ 144], 80.00th=[ 165], 90.00th=[ 199], 95.00th=[ 234], | 99.00th=[ 330], 99.50th=[ 451], 99.90th=[ 506], 99.95th=[ 506], | 99.99th=[ 506] bw ( KiB/s): min= 30, max= 192, per=24.88%, avg=100.76, stdev=35.31, samples=117 iops : min= 1, max= 12, avg= 5.51, stdev= 2.28, samples=117 lat (msec) : 10=3.52%, 20=23.44%, 50=17.97%, 100=19.92%, 250=33.33% lat (msec) : 500=1.69%, 750=0.13% cpu : usr=0.04%, sys=1.50%, ctx=1435, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147590: Thu Oct 23 13:09:08 2025 read: IOPS=6, BW=99.3KiB/s (102kB/s)(6032KiB/60749msec) clat (msec): min=7, max=257, avg=28.83, stdev=27.92 lat (msec): min=7, max=257, avg=28.84, stdev=27.92 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 17], 50.00th=[ 19], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 36], 90.00th=[ 64], 95.00th=[ 89], | 99.00th=[ 130], 99.50th=[ 228], 99.90th=[ 257], 99.95th=[ 257], | 99.99th=[ 257] bw ( KiB/s): min= 29, max= 320, per=28.43%, avg=109.44, stdev=71.03, samples=107 iops : min= 1, max= 20, avg= 6.07, stdev= 4.51, samples=107 write: IOPS=6, BW=103KiB/s (105kB/s)(6256KiB/60749msec); 0 zone resets clat (msec): min=49, max=357, avg=127.40, stdev=50.74 lat (msec): min=49, max=357, avg=127.41, stdev=50.75 clat percentiles (msec): | 1.00th=[ 58], 5.00th=[ 66], 10.00th=[ 75], 20.00th=[ 87], | 30.00th=[ 96], 40.00th=[ 107], 50.00th=[ 120], 60.00th=[ 132], | 70.00th=[ 146], 80.00th=[ 159], 90.00th=[ 186], 95.00th=[ 232], | 99.00th=[ 300], 99.50th=[ 347], 99.90th=[ 359], 99.95th=[ 359], | 99.99th=[ 359] bw ( KiB/s): min= 31, max= 191, per=25.30%, avg=102.45, stdev=38.41, samples=119 iops : min= 1, max= 11, avg= 5.63, stdev= 2.45, samples=119 lat (msec) : 10=2.21%, 20=24.35%, 50=16.41%, 100=22.40%, 250=32.94% lat (msec) : 500=1.69% cpu : usr=0.04%, sys=1.46%, ctx=1365, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=386KiB/s (395kB/s), 92.5KiB/s-104KiB/s (94.7kB/s-107kB/s), io=23.4MiB (24.5MB), run=60151-62081msec WRITE: bw=406KiB/s (416kB/s), 100KiB/s-105KiB/s (103kB/s-108kB/s), io=24.6MiB (25.8MB), run=60151-62081msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147607: Thu Oct 23 13:09:19 2025 read: IOPS=2, BW=2448KiB/s (2507kB/s)(7168KiB/2928msec) clat (usec): min=1505, max=445476, avg=198541.44, stdev=133328.76 lat (usec): min=1508, max=445478, avg=198543.71, stdev=133328.49 clat percentiles (usec): | 1.00th=[ 1500], 5.00th=[ 1500], 10.00th=[ 1500], 20.00th=[128451], | 30.00th=[185598], 40.00th=[185598], 50.00th=[189793], 60.00th=[198181], | 70.00th=[198181], 80.00th=[240124], 90.00th=[446694], 95.00th=[446694], | 99.00th=[446694], 99.50th=[446694], 99.90th=[446694], 99.95th=[446694], | 99.99th=[446694] bw ( KiB/s): min= 1838, max= 4010, per=44.97%, avg=2465.75, stdev=1032.64, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=1, BW=1749KiB/s (1791kB/s)(5120KiB/2928msec); 0 zone resets clat (msec): min=174, max=450, avg=296.21, stdev=112.00 lat (msec): min=175, max=450, avg=296.30, stdev=111.96 clat percentiles (msec): | 1.00th=[ 176], 5.00th=[ 176], 10.00th=[ 176], 20.00th=[ 176], | 30.00th=[ 203], 40.00th=[ 203], 50.00th=[ 309], 60.00th=[ 309], | 70.00th=[ 347], 80.00th=[ 347], 90.00th=[ 451], 95.00th=[ 451], | 99.00th=[ 451], 99.50th=[ 451], 99.90th=[ 451], 99.95th=[ 451], | 99.99th=[ 451] bw ( KiB/s): min= 1336, max= 6144, per=48.72%, avg=3740.00, stdev=3399.77, samples=2 iops : min= 1, max= 6, avg= 3.50, stdev= 3.54, samples=2 lat (msec) : 2=8.33%, 250=58.33%, 500=33.33% cpu : usr=0.03%, sys=22.00%, ctx=215, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147608: Thu Oct 23 13:09:19 2025 read: IOPS=0, BW=548KiB/s (561kB/s)(2048KiB/3735msec) clat (msec): min=173, max=201, avg=187.35, stdev=19.72 lat (msec): min=173, max=201, avg=187.35, stdev=19.72 clat percentiles (msec): | 1.00th=[ 174], 5.00th=[ 174], 10.00th=[ 174], 20.00th=[ 174], | 30.00th=[ 174], 40.00th=[ 174], 50.00th=[ 174], 60.00th=[ 201], | 70.00th=[ 201], 80.00th=[ 201], 90.00th=[ 201], 95.00th=[ 201], | 99.00th=[ 201], 99.50th=[ 201], 99.90th=[ 201], 99.95th=[ 201], | 99.99th=[ 201] bw ( KiB/s): min= 1434, max= 2048, per=31.75%, avg=1741.00, stdev=434.16, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2742KiB/s (2807kB/s)(10.0MiB/3735msec); 0 zone resets clat (msec): min=150, max=543, avg=320.87, stdev=128.46 lat (msec): min=151, max=543, avg=321.00, stdev=128.45 clat percentiles (msec): | 1.00th=[ 150], 5.00th=[ 150], 10.00th=[ 150], 20.00th=[ 155], | 30.00th=[ 234], 40.00th=[ 300], 50.00th=[ 313], 60.00th=[ 321], | 70.00th=[ 334], 80.00th=[ 363], 90.00th=[ 498], 95.00th=[ 542], | 99.00th=[ 542], 99.50th=[ 542], 99.90th=[ 542], 99.95th=[ 542], | 99.99th=[ 542] bw ( KiB/s): min= 1838, max= 4039, per=36.35%, avg=2790.40, stdev=1121.57, samples=5 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=5 lat (msec) : 250=41.67%, 500=50.00%, 750=8.33% cpu : usr=0.08%, sys=22.07%, ctx=209, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147609: Thu Oct 23 13:09:19 2025 read: IOPS=1, BW=2008KiB/s (2057kB/s)(6144KiB/3059msec) clat (usec): min=452, max=252994, avg=120554.98, stdev=130958.94 lat (usec): min=452, max=252995, avg=120556.23, stdev=130959.08 clat percentiles (usec): | 1.00th=[ 453], 5.00th=[ 453], 10.00th=[ 453], 20.00th=[ 717], | 30.00th=[ 717], 40.00th=[ 2507], 50.00th=[ 2507], 60.00th=[227541], | 70.00th=[238027], 80.00th=[238027], 90.00th=[252707], 95.00th=[252707], | 99.00th=[252707], 99.50th=[252707], 99.90th=[252707], 99.95th=[252707], | 99.99th=[252707] bw ( KiB/s): min= 1838, max= 5988, per=60.03%, avg=3291.33, stdev=2337.74, samples=3 iops : min= 1, max= 5, avg= 2.67, stdev= 2.08, samples=3 write: IOPS=1, BW=2008KiB/s (2057kB/s)(6144KiB/3059msec); 0 zone resets clat (msec): min=247, max=653, avg=370.57, stdev=154.03 lat (msec): min=247, max=653, avg=370.67, stdev=154.01 clat percentiles (msec): | 1.00th=[ 247], 5.00th=[ 247], 10.00th=[ 247], 20.00th=[ 251], | 30.00th=[ 251], 40.00th=[ 279], 50.00th=[ 279], 60.00th=[ 393], | 70.00th=[ 401], 80.00th=[ 401], 90.00th=[ 651], 95.00th=[ 651], | 99.00th=[ 651], 99.50th=[ 651], 99.90th=[ 651], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 1395, max= 3992, per=30.79%, avg=2363.50, stdev=1126.68, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (usec) : 500=8.33%, 750=8.33% lat (msec) : 4=8.33%, 250=25.00%, 500=41.67%, 750=8.33% cpu : usr=0.07%, sys=23.68%, ctx=170, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147610: Thu Oct 23 13:09:19 2025 read: IOPS=1, BW=1383KiB/s (1416kB/s)(5120KiB/3703msec) clat (usec): min=481, max=304133, avg=62155.88, stdev=135282.99 lat (usec): min=481, max=304134, avg=62156.93, stdev=135283.20 clat percentiles (usec): | 1.00th=[ 482], 5.00th=[ 482], 10.00th=[ 482], 20.00th=[ 482], | 30.00th=[ 490], 40.00th=[ 490], 50.00th=[ 693], 60.00th=[ 693], | 70.00th=[ 5014], 80.00th=[ 5014], 90.00th=[304088], 95.00th=[304088], | 99.00th=[304088], 99.50th=[304088], 99.90th=[304088], 99.95th=[304088], | 99.99th=[304088] bw ( KiB/s): min= 7353, max= 7353, per=100.00%, avg=7353.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=1, BW=1936KiB/s (1982kB/s)(7168KiB/3703msec); 0 zone resets clat (msec): min=168, max=894, avg=446.17, stdev=247.22 lat (msec): min=168, max=894, avg=446.28, stdev=247.20 clat percentiles (msec): | 1.00th=[ 169], 5.00th=[ 169], 10.00th=[ 169], 20.00th=[ 192], | 30.00th=[ 347], 40.00th=[ 347], 50.00th=[ 481], 60.00th=[ 502], | 70.00th=[ 502], 80.00th=[ 542], 90.00th=[ 894], 95.00th=[ 894], | 99.00th=[ 894], 99.50th=[ 894], 99.90th=[ 894], 99.95th=[ 894], | 99.99th=[ 894] bw ( KiB/s): min= 1838, max= 4071, per=31.19%, avg=2394.40, stdev=940.77, samples=5 iops : min= 1, max= 3, avg= 1.60, stdev= 0.89, samples=5 lat (usec) : 500=16.67%, 750=8.33% lat (msec) : 10=8.33%, 250=16.67%, 500=33.33%, 750=8.33%, 1000=8.33% cpu : usr=0.05%, sys=17.48%, ctx=224, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5483KiB/s (5615kB/s), 548KiB/s-2448KiB/s (561kB/s-2507kB/s), io=20.0MiB (20.0MB), run=2928-3735msec WRITE: bw=7677KiB/s (7861kB/s), 1749KiB/s-2742KiB/s (1791kB/s-2807kB/s), io=28.0MiB (29.4MB), run=2928-3735msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147611: Thu Oct 23 13:09:20 2025 read: IOPS=2, BW=2119KiB/s (2170kB/s)(7168KiB/3382msec) clat (msec): min=88, max=273, avg=193.97, stdev=69.85 lat (msec): min=88, max=273, avg=193.97, stdev=69.85 clat percentiles (msec): | 1.00th=[ 89], 5.00th=[ 89], 10.00th=[ 89], 20.00th=[ 114], | 30.00th=[ 192], 40.00th=[ 192], 50.00th=[ 213], 60.00th=[ 218], | 70.00th=[ 218], 80.00th=[ 262], 90.00th=[ 275], 95.00th=[ 275], | 99.00th=[ 275], 99.50th=[ 275], 99.90th=[ 275], 99.95th=[ 275], | 99.99th=[ 275] bw ( KiB/s): min= 2000, max= 4047, per=52.67%, avg=2745.75, stdev=969.77, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 write: IOPS=1, BW=1514KiB/s (1550kB/s)(5120KiB/3382msec); 0 zone resets clat (msec): min=249, max=722, avg=363.05, stdev=201.35 lat (msec): min=249, max=722, avg=363.17, stdev=201.35 clat percentiles (msec): | 1.00th=[ 249], 5.00th=[ 249], 10.00th=[ 249], 20.00th=[ 249], | 30.00th=[ 271], 40.00th=[ 271], 50.00th=[ 279], 60.00th=[ 279], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 726], 95.00th=[ 726], | 99.00th=[ 726], 99.50th=[ 726], 99.90th=[ 726], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 2000, max= 3799, per=35.69%, avg=2604.67, stdev=1034.35, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 100=8.33%, 250=41.67%, 500=41.67%, 750=8.33% cpu : usr=0.00%, sys=9.23%, ctx=111, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147612: Thu Oct 23 13:09:20 2025 read: IOPS=0, BW=521KiB/s (534kB/s)(2048KiB/3928msec) clat (msec): min=133, max=206, avg=169.69, stdev=51.55 lat (msec): min=133, max=206, avg=169.69, stdev=51.55 clat percentiles (msec): | 1.00th=[ 134], 5.00th=[ 134], 10.00th=[ 134], 20.00th=[ 134], | 30.00th=[ 134], 40.00th=[ 134], 50.00th=[ 134], 60.00th=[ 207], | 70.00th=[ 207], 80.00th=[ 207], 90.00th=[ 207], 95.00th=[ 207], | 99.00th=[ 207], 99.50th=[ 207], 99.90th=[ 207], 99.95th=[ 207], | 99.99th=[ 207] bw ( KiB/s): min= 1473, max= 1899, per=32.34%, avg=1686.00, stdev=301.23, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2607KiB/s (2669kB/s)(10.0MiB/3928msec); 0 zone resets clat (msec): min=122, max=768, avg=340.50, stdev=186.58 lat (msec): min=122, max=769, avg=340.69, stdev=186.54 clat percentiles (msec): | 1.00th=[ 123], 5.00th=[ 123], 10.00th=[ 123], 20.00th=[ 134], | 30.00th=[ 236], 40.00th=[ 292], 50.00th=[ 300], 60.00th=[ 326], | 70.00th=[ 326], 80.00th=[ 435], 90.00th=[ 468], 95.00th=[ 768], | 99.00th=[ 768], 99.50th=[ 768], 99.90th=[ 768], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 1899, max= 4031, per=36.42%, avg=2658.17, stdev=1061.12, samples=6 iops : min= 1, max= 3, avg= 1.67, stdev= 1.03, samples=6 lat (msec) : 250=41.67%, 500=50.00%, 1000=8.33% cpu : usr=0.00%, sys=9.40%, ctx=126, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147613: Thu Oct 23 13:09:20 2025 read: IOPS=1, BW=1714KiB/s (1755kB/s)(6144KiB/3584msec) clat (msec): min=85, max=393, avg=226.95, stdev=130.60 lat (msec): min=85, max=393, avg=226.95, stdev=130.60 clat percentiles (msec): | 1.00th=[ 86], 5.00th=[ 86], 10.00th=[ 86], 20.00th=[ 107], | 30.00th=[ 107], 40.00th=[ 180], 50.00th=[ 180], 60.00th=[ 226], | 70.00th=[ 372], 80.00th=[ 372], 90.00th=[ 393], 95.00th=[ 393], | 99.00th=[ 393], 99.50th=[ 393], 99.90th=[ 393], 99.95th=[ 393], | 99.99th=[ 393] bw ( KiB/s): min= 2000, max= 3799, per=47.16%, avg=2458.25, stdev=893.88, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 write: IOPS=1, BW=1714KiB/s (1755kB/s)(6144KiB/3584msec); 0 zone resets clat (msec): min=160, max=559, avg=361.41, stdev=153.70 lat (msec): min=161, max=559, avg=361.54, stdev=153.70 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 292], | 30.00th=[ 292], 40.00th=[ 296], 50.00th=[ 296], 60.00th=[ 330], | 70.00th=[ 531], 80.00th=[ 531], 90.00th=[ 558], 95.00th=[ 558], | 99.00th=[ 558], 99.50th=[ 558], 99.90th=[ 558], 99.95th=[ 558], | 99.99th=[ 558] bw ( KiB/s): min= 2000, max= 2775, per=30.14%, avg=2200.25, stdev=383.22, samples=4 iops : min= 1, max= 2, avg= 1.25, stdev= 0.50, samples=4 lat (msec) : 100=8.33%, 250=33.33%, 500=41.67%, 750=16.67% cpu : usr=0.00%, sys=6.22%, ctx=85, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147614: Thu Oct 23 13:09:20 2025 read: IOPS=1, BW=1435KiB/s (1470kB/s)(5120KiB/3567msec) clat (msec): min=118, max=460, avg=268.94, stdev=163.78 lat (msec): min=118, max=460, avg=268.94, stdev=163.78 clat percentiles (msec): | 1.00th=[ 120], 5.00th=[ 120], 10.00th=[ 120], 20.00th=[ 120], | 30.00th=[ 136], 40.00th=[ 136], 50.00th=[ 201], 60.00th=[ 201], | 70.00th=[ 430], 80.00th=[ 430], 90.00th=[ 460], 95.00th=[ 460], | 99.00th=[ 460], 99.50th=[ 460], 99.90th=[ 460], 99.95th=[ 460], | 99.99th=[ 460] bw ( KiB/s): min= 1899, max= 6047, per=76.21%, avg=3973.00, stdev=2933.08, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=1, BW=2010KiB/s (2058kB/s)(7168KiB/3567msec); 0 zone resets clat (msec): min=130, max=452, avg=302.17, stdev=120.51 lat (msec): min=130, max=452, avg=302.85, stdev=120.77 clat percentiles (msec): | 1.00th=[ 131], 5.00th=[ 131], 10.00th=[ 131], 20.00th=[ 174], | 30.00th=[ 288], 40.00th=[ 288], 50.00th=[ 300], 60.00th=[ 342], | 70.00th=[ 342], 80.00th=[ 430], 90.00th=[ 451], 95.00th=[ 451], | 99.00th=[ 451], 99.50th=[ 451], 99.90th=[ 451], 99.95th=[ 451], | 99.99th=[ 451] bw ( KiB/s): min= 1416, max= 4023, per=36.61%, avg=2672.20, stdev=1243.49, samples=5 iops : min= 1, max= 3, avg= 1.80, stdev= 1.10, samples=5 lat (msec) : 250=41.67%, 500=58.33% cpu : usr=0.06%, sys=5.55%, ctx=84, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5214KiB/s (5339kB/s), 521KiB/s-2119KiB/s (534kB/s-2170kB/s), io=20.0MiB (20.0MB), run=3382-3928msec WRITE: bw=7299KiB/s (7475kB/s), 1514KiB/s-2607KiB/s (1550kB/s-2669kB/s), io=28.0MiB (29.4MB), run=3382-3928msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147620: Thu Oct 23 13:09:31 2025 read: IOPS=1, BW=4741KiB/s (4855kB/s)(12.0MiB/2592msec) clat (msec): min=193, max=1322, avg=834.25, stdev=579.56 lat (msec): min=193, max=1322, avg=834.25, stdev=579.56 clat percentiles (msec): | 1.00th=[ 194], 5.00th=[ 194], 10.00th=[ 194], 20.00th=[ 194], | 30.00th=[ 194], 40.00th=[ 986], 50.00th=[ 986], 60.00th=[ 986], | 70.00th=[ 1318], 80.00th=[ 1318], 90.00th=[ 1318], 95.00th=[ 1318], | 99.00th=[ 1318], 99.50th=[ 1318], 99.90th=[ 1318], 99.95th=[ 1318], | 99.99th=[ 1318] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=5.33%, ctx=44, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147621: Thu Oct 23 13:09:31 2025 read: IOPS=0, BW=1459KiB/s (1494kB/s)(4096KiB/2808msec) clat (nsec): min=178703k, max=178703k, avg=178702529.00, stdev= 0.00 lat (nsec): min=178705k, max=178705k, avg=178705441.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 180], 5.00th=[ 180], 10.00th=[ 180], 20.00th=[ 180], | 30.00th=[ 180], 40.00th=[ 180], 50.00th=[ 180], 60.00th=[ 180], | 70.00th=[ 180], 80.00th=[ 180], 90.00th=[ 180], 95.00th=[ 180], | 99.00th=[ 180], 99.50th=[ 180], 99.90th=[ 180], 99.95th=[ 180], | 99.99th=[ 180] bw ( KiB/s): min= 8000, max= 8000, per=100.00%, avg=8000.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2917KiB/s (2987kB/s)(8192KiB/2808msec); 0 zone resets clat (msec): min=434, max=2104, avg=1269.76, stdev=1181.00 lat (msec): min=435, max=2105, avg=1270.18, stdev=1180.93 clat percentiles (msec): | 1.00th=[ 435], 5.00th=[ 435], 10.00th=[ 435], 20.00th=[ 435], | 30.00th=[ 435], 40.00th=[ 435], 50.00th=[ 435], 60.00th=[ 2106], | 70.00th=[ 2106], 80.00th=[ 2106], 90.00th=[ 2106], 95.00th=[ 2106], | 99.00th=[ 2106], 99.50th=[ 2106], 99.90th=[ 2106], 99.95th=[ 2106], | 99.99th=[ 2106] bw ( KiB/s): min= 8175, max= 8175, per=70.06%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, >=2000=33.33% cpu : usr=0.07%, sys=5.95%, ctx=61, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147622: Thu Oct 23 13:09:31 2025 write: IOPS=1, BW=6678KiB/s (6839kB/s)(12.0MiB/1840msec); 0 zone resets clat (msec): min=388, max=793, avg=592.40, stdev=202.64 lat (msec): min=388, max=793, avg=593.13, stdev=202.59 clat percentiles (msec): | 1.00th=[ 388], 5.00th=[ 388], 10.00th=[ 388], 20.00th=[ 388], | 30.00th=[ 388], 40.00th=[ 600], 50.00th=[ 600], 60.00th=[ 600], | 70.00th=[ 793], 80.00th=[ 793], 90.00th=[ 793], 95.00th=[ 793], | 99.00th=[ 793], 99.50th=[ 793], 99.90th=[ 793], 99.95th=[ 793], | 99.99th=[ 793] bw ( KiB/s): min= 5834, max= 8175, per=60.03%, avg=7004.50, stdev=1655.34, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=6.85%, ctx=104, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147623: Thu Oct 23 13:09:31 2025 write: IOPS=1, BW=6678KiB/s (6839kB/s)(12.0MiB/1840msec); 0 zone resets clat (msec): min=292, max=1147, avg=579.42, stdev=491.89 lat (msec): min=293, max=1147, avg=579.84, stdev=491.73 clat percentiles (msec): | 1.00th=[ 292], 5.00th=[ 292], 10.00th=[ 292], 20.00th=[ 292], | 30.00th=[ 292], 40.00th=[ 300], 50.00th=[ 300], 60.00th=[ 300], | 70.00th=[ 1150], 80.00th=[ 1150], 90.00th=[ 1150], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min=11299, max=11299, per=96.83%, avg=11299.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=7.18%, ctx=46, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5835KiB/s (5975kB/s), 1459KiB/s-4741KiB/s (1494kB/s-4855kB/s), io=16.0MiB (16.8MB), run=2592-2808msec WRITE: bw=11.4MiB/s (11.9MB/s), 2917KiB/s-6678KiB/s (2987kB/s-6839kB/s), io=32.0MiB (33.6MB), run=1840-2808msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147625: Thu Oct 23 13:09:31 2025 read: IOPS=1, BW=6774KiB/s (6937kB/s)(12.0MiB/1814msec) clat (msec): min=504, max=674, avg=604.34, stdev=89.20 lat (msec): min=504, max=674, avg=604.34, stdev=89.20 clat percentiles (msec): | 1.00th=[ 506], 5.00th=[ 506], 10.00th=[ 506], 20.00th=[ 506], | 30.00th=[ 506], 40.00th=[ 634], 50.00th=[ 634], 60.00th=[ 634], | 70.00th=[ 676], 80.00th=[ 676], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 6736, max= 8159, per=100.00%, avg=7447.50, stdev=1006.21, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=100.00% cpu : usr=0.11%, sys=23.99%, ctx=645, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147626: Thu Oct 23 13:09:31 2025 read: IOPS=0, BW=1539KiB/s (1576kB/s)(4096KiB/2661msec) clat (nsec): min=579513k, max=579513k, avg=579513012.00, stdev= 0.00 lat (nsec): min=579515k, max=579515k, avg=579515029.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 584], 5.00th=[ 584], 10.00th=[ 584], 20.00th=[ 584], | 30.00th=[ 584], 40.00th=[ 584], 50.00th=[ 584], 60.00th=[ 584], | 70.00th=[ 584], 80.00th=[ 584], 90.00th=[ 584], 95.00th=[ 584], | 99.00th=[ 584], 99.50th=[ 584], 99.90th=[ 584], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 5970, max= 5970, per=96.96%, avg=5970.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3079KiB/s (3152kB/s)(8192KiB/2661msec); 0 zone resets clat (msec): min=568, max=1504, avg=1036.55, stdev=662.13 lat (msec): min=568, max=1511, avg=1039.88, stdev=666.44 clat percentiles (msec): | 1.00th=[ 567], 5.00th=[ 567], 10.00th=[ 567], 20.00th=[ 567], | 30.00th=[ 567], 40.00th=[ 567], 50.00th=[ 567], 60.00th=[ 1502], | 70.00th=[ 1502], 80.00th=[ 1502], 90.00th=[ 1502], 95.00th=[ 1502], | 99.00th=[ 1502], 99.50th=[ 1502], 99.90th=[ 1502], 99.95th=[ 1502], | 99.99th=[ 1502] bw ( KiB/s): min= 8143, max= 8143, per=66.13%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.04%, sys=10.49%, ctx=75, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147627: Thu Oct 23 13:09:31 2025 write: IOPS=1, BW=4723KiB/s (4836kB/s)(12.0MiB/2602msec); 0 zone resets clat (msec): min=290, max=1297, avg=865.26, stdev=518.60 lat (msec): min=290, max=1301, avg=867.06, stdev=520.07 clat percentiles (msec): | 1.00th=[ 292], 5.00th=[ 292], 10.00th=[ 292], 20.00th=[ 292], | 30.00th=[ 292], 40.00th=[ 1011], 50.00th=[ 1011], 60.00th=[ 1011], | 70.00th=[ 1301], 80.00th=[ 1301], 90.00th=[ 1301], 95.00th=[ 1301], | 99.00th=[ 1301], 99.50th=[ 1301], 99.90th=[ 1301], 99.95th=[ 1301], | 99.99th=[ 1301] bw ( KiB/s): min= 8143, max= 8143, per=66.13%, avg=8143.00, stdev= 0.00, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.00%, sys=7.54%, ctx=55, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147628: Thu Oct 23 13:09:31 2025 write: IOPS=1, BW=6827KiB/s (6991kB/s)(12.0MiB/1800msec); 0 zone resets clat (msec): min=500, max=765, avg=598.90, stdev=144.95 lat (msec): min=500, max=765, avg=599.45, stdev=144.96 clat percentiles (msec): | 1.00th=[ 502], 5.00th=[ 502], 10.00th=[ 502], 20.00th=[ 502], | 30.00th=[ 502], 40.00th=[ 531], 50.00th=[ 531], 60.00th=[ 531], | 70.00th=[ 768], 80.00th=[ 768], 90.00th=[ 768], 95.00th=[ 768], | 99.00th=[ 768], 99.50th=[ 768], 99.90th=[ 768], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 7670, max= 8159, per=64.27%, avg=7914.50, stdev=345.78, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=66.67%, 1000=33.33% cpu : usr=0.00%, sys=10.84%, ctx=42, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6157KiB/s (6305kB/s), 1539KiB/s-6774KiB/s (1576kB/s-6937kB/s), io=16.0MiB (16.8MB), run=1814-2661msec WRITE: bw=12.0MiB/s (12.6MB/s), 3079KiB/s-6827KiB/s (3152kB/s-6991kB/s), io=32.0MiB (33.6MB), run=1800-2661msec