== sanity test 398b: DIO and buffer IO race ============== 21:25:26 (1763346326) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 4.18382 s, 12.0 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147314: Sun Nov 16 21:27:24 2025 read: IOPS=14, BW=57.6KiB/s (58.0kB/s)(5992KiB/104048msec) clat (usec): min=105, max=880727, avg=34182.68, stdev=33934.70 lat (usec): min=105, max=880728, avg=34184.54, stdev=33934.52 clat percentiles (usec): | 1.00th=[ 133], 5.00th=[ 176], 10.00th=[ 383], 20.00th=[ 16319], | 30.00th=[ 20317], 40.00th=[ 24249], 50.00th=[ 28443], 60.00th=[ 34341], | 70.00th=[ 41157], 80.00th=[ 50070], 90.00th=[ 63701], 95.00th=[ 76022], | 99.00th=[127402], 99.50th=[164627], 99.90th=[265290], 99.95th=[876610], | 99.99th=[876610] bw ( KiB/s): min= 7, max= 111, per=25.55%, avg=58.50, stdev=22.91, samples=200 iops : min= 1, max= 27, avg=14.00, stdev= 5.79, samples=200 write: IOPS=15, BW=60.5KiB/s (61.0kB/s)(6296KiB/104048msec); 0 zone resets clat (usec): min=909, max=841087, avg=33255.94, stdev=40487.48 lat (usec): min=910, max=841088, avg=33259.09, stdev=40488.23 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 1172], 10.00th=[ 1270], 20.00th=[ 2900], | 30.00th=[ 7701], 40.00th=[ 16909], 50.00th=[ 26084], 60.00th=[ 34341], | 70.00th=[ 43254], 80.00th=[ 54789], 90.00th=[ 72877], 95.00th=[ 93848], | 99.00th=[152044], 99.50th=[181404], 99.90th=[557843], 99.95th=[843056], | 99.99th=[843056] bw ( KiB/s): min= 7, max= 112, per=25.48%, avg=61.14, stdev=22.44, samples=201 iops : min= 1, max= 28, avg=14.65, stdev= 5.65, samples=201 lat (usec) : 250=3.97%, 500=0.98%, 750=0.07%, 1000=0.20% lat (msec) : 2=9.18%, 4=2.51%, 10=5.86%, 20=13.67%, 50=41.86% lat (msec) : 100=18.82%, 250=2.70%, 500=0.10%, 750=0.03%, 1000=0.07% cpu : usr=0.10%, sys=7.12%, ctx=6543, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147315: Sun Nov 16 21:27:24 2025 read: IOPS=14, BW=57.7KiB/s (59.1kB/s)(5940KiB/102865msec) clat (usec): min=103, max=461168, avg=32888.90, stdev=27016.80 lat (usec): min=104, max=461170, avg=32891.84, stdev=27016.60 clat percentiles (usec): | 1.00th=[ 135], 5.00th=[ 176], 10.00th=[ 289], 20.00th=[ 15401], | 30.00th=[ 19792], 40.00th=[ 23725], 50.00th=[ 28181], 60.00th=[ 33424], | 70.00th=[ 40633], 80.00th=[ 49021], 90.00th=[ 61604], 95.00th=[ 74974], | 99.00th=[112722], 99.50th=[154141], 99.90th=[256902], 99.95th=[459277], | 99.99th=[459277] bw ( KiB/s): min= 7, max= 126, per=25.14%, avg=57.56, stdev=23.30, samples=202 iops : min= 1, max= 31, avg=13.76, stdev= 5.85, samples=202 write: IOPS=15, BW=61.7KiB/s (63.2kB/s)(6348KiB/102865msec); 0 zone resets clat (usec): min=903, max=763279, avg=33732.77, stdev=39303.77 lat (usec): min=904, max=763280, avg=33738.53, stdev=39301.82 clat percentiles (usec): | 1.00th=[ 1037], 5.00th=[ 1172], 10.00th=[ 1287], 20.00th=[ 3163], | 30.00th=[ 8160], 40.00th=[ 16581], 50.00th=[ 24773], 60.00th=[ 33162], | 70.00th=[ 42206], 80.00th=[ 56361], 90.00th=[ 78119], 95.00th=[ 98042], | 99.00th=[160433], 99.50th=[196084], 99.90th=[417334], 99.95th=[759170], | 99.99th=[759170] bw ( KiB/s): min= 8, max= 167, per=25.64%, avg=61.54, stdev=24.02, samples=202 iops : min= 2, max= 41, avg=14.77, stdev= 6.07, samples=202 lat (usec) : 250=4.72%, 500=0.59%, 750=0.20%, 1000=0.29% lat (msec) : 2=8.69%, 4=2.47%, 10=6.05%, 20=14.03%, 50=41.21% lat (msec) : 100=19.01%, 250=2.60%, 500=0.10%, 1000=0.03% cpu : usr=0.10%, sys=6.83%, ctx=6430, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147317: Sun Nov 16 21:27:24 2025 read: IOPS=14, BW=57.6KiB/s (58.0kB/s)(6028KiB/104649msec) clat (usec): min=94, max=434453, avg=32871.12, stdev=28889.66 lat (usec): min=94, max=434455, avg=32877.71, stdev=28893.64 clat percentiles (usec): | 1.00th=[ 133], 5.00th=[ 174], 10.00th=[ 302], 20.00th=[ 15139], | 30.00th=[ 20055], 40.00th=[ 23200], 50.00th=[ 27132], 60.00th=[ 32375], | 70.00th=[ 39584], 80.00th=[ 47973], 90.00th=[ 64226], 95.00th=[ 74974], | 99.00th=[114820], 99.50th=[152044], 99.90th=[392168], 99.95th=[434111], | 99.99th=[434111] bw ( KiB/s): min= 7, max= 134, per=25.07%, avg=57.41, stdev=24.77, samples=204 iops : min= 1, max= 33, avg=13.72, stdev= 6.23, samples=204 write: IOPS=14, BW=59.8KiB/s (61.3kB/s)(6260KiB/104649msec); 0 zone resets clat (usec): min=970, max=859407, avg=34890.00, stdev=40767.23 lat (usec): min=971, max=859411, avg=34899.20, stdev=40775.17 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 1172], 10.00th=[ 1270], 20.00th=[ 2769], | 30.00th=[ 8455], 40.00th=[ 18220], 50.00th=[ 27657], 60.00th=[ 35914], | 70.00th=[ 46400], 80.00th=[ 58459], 90.00th=[ 77071], 95.00th=[ 98042], | 99.00th=[139461], 99.50th=[158335], 99.90th=[467665], 99.95th=[859833], | 99.99th=[859833] bw ( KiB/s): min= 8, max= 127, per=24.89%, avg=59.72, stdev=21.62, samples=203 iops : min= 2, max= 31, avg=14.31, stdev= 5.43, samples=203 lat (usec) : 100=0.03%, 250=4.39%, 500=0.88%, 750=0.10%, 1000=0.20% lat (msec) : 2=9.15%, 4=2.57%, 10=5.53%, 20=13.18%, 50=40.92% lat (msec) : 100=19.82%, 250=2.99%, 500=0.20%, 1000=0.03% cpu : usr=0.10%, sys=6.63%, ctx=6420, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147319: Sun Nov 16 21:27:24 2025 read: IOPS=14, BW=58.3KiB/s (59.6kB/s)(6052KiB/103895msec) clat (usec): min=92, max=795812, avg=34617.12, stdev=36433.58 lat (usec): min=92, max=795814, avg=34618.49, stdev=36433.64 clat percentiles (usec): | 1.00th=[ 139], 5.00th=[ 180], 10.00th=[ 486], 20.00th=[ 16057], | 30.00th=[ 19792], 40.00th=[ 23987], 50.00th=[ 28443], 60.00th=[ 33817], | 70.00th=[ 40633], 80.00th=[ 50594], 90.00th=[ 64226], 95.00th=[ 78119], | 99.00th=[124257], 99.50th=[154141], 99.90th=[522191], 99.95th=[792724], | 99.99th=[792724] bw ( KiB/s): min= 7, max= 127, per=25.49%, avg=58.37, stdev=22.87, samples=203 iops : min= 1, max= 31, avg=13.96, stdev= 5.76, samples=203 write: IOPS=15, BW=60.0KiB/s (61.5kB/s)(6236KiB/103895msec); 0 zone resets clat (usec): min=939, max=234431, avg=32761.42, stdev=32016.73 lat (usec): min=940, max=234433, avg=32763.05, stdev=32016.70 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 1188], 10.00th=[ 1303], 20.00th=[ 2507], | 30.00th=[ 7373], 40.00th=[ 16188], 50.00th=[ 26084], 60.00th=[ 34866], | 70.00th=[ 44303], 80.00th=[ 57934], 90.00th=[ 76022], 95.00th=[ 93848], | 99.00th=[135267], 99.50th=[154141], 99.90th=[170918], 99.95th=[233833], | 99.99th=[233833] bw ( KiB/s): min= 7, max= 127, per=25.03%, avg=60.08, stdev=22.13, samples=203 iops : min= 1, max= 31, avg=14.39, stdev= 5.57, samples=203 lat (usec) : 100=0.07%, 250=4.13%, 500=0.75%, 750=0.20%, 1000=0.16% lat (msec) : 2=9.02%, 4=3.32%, 10=5.18%, 20=14.19%, 50=40.46% lat (msec) : 100=19.43%, 250=2.93%, 500=0.10%, 750=0.03%, 1000=0.03% cpu : usr=0.11%, sys=6.78%, ctx=6585, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=229KiB/s (235kB/s), 57.6KiB/s-58.3KiB/s (58.0kB/s-59.6kB/s), io=23.4MiB (24.6MB), run=102865-104649msec WRITE: bw=240KiB/s (246kB/s), 59.8KiB/s-61.7KiB/s (61.3kB/s-63.2kB/s), io=24.6MiB (25.7MB), run=102865-104649msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147313: Sun Nov 16 21:29:46 2025 read: IOPS=6, BW=24.3KiB/s (24.9kB/s)(5992KiB/246740msec) clat (msec): min=5, max=920, avg=33.68, stdev=38.36 lat (msec): min=5, max=920, avg=33.68, stdev=38.36 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 30], | 70.00th=[ 39], 80.00th=[ 49], 90.00th=[ 63], 95.00th=[ 83], | 99.00th=[ 142], 99.50th=[ 194], 99.90th=[ 531], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 7, max= 96, per=26.86%, avg=26.05, stdev=17.13, samples=445 iops : min= 1, max= 24, avg= 5.87, stdev= 4.31, samples=445 write: IOPS=6, BW=25.5KiB/s (26.1kB/s)(6296KiB/246740msec); 0 zone resets clat (msec): min=41, max=423, avg=124.44, stdev=51.39 lat (msec): min=41, max=423, avg=124.44, stdev=51.39 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 65], 10.00th=[ 73], 20.00th=[ 84], | 30.00th=[ 94], 40.00th=[ 104], 50.00th=[ 115], 60.00th=[ 125], | 70.00th=[ 138], 80.00th=[ 159], 90.00th=[ 190], 95.00th=[ 224], | 99.00th=[ 300], 99.50th=[ 347], 99.90th=[ 414], 99.95th=[ 422], | 99.99th=[ 422] bw ( KiB/s): min= 7, max= 48, per=24.91%, avg=25.16, stdev= 9.60, samples=483 iops : min= 1, max= 12, avg= 5.64, stdev= 2.45, samples=483 lat (msec) : 10=3.39%, 20=16.83%, 50=19.95%, 100=25.98%, 250=32.39% lat (msec) : 500=1.40%, 750=0.03%, 1000=0.03% cpu : usr=0.05%, sys=1.44%, ctx=5607, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147316: Sun Nov 16 21:29:46 2025 read: IOPS=6, BW=24.1KiB/s (24.7kB/s)(5940KiB/246658msec) clat (msec): min=5, max=811, avg=32.96, stdev=32.54 lat (msec): min=5, max=811, avg=32.97, stdev=32.54 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 30], | 70.00th=[ 39], 80.00th=[ 50], 90.00th=[ 65], 95.00th=[ 79], | 99.00th=[ 126], 99.50th=[ 161], 99.90th=[ 255], 99.95th=[ 810], | 99.99th=[ 810] bw ( KiB/s): min= 7, max= 95, per=27.06%, avg=26.25, stdev=16.59, samples=438 iops : min= 1, max= 23, avg= 5.92, stdev= 4.19, samples=438 write: IOPS=6, BW=25.7KiB/s (26.4kB/s)(6348KiB/246658msec); 0 zone resets clat (msec): min=40, max=909, avg=124.28, stdev=55.10 lat (msec): min=40, max=909, avg=124.28, stdev=55.10 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 64], 10.00th=[ 70], 20.00th=[ 83], | 30.00th=[ 92], 40.00th=[ 103], 50.00th=[ 113], 60.00th=[ 125], | 70.00th=[ 140], 80.00th=[ 161], 90.00th=[ 190], 95.00th=[ 222], | 99.00th=[ 292], 99.50th=[ 338], 99.90th=[ 567], 99.95th=[ 911], | 99.99th=[ 911] bw ( KiB/s): min= 7, max= 55, per=25.15%, avg=25.40, stdev= 9.70, samples=483 iops : min= 1, max= 13, avg= 5.70, stdev= 2.47, samples=483 lat (msec) : 10=2.77%, 20=17.42%, 50=19.17%, 100=27.73%, 250=31.64% lat (msec) : 500=1.17%, 750=0.03%, 1000=0.07% cpu : usr=0.05%, sys=1.50%, ctx=5645, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147318: Sun Nov 16 21:29:46 2025 read: IOPS=6, BW=24.4KiB/s (24.0kB/s)(6028KiB/246921msec) clat (msec): min=6, max=509, avg=33.69, stdev=30.23 lat (msec): min=6, max=509, avg=33.70, stdev=30.23 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 25], 60.00th=[ 32], | 70.00th=[ 40], 80.00th=[ 50], 90.00th=[ 66], 95.00th=[ 82], | 99.00th=[ 125], 99.50th=[ 176], 99.90th=[ 338], 99.95th=[ 510], | 99.99th=[ 510] bw ( KiB/s): min= 7, max= 94, per=26.99%, avg=26.18, stdev=16.17, samples=445 iops : min= 1, max= 23, avg= 5.90, stdev= 4.07, samples=445 write: IOPS=6, BW=25.4KiB/s (25.0kB/s)(6260KiB/246921msec); 0 zone resets clat (msec): min=38, max=853, avg=125.05, stdev=56.49 lat (msec): min=38, max=853, avg=125.05, stdev=56.49 clat percentiles (msec): | 1.00th=[ 53], 5.00th=[ 64], 10.00th=[ 72], 20.00th=[ 85], | 30.00th=[ 93], 40.00th=[ 103], 50.00th=[ 112], 60.00th=[ 124], | 70.00th=[ 138], 80.00th=[ 159], 90.00th=[ 194], 95.00th=[ 228], | 99.00th=[ 300], 99.50th=[ 326], 99.90th=[ 567], 99.95th=[ 852], | 99.99th=[ 852] bw ( KiB/s): min= 7, max= 48, per=24.73%, avg=24.98, stdev=10.03, samples=483 iops : min= 1, max= 12, avg= 5.60, stdev= 2.56, samples=483 lat (msec) : 10=2.41%, 20=17.74%, 50=19.86%, 100=26.92%, 250=31.51% lat (msec) : 500=1.43%, 750=0.10%, 1000=0.03% cpu : usr=0.06%, sys=1.49%, ctx=5667, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147320: Sun Nov 16 21:29:46 2025 read: IOPS=6, BW=24.6KiB/s (25.2kB/s)(6052KiB/246147msec) clat (msec): min=5, max=245, avg=32.85, stdev=26.63 lat (msec): min=5, max=245, avg=32.85, stdev=26.63 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 23], 60.00th=[ 30], | 70.00th=[ 40], 80.00th=[ 51], 90.00th=[ 65], 95.00th=[ 81], | 99.00th=[ 127], 99.50th=[ 174], 99.90th=[ 245], 99.95th=[ 247], | 99.99th=[ 247] bw ( KiB/s): min= 7, max= 95, per=27.52%, avg=26.69, stdev=16.92, samples=439 iops : min= 1, max= 23, avg= 6.03, stdev= 4.24, samples=439 write: IOPS=6, BW=25.3KiB/s (25.9kB/s)(6236KiB/246147msec); 0 zone resets clat (msec): min=37, max=907, avg=125.73, stdev=56.76 lat (msec): min=37, max=907, avg=125.74, stdev=56.76 clat percentiles (msec): | 1.00th=[ 53], 5.00th=[ 66], 10.00th=[ 74], 20.00th=[ 85], | 30.00th=[ 95], 40.00th=[ 105], 50.00th=[ 114], 60.00th=[ 126], | 70.00th=[ 140], 80.00th=[ 161], 90.00th=[ 190], 95.00th=[ 215], | 99.00th=[ 296], 99.50th=[ 342], 99.90th=[ 869], 99.95th=[ 911], | 99.99th=[ 911] bw ( KiB/s): min= 7, max= 55, per=24.74%, avg=24.99, stdev= 9.53, samples=482 iops : min= 1, max= 13, avg= 5.60, stdev= 2.44, samples=482 lat (msec) : 10=3.52%, 20=17.77%, 50=18.49%, 100=26.30%, 250=32.85% lat (msec) : 500=0.98%, 750=0.03%, 1000=0.07% cpu : usr=0.05%, sys=1.51%, ctx=5567, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=97.2KiB/s (99.6kB/s), 24.1KiB/s-24.6KiB/s (24.7kB/s-25.2kB/s), io=23.4MiB (24.6MB), run=246147-246921msec WRITE: bw=102KiB/s (104kB/s), 25.3KiB/s-25.7KiB/s (25.9kB/s-26.4kB/s), io=24.6MiB (25.7MB), run=246147-246921msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147375: Sun Nov 16 21:30:19 2025 read: IOPS=16, BW=259KiB/s (265kB/s)(5920KiB/22863msec) clat (usec): min=107, max=297502, avg=31554.25, stdev=26823.18 lat (usec): min=108, max=297504, avg=31555.81, stdev=26823.11 clat percentiles (usec): | 1.00th=[ 129], 5.00th=[ 172], 10.00th=[ 202], 20.00th=[ 13829], | 30.00th=[ 20317], 40.00th=[ 23987], 50.00th=[ 27395], 60.00th=[ 33817], | 70.00th=[ 40109], 80.00th=[ 48497], 90.00th=[ 60031], 95.00th=[ 68682], | 99.00th=[128451], 99.50th=[143655], 99.90th=[295699], 99.95th=[295699], | 99.99th=[295699] bw ( KiB/s): min= 95, max= 540, per=26.01%, avg=258.80, stdev=94.62, samples=45 iops : min= 5, max= 33, avg=15.49, stdev= 5.93, samples=45 write: IOPS=17, BW=279KiB/s (285kB/s)(6368KiB/22863msec); 0 zone resets clat (usec): min=1479, max=171110, avg=27517.90, stdev=28139.03 lat (usec): min=1482, max=171113, avg=27550.68, stdev=28118.61 clat percentiles (usec): | 1.00th=[ 1582], 5.00th=[ 1844], 10.00th=[ 2057], 20.00th=[ 4621], | 30.00th=[ 7767], 40.00th=[ 12256], 50.00th=[ 18744], 60.00th=[ 25035], | 70.00th=[ 32900], 80.00th=[ 47449], 90.00th=[ 65274], 95.00th=[ 86508], | 99.00th=[126354], 99.50th=[147850], 99.90th=[170918], 99.95th=[170918], | 99.99th=[170918] bw ( KiB/s): min= 29, max= 479, per=26.57%, avg=278.18, stdev=98.48, samples=45 iops : min= 1, max= 29, avg=16.69, stdev= 6.19, samples=45 lat (usec) : 250=6.90%, 500=1.04%, 750=0.26%, 1000=0.26% lat (msec) : 2=4.82%, 4=4.95%, 10=9.38%, 20=12.63%, 50=41.28% lat (msec) : 100=16.28%, 250=2.08%, 500=0.13% cpu : usr=0.11%, sys=8.49%, ctx=1535, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147376: Sun Nov 16 21:30:19 2025 read: IOPS=15, BW=247KiB/s (253kB/s)(5744KiB/23216msec) clat (usec): min=151, max=184364, avg=33750.91, stdev=24617.25 lat (usec): min=152, max=184364, avg=33752.47, stdev=24617.28 clat percentiles (usec): | 1.00th=[ 167], 5.00th=[ 192], 10.00th=[ 245], 20.00th=[ 15401], | 30.00th=[ 21890], 40.00th=[ 26870], 50.00th=[ 32113], 60.00th=[ 36439], | 70.00th=[ 41681], 80.00th=[ 50594], 90.00th=[ 63701], 95.00th=[ 73925], | 99.00th=[123208], 99.50th=[156238], 99.90th=[183501], 99.95th=[183501], | 99.99th=[183501] bw ( KiB/s): min= 29, max= 447, per=24.78%, avg=246.53, stdev=97.32, samples=45 iops : min= 1, max= 27, avg=14.69, stdev= 6.10, samples=45 write: IOPS=17, BW=282KiB/s (289kB/s)(6544KiB/23216msec); 0 zone resets clat (usec): min=1414, max=265707, avg=26498.88, stdev=28673.16 lat (usec): min=1416, max=265711, avg=26508.04, stdev=28677.71 clat percentiles (usec): | 1.00th=[ 1549], 5.00th=[ 1762], 10.00th=[ 1893], 20.00th=[ 3490], | 30.00th=[ 7373], 40.00th=[ 11207], 50.00th=[ 18220], 60.00th=[ 24511], | 70.00th=[ 31851], 80.00th=[ 45351], 90.00th=[ 66323], 95.00th=[ 82314], | 99.00th=[108528], 99.50th=[128451], 99.90th=[265290], 99.95th=[265290], | 99.99th=[265290] bw ( KiB/s): min= 92, max= 474, per=26.69%, avg=279.40, stdev=88.65, samples=45 iops : min= 5, max= 29, avg=16.73, stdev= 5.56, samples=45 lat (usec) : 250=4.82%, 500=1.69%, 750=0.13%, 1000=0.13% lat (msec) : 2=6.77%, 4=5.21%, 10=7.94%, 20=14.32%, 50=39.97% lat (msec) : 100=17.45%, 250=1.43%, 500=0.13% cpu : usr=0.12%, sys=8.99%, ctx=1548, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147377: Sun Nov 16 21:30:19 2025 read: IOPS=16, BW=260KiB/s (266kB/s)(6256KiB/24067msec) clat (usec): min=98, max=153930, avg=32908.37, stdev=25678.04 lat (usec): min=99, max=153932, avg=32915.03, stdev=25671.97 clat percentiles (usec): | 1.00th=[ 115], 5.00th=[ 169], 10.00th=[ 196], 20.00th=[ 8160], | 30.00th=[ 20317], 40.00th=[ 25035], 50.00th=[ 30278], 60.00th=[ 36439], | 70.00th=[ 43254], 80.00th=[ 50070], 90.00th=[ 62129], 95.00th=[ 74974], | 99.00th=[120062], 99.50th=[133694], 99.90th=[154141], 99.95th=[154141], | 99.99th=[154141] bw ( KiB/s): min= 63, max= 432, per=26.73%, avg=265.96, stdev=86.20, samples=46 iops : min= 3, max= 27, avg=15.93, stdev= 5.44, samples=46 write: IOPS=15, BW=251KiB/s (257kB/s)(6032KiB/24067msec); 0 zone resets clat (usec): min=1491, max=319570, avg=29391.30, stdev=36119.65 lat (usec): min=1491, max=319572, avg=29398.77, stdev=36118.52 clat percentiles (usec): | 1.00th=[ 1516], 5.00th=[ 1762], 10.00th=[ 1975], 20.00th=[ 3359], | 30.00th=[ 7177], 40.00th=[ 10814], 50.00th=[ 18744], 60.00th=[ 27919], | 70.00th=[ 37487], 80.00th=[ 52167], 90.00th=[ 63701], 95.00th=[ 80217], | 99.00th=[170918], 99.50th=[287310], 99.90th=[320865], 99.95th=[320865], | 99.99th=[320865] bw ( KiB/s): min= 31, max= 555, per=23.53%, avg=246.40, stdev=109.55, samples=47 iops : min= 1, max= 34, avg=14.70, stdev= 6.82, samples=47 lat (usec) : 100=0.13%, 250=7.29%, 500=1.56%, 750=0.13%, 1000=0.26% lat (msec) : 2=6.12%, 4=5.34%, 10=8.07%, 20=11.46%, 50=38.67% lat (msec) : 100=18.36%, 250=2.34%, 500=0.26% cpu : usr=0.13%, sys=8.41%, ctx=1573, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147378: Sun Nov 16 21:30:19 2025 read: IOPS=16, BW=258KiB/s (264kB/s)(6032KiB/23357msec) clat (usec): min=105, max=204516, avg=31666.84, stdev=25531.55 lat (usec): min=105, max=204519, avg=31668.60, stdev=25531.44 clat percentiles (usec): | 1.00th=[ 122], 5.00th=[ 178], 10.00th=[ 200], 20.00th=[ 2343], | 30.00th=[ 19006], 40.00th=[ 24773], 50.00th=[ 31327], 60.00th=[ 36439], | 70.00th=[ 41157], 80.00th=[ 48497], 90.00th=[ 58983], 95.00th=[ 70779], | 99.00th=[101188], 99.50th=[179307], 99.90th=[204473], 99.95th=[204473], | 99.99th=[204473] bw ( KiB/s): min= 93, max= 415, per=26.33%, avg=262.02, stdev=89.29, samples=44 iops : min= 5, max= 25, avg=15.66, stdev= 5.60, samples=44 write: IOPS=16, BW=268KiB/s (274kB/s)(6256KiB/23357msec); 0 zone resets clat (usec): min=1325, max=201116, avg=28605.27, stdev=28887.38 lat (usec): min=1326, max=201119, avg=28608.09, stdev=28887.54 clat percentiles (usec): | 1.00th=[ 1549], 5.00th=[ 1762], 10.00th=[ 2040], 20.00th=[ 4752], | 30.00th=[ 8455], 40.00th=[ 13042], 50.00th=[ 18744], 60.00th=[ 26608], | 70.00th=[ 35390], 80.00th=[ 50070], 90.00th=[ 66847], 95.00th=[ 84411], | 99.00th=[122160], 99.50th=[145753], 99.90th=[200279], 99.95th=[200279], | 99.99th=[200279] bw ( KiB/s): min= 89, max= 446, per=25.45%, avg=266.42, stdev=84.72, samples=45 iops : min= 5, max= 27, avg=15.98, stdev= 5.29, samples=45 lat (usec) : 250=7.68%, 500=1.43%, 750=0.39% lat (msec) : 2=5.34%, 4=4.69%, 10=8.07%, 20=14.45%, 50=38.15% lat (msec) : 100=18.10%, 250=1.69% cpu : usr=0.15%, sys=8.16%, ctx=1534, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=995KiB/s (1019kB/s), 247KiB/s-260KiB/s (253kB/s-266kB/s), io=23.4MiB (24.5MB), run=22863-24067msec WRITE: bw=1047KiB/s (1072kB/s), 251KiB/s-282KiB/s (257kB/s-289kB/s), io=24.6MiB (25.8MB), run=22863-24067msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147371: Sun Nov 16 21:30:56 2025 read: IOPS=6, BW=98.1KiB/s (100kB/s)(5920KiB/60369msec) clat (msec): min=6, max=235, avg=31.18, stdev=30.48 lat (msec): min=6, max=235, avg=31.18, stdev=30.48 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 19], 50.00th=[ 21], 60.00th=[ 24], | 70.00th=[ 28], 80.00th=[ 41], 90.00th=[ 68], 95.00th=[ 94], | 99.00th=[ 161], 99.50th=[ 182], 99.90th=[ 236], 99.95th=[ 236], | 99.99th=[ 236] bw ( KiB/s): min= 31, max= 333, per=26.23%, avg=103.61, stdev=64.48, samples=112 iops : min= 1, max= 20, avg= 5.70, stdev= 4.05, samples=112 write: IOPS=6, BW=105KiB/s (108kB/s)(6368KiB/60369msec); 0 zone resets clat (msec): min=44, max=469, avg=122.20, stdev=46.19 lat (msec): min=44, max=469, avg=122.20, stdev=46.19 clat percentiles (msec): | 1.00th=[ 57], 5.00th=[ 68], 10.00th=[ 77], 20.00th=[ 87], | 30.00th=[ 95], 40.00th=[ 105], 50.00th=[ 113], 60.00th=[ 125], | 70.00th=[ 136], 80.00th=[ 153], 90.00th=[ 178], 95.00th=[ 209], | 99.00th=[ 266], 99.50th=[ 292], 99.90th=[ 468], 99.95th=[ 468], | 99.99th=[ 468] bw ( KiB/s): min= 30, max= 191, per=25.48%, avg=105.76, stdev=37.44, samples=118 iops : min= 1, max= 11, avg= 5.82, stdev= 2.36, samples=118 lat (msec) : 10=2.34%, 20=20.83%, 50=18.36%, 100=22.66%, 250=34.64% lat (msec) : 500=1.17% cpu : usr=0.06%, sys=1.55%, ctx=1404, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147372: Sun Nov 16 21:30:56 2025 read: IOPS=5, BW=94.8KiB/s (97.0kB/s)(5744KiB/60617msec) clat (msec): min=5, max=323, avg=30.49, stdev=33.02 lat (msec): min=5, max=323, avg=30.51, stdev=33.02 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 24], | 70.00th=[ 28], 80.00th=[ 35], 90.00th=[ 60], 95.00th=[ 102], | 99.00th=[ 161], 99.50th=[ 232], 99.90th=[ 326], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 30, max= 415, per=27.90%, avg=110.19, stdev=65.58, samples=103 iops : min= 1, max= 25, avg= 6.12, stdev= 4.12, samples=103 write: IOPS=6, BW=108KiB/s (111kB/s)(6544KiB/60617msec); 0 zone resets clat (msec): min=51, max=411, avg=120.98, stdev=45.99 lat (msec): min=51, max=411, avg=120.98, stdev=45.99 clat percentiles (msec): | 1.00th=[ 58], 5.00th=[ 66], 10.00th=[ 73], 20.00th=[ 86], | 30.00th=[ 95], 40.00th=[ 103], 50.00th=[ 110], 60.00th=[ 122], | 70.00th=[ 134], 80.00th=[ 153], 90.00th=[ 184], 95.00th=[ 213], | 99.00th=[ 271], 99.50th=[ 296], 99.90th=[ 414], 99.95th=[ 414], | 99.99th=[ 414] bw ( KiB/s): min= 30, max= 191, per=26.10%, avg=108.31, stdev=38.13, samples=119 iops : min= 1, max= 11, avg= 6.00, stdev= 2.40, samples=119 lat (msec) : 10=2.21%, 20=20.83%, 50=17.58%, 100=23.18%, 250=35.16% lat (msec) : 500=1.04% cpu : usr=0.04%, sys=1.60%, ctx=1383, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147373: Sun Nov 16 21:30:56 2025 read: IOPS=6, BW=108KiB/s (110kB/s)(6256KiB/58061msec) clat (msec): min=5, max=244, avg=29.53, stdev=29.73 lat (msec): min=5, max=244, avg=29.54, stdev=29.73 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 24], | 70.00th=[ 28], 80.00th=[ 37], 90.00th=[ 55], 95.00th=[ 99], | 99.00th=[ 161], 99.50th=[ 182], 99.90th=[ 245], 99.95th=[ 245], | 99.99th=[ 245] bw ( KiB/s): min= 30, max= 318, per=28.43%, avg=112.30, stdev=71.92, samples=110 iops : min= 1, max= 19, avg= 6.25, stdev= 4.52, samples=110 write: IOPS=6, BW=104KiB/s (106kB/s)(6032KiB/58061msec); 0 zone resets clat (msec): min=34, max=312, avg=123.16, stdev=42.86 lat (msec): min=34, max=312, avg=123.16, stdev=42.86 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 67], 10.00th=[ 72], 20.00th=[ 87], | 30.00th=[ 96], 40.00th=[ 108], 50.00th=[ 117], 60.00th=[ 128], | 70.00th=[ 140], 80.00th=[ 157], 90.00th=[ 178], 95.00th=[ 207], | 99.00th=[ 241], 99.50th=[ 266], 99.90th=[ 313], 99.95th=[ 313], | 99.99th=[ 313] bw ( KiB/s): min= 30, max= 192, per=24.90%, avg=103.32, stdev=37.32, samples=114 iops : min= 1, max= 12, avg= 5.68, stdev= 2.37, samples=114 lat (msec) : 10=4.69%, 20=20.18%, 50=20.44%, 100=19.79%, 250=34.51% lat (msec) : 500=0.39% cpu : usr=0.06%, sys=1.66%, ctx=1391, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147374: Sun Nov 16 21:30:56 2025 read: IOPS=6, BW=101KiB/s (103kB/s)(6032KiB/59967msec) clat (msec): min=6, max=336, avg=29.29, stdev=32.31 lat (msec): min=6, max=336, avg=29.29, stdev=32.31 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 23], | 70.00th=[ 26], 80.00th=[ 36], 90.00th=[ 57], 95.00th=[ 83], | 99.00th=[ 186], 99.50th=[ 264], 99.90th=[ 338], 99.95th=[ 338], | 99.99th=[ 338] bw ( KiB/s): min= 31, max= 318, per=26.72%, avg=105.55, stdev=64.13, samples=111 iops : min= 1, max= 19, avg= 5.82, stdev= 4.00, samples=111 write: IOPS=6, BW=104KiB/s (107kB/s)(6256KiB/59967msec); 0 zone resets clat (msec): min=46, max=352, avg=124.72, stdev=46.64 lat (msec): min=46, max=352, avg=124.73, stdev=46.64 clat percentiles (msec): | 1.00th=[ 57], 5.00th=[ 69], 10.00th=[ 78], 20.00th=[ 88], | 30.00th=[ 97], 40.00th=[ 106], 50.00th=[ 116], 60.00th=[ 127], | 70.00th=[ 138], 80.00th=[ 155], 90.00th=[ 184], 95.00th=[ 213], | 99.00th=[ 275], 99.50th=[ 321], 99.90th=[ 351], 99.95th=[ 351], | 99.99th=[ 351] bw ( KiB/s): min= 31, max= 192, per=25.16%, avg=104.42, stdev=37.05, samples=117 iops : min= 1, max= 12, avg= 5.75, stdev= 2.34, samples=117 lat (msec) : 10=3.12%, 20=22.27%, 50=17.58%, 100=21.35%, 250=34.24% lat (msec) : 500=1.43% cpu : usr=0.05%, sys=1.67%, ctx=1396, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=395KiB/s (405kB/s), 94.8KiB/s-108KiB/s (97.0kB/s-110kB/s), io=23.4MiB (24.5MB), run=58061-60617msec WRITE: bw=416KiB/s (426kB/s), 104KiB/s-108KiB/s (106kB/s-111kB/s), io=24.6MiB (25.8MB), run=58061-60617msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147392: Sun Nov 16 21:31:12 2025 read: IOPS=1, BW=1667KiB/s (1707kB/s)(7168KiB/4301msec) clat (msec): min=121, max=919, avg=308.55, stdev=280.86 lat (msec): min=121, max=919, avg=308.55, stdev=280.86 clat percentiles (msec): | 1.00th=[ 123], 5.00th=[ 123], 10.00th=[ 123], 20.00th=[ 153], | 30.00th=[ 165], 40.00th=[ 165], 50.00th=[ 180], 60.00th=[ 275], | 70.00th=[ 275], 80.00th=[ 351], 90.00th=[ 919], 95.00th=[ 919], | 99.00th=[ 919], 99.50th=[ 919], 99.90th=[ 919], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 1988, max= 4231, per=97.82%, avg=3432.67, stdev=1253.42, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=1, BW=1190KiB/s (1219kB/s)(5120KiB/4301msec); 0 zone resets clat (msec): min=280, max=582, avg=410.28, stdev=131.55 lat (msec): min=281, max=582, avg=410.42, stdev=131.58 clat percentiles (msec): | 1.00th=[ 279], 5.00th=[ 279], 10.00th=[ 279], 20.00th=[ 279], | 30.00th=[ 321], 40.00th=[ 321], 50.00th=[ 351], 60.00th=[ 351], | 70.00th=[ 518], 80.00th=[ 518], 90.00th=[ 584], 95.00th=[ 584], | 99.00th=[ 584], 99.50th=[ 584], 99.90th=[ 584], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 1954, max= 3938, per=50.38%, avg=2474.75, stdev=975.91, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 250=33.33%, 500=41.67%, 750=16.67%, 1000=8.33% cpu : usr=0.02%, sys=4.42%, ctx=69, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147393: Sun Nov 16 21:31:12 2025 read: IOPS=0, BW=351KiB/s (359kB/s)(2048KiB/5836msec) clat (msec): min=208, max=490, avg=349.38, stdev=199.87 lat (msec): min=208, max=490, avg=349.38, stdev=199.87 clat percentiles (msec): | 1.00th=[ 209], 5.00th=[ 209], 10.00th=[ 209], 20.00th=[ 209], | 30.00th=[ 209], 40.00th=[ 209], 50.00th=[ 209], 60.00th=[ 489], | 70.00th=[ 489], 80.00th=[ 489], 90.00th=[ 489], 95.00th=[ 489], | 99.00th=[ 489], 99.50th=[ 489], 99.90th=[ 489], 99.95th=[ 489], | 99.99th=[ 489] bw ( KiB/s): min= 1845, max= 1943, per=53.98%, avg=1894.00, stdev=69.30, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1755KiB/s (1797kB/s)(10.0MiB/5836msec); 0 zone resets clat (msec): min=207, max=1118, avg=507.11, stdev=302.10 lat (msec): min=207, max=1119, avg=507.48, stdev=302.29 clat percentiles (msec): | 1.00th=[ 207], 5.00th=[ 207], 10.00th=[ 207], 20.00th=[ 245], | 30.00th=[ 321], 40.00th=[ 330], 50.00th=[ 355], 60.00th=[ 384], | 70.00th=[ 518], 80.00th=[ 718], 90.00th=[ 877], 95.00th=[ 1116], | 99.00th=[ 1116], 99.50th=[ 1116], 99.90th=[ 1116], 99.95th=[ 1116], | 99.99th=[ 1116] bw ( KiB/s): min= 1969, max= 4096, per=52.99%, avg=2603.00, stdev=1006.32, samples=7 iops : min= 1, max= 4, avg= 1.71, stdev= 1.25, samples=7 lat (msec) : 250=25.00%, 500=41.67%, 750=16.67%, 1000=8.33%, 2000=8.33% cpu : usr=0.02%, sys=3.02%, ctx=66, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147394: Sun Nov 16 21:31:12 2025 read: IOPS=1, BW=1074KiB/s (1100kB/s)(6144KiB/5719msec) clat (msec): min=78, max=471, avg=244.66, stdev=166.54 lat (msec): min=78, max=471, avg=244.67, stdev=166.54 clat percentiles (msec): | 1.00th=[ 79], 5.00th=[ 79], 10.00th=[ 79], 20.00th=[ 120], | 30.00th=[ 120], 40.00th=[ 146], 50.00th=[ 146], 60.00th=[ 226], | 70.00th=[ 430], 80.00th=[ 430], 90.00th=[ 472], 95.00th=[ 472], | 99.00th=[ 472], 99.50th=[ 472], 99.90th=[ 472], 99.95th=[ 472], | 99.99th=[ 472] bw ( KiB/s): min= 1954, max= 2048, per=57.37%, avg=2013.00, stdev=41.09, samples=5 iops : min= 1, max= 2, avg= 1.40, stdev= 0.55, samples=5 write: IOPS=1, BW=1074KiB/s (1100kB/s)(6144KiB/5719msec); 0 zone resets clat (msec): min=304, max=1430, avg=668.14, stdev=408.24 lat (msec): min=304, max=1430, avg=668.26, stdev=408.27 clat percentiles (msec): | 1.00th=[ 305], 5.00th=[ 305], 10.00th=[ 305], 20.00th=[ 384], | 30.00th=[ 384], 40.00th=[ 527], 50.00th=[ 527], 60.00th=[ 584], | 70.00th=[ 776], 80.00th=[ 776], 90.00th=[ 1435], 95.00th=[ 1435], | 99.00th=[ 1435], 99.50th=[ 1435], 99.90th=[ 1435], 99.95th=[ 1435], | 99.99th=[ 1435] bw ( KiB/s): min= 1954, max= 2048, per=40.91%, avg=2009.60, stdev=39.58, samples=5 iops : min= 1, max= 2, avg= 1.20, stdev= 0.45, samples=5 lat (msec) : 100=8.33%, 250=25.00%, 500=33.33%, 750=16.67%, 1000=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=4.49%, ctx=88, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147395: Sun Nov 16 21:31:12 2025 read: IOPS=1, BW=1173KiB/s (1201kB/s)(5120KiB/4365msec) clat (msec): min=84, max=192, avg=130.94, stdev=39.89 lat (msec): min=84, max=192, avg=130.94, stdev=39.89 clat percentiles (msec): | 1.00th=[ 85], 5.00th=[ 85], 10.00th=[ 85], 20.00th=[ 85], | 30.00th=[ 118], 40.00th=[ 118], 50.00th=[ 120], 60.00th=[ 120], | 70.00th=[ 142], 80.00th=[ 142], 90.00th=[ 192], 95.00th=[ 192], | 99.00th=[ 192], 99.50th=[ 192], 99.90th=[ 192], 99.95th=[ 192], | 99.99th=[ 192] bw ( KiB/s): min= 3938, max= 3984, per=100.00%, avg=3961.00, stdev=32.53, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1642KiB/s (1682kB/s)(7168KiB/4365msec); 0 zone resets clat (msec): min=276, max=1188, avg=508.22, stdev=311.00 lat (msec): min=276, max=1188, avg=508.34, stdev=311.00 clat percentiles (msec): | 1.00th=[ 279], 5.00th=[ 279], 10.00th=[ 279], 20.00th=[ 330], | 30.00th=[ 372], 40.00th=[ 372], 50.00th=[ 401], 60.00th=[ 468], | 70.00th=[ 468], 80.00th=[ 523], 90.00th=[ 1183], 95.00th=[ 1183], | 99.00th=[ 1183], 99.50th=[ 1183], 99.90th=[ 1183], 99.95th=[ 1183], | 99.99th=[ 1183] bw ( KiB/s): min= 1943, max= 4055, per=48.91%, avg=2402.60, stdev=924.25, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 lat (msec) : 100=8.33%, 250=33.33%, 500=41.67%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=4.19%, ctx=60, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3509KiB/s (3593kB/s), 351KiB/s-1667KiB/s (359kB/s-1707kB/s), io=20.0MiB (20.0MB), run=4301-5836msec WRITE: bw=4913KiB/s (5031kB/s), 1074KiB/s-1755KiB/s (1100kB/s-1797kB/s), io=28.0MiB (29.4MB), run=4301-5836msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147397: Sun Nov 16 21:31:12 2025 read: IOPS=1, BW=1769KiB/s (1812kB/s)(7168KiB/4051msec) clat (msec): min=123, max=322, avg=233.04, stdev=64.27 lat (msec): min=123, max=322, avg=233.05, stdev=64.27 clat percentiles (msec): | 1.00th=[ 124], 5.00th=[ 124], 10.00th=[ 124], 20.00th=[ 205], | 30.00th=[ 215], 40.00th=[ 215], 50.00th=[ 222], 60.00th=[ 257], | 70.00th=[ 257], 80.00th=[ 288], 90.00th=[ 321], 95.00th=[ 321], | 99.00th=[ 321], 99.50th=[ 321], 99.90th=[ 321], 99.95th=[ 321], | 99.99th=[ 321] bw ( KiB/s): min= 1381, max= 4015, per=64.53%, avg=2270.20, stdev=1010.41, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 write: IOPS=1, BW=1264KiB/s (1294kB/s)(5120KiB/4051msec); 0 zone resets clat (msec): min=241, max=690, avg=382.52, stdev=190.11 lat (msec): min=241, max=690, avg=382.63, stdev=190.10 clat percentiles (msec): | 1.00th=[ 241], 5.00th=[ 241], 10.00th=[ 241], 20.00th=[ 241], | 30.00th=[ 259], 40.00th=[ 259], 50.00th=[ 279], 60.00th=[ 279], | 70.00th=[ 443], 80.00th=[ 443], 90.00th=[ 693], 95.00th=[ 693], | 99.00th=[ 693], 99.50th=[ 693], 99.90th=[ 693], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 1932, max= 4096, per=54.52%, avg=2685.00, stdev=1222.88, samples=3 iops : min= 1, max= 4, avg= 2.00, stdev= 1.73, samples=3 lat (msec) : 250=41.67%, 500=50.00%, 750=8.33% cpu : usr=0.00%, sys=12.79%, ctx=289, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147398: Sun Nov 16 21:31:12 2025 read: IOPS=0, BW=352KiB/s (360kB/s)(2048KiB/5821msec) clat (msec): min=171, max=528, avg=349.68, stdev=252.21 lat (msec): min=171, max=528, avg=349.68, stdev=252.21 clat percentiles (msec): | 1.00th=[ 171], 5.00th=[ 171], 10.00th=[ 171], 20.00th=[ 171], | 30.00th=[ 171], 40.00th=[ 171], 50.00th=[ 171], 60.00th=[ 527], | 70.00th=[ 527], 80.00th=[ 527], 90.00th=[ 527], 95.00th=[ 527], | 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527], | 99.99th=[ 527] bw ( KiB/s): min= 1428, max= 1969, per=48.28%, avg=1698.50, stdev=382.54, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1759KiB/s (1801kB/s)(10.0MiB/5821msec); 0 zone resets clat (msec): min=163, max=1236, avg=470.24, stdev=309.75 lat (msec): min=163, max=1236, avg=470.35, stdev=309.74 clat percentiles (msec): | 1.00th=[ 163], 5.00th=[ 163], 10.00th=[ 163], 20.00th=[ 232], | 30.00th=[ 292], 40.00th=[ 317], 50.00th=[ 347], 60.00th=[ 363], | 70.00th=[ 542], 80.00th=[ 609], 90.00th=[ 609], 95.00th=[ 1234], | 99.00th=[ 1234], 99.50th=[ 1234], 99.90th=[ 1234], 99.95th=[ 1234], | 99.99th=[ 1234] bw ( KiB/s): min= 1992, max= 3946, per=46.57%, avg=2293.57, stdev=728.96, samples=7 iops : min= 1, max= 3, avg= 1.43, stdev= 0.79, samples=7 lat (msec) : 250=25.00%, 500=33.33%, 750=33.33%, 2000=8.33% cpu : usr=0.00%, sys=12.96%, ctx=159, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147399: Sun Nov 16 21:31:12 2025 read: IOPS=1, BW=1499KiB/s (1535kB/s)(6144KiB/4098msec) clat (usec): min=668, max=604790, avg=233946.31, stdev=224008.84 lat (usec): min=670, max=604791, avg=233947.64, stdev=224008.81 clat percentiles (usec): | 1.00th=[ 668], 5.00th=[ 668], 10.00th=[ 668], 20.00th=[ 840], | 30.00th=[ 840], 40.00th=[238027], 50.00th=[238027], 60.00th=[263193], | 70.00th=[295699], 80.00th=[295699], 90.00th=[608175], 95.00th=[608175], | 99.00th=[608175], 99.50th=[608175], 99.90th=[608175], 99.95th=[608175], | 99.99th=[608175] bw ( KiB/s): min= 1932, max= 4096, per=95.16%, avg=3347.67, stdev=1226.67, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=1, BW=1499KiB/s (1535kB/s)(6144KiB/4098msec); 0 zone resets clat (msec): min=273, max=930, avg=425.67, stdev=250.26 lat (msec): min=274, max=930, avg=425.76, stdev=250.27 clat percentiles (msec): | 1.00th=[ 275], 5.00th=[ 275], 10.00th=[ 275], 20.00th=[ 296], | 30.00th=[ 296], 40.00th=[ 326], 50.00th=[ 326], 60.00th=[ 347], | 70.00th=[ 380], 80.00th=[ 380], 90.00th=[ 927], 95.00th=[ 927], | 99.00th=[ 927], 99.50th=[ 927], 99.90th=[ 927], 99.95th=[ 927], | 99.99th=[ 927] bw ( KiB/s): min= 1404, max= 2048, per=38.73%, avg=1907.67, stdev=250.30, samples=6 iops : min= 1, max= 2, avg= 1.17, stdev= 0.41, samples=6 lat (usec) : 750=8.33%, 1000=8.33% lat (msec) : 250=8.33%, 500=58.33%, 750=8.33%, 1000=8.33% cpu : usr=0.12%, sys=12.62%, ctx=136, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147400: Sun Nov 16 21:31:12 2025 read: IOPS=1, BW=1116KiB/s (1143kB/s)(5120KiB/4588msec) clat (usec): min=748, max=404969, avg=236297.05, stdev=214663.98 lat (usec): min=749, max=404971, avg=236298.73, stdev=214664.03 clat percentiles (usec): | 1.00th=[ 750], 5.00th=[ 750], 10.00th=[ 750], 20.00th=[ 750], | 30.00th=[ 1876], 40.00th=[ 1876], 50.00th=[383779], 60.00th=[383779], | 70.00th=[392168], 80.00th=[392168], 90.00th=[404751], 95.00th=[404751], | 99.00th=[404751], 99.50th=[404751], 99.90th=[404751], 99.95th=[404751], | 99.99th=[404751] bw ( KiB/s): min= 1932, max= 3938, per=74.94%, avg=2636.33, stdev=1128.55, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=1, BW=1562KiB/s (1600kB/s)(7168KiB/4588msec); 0 zone resets clat (msec): min=175, max=931, avg=425.04, stdev=276.18 lat (msec): min=175, max=932, avg=425.13, stdev=276.17 clat percentiles (msec): | 1.00th=[ 176], 5.00th=[ 176], 10.00th=[ 176], 20.00th=[ 234], | 30.00th=[ 251], 40.00th=[ 251], 50.00th=[ 255], 60.00th=[ 523], | 70.00th=[ 523], 80.00th=[ 609], 90.00th=[ 936], 95.00th=[ 936], | 99.00th=[ 936], 99.50th=[ 936], 99.90th=[ 936], 99.95th=[ 936], | 99.99th=[ 936] bw ( KiB/s): min= 1932, max= 2048, per=40.41%, avg=1990.40, stdev=43.28, samples=5 iops : min= 1, max= 2, avg= 1.20, stdev= 0.45, samples=5 lat (usec) : 750=8.33% lat (msec) : 2=8.33%, 250=16.67%, 500=41.67%, 750=16.67%, 1000=8.33% cpu : usr=0.00%, sys=9.40%, ctx=180, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3518KiB/s (3603kB/s), 352KiB/s-1769KiB/s (360kB/s-1812kB/s), io=20.0MiB (20.0MB), run=4051-5821msec WRITE: bw=4926KiB/s (5044kB/s), 1264KiB/s-1759KiB/s (1294kB/s-1801kB/s), io=28.0MiB (29.4MB), run=4051-5821msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=147412: Sun Nov 16 21:31:28 2025 read: IOPS=0, BW=2521KiB/s (2581kB/s)(12.0MiB/4875msec) clat (msec): min=388, max=3201, avg=1597.75, stdev=1447.72 lat (msec): min=388, max=3201, avg=1597.75, stdev=1447.72 clat percentiles (msec): | 1.00th=[ 388], 5.00th=[ 388], 10.00th=[ 388], 20.00th=[ 388], | 30.00th=[ 388], 40.00th=[ 1200], 50.00th=[ 1200], 60.00th=[ 1200], | 70.00th=[ 3205], 80.00th=[ 3205], 90.00th=[ 3205], 95.00th=[ 3205], | 99.00th=[ 3205], 99.50th=[ 3205], 99.90th=[ 3205], 99.95th=[ 3205], | 99.99th=[ 3205] bw ( KiB/s): min= 8094, max= 8143, per=100.00%, avg=8118.50, stdev=34.65, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=2.85%, ctx=41, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147413: Sun Nov 16 21:31:28 2025 read: IOPS=0, BW=885KiB/s (907kB/s)(4096KiB/4626msec) clat (nsec): min=1044.9M, max=1044.9M, avg=1044888290.00, stdev= 0.00 lat (nsec): min=1044.9M, max=1044.9M, avg=1044890682.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1045], 5.00th=[ 1045], 10.00th=[ 1045], 20.00th=[ 1045], | 30.00th=[ 1045], 40.00th=[ 1045], 50.00th=[ 1045], 60.00th=[ 1045], | 70.00th=[ 1045], 80.00th=[ 1045], 90.00th=[ 1045], 95.00th=[ 1045], | 99.00th=[ 1045], 99.50th=[ 1045], 99.90th=[ 1045], 99.95th=[ 1045], | 99.99th=[ 1045] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1771KiB/s (1813kB/s)(8192KiB/4626msec); 0 zone resets clat (msec): min=405, max=3162, avg=1783.86, stdev=1949.45 lat (msec): min=406, max=3162, avg=1784.92, stdev=1948.69 clat percentiles (msec): | 1.00th=[ 405], 5.00th=[ 405], 10.00th=[ 405], 20.00th=[ 405], | 30.00th=[ 405], 40.00th=[ 405], 50.00th=[ 405], 60.00th=[ 3171], | 70.00th=[ 3171], 80.00th=[ 3171], 90.00th=[ 3171], 95.00th=[ 3171], | 99.00th=[ 3171], 99.50th=[ 3171], 99.90th=[ 3171], 99.95th=[ 3171], | 99.99th=[ 3171] bw ( KiB/s): min= 8094, max= 8094, per=100.00%, avg=8094.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.06%, sys=2.36%, ctx=32, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147414: Sun Nov 16 21:31:28 2025 write: IOPS=0, BW=2461KiB/s (2520kB/s)(12.0MiB/4994msec); 0 zone resets clat (msec): min=692, max=3338, avg=1662.86, stdev=1457.25 lat (msec): min=693, max=3339, avg=1663.43, stdev=1457.24 clat percentiles (msec): | 1.00th=[ 693], 5.00th=[ 693], 10.00th=[ 693], 20.00th=[ 693], | 30.00th=[ 693], 40.00th=[ 961], 50.00th=[ 961], 60.00th=[ 961], | 70.00th=[ 3339], 80.00th=[ 3339], 90.00th=[ 3339], 95.00th=[ 3339], | 99.00th=[ 3339], 99.50th=[ 3339], 99.90th=[ 3339], 99.95th=[ 3339], | 99.99th=[ 3339] bw ( KiB/s): min= 8143, max= 8159, per=100.00%, avg=8151.00, stdev=11.31, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=3.46%, ctx=59, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147415: Sun Nov 16 21:31:28 2025 write: IOPS=0, BW=3120KiB/s (3195kB/s)(12.0MiB/3938msec); 0 zone resets clat (msec): min=323, max=3203, avg=1311.88, stdev=1639.09 lat (msec): min=324, max=3204, avg=1312.28, stdev=1639.18 clat percentiles (msec): | 1.00th=[ 326], 5.00th=[ 326], 10.00th=[ 326], 20.00th=[ 326], | 30.00th=[ 326], 40.00th=[ 409], 50.00th=[ 409], 60.00th=[ 409], | 70.00th=[ 3205], 80.00th=[ 3205], 90.00th=[ 3205], 95.00th=[ 3205], | 99.00th=[ 3205], 99.50th=[ 3205], 99.90th=[ 3205], 99.95th=[ 3205], | 99.99th=[ 3205] bw ( KiB/s): min= 7301, max= 7301, per=100.00%, avg=7301.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=3.68%, ctx=32, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3361KiB/s (3441kB/s), 885KiB/s-2521KiB/s (907kB/s-2581kB/s), io=16.0MiB (16.8MB), run=4626-4875msec WRITE: bw=6561KiB/s (6719kB/s), 1771KiB/s-3120KiB/s (1813kB/s-3195kB/s), io=32.0MiB (33.6MB), run=3938-4994msec rand-rw: (groupid=0, jobs=1): err= 0: pid=147407: Sun Nov 16 21:31:29 2025 read: IOPS=0, BW=2555KiB/s (2616kB/s)(12.0MiB/4810msec) clat (msec): min=634, max=3316, avg=1553.27, stdev=1527.09 lat (msec): min=634, max=3316, avg=1553.27, stdev=1527.09 clat percentiles (msec): | 1.00th=[ 634], 5.00th=[ 634], 10.00th=[ 634], 20.00th=[ 634], | 30.00th=[ 634], 40.00th=[ 709], 50.00th=[ 709], 60.00th=[ 709], | 70.00th=[ 3306], 80.00th=[ 3306], 90.00th=[ 3306], 95.00th=[ 3306], | 99.00th=[ 3306], 99.50th=[ 3306], 99.90th=[ 3306], 99.95th=[ 3306], | 99.99th=[ 3306] bw ( KiB/s): min= 8031, max= 8192, per=100.00%, avg=8111.50, stdev=113.84, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=9.50%, ctx=891, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147408: Sun Nov 16 21:31:29 2025 read: IOPS=0, BW=756KiB/s (774kB/s)(4096KiB/5419msec) clat (nsec): min=719570k, max=719570k, avg=719570280.00, stdev= 0.00 lat (nsec): min=719574k, max=719574k, avg=719573635.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 718], 5.00th=[ 718], 10.00th=[ 718], 20.00th=[ 718], | 30.00th=[ 718], 40.00th=[ 718], 50.00th=[ 718], 60.00th=[ 718], | 70.00th=[ 718], 80.00th=[ 718], 90.00th=[ 718], 95.00th=[ 718], | 99.00th=[ 718], 99.50th=[ 718], 99.90th=[ 718], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 8031, max= 8031, per=100.00%, avg=8031.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1512KiB/s (1548kB/s)(8192KiB/5419msec); 0 zone resets clat (msec): min=318, max=4266, avg=2292.71, stdev=2791.69 lat (msec): min=319, max=4267, avg=2293.07, stdev=2791.62 clat percentiles (msec): | 1.00th=[ 317], 5.00th=[ 317], 10.00th=[ 317], 20.00th=[ 317], | 30.00th=[ 317], 40.00th=[ 317], 50.00th=[ 317], 60.00th=[ 4279], | 70.00th=[ 4279], 80.00th=[ 4279], 90.00th=[ 4279], 95.00th=[ 4279], | 99.00th=[ 4279], 99.50th=[ 4279], 99.90th=[ 4279], 99.95th=[ 4279], | 99.99th=[ 4279] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=4.56%, ctx=250, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147409: Sun Nov 16 21:31:29 2025 write: IOPS=0, BW=2208KiB/s (2261kB/s)(12.0MiB/5565msec); 0 zone resets clat (msec): min=258, max=3289, avg=1804.00, stdev=1516.50 lat (msec): min=259, max=3290, avg=1804.62, stdev=1516.52 clat percentiles (msec): | 1.00th=[ 259], 5.00th=[ 259], 10.00th=[ 259], 20.00th=[ 259], | 30.00th=[ 259], 40.00th=[ 1871], 50.00th=[ 1871], 60.00th=[ 1871], | 70.00th=[ 3306], 80.00th=[ 3306], 90.00th=[ 3306], 95.00th=[ 3306], | 99.00th=[ 3306], 99.50th=[ 3306], 99.90th=[ 3306], 99.95th=[ 3306], | 99.99th=[ 3306] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.04%, sys=3.11%, ctx=55, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=147410: Sun Nov 16 21:31:29 2025 write: IOPS=1, BW=6440KiB/s (6595kB/s)(12.0MiB/1908msec); 0 zone resets clat (msec): min=344, max=1001, avg=585.09, stdev=361.79 lat (msec): min=344, max=1001, avg=585.47, stdev=361.82 clat percentiles (msec): | 1.00th=[ 347], 5.00th=[ 347], 10.00th=[ 347], 20.00th=[ 347], | 30.00th=[ 347], 40.00th=[ 409], 50.00th=[ 409], 60.00th=[ 409], | 70.00th=[ 1003], 80.00th=[ 1003], 90.00th=[ 1003], 95.00th=[ 1003], | 99.00th=[ 1003], 99.50th=[ 1003], 99.90th=[ 1003], 99.95th=[ 1003], | 99.99th=[ 1003] bw ( KiB/s): min= 7529, max= 8031, per=100.00%, avg=7780.00, stdev=354.97, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=5.93%, ctx=47, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3023KiB/s (3096kB/s), 756KiB/s-2555KiB/s (774kB/s-2616kB/s), io=16.0MiB (16.8MB), run=4810-5419msec WRITE: bw=5888KiB/s (6030kB/s), 1512KiB/s-6440KiB/s (1548kB/s-6595kB/s), io=32.0MiB (33.6MB), run=1908-5565msec