== sanity test 398b: DIO and buffer IO race ============== 16:40:28 (1763156428) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.76658 s, 13.4 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148654: Fri Nov 14 16:42:03 2025 read: IOPS=18, BW=73.7KiB/s (75.5kB/s)(5992KiB/81309msec) clat (usec): min=100, max=720776, avg=26600.93, stdev=29626.31 lat (usec): min=100, max=720777, avg=26602.90, stdev=29626.15 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 174], 10.00th=[ 326], 20.00th=[ 12649], | 30.00th=[ 15926], 40.00th=[ 19006], 50.00th=[ 21890], 60.00th=[ 26084], | 70.00th=[ 31327], 80.00th=[ 38011], 90.00th=[ 49021], 95.00th=[ 60556], | 99.00th=[ 96994], 99.50th=[132645], 99.90th=[396362], 99.95th=[717226], | 99.99th=[717226] bw ( KiB/s): min= 7, max= 143, per=25.17%, avg=73.25, stdev=26.28, samples=161 iops : min= 1, max= 35, avg=17.71, stdev= 6.61, samples=161 write: IOPS=19, BW=77.4KiB/s (79.3kB/s)(6296KiB/81309msec); 0 zone resets clat (usec): min=793, max=200345, avg=26133.90, stdev=26059.94 lat (usec): min=793, max=200347, avg=26143.77, stdev=26075.79 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1188], 10.00th=[ 1434], 20.00th=[ 2671], | 30.00th=[ 5473], 40.00th=[ 11600], 50.00th=[ 19530], 60.00th=[ 26608], | 70.00th=[ 35390], 80.00th=[ 46924], 90.00th=[ 61604], 95.00th=[ 77071], | 99.00th=[106431], 99.50th=[116917], 99.90th=[158335], 99.95th=[200279], | 99.99th=[200279] bw ( KiB/s): min= 8, max= 173, per=25.43%, avg=77.31, stdev=28.27, samples=160 iops : min= 2, max= 43, avg=18.74, stdev= 7.12, samples=160 lat (usec) : 250=3.97%, 500=1.63%, 750=0.33%, 1000=0.42% lat (msec) : 2=7.52%, 4=5.86%, 10=7.16%, 20=20.02%, 50=39.65% lat (msec) : 100=12.27%, 250=1.04%, 500=0.10%, 750=0.03% cpu : usr=0.10%, sys=9.91%, ctx=6584, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148655: Fri Nov 14 16:42:03 2025 read: IOPS=17, BW=71.0KiB/s (73.7kB/s)(5940KiB/82505msec) clat (usec): min=91, max=784420, avg=27384.55, stdev=28885.93 lat (usec): min=92, max=784423, avg=27386.92, stdev=28885.20 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 200], 10.00th=[ 644], 20.00th=[ 13042], | 30.00th=[ 16057], 40.00th=[ 19268], 50.00th=[ 22414], 60.00th=[ 26870], | 70.00th=[ 33162], 80.00th=[ 40109], 90.00th=[ 50594], 95.00th=[ 62129], | 99.00th=[102237], 99.50th=[139461], 99.90th=[204473], 99.95th=[784335], | 99.99th=[784335] bw ( KiB/s): min= 8, max= 150, per=24.82%, avg=72.22, stdev=25.02, samples=160 iops : min= 2, max= 37, avg=17.46, stdev= 6.28, samples=160 write: IOPS=19, BW=76.9KiB/s (78.8kB/s)(6348KiB/82505msec); 0 zone resets clat (usec): min=929, max=240381, avg=26141.85, stdev=26929.16 lat (usec): min=930, max=240383, avg=26147.29, stdev=26930.28 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1156], 10.00th=[ 1352], 20.00th=[ 2638], | 30.00th=[ 5997], 40.00th=[ 12649], 50.00th=[ 20055], 60.00th=[ 26084], | 70.00th=[ 33817], 80.00th=[ 44303], 90.00th=[ 60556], 95.00th=[ 76022], | 99.00th=[120062], 99.50th=[137364], 99.90th=[193987], 99.95th=[240124], | 99.99th=[240124] bw ( KiB/s): min= 15, max= 159, per=25.05%, avg=76.14, stdev=27.57, samples=161 iops : min= 3, max= 39, avg=18.43, stdev= 6.93, samples=161 lat (usec) : 100=0.03%, 250=3.09%, 500=1.60%, 750=0.20%, 1000=0.33% lat (msec) : 2=7.91%, 4=5.44%, 10=7.55%, 20=20.28%, 50=40.43% lat (msec) : 100=11.43%, 250=1.69%, 1000=0.03% cpu : usr=0.12%, sys=9.82%, ctx=6712, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148656: Fri Nov 14 16:42:03 2025 read: IOPS=18, BW=73.3KiB/s (75.1kB/s)(6028KiB/82228msec) clat (usec): min=84, max=723563, avg=26276.42, stdev=26926.34 lat (usec): min=84, max=723565, avg=26277.80, stdev=26926.29 clat percentiles (usec): | 1.00th=[ 133], 5.00th=[ 167], 10.00th=[ 277], 20.00th=[ 11994], | 30.00th=[ 15533], 40.00th=[ 18744], 50.00th=[ 22676], 60.00th=[ 26870], | 70.00th=[ 31851], 80.00th=[ 38536], 90.00th=[ 49021], 95.00th=[ 59507], | 99.00th=[ 88605], 99.50th=[143655], 99.90th=[187696], 99.95th=[725615], | 99.99th=[725615] bw ( KiB/s): min= 7, max= 175, per=25.15%, avg=73.19, stdev=26.21, samples=162 iops : min= 1, max= 43, avg=17.69, stdev= 6.57, samples=162 write: IOPS=19, BW=76.1KiB/s (77.0kB/s)(6260KiB/82228msec); 0 zone resets clat (usec): min=891, max=295366, avg=27026.19, stdev=28093.58 lat (usec): min=892, max=295368, avg=27032.29, stdev=28095.12 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1172], 10.00th=[ 1401], 20.00th=[ 2966], | 30.00th=[ 6194], 40.00th=[ 13566], 50.00th=[ 20841], 60.00th=[ 28181], | 70.00th=[ 36439], 80.00th=[ 45876], 90.00th=[ 60031], 95.00th=[ 76022], | 99.00th=[112722], 99.50th=[152044], 99.90th=[291505], 99.95th=[295699], | 99.99th=[295699] bw ( KiB/s): min= 8, max= 191, per=24.84%, avg=75.53, stdev=27.93, samples=163 iops : min= 2, max= 47, avg=18.28, stdev= 6.99, samples=163 lat (usec) : 100=0.07%, 250=4.52%, 500=1.40%, 750=0.39%, 1000=0.36% lat (msec) : 2=7.10%, 4=4.59%, 10=7.78%, 20=20.41%, 50=40.59% lat (msec) : 100=11.56%, 250=1.14%, 500=0.07%, 750=0.03% cpu : usr=0.11%, sys=10.05%, ctx=6612, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148657: Fri Nov 14 16:42:03 2025 read: IOPS=18, BW=74.2KiB/s (75.0kB/s)(6052KiB/81556msec) clat (usec): min=95, max=220355, avg=24875.95, stdev=19359.04 lat (usec): min=95, max=220356, avg=24878.13, stdev=19358.97 clat percentiles (usec): | 1.00th=[ 135], 5.00th=[ 182], 10.00th=[ 338], 20.00th=[ 11994], | 30.00th=[ 15533], 40.00th=[ 18482], 50.00th=[ 21627], 60.00th=[ 25297], | 70.00th=[ 30540], 80.00th=[ 36963], 90.00th=[ 46400], 95.00th=[ 56886], | 99.00th=[ 89654], 99.50th=[107480], 99.90th=[214959], 99.95th=[221250], | 99.99th=[221250] bw ( KiB/s): min= 7, max= 135, per=25.63%, avg=74.58, stdev=26.57, samples=159 iops : min= 1, max= 33, avg=18.04, stdev= 6.65, samples=159 write: IOPS=19, BW=76.5KiB/s (78.3kB/s)(6236KiB/81556msec); 0 zone resets clat (usec): min=809, max=762321, avg=27965.88, stdev=33083.45 lat (usec): min=811, max=762323, avg=27967.45, stdev=33083.47 clat percentiles (usec): | 1.00th=[ 1020], 5.00th=[ 1188], 10.00th=[ 1450], 20.00th=[ 2737], | 30.00th=[ 6259], 40.00th=[ 14615], 50.00th=[ 22152], 60.00th=[ 28181], | 70.00th=[ 36439], 80.00th=[ 46400], 90.00th=[ 62129], 95.00th=[ 78119], | 99.00th=[122160], 99.50th=[143655], 99.90th=[274727], 99.95th=[759170], | 99.99th=[759170] bw ( KiB/s): min= 14, max= 191, per=24.96%, avg=75.88, stdev=26.81, samples=161 iops : min= 3, max= 47, avg=18.38, stdev= 6.74, samples=161 lat (usec) : 100=0.03%, 250=3.74%, 500=1.99%, 750=0.29%, 1000=0.46% lat (msec) : 2=7.58%, 4=5.11%, 10=6.61%, 20=20.57%, 50=40.95% lat (msec) : 100=11.13%, 250=1.46%, 500=0.03%, 1000=0.03% cpu : usr=0.10%, sys=9.98%, ctx=6675, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=291KiB/s (298kB/s), 71.0KiB/s-74.2KiB/s (73.7kB/s-75.0kB/s), io=23.4MiB (24.6MB), run=81309-82505msec WRITE: bw=305KiB/s (312kB/s), 76.1KiB/s-77.4KiB/s (77.0kB/s-79.3kB/s), io=24.6MiB (25.7MB), run=81309-82505msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148650: Fri Nov 14 16:43:50 2025 read: IOPS=7, BW=31.9KiB/s (32.7kB/s)(5992KiB/187622msec) clat (msec): min=4, max=742, avg=27.56, stdev=27.32 lat (msec): min=4, max=742, avg=27.56, stdev=27.32 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 20], 50.00th=[ 24], 60.00th=[ 29], | 70.00th=[ 33], 80.00th=[ 38], 90.00th=[ 48], 95.00th=[ 62], | 99.00th=[ 99], 99.50th=[ 138], 99.90th=[ 239], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 7, max= 111, per=26.59%, avg=33.76, stdev=20.51, samples=347 iops : min= 1, max= 27, avg= 7.87, stdev= 5.14, samples=347 write: IOPS=8, BW=33.6KiB/s (34.4kB/s)(6296KiB/187622msec); 0 zone resets clat (msec): min=31, max=399, avg=92.76, stdev=43.07 lat (msec): min=31, max=399, avg=92.76, stdev=43.07 clat percentiles (msec): | 1.00th=[ 38], 5.00th=[ 48], 10.00th=[ 53], 20.00th=[ 60], | 30.00th=[ 67], 40.00th=[ 74], 50.00th=[ 82], 60.00th=[ 89], | 70.00th=[ 103], 80.00th=[ 121], 90.00th=[ 155], 95.00th=[ 180], | 99.00th=[ 230], 99.50th=[ 271], 99.90th=[ 338], 99.95th=[ 401], | 99.99th=[ 401] bw ( KiB/s): min= 7, max= 71, per=24.92%, avg=33.14, stdev=13.35, samples=371 iops : min= 1, max= 17, avg= 7.71, stdev= 3.41, samples=371 lat (msec) : 10=7.06%, 20=13.51%, 50=27.60%, 100=35.45%, 250=15.95% lat (msec) : 500=0.39%, 750=0.03% cpu : usr=0.05%, sys=2.32%, ctx=6157, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148651: Fri Nov 14 16:43:50 2025 read: IOPS=7, BW=31.5KiB/s (32.2kB/s)(5940KiB/188634msec) clat (msec): min=4, max=782, avg=28.17, stdev=30.37 lat (msec): min=4, max=782, avg=28.17, stdev=30.37 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 19], 50.00th=[ 24], 60.00th=[ 29], | 70.00th=[ 33], 80.00th=[ 40], 90.00th=[ 50], 95.00th=[ 64], | 99.00th=[ 108], 99.50th=[ 123], 99.90th=[ 347], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 7, max= 111, per=26.35%, avg=33.47, stdev=20.21, samples=347 iops : min= 1, max= 27, avg= 7.80, stdev= 5.10, samples=347 write: IOPS=8, BW=33.7KiB/s (34.5kB/s)(6348KiB/188634msec); 0 zone resets clat (msec): min=25, max=344, avg=92.29, stdev=44.17 lat (msec): min=25, max=344, avg=92.29, stdev=44.17 clat percentiles (msec): | 1.00th=[ 35], 5.00th=[ 46], 10.00th=[ 52], 20.00th=[ 59], | 30.00th=[ 66], 40.00th=[ 73], 50.00th=[ 81], 60.00th=[ 89], | 70.00th=[ 101], 80.00th=[ 121], 90.00th=[ 153], 95.00th=[ 184], | 99.00th=[ 245], 99.50th=[ 266], 99.90th=[ 342], 99.95th=[ 347], | 99.99th=[ 347] bw ( KiB/s): min= 7, max= 87, per=24.99%, avg=33.24, stdev=13.82, samples=373 iops : min= 1, max= 21, avg= 7.73, stdev= 3.51, samples=373 lat (msec) : 10=7.39%, 20=13.28%, 50=27.57%, 100=35.61%, 250=15.63% lat (msec) : 500=0.49%, 1000=0.03% cpu : usr=0.05%, sys=2.30%, ctx=6041, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148652: Fri Nov 14 16:43:50 2025 read: IOPS=8, BW=32.3KiB/s (33.0kB/s)(6028KiB/186906msec) clat (msec): min=4, max=285, avg=27.81, stdev=22.48 lat (msec): min=4, max=285, avg=27.82, stdev=22.48 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 19], 50.00th=[ 24], 60.00th=[ 28], | 70.00th=[ 33], 80.00th=[ 40], 90.00th=[ 51], 95.00th=[ 64], | 99.00th=[ 101], 99.50th=[ 126], 99.90th=[ 266], 99.95th=[ 288], | 99.99th=[ 288] bw ( KiB/s): min= 7, max= 135, per=26.02%, avg=33.04, stdev=20.00, samples=356 iops : min= 1, max= 33, avg= 7.70, stdev= 5.03, samples=356 write: IOPS=8, BW=33.5KiB/s (34.3kB/s)(6260KiB/186906msec); 0 zone resets clat (msec): min=30, max=947, avg=92.41, stdev=48.30 lat (msec): min=30, max=947, avg=92.41, stdev=48.30 clat percentiles (msec): | 1.00th=[ 38], 5.00th=[ 47], 10.00th=[ 51], 20.00th=[ 59], | 30.00th=[ 66], 40.00th=[ 73], 50.00th=[ 80], 60.00th=[ 89], | 70.00th=[ 101], 80.00th=[ 121], 90.00th=[ 153], 95.00th=[ 180], | 99.00th=[ 236], 99.50th=[ 259], 99.90th=[ 393], 99.95th=[ 944], | 99.99th=[ 944] bw ( KiB/s): min= 7, max= 72, per=25.02%, avg=33.28, stdev=13.91, samples=367 iops : min= 1, max= 18, avg= 7.75, stdev= 3.54, samples=367 lat (msec) : 10=7.65%, 20=13.38%, 50=27.80%, 100=35.45%, 250=15.30% lat (msec) : 500=0.39%, 1000=0.03% cpu : usr=0.06%, sys=2.27%, ctx=6037, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148653: Fri Nov 14 16:43:50 2025 read: IOPS=8, BW=32.3KiB/s (33.1kB/s)(6052KiB/187388msec) clat (msec): min=4, max=344, avg=27.30, stdev=23.54 lat (msec): min=4, max=344, avg=27.30, stdev=23.54 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 18], 50.00th=[ 24], 60.00th=[ 28], | 70.00th=[ 33], 80.00th=[ 39], 90.00th=[ 49], 95.00th=[ 63], | 99.00th=[ 105], 99.50th=[ 144], 99.90th=[ 317], 99.95th=[ 347], | 99.99th=[ 347] bw ( KiB/s): min= 7, max= 103, per=26.43%, avg=33.56, stdev=20.25, samples=352 iops : min= 1, max= 25, avg= 7.82, stdev= 5.10, samples=352 write: IOPS=8, BW=33.3KiB/s (34.1kB/s)(6236KiB/187388msec); 0 zone resets clat (msec): min=34, max=855, avg=93.48, stdev=47.18 lat (msec): min=34, max=855, avg=93.48, stdev=47.18 clat percentiles (msec): | 1.00th=[ 40], 5.00th=[ 47], 10.00th=[ 53], 20.00th=[ 60], | 30.00th=[ 67], 40.00th=[ 74], 50.00th=[ 82], 60.00th=[ 90], | 70.00th=[ 102], 80.00th=[ 122], 90.00th=[ 157], 95.00th=[ 180], | 99.00th=[ 243], 99.50th=[ 275], 99.90th=[ 359], 99.95th=[ 852], | 99.99th=[ 852] bw ( KiB/s): min= 7, max= 64, per=24.68%, avg=32.83, stdev=13.56, samples=370 iops : min= 1, max= 16, avg= 7.63, stdev= 3.47, samples=370 lat (msec) : 10=7.98%, 20=13.74%, 50=26.60%, 100=35.58%, 250=15.66% lat (msec) : 500=0.42%, 1000=0.03% cpu : usr=0.05%, sys=2.25%, ctx=6096, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=127KiB/s (130kB/s), 31.5KiB/s-32.3KiB/s (32.2kB/s-33.1kB/s), io=23.4MiB (24.6MB), run=186906-188634msec WRITE: bw=133KiB/s (136kB/s), 33.3KiB/s-33.7KiB/s (34.1kB/s-34.5kB/s), io=24.6MiB (25.7MB), run=186906-188634msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148699: Fri Nov 14 16:44:11 2025 read: IOPS=24, BW=395KiB/s (404kB/s)(5920KiB/15003msec) clat (usec): min=109, max=90642, avg=19166.57, stdev=15785.05 lat (usec): min=109, max=90643, avg=19167.88, stdev=15785.10 clat percentiles (usec): | 1.00th=[ 149], 5.00th=[ 188], 10.00th=[ 245], 20.00th=[ 644], | 30.00th=[11863], 40.00th=[15139], 50.00th=[17695], 60.00th=[20579], | 70.00th=[25297], 80.00th=[29492], 90.00th=[38011], 95.00th=[45351], | 99.00th=[73925], 99.50th=[80217], 99.90th=[90702], 99.95th=[90702], | 99.99th=[90702] bw ( KiB/s): min= 252, max= 670, per=25.67%, avg=392.48, stdev=105.20, samples=29 iops : min= 15, max= 41, avg=23.86, stdev= 6.60, samples=29 write: IOPS=26, BW=424KiB/s (435kB/s)(6368KiB/15003msec); 0 zone resets clat (usec): min=1216, max=119005, avg=19421.07, stdev=18984.56 lat (usec): min=1218, max=119007, avg=19423.48, stdev=18984.34 clat percentiles (usec): | 1.00th=[ 1467], 5.00th=[ 1827], 10.00th=[ 2802], 20.00th=[ 4424], | 30.00th=[ 6063], 40.00th=[ 8586], 50.00th=[ 13960], 60.00th=[ 18220], | 70.00th=[ 23987], 80.00th=[ 32900], 90.00th=[ 44303], 95.00th=[ 57410], | 99.00th=[ 96994], 99.50th=[112722], 99.90th=[119014], 99.95th=[119014], | 99.99th=[119014] bw ( KiB/s): min= 159, max= 702, per=26.55%, avg=427.21, stdev=119.10, samples=29 iops : min= 9, max= 43, avg=26.03, stdev= 7.38, samples=29 lat (usec) : 250=5.08%, 500=4.17%, 750=0.52%, 1000=0.65% lat (msec) : 2=3.91%, 4=7.16%, 10=14.45%, 20=24.48%, 50=33.72% lat (msec) : 100=5.47%, 250=0.39% cpu : usr=0.14%, sys=17.69%, ctx=1709, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148700: Fri Nov 14 16:44:11 2025 read: IOPS=22, BW=367KiB/s (376kB/s)(5744KiB/15656msec) clat (usec): min=118, max=128119, avg=19818.83, stdev=15910.66 lat (usec): min=119, max=128120, avg=19820.05, stdev=15910.61 clat percentiles (usec): | 1.00th=[ 133], 5.00th=[ 182], 10.00th=[ 269], 20.00th=[ 437], | 30.00th=[ 12518], 40.00th=[ 15926], 50.00th=[ 18744], 60.00th=[ 22152], | 70.00th=[ 27132], 80.00th=[ 32375], 90.00th=[ 39584], 95.00th=[ 47449], | 99.00th=[ 57934], 99.50th=[ 66323], 99.90th=[128451], 99.95th=[128451], | 99.99th=[128451] bw ( KiB/s): min= 26, max= 637, per=23.75%, avg=363.20, stdev=146.95, samples=30 iops : min= 1, max= 39, avg=22.00, stdev= 9.22, samples=30 write: IOPS=26, BW=418KiB/s (428kB/s)(6544KiB/15656msec); 0 zone resets clat (usec): min=1110, max=185634, avg=20498.80, stdev=25282.11 lat (usec): min=1112, max=185636, avg=20504.02, stdev=25281.88 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 1713], 10.00th=[ 2180], 20.00th=[ 3654], | 30.00th=[ 5145], 40.00th=[ 7177], 50.00th=[ 11076], 60.00th=[ 16909], | 70.00th=[ 23725], 80.00th=[ 32900], 90.00th=[ 48497], 95.00th=[ 60031], | 99.00th=[127402], 99.50th=[149947], 99.90th=[185598], 99.95th=[185598], | 99.99th=[185598] bw ( KiB/s): min= 95, max= 768, per=25.23%, avg=405.97, stdev=154.86, samples=30 iops : min= 5, max= 48, avg=24.67, stdev= 9.78, samples=30 lat (usec) : 250=4.17%, 500=5.99%, 750=0.91% lat (msec) : 2=4.17%, 4=8.46%, 10=14.45%, 20=22.14%, 50=33.59% lat (msec) : 100=4.69%, 250=1.43% cpu : usr=0.14%, sys=17.02%, ctx=1647, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148701: Fri Nov 14 16:44:11 2025 read: IOPS=26, BW=421KiB/s (431kB/s)(6256KiB/14867msec) clat (usec): min=118, max=105282, avg=19361.07, stdev=14573.41 lat (usec): min=118, max=105283, avg=19383.73, stdev=14594.18 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 186], 10.00th=[ 330], 20.00th=[ 1156], | 30.00th=[ 13304], 40.00th=[ 16188], 50.00th=[ 18744], 60.00th=[ 21627], | 70.00th=[ 26346], 80.00th=[ 29492], 90.00th=[ 34866], 95.00th=[ 44827], | 99.00th=[ 67634], 99.50th=[ 86508], 99.90th=[105382], 99.95th=[105382], | 99.99th=[105382] bw ( KiB/s): min= 222, max= 669, per=27.86%, avg=426.00, stdev=106.23, samples=29 iops : min= 13, max= 41, avg=25.97, stdev= 6.65, samples=29 write: IOPS=25, BW=406KiB/s (415kB/s)(6032KiB/14867msec); 0 zone resets clat (usec): min=1431, max=179989, avg=19205.54, stdev=18885.07 lat (usec): min=1433, max=179993, avg=19207.56, stdev=18885.13 clat percentiles (usec): | 1.00th=[ 1532], 5.00th=[ 2311], 10.00th=[ 3032], 20.00th=[ 4424], | 30.00th=[ 6325], 40.00th=[ 8979], 50.00th=[ 13960], 60.00th=[ 18482], | 70.00th=[ 23462], 80.00th=[ 30802], 90.00th=[ 44303], 95.00th=[ 51119], | 99.00th=[ 87557], 99.50th=[ 94897], 99.90th=[179307], 99.95th=[179307], | 99.99th=[179307] bw ( KiB/s): min= 191, max= 736, per=25.16%, avg=404.86, stdev=126.48, samples=29 iops : min= 11, max= 46, avg=24.62, stdev= 7.98, samples=29 lat (usec) : 250=3.52%, 500=4.56%, 750=1.56%, 1000=0.39% lat (msec) : 2=2.73%, 4=7.16%, 10=12.89%, 20=26.17%, 50=37.37% lat (msec) : 100=3.39%, 250=0.26% cpu : usr=0.11%, sys=17.74%, ctx=1763, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148702: Fri Nov 14 16:44:11 2025 read: IOPS=24, BW=394KiB/s (403kB/s)(6032KiB/15327msec) clat (usec): min=119, max=86259, avg=19535.17, stdev=15101.09 lat (usec): min=119, max=86260, avg=19537.54, stdev=15099.80 clat percentiles (usec): | 1.00th=[ 145], 5.00th=[ 180], 10.00th=[ 293], 20.00th=[ 660], | 30.00th=[12256], 40.00th=[15270], 50.00th=[18482], 60.00th=[21627], | 70.00th=[26346], 80.00th=[31589], 90.00th=[40109], 95.00th=[46400], | 99.00th=[57934], 99.50th=[77071], 99.90th=[86508], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 63, max= 736, per=26.60%, avg=406.72, stdev=121.30, samples=29 iops : min= 3, max= 46, avg=24.72, stdev= 7.75, samples=29 write: IOPS=25, BW=408KiB/s (418kB/s)(6256KiB/15327msec); 0 zone resets clat (usec): min=1350, max=148946, avg=19849.02, stdev=21297.39 lat (usec): min=1352, max=148950, avg=19852.24, stdev=21296.70 clat percentiles (usec): | 1.00th=[ 1434], 5.00th=[ 1958], 10.00th=[ 2507], 20.00th=[ 3949], | 30.00th=[ 5211], 40.00th=[ 7898], 50.00th=[ 13173], 60.00th=[ 17957], | 70.00th=[ 24773], 80.00th=[ 33424], 90.00th=[ 46924], 95.00th=[ 54789], | 99.00th=[114820], 99.50th=[147850], 99.90th=[149947], 99.95th=[149947], | 99.99th=[149947] bw ( KiB/s): min= 80, max= 764, per=25.44%, avg=409.30, stdev=127.03, samples=30 iops : min= 5, max= 47, avg=24.93, stdev= 7.90, samples=30 lat (usec) : 250=3.52%, 500=5.21%, 750=1.43%, 1000=0.39% lat (msec) : 2=3.52%, 4=7.55%, 10=14.58%, 20=23.05%, 50=35.16% lat (msec) : 100=4.95%, 250=0.65% cpu : usr=0.12%, sys=17.53%, ctx=1672, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1530KiB/s (1567kB/s), 367KiB/s-421KiB/s (376kB/s-431kB/s), io=23.4MiB (24.5MB), run=14867-15656msec WRITE: bw=1610KiB/s (1648kB/s), 406KiB/s-424KiB/s (415kB/s-435kB/s), io=24.6MiB (25.8MB), run=14867-15656msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148703: Fri Nov 14 16:44:38 2025 read: IOPS=8, BW=140KiB/s (143kB/s)(5920KiB/42409msec) clat (msec): min=5, max=232, avg=21.24, stdev=19.98 lat (msec): min=5, max=232, avg=21.24, stdev=19.98 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 21], 80.00th=[ 26], 90.00th=[ 37], 95.00th=[ 56], | 99.00th=[ 97], 99.50th=[ 108], 99.90th=[ 232], 99.95th=[ 232], | 99.99th=[ 232] bw ( KiB/s): min= 31, max= 479, per=25.88%, avg=145.99, stdev=84.49, samples=80 iops : min= 1, max= 29, avg= 8.46, stdev= 5.35, samples=80 write: IOPS=9, BW=150KiB/s (154kB/s)(6368KiB/42409msec); 0 zone resets clat (msec): min=24, max=210, avg=86.46, stdev=33.88 lat (msec): min=24, max=210, avg=86.46, stdev=33.88 clat percentiles (msec): | 1.00th=[ 34], 5.00th=[ 45], 10.00th=[ 51], 20.00th=[ 59], | 30.00th=[ 66], 40.00th=[ 71], 50.00th=[ 80], 60.00th=[ 89], | 70.00th=[ 97], 80.00th=[ 110], 90.00th=[ 136], 95.00th=[ 155], | 99.00th=[ 190], 99.50th=[ 201], 99.90th=[ 211], 99.95th=[ 211], | 99.99th=[ 211] bw ( KiB/s): min= 53, max= 256, per=25.22%, avg=149.81, stdev=50.51, samples=84 iops : min= 3, max= 16, avg= 8.70, stdev= 3.17, samples=84 lat (msec) : 10=8.59%, 20=24.74%, 50=16.93%, 100=35.81%, 250=13.93% cpu : usr=0.04%, sys=2.83%, ctx=1536, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148704: Fri Nov 14 16:44:38 2025 read: IOPS=8, BW=136KiB/s (139kB/s)(5744KiB/42307msec) clat (msec): min=5, max=230, avg=21.22, stdev=19.14 lat (msec): min=5, max=230, avg=21.22, stdev=19.14 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 12], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 29], 90.00th=[ 41], 95.00th=[ 55], | 99.00th=[ 97], 99.50th=[ 102], 99.90th=[ 230], 99.95th=[ 230], | 99.99th=[ 230] bw ( KiB/s): min= 31, max= 416, per=25.27%, avg=142.54, stdev=84.83, samples=80 iops : min= 1, max= 26, avg= 8.25, stdev= 5.42, samples=80 write: IOPS=9, BW=155KiB/s (158kB/s)(6544KiB/42307msec); 0 zone resets clat (msec): min=31, max=185, avg=84.60, stdev=30.00 lat (msec): min=31, max=185, avg=84.61, stdev=30.00 clat percentiles (msec): | 1.00th=[ 37], 5.00th=[ 45], 10.00th=[ 50], 20.00th=[ 57], | 30.00th=[ 65], 40.00th=[ 71], 50.00th=[ 82], 60.00th=[ 90], | 70.00th=[ 100], 80.00th=[ 110], 90.00th=[ 125], 95.00th=[ 142], | 99.00th=[ 167], 99.50th=[ 174], 99.90th=[ 186], 99.95th=[ 186], | 99.99th=[ 186] bw ( KiB/s): min= 62, max= 286, per=25.95%, avg=154.17, stdev=49.47, samples=84 iops : min= 3, max= 17, avg= 8.98, stdev= 3.14, samples=84 lat (msec) : 10=8.98%, 20=22.01%, 50=18.36%, 100=34.77%, 250=15.89% cpu : usr=0.08%, sys=2.81%, ctx=1428, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148705: Fri Nov 14 16:44:38 2025 read: IOPS=9, BW=153KiB/s (156kB/s)(6256KiB/40979msec) clat (msec): min=5, max=167, avg=22.38, stdev=19.08 lat (msec): min=5, max=167, avg=22.38, stdev=19.08 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 23], 80.00th=[ 29], 90.00th=[ 42], 95.00th=[ 53], | 99.00th=[ 97], 99.50th=[ 161], 99.90th=[ 167], 99.95th=[ 167], | 99.99th=[ 167] bw ( KiB/s): min= 31, max= 446, per=27.20%, avg=153.42, stdev=90.61, samples=81 iops : min= 1, max= 27, avg= 8.93, stdev= 5.69, samples=81 write: IOPS=9, BW=147KiB/s (151kB/s)(6032KiB/40979msec); 0 zone resets clat (msec): min=29, max=207, avg=85.08, stdev=30.57 lat (msec): min=29, max=207, avg=85.08, stdev=30.57 clat percentiles (msec): | 1.00th=[ 38], 5.00th=[ 45], 10.00th=[ 51], 20.00th=[ 59], | 30.00th=[ 66], 40.00th=[ 74], 50.00th=[ 81], 60.00th=[ 88], | 70.00th=[ 99], 80.00th=[ 110], 90.00th=[ 127], 95.00th=[ 140], | 99.00th=[ 174], 99.50th=[ 190], 99.90th=[ 207], 99.95th=[ 207], | 99.99th=[ 207] bw ( KiB/s): min= 63, max= 286, per=24.62%, avg=146.27, stdev=45.69, samples=81 iops : min= 3, max= 17, avg= 8.48, stdev= 2.90, samples=81 lat (msec) : 10=6.77%, 20=25.00%, 50=20.70%, 100=33.72%, 250=13.80% cpu : usr=0.07%, sys=2.95%, ctx=1527, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148706: Fri Nov 14 16:44:38 2025 read: IOPS=8, BW=143KiB/s (146kB/s)(6032KiB/42168msec) clat (msec): min=5, max=236, avg=22.72, stdev=22.25 lat (msec): min=5, max=236, avg=22.72, stdev=22.25 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 14], 50.00th=[ 16], 60.00th=[ 19], | 70.00th=[ 22], 80.00th=[ 27], 90.00th=[ 43], 95.00th=[ 68], | 99.00th=[ 116], 99.50th=[ 142], 99.90th=[ 236], 99.95th=[ 236], | 99.99th=[ 236] bw ( KiB/s): min= 31, max= 416, per=25.69%, avg=144.90, stdev=81.86, samples=81 iops : min= 1, max= 26, avg= 8.38, stdev= 5.16, samples=81 write: IOPS=9, BW=148KiB/s (152kB/s)(6256KiB/42168msec); 0 zone resets clat (msec): min=32, max=223, avg=85.48, stdev=32.93 lat (msec): min=32, max=223, avg=85.49, stdev=32.93 clat percentiles (msec): | 1.00th=[ 37], 5.00th=[ 43], 10.00th=[ 50], 20.00th=[ 57], | 30.00th=[ 66], 40.00th=[ 71], 50.00th=[ 79], 60.00th=[ 89], | 70.00th=[ 100], 80.00th=[ 111], 90.00th=[ 132], 95.00th=[ 153], | 99.00th=[ 171], 99.50th=[ 207], 99.90th=[ 224], 99.95th=[ 224], | 99.99th=[ 224] bw ( KiB/s): min= 32, max= 255, per=24.80%, avg=147.34, stdev=50.06, samples=83 iops : min= 2, max= 15, avg= 8.54, stdev= 3.12, samples=83 lat (msec) : 10=8.59%, 20=23.18%, 50=19.66%, 100=33.20%, 250=15.36% cpu : usr=0.05%, sys=3.04%, ctx=1556, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=565KiB/s (578kB/s), 136KiB/s-153KiB/s (139kB/s-156kB/s), io=23.4MiB (24.5MB), run=40979-42409msec WRITE: bw=594KiB/s (608kB/s), 147KiB/s-155KiB/s (151kB/s-158kB/s), io=24.6MiB (25.8MB), run=40979-42409msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148720: Fri Nov 14 16:44:48 2025 read: IOPS=1, BW=1509KiB/s (1546kB/s)(7168KiB/4749msec) clat (usec): min=994, max=2645.7k, avg=594467.15, stdev=963850.77 lat (usec): min=995, max=2645.7k, avg=594468.55, stdev=963851.19 clat percentiles (usec): | 1.00th=[ 996], 5.00th=[ 996], 10.00th=[ 996], | 20.00th=[ 68682], 30.00th=[ 95945], 40.00th=[ 95945], | 50.00th=[ 177210], 60.00th=[ 187696], 70.00th=[ 187696], | 80.00th=[ 985662], 90.00th=[2634023], 95.00th=[2634023], | 99.00th=[2634023], 99.50th=[2634023], 99.90th=[2634023], | 99.95th=[2634023], 99.99th=[2634023] bw ( KiB/s): min= 2031, max= 6119, per=82.59%, avg=3056.00, stdev=2042.00, samples=4 iops : min= 1, max= 5, avg= 2.00, stdev= 2.00, samples=4 write: IOPS=1, BW=1078KiB/s (1104kB/s)(5120KiB/4749msec); 0 zone resets clat (msec): min=49, max=157, avg=102.28, stdev=41.25 lat (msec): min=49, max=157, avg=102.40, stdev=41.20 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 51], 10.00th=[ 51], 20.00th=[ 51], | 30.00th=[ 80], 40.00th=[ 80], 50.00th=[ 102], 60.00th=[ 102], | 70.00th=[ 124], 80.00th=[ 124], 90.00th=[ 159], 95.00th=[ 159], | 99.00th=[ 159], 99.50th=[ 159], 99.90th=[ 159], 99.95th=[ 159], | 99.99th=[ 159] bw ( KiB/s): min= 2039, max= 8143, per=98.28%, avg=5091.00, stdev=4316.18, samples=2 iops : min= 1, max= 7, avg= 4.00, stdev= 4.24, samples=2 lat (usec) : 1000=8.33% lat (msec) : 50=8.33%, 100=25.00%, 250=41.67%, 1000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=18.22%, ctx=763, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148721: Fri Nov 14 16:44:48 2025 read: IOPS=0, BW=399KiB/s (409kB/s)(2048KiB/5129msec) clat (msec): min=121, max=136, avg=128.89, stdev=10.66 lat (msec): min=121, max=136, avg=128.89, stdev=10.66 clat percentiles (msec): | 1.00th=[ 122], 5.00th=[ 122], 10.00th=[ 122], 20.00th=[ 122], | 30.00th=[ 122], 40.00th=[ 122], 50.00th=[ 122], 60.00th=[ 138], | 70.00th=[ 138], 80.00th=[ 138], 90.00th=[ 138], 95.00th=[ 138], | 99.00th=[ 138], 99.50th=[ 138], 99.90th=[ 138], 99.95th=[ 138], | 99.99th=[ 138] bw ( KiB/s): min= 1774, max= 2043, per=51.58%, avg=1908.50, stdev=190.21, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1996KiB/s (2044kB/s)(10.0MiB/5129msec); 0 zone resets clat (msec): min=109, max=1899, avg=473.86, stdev=589.50 lat (msec): min=109, max=1899, avg=473.94, stdev=589.51 clat percentiles (msec): | 1.00th=[ 110], 5.00th=[ 110], 10.00th=[ 110], 20.00th=[ 122], | 30.00th=[ 123], 40.00th=[ 138], 50.00th=[ 174], 60.00th=[ 251], | 70.00th=[ 259], 80.00th=[ 558], 90.00th=[ 1116], 95.00th=[ 1905], | 99.00th=[ 1905], 99.50th=[ 1905], 99.90th=[ 1905], 99.95th=[ 1905], | 99.99th=[ 1905] bw ( KiB/s): min= 2043, max= 6131, per=71.03%, avg=3679.40, stdev=1709.01, samples=5 iops : min= 1, max= 5, avg= 3.00, stdev= 1.58, samples=5 lat (msec) : 250=66.67%, 500=8.33%, 750=8.33%, 2000=16.67% cpu : usr=0.02%, sys=16.71%, ctx=187, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148722: Fri Nov 14 16:44:48 2025 read: IOPS=1, BW=1110KiB/s (1137kB/s)(6144KiB/5535msec) clat (msec): min=94, max=281, avg=174.95, stdev=76.41 lat (msec): min=94, max=281, avg=174.95, stdev=76.41 clat percentiles (msec): | 1.00th=[ 94], 5.00th=[ 94], 10.00th=[ 94], 20.00th=[ 122], | 30.00th=[ 122], 40.00th=[ 140], 50.00th=[ 140], 60.00th=[ 157], | 70.00th=[ 259], 80.00th=[ 259], 90.00th=[ 284], 95.00th=[ 284], | 99.00th=[ 284], 99.50th=[ 284], 99.90th=[ 284], 99.95th=[ 284], | 99.99th=[ 284] bw ( KiB/s): min= 2048, max= 4087, per=92.09%, avg=3407.33, stdev=1177.22, samples=3 iops : min= 2, max= 3, avg= 2.67, stdev= 0.58, samples=3 write: IOPS=1, BW=1110KiB/s (1137kB/s)(6144KiB/5535msec); 0 zone resets clat (msec): min=185, max=1819, avg=725.89, stdev=681.44 lat (msec): min=185, max=1819, avg=725.97, stdev=681.45 clat percentiles (msec): | 1.00th=[ 186], 5.00th=[ 186], 10.00th=[ 186], 20.00th=[ 309], | 30.00th=[ 309], 40.00th=[ 317], 50.00th=[ 317], 60.00th=[ 388], | 70.00th=[ 1334], 80.00th=[ 1334], 90.00th=[ 1821], 95.00th=[ 1821], | 99.00th=[ 1821], 99.50th=[ 1821], 99.90th=[ 1821], 99.95th=[ 1821], | 99.99th=[ 1821] bw ( KiB/s): min= 1793, max= 2048, per=38.47%, avg=1992.60, stdev=111.80, samples=5 iops : min= 1, max= 2, avg= 1.40, stdev= 0.55, samples=5 lat (msec) : 100=8.33%, 250=33.33%, 500=41.67%, 2000=16.67% cpu : usr=0.02%, sys=15.18%, ctx=542, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148723: Fri Nov 14 16:44:48 2025 read: IOPS=1, BW=1152KiB/s (1180kB/s)(5120KiB/4444msec) clat (usec): min=993, max=243070, avg=142391.98, stdev=114152.26 lat (usec): min=993, max=243071, avg=142392.81, stdev=114152.38 clat percentiles (usec): | 1.00th=[ 996], 5.00th=[ 996], 10.00th=[ 996], 20.00th=[ 996], | 30.00th=[ 40633], 40.00th=[ 40633], 50.00th=[185598], 60.00th=[185598], | 70.00th=[240124], 80.00th=[240124], 90.00th=[242222], 95.00th=[242222], | 99.00th=[242222], 99.50th=[242222], 99.90th=[242222], 99.95th=[242222], | 99.99th=[242222] bw ( KiB/s): min= 2043, max= 6144, per=100.00%, avg=4093.50, stdev=2899.84, samples=2 iops : min= 1, max= 6, avg= 3.50, stdev= 3.54, samples=2 write: IOPS=1, BW=1613KiB/s (1652kB/s)(7168KiB/4444msec); 0 zone resets clat (msec): min=101, max=1826, avg=528.85, stdev=711.17 lat (msec): min=101, max=1826, avg=529.29, stdev=710.93 clat percentiles (msec): | 1.00th=[ 102], 5.00th=[ 102], 10.00th=[ 102], 20.00th=[ 111], | 30.00th=[ 126], 40.00th=[ 126], 50.00th=[ 138], 60.00th=[ 144], | 70.00th=[ 144], 80.00th=[ 1250], 90.00th=[ 1821], 95.00th=[ 1821], | 99.00th=[ 1821], 99.50th=[ 1821], 99.90th=[ 1821], 99.95th=[ 1821], | 99.99th=[ 1821] bw ( KiB/s): min= 1774, max= 4079, per=57.82%, avg=2995.00, stdev=1256.68, samples=4 iops : min= 1, max= 3, avg= 2.25, stdev= 0.96, samples=4 lat (usec) : 1000=8.33% lat (msec) : 50=8.33%, 250=66.67%, 2000=16.67% cpu : usr=0.00%, sys=18.05%, ctx=312, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3700KiB/s (3789kB/s), 399KiB/s-1509KiB/s (409kB/s-1546kB/s), io=20.0MiB (20.0MB), run=4444-5535msec WRITE: bw=5180KiB/s (5304kB/s), 1078KiB/s-1996KiB/s (1104kB/s-2044kB/s), io=28.0MiB (29.4MB), run=4444-5535msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148724: Fri Nov 14 16:44:49 2025 read: IOPS=1, BW=1270KiB/s (1301kB/s)(7168KiB/5643msec) clat (msec): min=47, max=2185, avg=668.92, stdev=789.73 lat (msec): min=47, max=2185, avg=668.93, stdev=789.73 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 48], 10.00th=[ 48], 20.00th=[ 67], | 30.00th=[ 165], 40.00th=[ 165], 50.00th=[ 213], 60.00th=[ 936], | 70.00th=[ 936], 80.00th=[ 1070], 90.00th=[ 2198], 95.00th=[ 2198], | 99.00th=[ 2198], 99.50th=[ 2198], 99.90th=[ 2198], 99.95th=[ 2198], | 99.99th=[ 2198] bw ( KiB/s): min= 2015, max= 4079, per=67.42%, avg=2446.60, stdev=912.64, samples=5 iops : min= 1, max= 3, avg= 1.80, stdev= 0.84, samples=5 write: IOPS=0, BW=907KiB/s (929kB/s)(5120KiB/5643msec); 0 zone resets clat (msec): min=96, max=331, avg=163.11, stdev=97.44 lat (msec): min=96, max=331, avg=163.19, stdev=97.45 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 96], 10.00th=[ 96], 20.00th=[ 96], | 30.00th=[ 111], 40.00th=[ 111], 50.00th=[ 115], 60.00th=[ 115], | 70.00th=[ 163], 80.00th=[ 163], 90.00th=[ 334], 95.00th=[ 334], | 99.00th=[ 334], 99.50th=[ 334], 99.90th=[ 334], 99.95th=[ 334], | 99.99th=[ 334] bw ( KiB/s): min= 2039, max= 6131, per=80.41%, avg=4085.00, stdev=2893.48, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 lat (msec) : 50=8.33%, 100=16.67%, 250=41.67%, 500=8.33%, 1000=8.33% lat (msec) : 2000=8.33%, >=2000=8.33% cpu : usr=0.00%, sys=3.85%, ctx=75, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148725: Fri Nov 14 16:44:49 2025 read: IOPS=0, BW=389KiB/s (398kB/s)(2048KiB/5269msec) clat (msec): min=113, max=145, avg=129.60, stdev=22.45 lat (msec): min=113, max=145, avg=129.60, stdev=22.45 clat percentiles (msec): | 1.00th=[ 114], 5.00th=[ 114], 10.00th=[ 114], 20.00th=[ 114], | 30.00th=[ 114], 40.00th=[ 114], 50.00th=[ 114], 60.00th=[ 146], | 70.00th=[ 146], 80.00th=[ 146], 90.00th=[ 146], 95.00th=[ 146], | 99.00th=[ 146], 99.50th=[ 146], 99.90th=[ 146], 99.95th=[ 146], | 99.99th=[ 146] bw ( KiB/s): min= 1780, max= 2048, per=52.74%, avg=1914.00, stdev=189.50, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=1, BW=1943KiB/s (1990kB/s)(10.0MiB/5269msec); 0 zone resets clat (msec): min=64, max=1818, avg=487.36, stdev=600.80 lat (msec): min=65, max=1818, avg=487.45, stdev=600.81 clat percentiles (msec): | 1.00th=[ 65], 5.00th=[ 65], 10.00th=[ 65], 20.00th=[ 77], | 30.00th=[ 79], 40.00th=[ 130], 50.00th=[ 201], 60.00th=[ 313], | 70.00th=[ 426], 80.00th=[ 439], 90.00th=[ 1334], 95.00th=[ 1821], | 99.00th=[ 1821], 99.50th=[ 1821], 99.90th=[ 1821], 99.95th=[ 1821], | 99.99th=[ 1821] bw ( KiB/s): min= 2015, max= 6144, per=80.38%, avg=4083.50, stdev=1685.68, samples=4 iops : min= 1, max= 6, avg= 3.50, stdev= 2.08, samples=4 lat (msec) : 100=25.00%, 250=33.33%, 500=25.00%, 2000=16.67% cpu : usr=0.02%, sys=6.15%, ctx=89, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148726: Fri Nov 14 16:44:49 2025 read: IOPS=1, BW=1147KiB/s (1175kB/s)(6144KiB/5356msec) clat (msec): min=58, max=246, avg=136.08, stdev=77.16 lat (msec): min=58, max=246, avg=136.08, stdev=77.16 clat percentiles (msec): | 1.00th=[ 59], 5.00th=[ 59], 10.00th=[ 59], 20.00th=[ 75], | 30.00th=[ 75], 40.00th=[ 75], 50.00th=[ 75], 60.00th=[ 178], | 70.00th=[ 184], 80.00th=[ 184], 90.00th=[ 247], 95.00th=[ 247], | 99.00th=[ 247], 99.50th=[ 247], 99.90th=[ 247], 99.95th=[ 247], | 99.99th=[ 247] bw ( KiB/s): min= 2048, max= 6047, per=93.17%, avg=3381.00, stdev=2308.82, samples=3 iops : min= 2, max= 5, avg= 3.00, stdev= 1.73, samples=3 write: IOPS=1, BW=1147KiB/s (1175kB/s)(6144KiB/5356msec); 0 zone resets clat (msec): min=191, max=2038, avg=750.31, stdev=738.90 lat (msec): min=191, max=2038, avg=750.42, stdev=738.97 clat percentiles (msec): | 1.00th=[ 192], 5.00th=[ 192], 10.00th=[ 192], 20.00th=[ 275], | 30.00th=[ 275], 40.00th=[ 300], 50.00th=[ 300], 60.00th=[ 451], | 70.00th=[ 1250], 80.00th=[ 1250], 90.00th=[ 2039], 95.00th=[ 2039], | 99.00th=[ 2039], 99.50th=[ 2039], 99.90th=[ 2039], 99.95th=[ 2039], | 99.99th=[ 2039] bw ( KiB/s): min= 2015, max= 3561, per=47.60%, avg=2418.00, stdev=762.16, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (msec) : 100=25.00%, 250=33.33%, 500=25.00%, 2000=8.33%, >=2000=8.33% cpu : usr=0.04%, sys=4.01%, ctx=79, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148727: Fri Nov 14 16:44:49 2025 read: IOPS=0, BW=919KiB/s (941kB/s)(5120KiB/5569msec) clat (msec): min=45, max=492, avg=166.65, stdev=188.73 lat (msec): min=45, max=492, avg=166.66, stdev=188.73 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 46], 10.00th=[ 46], 20.00th=[ 46], | 30.00th=[ 51], 40.00th=[ 51], 50.00th=[ 77], 60.00th=[ 77], | 70.00th=[ 169], 80.00th=[ 169], 90.00th=[ 493], 95.00th=[ 493], | 99.00th=[ 493], 99.50th=[ 493], 99.90th=[ 493], 99.95th=[ 493], | 99.99th=[ 493] bw ( KiB/s): min= 4031, max= 4079, per=100.00%, avg=4055.00, stdev=33.94, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1287KiB/s (1318kB/s)(7168KiB/5569msec); 0 zone resets clat (msec): min=92, max=2244, avg=646.11, stdev=825.96 lat (msec): min=92, max=2244, avg=646.19, stdev=825.95 clat percentiles (msec): | 1.00th=[ 93], 5.00th=[ 93], 10.00th=[ 93], 20.00th=[ 140], | 30.00th=[ 142], 40.00th=[ 142], 50.00th=[ 222], 60.00th=[ 359], | 70.00th=[ 359], 80.00th=[ 1318], 90.00th=[ 2232], 95.00th=[ 2232], | 99.00th=[ 2232], 99.50th=[ 2232], 99.90th=[ 2232], 99.95th=[ 2232], | 99.99th=[ 2232] bw ( KiB/s): min= 1796, max= 4096, per=51.88%, avg=2635.67, stdev=1269.42, samples=3 iops : min= 1, max= 4, avg= 2.00, stdev= 1.73, samples=3 lat (msec) : 50=8.33%, 100=25.00%, 250=33.33%, 500=16.67%, 2000=8.33% lat (msec) : >=2000=8.33% cpu : usr=0.00%, sys=6.21%, ctx=244, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3629KiB/s (3716kB/s), 389KiB/s-1270KiB/s (398kB/s-1301kB/s), io=20.0MiB (20.0MB), run=5269-5643msec WRITE: bw=5081KiB/s (5203kB/s), 907KiB/s-1943KiB/s (929kB/s-1990kB/s), io=28.0MiB (29.4MB), run=5269-5643msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=148735: Fri Nov 14 16:44:57 2025 read: IOPS=1, BW=5832KiB/s (5972kB/s)(12.0MiB/2107msec) clat (msec): min=579, max=825, avg=686.90, stdev=125.69 lat (msec): min=579, max=825, avg=686.91, stdev=125.69 clat percentiles (msec): | 1.00th=[ 584], 5.00th=[ 584], 10.00th=[ 584], 20.00th=[ 584], | 30.00th=[ 584], 40.00th=[ 659], 50.00th=[ 659], 60.00th=[ 659], | 70.00th=[ 827], 80.00th=[ 827], 90.00th=[ 827], 95.00th=[ 827], | 99.00th=[ 827], 99.50th=[ 827], 99.90th=[ 827], 99.95th=[ 827], | 99.99th=[ 827] bw ( KiB/s): min= 8031, max= 8192, per=100.00%, avg=8111.50, stdev=113.84, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 1000=33.33% cpu : usr=0.00%, sys=9.35%, ctx=65, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148736: Fri Nov 14 16:44:57 2025 read: IOPS=1, BW=6575KiB/s (6732kB/s)(4096KiB/623msec) clat (nsec): min=146984k, max=146984k, avg=146983973.00, stdev= 0.00 lat (nsec): min=146986k, max=146986k, avg=146985931.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 148], 5.00th=[ 148], 10.00th=[ 148], 20.00th=[ 148], | 30.00th=[ 148], 40.00th=[ 148], 50.00th=[ 148], 60.00th=[ 148], | 70.00th=[ 148], 80.00th=[ 148], 90.00th=[ 148], 95.00th=[ 148], | 99.00th=[ 148], 99.50th=[ 148], 99.90th=[ 148], 99.95th=[ 148], | 99.99th=[ 148] write: IOPS=3, BW=12.8MiB/s (13.5MB/s)(8192KiB/623msec); 0 zone resets clat (msec): min=226, max=228, avg=227.57, stdev= 1.94 lat (msec): min=226, max=229, avg=227.88, stdev= 1.99 clat percentiles (msec): | 1.00th=[ 226], 5.00th=[ 226], 10.00th=[ 226], 20.00th=[ 226], | 30.00th=[ 226], 40.00th=[ 226], 50.00th=[ 226], 60.00th=[ 230], | 70.00th=[ 230], 80.00th=[ 230], 90.00th=[ 230], 95.00th=[ 230], | 99.00th=[ 230], 99.50th=[ 230], 99.90th=[ 230], 99.95th=[ 230], | 99.99th=[ 230] lat (msec) : 250=100.00% cpu : usr=0.00%, sys=22.99%, ctx=80, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148737: Fri Nov 14 16:44:57 2025 write: IOPS=2, BW=9218KiB/s (9440kB/s)(12.0MiB/1333msec); 0 zone resets clat (msec): min=335, max=529, avg=411.14, stdev=103.55 lat (msec): min=335, max=529, avg=411.55, stdev=103.47 clat percentiles (msec): | 1.00th=[ 334], 5.00th=[ 334], 10.00th=[ 334], 20.00th=[ 334], | 30.00th=[ 334], 40.00th=[ 368], 50.00th=[ 368], 60.00th=[ 368], | 70.00th=[ 531], 80.00th=[ 531], 90.00th=[ 531], 95.00th=[ 531], | 99.00th=[ 531], 99.50th=[ 531], 99.90th=[ 531], 99.95th=[ 531], | 99.99th=[ 531] bw ( KiB/s): min= 5818, max= 8192, per=28.65%, avg=7005.00, stdev=1678.67, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.00%, sys=11.56%, ctx=74, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148738: Fri Nov 14 16:44:57 2025 write: IOPS=2, BW=9170KiB/s (9390kB/s)(12.0MiB/1340msec); 0 zone resets clat (msec): min=220, max=640, avg=435.40, stdev=210.06 lat (msec): min=221, max=641, avg=435.88, stdev=210.30 clat percentiles (msec): | 1.00th=[ 222], 5.00th=[ 222], 10.00th=[ 222], 20.00th=[ 222], | 30.00th=[ 222], 40.00th=[ 447], 50.00th=[ 447], 60.00th=[ 447], | 70.00th=[ 642], 80.00th=[ 642], 90.00th=[ 642], 95.00th=[ 642], | 99.00th=[ 642], 99.50th=[ 642], 99.90th=[ 642], 99.95th=[ 642], | 99.99th=[ 642] bw ( KiB/s): min=16384, max=16384, per=67.00%, avg=16384.00, stdev= 0.00, samples=1 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 750=33.33% cpu : usr=0.15%, sys=14.86%, ctx=70, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7776KiB/s (7963kB/s), 5832KiB/s-6575KiB/s (5972kB/s-6732kB/s), io=16.0MiB (16.8MB), run=623-2107msec WRITE: bw=23.9MiB/s (25.0MB/s), 9170KiB/s-12.8MiB/s (9390kB/s-13.5MB/s), io=32.0MiB (33.6MB), run=623-1340msec rand-rw: (groupid=0, jobs=1): err= 0: pid=148739: Fri Nov 14 16:44:58 2025 read: IOPS=1, BW=6327KiB/s (6479kB/s)(12.0MiB/1942msec) clat (msec): min=564, max=689, avg=612.11, stdev=67.93 lat (msec): min=564, max=689, avg=612.11, stdev=67.93 clat percentiles (msec): | 1.00th=[ 567], 5.00th=[ 567], 10.00th=[ 567], 20.00th=[ 567], | 30.00th=[ 567], 40.00th=[ 584], 50.00th=[ 584], 60.00th=[ 584], | 70.00th=[ 693], 80.00th=[ 693], 90.00th=[ 693], 95.00th=[ 693], | 99.00th=[ 693], 99.50th=[ 693], 99.90th=[ 693], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 7892, max= 8126, per=100.00%, avg=8009.00, stdev=165.46, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=100.00% cpu : usr=0.00%, sys=28.03%, ctx=722, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148740: Fri Nov 14 16:44:58 2025 read: IOPS=0, BW=1711KiB/s (1752kB/s)(4096KiB/2394msec) clat (nsec): min=677823k, max=677823k, avg=677823414.00, stdev= 0.00 lat (nsec): min=677826k, max=677826k, avg=677825759.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 676], 5.00th=[ 676], 10.00th=[ 676], 20.00th=[ 676], | 30.00th=[ 676], 40.00th=[ 676], 50.00th=[ 676], 60.00th=[ 676], | 70.00th=[ 676], 80.00th=[ 676], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 7892, max= 7892, per=100.00%, avg=7892.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3422KiB/s (3504kB/s)(8192KiB/2394msec); 0 zone resets clat (msec): min=636, max=1004, avg=820.39, stdev=260.06 lat (msec): min=636, max=1004, avg=820.76, stdev=260.15 clat percentiles (msec): | 1.00th=[ 634], 5.00th=[ 634], 10.00th=[ 634], 20.00th=[ 634], | 30.00th=[ 634], 40.00th=[ 634], 50.00th=[ 634], 60.00th=[ 1003], | 70.00th=[ 1003], 80.00th=[ 1003], 90.00th=[ 1003], 95.00th=[ 1003], | 99.00th=[ 1003], 99.50th=[ 1003], 99.90th=[ 1003], 99.95th=[ 1003], | 99.99th=[ 1003] bw ( KiB/s): min= 8126, max= 8126, per=59.62%, avg=8126.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=14.25%, ctx=256, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148741: Fri Nov 14 16:44:58 2025 write: IOPS=1, BW=5111KiB/s (5234kB/s)(12.0MiB/2404msec); 0 zone resets clat (msec): min=191, max=1138, avg=791.35, stdev=521.58 lat (msec): min=191, max=1138, avg=791.79, stdev=521.69 clat percentiles (msec): | 1.00th=[ 192], 5.00th=[ 192], 10.00th=[ 192], 20.00th=[ 192], | 30.00th=[ 192], 40.00th=[ 1045], 50.00th=[ 1045], 60.00th=[ 1045], | 70.00th=[ 1133], 80.00th=[ 1133], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 7892, max= 7892, per=57.90%, avg=7892.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 2000=66.67% cpu : usr=0.00%, sys=11.11%, ctx=70, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=148742: Fri Nov 14 16:44:58 2025 write: IOPS=1, BW=5500KiB/s (5632kB/s)(12.0MiB/2234msec); 0 zone resets clat (msec): min=224, max=1025, avg=716.22, stdev=430.15 lat (msec): min=225, max=1025, avg=716.54, stdev=430.25 clat percentiles (msec): | 1.00th=[ 226], 5.00th=[ 226], 10.00th=[ 226], 20.00th=[ 226], | 30.00th=[ 226], 40.00th=[ 902], 50.00th=[ 902], 60.00th=[ 902], | 70.00th=[ 1028], 80.00th=[ 1028], 90.00th=[ 1028], 95.00th=[ 1028], | 99.00th=[ 1028], 99.50th=[ 1028], 99.90th=[ 1028], 99.95th=[ 1028], | 99.99th=[ 1028] bw ( KiB/s): min= 7892, max= 8094, per=58.64%, avg=7993.00, stdev=142.84, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=14.82%, ctx=72, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6844KiB/s (7008kB/s), 1711KiB/s-6327KiB/s (1752kB/s-6479kB/s), io=16.0MiB (16.8MB), run=1942-2394msec WRITE: bw=13.3MiB/s (13.0MB/s), 3422KiB/s-5500KiB/s (3504kB/s-5632kB/s), io=32.0MiB (33.6MB), run=2234-2404msec