== sanity test 398b: DIO and buffer IO race ============== 10:33:20 (1753367600) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.32552 s, 15.1 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=190412: Thu Jul 24 10:34:29 2025 read: IOPS=25, BW=102KiB/s (105kB/s)(5992KiB/58579msec) clat (usec): min=127, max=723827, avg=21405.53, stdev=21985.69 lat (usec): min=128, max=723830, avg=21407.30, stdev=21985.80 clat percentiles (usec): | 1.00th=[ 302], 5.00th=[ 7963], 10.00th=[ 9634], 20.00th=[ 11731], | 30.00th=[ 13698], 40.00th=[ 15533], 50.00th=[ 17957], 60.00th=[ 20841], | 70.00th=[ 24249], 80.00th=[ 28705], 90.00th=[ 36439], 95.00th=[ 44827], | 99.00th=[ 65274], 99.50th=[ 77071], 99.90th=[100140], 99.95th=[725615], | 99.99th=[725615] bw ( KiB/s): min= 14, max= 183, per=24.85%, avg=101.62, stdev=28.98, samples=116 iops : min= 3, max= 45, avg=24.94, stdev= 7.27, samples=116 write: IOPS=26, BW=107KiB/s (110kB/s)(6296KiB/58579msec); 0 zone resets clat (usec): min=1021, max=116556, avg=16661.44, stdev=14665.48 lat (usec): min=1022, max=116559, avg=16663.30, stdev=14665.59 clat percentiles (usec): | 1.00th=[ 1254], 5.00th=[ 2073], 10.00th=[ 2671], 20.00th=[ 3392], | 30.00th=[ 5866], 40.00th=[ 10028], 50.00th=[ 13042], 60.00th=[ 16450], | 70.00th=[ 21103], 80.00th=[ 27657], 90.00th=[ 35914], 95.00th=[ 44303], | 99.00th=[ 63701], 99.50th=[ 71828], 99.90th=[102237], 99.95th=[116917], | 99.99th=[116917] bw ( KiB/s): min= 15, max= 199, per=25.09%, avg=107.63, stdev=31.65, samples=115 iops : min= 3, max= 49, avg=26.45, stdev= 7.90, samples=115 lat (usec) : 250=0.26%, 500=0.81%, 750=0.29%, 1000=0.07% lat (msec) : 2=2.28%, 4=9.73%, 10=12.50%, 20=36.78%, 50=33.95% lat (msec) : 100=3.19%, 250=0.10%, 750=0.03% cpu : usr=0.22%, sys=21.89%, ctx=7356, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190413: Thu Jul 24 10:34:29 2025 read: IOPS=25, BW=102KiB/s (104kB/s)(5940KiB/58230msec) clat (usec): min=151, max=92030, avg=21257.45, stdev=11907.55 lat (usec): min=153, max=92031, avg=21264.34, stdev=11904.04 clat percentiles (usec): | 1.00th=[ 338], 5.00th=[ 8356], 10.00th=[10028], 20.00th=[11863], | 30.00th=[13829], 40.00th=[16188], 50.00th=[18744], 60.00th=[21890], | 70.00th=[24773], 80.00th=[28967], 90.00th=[35914], 95.00th=[43779], | 99.00th=[63701], 99.50th=[71828], 99.90th=[85459], 99.95th=[91751], | 99.99th=[91751] bw ( KiB/s): min= 7, max= 176, per=25.04%, avg=102.43, stdev=28.72, samples=114 iops : min= 1, max= 44, avg=25.15, stdev= 7.25, samples=114 write: IOPS=27, BW=109KiB/s (112kB/s)(6348KiB/58230msec); 0 zone resets clat (usec): min=889, max=705258, avg=16591.83, stdev=22242.98 lat (usec): min=890, max=705260, avg=16594.49, stdev=22242.84 clat percentiles (usec): | 1.00th=[ 1336], 5.00th=[ 2073], 10.00th=[ 2606], 20.00th=[ 3261], | 30.00th=[ 5407], 40.00th=[ 9896], 50.00th=[ 12518], 60.00th=[ 16188], | 70.00th=[ 21103], 80.00th=[ 27132], 90.00th=[ 36439], 95.00th=[ 42730], | 99.00th=[ 58983], 99.50th=[ 66847], 99.90th=[127402], 99.95th=[708838], | 99.99th=[708838] bw ( KiB/s): min= 39, max= 223, per=25.43%, avg=109.11, stdev=33.54, samples=114 iops : min= 9, max= 55, avg=26.82, stdev= 8.36, samples=114 lat (usec) : 250=0.20%, 500=0.62%, 750=0.07%, 1000=0.26% lat (msec) : 2=2.38%, 4=10.16%, 10=12.08%, 20=35.81%, 50=35.68% lat (msec) : 100=2.67%, 250=0.07%, 750=0.03% cpu : usr=0.23%, sys=22.21%, ctx=7323, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190414: Thu Jul 24 10:34:29 2025 read: IOPS=25, BW=104KiB/s (106kB/s)(6028KiB/58230msec) clat (usec): min=132, max=718068, avg=21465.44, stdev=21492.19 lat (usec): min=133, max=718070, avg=21467.14, stdev=21492.26 clat percentiles (usec): | 1.00th=[ 338], 5.00th=[ 8225], 10.00th=[ 9634], 20.00th=[ 11863], | 30.00th=[ 13698], 40.00th=[ 16188], 50.00th=[ 18744], 60.00th=[ 21627], | 70.00th=[ 24511], 80.00th=[ 28443], 90.00th=[ 35390], 95.00th=[ 43779], | 99.00th=[ 60556], 99.50th=[ 68682], 99.90th=[ 99091], 99.95th=[717226], | 99.99th=[717226] bw ( KiB/s): min= 7, max= 175, per=25.22%, avg=103.13, stdev=27.88, samples=115 iops : min= 1, max= 43, avg=25.32, stdev= 6.97, samples=115 write: IOPS=26, BW=108KiB/s (110kB/s)(6260KiB/58230msec); 0 zone resets clat (usec): min=948, max=98517, avg=16352.46, stdev=14115.57 lat (usec): min=950, max=98520, avg=16354.65, stdev=14115.80 clat percentiles (usec): | 1.00th=[ 1270], 5.00th=[ 1958], 10.00th=[ 2540], 20.00th=[ 3228], | 30.00th=[ 5342], 40.00th=[ 9765], 50.00th=[13042], 60.00th=[16581], | 70.00th=[20841], 80.00th=[27132], 90.00th=[35390], 95.00th=[44303], | 99.00th=[61080], 99.50th=[71828], 99.90th=[82314], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 21, max= 215, per=24.90%, avg=106.83, stdev=34.05, samples=115 iops : min= 5, max= 53, avg=26.24, stdev= 8.57, samples=115 lat (usec) : 250=0.13%, 500=1.07%, 750=0.13%, 1000=0.10% lat (msec) : 2=2.70%, 4=10.12%, 10=12.08%, 20=34.99%, 50=35.90% lat (msec) : 100=2.73%, 750=0.03% cpu : usr=0.21%, sys=22.12%, ctx=7406, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190415: Thu Jul 24 10:34:29 2025 read: IOPS=25, BW=104KiB/s (106kB/s)(6052KiB/58425msec) clat (usec): min=146, max=769364, avg=20820.41, stdev=22463.26 lat (usec): min=146, max=769366, avg=20821.94, stdev=22463.31 clat percentiles (usec): | 1.00th=[ 285], 5.00th=[ 7504], 10.00th=[ 9372], 20.00th=[ 11863], | 30.00th=[ 13698], 40.00th=[ 16188], 50.00th=[ 18744], 60.00th=[ 21103], | 70.00th=[ 23462], 80.00th=[ 27132], 90.00th=[ 33817], 95.00th=[ 40633], | 99.00th=[ 61080], 99.50th=[ 69731], 99.90th=[143655], 99.95th=[767558], | 99.99th=[767558] bw ( KiB/s): min= 7, max= 167, per=25.20%, avg=103.08, stdev=29.48, samples=116 iops : min= 1, max= 41, avg=25.30, stdev= 7.37, samples=116 write: IOPS=26, BW=107KiB/s (109kB/s)(6236KiB/58425msec); 0 zone resets clat (usec): min=1052, max=125041, avg=17082.30, stdev=14792.07 lat (usec): min=1053, max=125045, avg=17084.41, stdev=14792.26 clat percentiles (usec): | 1.00th=[ 1352], 5.00th=[ 2311], 10.00th=[ 2704], 20.00th=[ 3490], | 30.00th=[ 5735], 40.00th=[ 10552], 50.00th=[ 13435], 60.00th=[ 16712], | 70.00th=[ 22152], 80.00th=[ 28705], 90.00th=[ 36963], 95.00th=[ 45876], | 99.00th=[ 63701], 99.50th=[ 69731], 99.90th=[106431], 99.95th=[125305], | 99.99th=[125305] bw ( KiB/s): min= 15, max= 199, per=24.74%, avg=106.15, stdev=30.54, samples=116 iops : min= 3, max= 49, avg=26.08, stdev= 7.65, samples=116 lat (usec) : 250=0.23%, 500=1.20%, 750=0.23%, 1000=0.20% lat (msec) : 2=1.76%, 4=10.12%, 10=11.30%, 20=36.07%, 50=36.17% lat (msec) : 100=2.60%, 250=0.10%, 1000=0.03% cpu : usr=0.20%, sys=22.15%, ctx=7496, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=410KiB/s (420kB/s), 102KiB/s-104KiB/s (104kB/s-106kB/s), io=23.4MiB (24.6MB), run=58230-58579msec WRITE: bw=429KiB/s (439kB/s), 107KiB/s-109KiB/s (109kB/s-112kB/s), io=24.6MiB (25.7MB), run=58230-58579msec rand-rw: (groupid=0, jobs=1): err= 0: pid=190416: Thu Jul 24 10:34:44 2025 read: IOPS=20, BW=82.7KiB/s (84.7kB/s)(5992KiB/72476msec) clat (msec): min=3, max=730, avg=20.56, stdev=25.23 lat (msec): min=3, max=731, avg=20.56, stdev=25.23 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 14], 60.00th=[ 18], | 70.00th=[ 23], 80.00th=[ 32], 90.00th=[ 46], 95.00th=[ 56], | 99.00th=[ 85], 99.50th=[ 101], 99.90th=[ 136], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 15, max= 262, per=25.20%, avg=82.92, stdev=53.60, samples=143 iops : min= 3, max= 65, avg=20.29, stdev=13.52, samples=143 write: IOPS=21, BW=86.9KiB/s (88.0kB/s)(6296KiB/72476msec); 0 zone resets clat (msec): min=5, max=128, avg=26.32, stdev=17.13 lat (msec): min=5, max=128, avg=26.32, stdev=17.13 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 26], | 70.00th=[ 33], 80.00th=[ 40], 90.00th=[ 50], 95.00th=[ 60], | 99.00th=[ 86], 99.50th=[ 95], 99.90th=[ 127], 99.95th=[ 129], | 99.99th=[ 129] bw ( KiB/s): min= 7, max= 240, per=25.29%, avg=87.01, stdev=53.32, samples=143 iops : min= 1, max= 60, avg=21.31, stdev=13.44, samples=143 lat (msec) : 4=0.07%, 10=21.91%, 20=34.51%, 50=35.22%, 100=7.94% lat (msec) : 250=0.33%, 750=0.03% cpu : usr=0.17%, sys=9.73%, ctx=4056, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190417: Thu Jul 24 10:34:44 2025 read: IOPS=20, BW=81.0KiB/s (83.0kB/s)(5940KiB/72445msec) clat (msec): min=3, max=773, avg=19.45, stdev=25.13 lat (msec): min=3, max=773, avg=19.45, stdev=25.13 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 21], 80.00th=[ 29], 90.00th=[ 41], 95.00th=[ 53], | 99.00th=[ 80], 99.50th=[ 87], 99.90th=[ 134], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 7, max= 280, per=25.20%, avg=82.90, stdev=55.91, samples=142 iops : min= 1, max= 70, avg=20.28, stdev=14.09, samples=142 write: IOPS=21, BW=87.6KiB/s (89.7kB/s)(6348KiB/72445msec); 0 zone resets clat (msec): min=6, max=169, avg=27.29, stdev=18.34 lat (msec): min=6, max=169, avg=27.30, stdev=18.34 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 22], 60.00th=[ 27], | 70.00th=[ 34], 80.00th=[ 42], 90.00th=[ 52], 95.00th=[ 60], | 99.00th=[ 90], 99.50th=[ 104], 99.90th=[ 140], 99.95th=[ 169], | 99.99th=[ 169] bw ( KiB/s): min= 15, max= 240, per=25.50%, avg=87.73, stdev=51.19, samples=143 iops : min= 3, max= 60, avg=21.49, stdev=12.92, samples=143 lat (msec) : 4=0.07%, 10=21.13%, 20=35.84%, 50=34.67%, 100=7.85% lat (msec) : 250=0.42%, 1000=0.03% cpu : usr=0.16%, sys=9.74%, ctx=4044, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190418: Thu Jul 24 10:34:44 2025 read: IOPS=20, BW=83.7KiB/s (85.8kB/s)(6028KiB/71979msec) clat (msec): min=4, max=124, avg=19.67, stdev=16.08 lat (msec): min=4, max=124, avg=19.67, stdev=16.08 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 30], 90.00th=[ 44], 95.00th=[ 55], | 99.00th=[ 74], 99.50th=[ 87], 99.90th=[ 103], 99.95th=[ 126], | 99.99th=[ 126] bw ( KiB/s): min= 23, max= 286, per=25.70%, avg=84.56, stdev=50.76, samples=141 iops : min= 5, max= 71, avg=20.70, stdev=12.79, samples=141 write: IOPS=21, BW=86.0KiB/s (89.1kB/s)(6260KiB/71979msec); 0 zone resets clat (msec): min=6, max=788, avg=26.89, stdev=26.61 lat (msec): min=6, max=788, avg=26.90, stdev=26.61 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 21], 60.00th=[ 26], | 70.00th=[ 32], 80.00th=[ 40], 90.00th=[ 51], 95.00th=[ 62], | 99.00th=[ 90], 99.50th=[ 111], 99.90th=[ 169], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 23, max= 248, per=25.33%, avg=87.12, stdev=53.11, samples=142 iops : min= 5, max= 62, avg=21.33, stdev=13.40, samples=142 lat (msec) : 10=21.16%, 20=36.52%, 50=33.89%, 100=7.98%, 250=0.42% lat (msec) : 1000=0.03% cpu : usr=0.18%, sys=9.82%, ctx=3959, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190419: Thu Jul 24 10:34:44 2025 read: IOPS=20, BW=83.0KiB/s (85.0kB/s)(6052KiB/72900msec) clat (msec): min=3, max=104, avg=19.62, stdev=16.15 lat (msec): min=3, max=104, avg=19.62, stdev=16.15 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 17], | 70.00th=[ 22], 80.00th=[ 30], 90.00th=[ 44], 95.00th=[ 54], | 99.00th=[ 74], 99.50th=[ 86], 99.90th=[ 100], 99.95th=[ 105], | 99.99th=[ 105] bw ( KiB/s): min= 7, max= 272, per=25.28%, avg=83.17, stdev=53.51, samples=144 iops : min= 1, max= 68, avg=20.35, stdev=13.49, samples=144 write: IOPS=21, BW=85.5KiB/s (87.6kB/s)(6236KiB/72900msec); 0 zone resets clat (msec): min=5, max=789, avg=27.56, stdev=27.03 lat (msec): min=5, max=789, avg=27.57, stdev=27.03 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 13], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 26], | 70.00th=[ 32], 80.00th=[ 41], 90.00th=[ 53], 95.00th=[ 66], | 99.00th=[ 94], 99.50th=[ 102], 99.90th=[ 142], 99.95th=[ 793], | 99.99th=[ 793] bw ( KiB/s): min= 15, max= 240, per=24.86%, avg=85.51, stdev=51.76, samples=144 iops : min= 3, max= 60, avg=20.93, stdev=13.05, samples=144 lat (msec) : 4=0.03%, 10=21.81%, 20=35.74%, 50=33.33%, 100=8.79% lat (msec) : 250=0.26%, 1000=0.03% cpu : usr=0.16%, sys=9.61%, ctx=4002, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=329KiB/s (337kB/s), 81.0KiB/s-83.7KiB/s (83.0kB/s-85.8kB/s), io=23.4MiB (24.6MB), run=71979-72900msec WRITE: bw=345KiB/s (353kB/s), 85.5KiB/s-87.6KiB/s (87.6kB/s-89.7kB/s), io=24.6MiB (25.7MB), run=71979-72900msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=190448: Thu Jul 24 10:35:07 2025 read: IOPS=22, BW=363KiB/s (372kB/s)(5920KiB/16309msec) clat (usec): min=157, max=145400, avg=22355.70, stdev=15338.16 lat (usec): min=158, max=145403, avg=22357.48, stdev=15338.14 clat percentiles (usec): | 1.00th=[ 322], 5.00th=[ 570], 10.00th=[ 10028], 20.00th=[ 12780], | 30.00th=[ 16057], 40.00th=[ 18220], 50.00th=[ 20055], 60.00th=[ 23200], | 70.00th=[ 26608], 80.00th=[ 30540], 90.00th=[ 33817], 95.00th=[ 41681], | 99.00th=[ 72877], 99.50th=[137364], 99.90th=[145753], 99.95th=[145753], | 99.99th=[145753] bw ( KiB/s): min= 159, max= 512, per=25.54%, avg=361.44, stdev=94.05, samples=32 iops : min= 9, max= 32, avg=21.97, stdev= 5.98, samples=32 write: IOPS=24, BW=390KiB/s (400kB/s)(6368KiB/16309msec); 0 zone resets clat (usec): min=1358, max=88081, avg=20030.69, stdev=15475.83 lat (usec): min=1361, max=88085, avg=20040.98, stdev=15476.86 clat percentiles (usec): | 1.00th=[ 1827], 5.00th=[ 3163], 10.00th=[ 4113], 20.00th=[ 5800], | 30.00th=[ 8848], 40.00th=[12256], 50.00th=[15139], 60.00th=[19792], | 70.00th=[26346], 80.00th=[34341], 90.00th=[43254], 95.00th=[47973], | 99.00th=[67634], 99.50th=[76022], 99.90th=[88605], 99.95th=[88605], | 99.99th=[88605] bw ( KiB/s): min= 191, max= 698, per=26.27%, avg=391.19, stdev=102.22, samples=32 iops : min= 11, max= 43, avg=23.84, stdev= 6.37, samples=32 lat (usec) : 250=0.13%, 500=1.95%, 750=0.65%, 1000=1.17% lat (msec) : 2=0.65%, 4=4.17%, 10=13.15%, 20=33.46%, 50=40.62% lat (msec) : 100=3.65%, 250=0.39% cpu : usr=0.18%, sys=26.74%, ctx=2259, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190449: Thu Jul 24 10:35:07 2025 read: IOPS=21, BW=340KiB/s (349kB/s)(5744KiB/16872msec) clat (usec): min=183, max=111924, avg=24437.06, stdev=15019.01 lat (usec): min=184, max=111925, avg=24438.92, stdev=15019.04 clat percentiles (usec): | 1.00th=[ 239], 5.00th=[ 799], 10.00th=[ 10683], 20.00th=[ 13960], | 30.00th=[ 16909], 40.00th=[ 19006], 50.00th=[ 21365], 60.00th=[ 24511], | 70.00th=[ 27919], 80.00th=[ 33817], 90.00th=[ 40633], 95.00th=[ 54789], | 99.00th=[ 76022], 99.50th=[ 88605], 99.90th=[111674], 99.95th=[111674], | 99.99th=[111674] bw ( KiB/s): min= 191, max= 542, per=23.93%, avg=338.64, stdev=82.78, samples=33 iops : min= 11, max= 33, avg=20.52, stdev= 5.30, samples=33 write: IOPS=24, BW=388KiB/s (397kB/s)(6544KiB/16872msec); 0 zone resets clat (usec): min=1632, max=100658, avg=19612.26, stdev=15678.89 lat (usec): min=1637, max=100662, avg=19615.52, stdev=15678.92 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 33], 90.00th=[ 42], 95.00th=[ 53], | 99.00th=[ 67], 99.50th=[ 69], 99.90th=[ 101], 99.95th=[ 101], | 99.99th=[ 101] bw ( KiB/s): min= 159, max= 702, per=25.72%, avg=382.94, stdev=116.57, samples=33 iops : min= 9, max= 43, avg=23.30, stdev= 7.31, samples=33 lat (usec) : 250=0.65%, 500=0.78%, 750=0.65%, 1000=0.52% lat (msec) : 2=0.39%, 4=4.43%, 10=13.80%, 20=31.64%, 50=41.54% lat (msec) : 100=5.34%, 250=0.26% cpu : usr=0.16%, sys=26.24%, ctx=2347, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190450: Thu Jul 24 10:35:07 2025 read: IOPS=23, BW=370KiB/s (379kB/s)(6256KiB/16920msec) clat (usec): min=168, max=92941, avg=23525.11, stdev=15024.87 lat (usec): min=170, max=92944, avg=23527.15, stdev=15024.84 clat percentiles (usec): | 1.00th=[ 255], 5.00th=[ 416], 10.00th=[ 9634], 20.00th=[12780], | 30.00th=[15401], 40.00th=[17957], 50.00th=[20055], 60.00th=[23462], | 70.00th=[28181], 80.00th=[33424], 90.00th=[41681], 95.00th=[54789], | 99.00th=[71828], 99.50th=[83362], 99.90th=[92799], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 222, max= 603, per=25.96%, avg=367.36, stdev=85.02, samples=33 iops : min= 13, max= 37, avg=22.33, stdev= 5.37, samples=33 write: IOPS=22, BW=357KiB/s (365kB/s)(6032KiB/16920msec); 0 zone resets clat (usec): min=1858, max=98776, avg=20197.33, stdev=14744.01 lat (usec): min=1860, max=98780, avg=20201.29, stdev=14744.89 clat percentiles (usec): | 1.00th=[ 2008], 5.00th=[ 3392], 10.00th=[ 4293], 20.00th=[ 6783], | 30.00th=[10290], 40.00th=[13829], 50.00th=[16909], 60.00th=[20317], | 70.00th=[25822], 80.00th=[31851], 90.00th=[41681], 95.00th=[46400], | 99.00th=[60556], 99.50th=[96994], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 191, max= 508, per=23.44%, avg=348.97, stdev=92.22, samples=33 iops : min= 11, max= 31, avg=21.18, stdev= 5.87, samples=33 lat (usec) : 250=0.39%, 500=2.60%, 750=0.39%, 1000=0.39% lat (msec) : 2=0.78%, 4=3.78%, 10=11.33%, 20=33.85%, 50=41.54% lat (msec) : 100=4.95% cpu : usr=0.13%, sys=25.92%, ctx=2318, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190451: Thu Jul 24 10:35:07 2025 read: IOPS=22, BW=359KiB/s (368kB/s)(6032KiB/16797msec) clat (usec): min=226, max=79020, avg=23023.82, stdev=12394.75 lat (usec): min=227, max=79022, avg=23025.54, stdev=12394.81 clat percentiles (usec): | 1.00th=[ 297], 5.00th=[ 734], 10.00th=[10421], 20.00th=[13435], | 30.00th=[16909], 40.00th=[19268], 50.00th=[21627], 60.00th=[25035], | 70.00th=[27657], 80.00th=[30278], 90.00th=[37487], 95.00th=[44827], | 99.00th=[66847], 99.50th=[76022], 99.90th=[79168], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 192, max= 665, per=25.16%, avg=355.97, stdev=93.75, samples=33 iops : min= 12, max= 41, avg=21.61, stdev= 5.83, samples=33 write: IOPS=23, BW=372KiB/s (381kB/s)(6256KiB/16797msec); 0 zone resets clat (usec): min=1779, max=142882, avg=20483.54, stdev=16832.12 lat (usec): min=1781, max=142885, avg=20486.93, stdev=16832.20 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 7], | 30.00th=[ 11], 40.00th=[ 14], 50.00th=[ 17], 60.00th=[ 20], | 70.00th=[ 25], 80.00th=[ 32], 90.00th=[ 42], 95.00th=[ 49], | 99.00th=[ 75], 99.50th=[ 99], 99.90th=[ 144], 99.95th=[ 144], | 99.99th=[ 144] bw ( KiB/s): min= 191, max= 576, per=25.00%, avg=372.18, stdev=89.74, samples=33 iops : min= 11, max= 36, avg=22.64, stdev= 5.76, samples=33 lat (usec) : 250=0.13%, 500=1.69%, 750=0.65%, 1000=0.39% lat (msec) : 2=0.39%, 4=2.73%, 10=13.54%, 20=31.90%, 50=44.53% lat (msec) : 100=3.91%, 250=0.13% cpu : usr=0.14%, sys=26.43%, ctx=2382, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1416KiB/s (1450kB/s), 340KiB/s-370KiB/s (349kB/s-379kB/s), io=23.4MiB (24.5MB), run=16309-16920msec WRITE: bw=1489KiB/s (1525kB/s), 357KiB/s-390KiB/s (365kB/s-400kB/s), io=24.6MiB (25.8MB), run=16309-16920msec rand-rw: (groupid=0, jobs=1): err= 0: pid=190443: Thu Jul 24 10:35:10 2025 read: IOPS=18, BW=291KiB/s (298kB/s)(5920KiB/20344msec) clat (msec): min=4, max=115, avg=22.08, stdev=18.00 lat (msec): min=4, max=115, avg=22.08, stdev=18.00 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 13], 50.00th=[ 15], 60.00th=[ 20], | 70.00th=[ 25], 80.00th=[ 35], 90.00th=[ 48], 95.00th=[ 63], | 99.00th=[ 79], 99.50th=[ 94], 99.90th=[ 116], 99.95th=[ 116], | 99.99th=[ 116] bw ( KiB/s): min= 63, max= 800, per=24.26%, avg=283.55, stdev=177.44, samples=40 iops : min= 3, max= 50, avg=17.23, stdev=11.23, samples=40 write: IOPS=19, BW=313KiB/s (321kB/s)(6368KiB/20344msec); 0 zone resets clat (msec): min=7, max=156, avg=30.41, stdev=21.65 lat (msec): min=7, max=156, avg=30.42, stdev=21.65 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 23], 60.00th=[ 29], | 70.00th=[ 36], 80.00th=[ 46], 90.00th=[ 59], 95.00th=[ 74], | 99.00th=[ 116], 99.50th=[ 150], 99.90th=[ 157], 99.95th=[ 157], | 99.99th=[ 157] bw ( KiB/s): min= 127, max= 800, per=25.09%, avg=308.62, stdev=168.76, samples=40 iops : min= 7, max= 50, avg=18.80, stdev=10.69, samples=40 lat (msec) : 10=15.89%, 20=35.68%, 50=35.68%, 100=11.98%, 250=0.78% cpu : usr=0.20%, sys=10.48%, ctx=1063, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190444: Thu Jul 24 10:35:10 2025 read: IOPS=17, BW=280KiB/s (287kB/s)(5744KiB/20480msec) clat (msec): min=4, max=120, avg=22.61, stdev=18.88 lat (msec): min=4, max=120, avg=22.61, stdev=18.88 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 19], | 70.00th=[ 28], 80.00th=[ 37], 90.00th=[ 52], 95.00th=[ 62], | 99.00th=[ 79], 99.50th=[ 115], 99.90th=[ 121], 99.95th=[ 121], | 99.99th=[ 121] bw ( KiB/s): min= 64, max= 768, per=23.46%, avg=274.20, stdev=165.24, samples=40 iops : min= 4, max= 48, avg=16.65, stdev=10.45, samples=40 write: IOPS=19, BW=320KiB/s (327kB/s)(6544KiB/20480msec); 0 zone resets clat (msec): min=6, max=171, avg=30.14, stdev=21.44 lat (msec): min=6, max=171, avg=30.15, stdev=21.44 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 23], 60.00th=[ 27], | 70.00th=[ 36], 80.00th=[ 43], 90.00th=[ 61], 95.00th=[ 74], | 99.00th=[ 104], 99.50th=[ 112], 99.90th=[ 171], 99.95th=[ 171], | 99.99th=[ 171] bw ( KiB/s): min= 159, max= 800, per=24.91%, avg=306.38, stdev=166.15, samples=40 iops : min= 9, max= 50, avg=18.65, stdev=10.56, samples=40 lat (msec) : 10=16.15%, 20=35.29%, 50=35.16%, 100=12.50%, 250=0.91% cpu : usr=0.15%, sys=10.40%, ctx=1094, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190445: Thu Jul 24 10:35:10 2025 read: IOPS=19, BW=311KiB/s (318kB/s)(6256KiB/20145msec) clat (usec): min=4056, max=94170, avg=21048.39, stdev=16954.78 lat (usec): min=4057, max=94171, avg=21050.12, stdev=16954.77 clat percentiles (usec): | 1.00th=[ 5211], 5.00th=[ 6587], 10.00th=[ 7439], 20.00th=[ 8979], | 30.00th=[10552], 40.00th=[12125], 50.00th=[14746], 60.00th=[17957], | 70.00th=[22152], 80.00th=[30802], 90.00th=[47449], 95.00th=[61080], | 99.00th=[82314], 99.50th=[88605], 99.90th=[93848], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 127, max= 864, per=26.16%, avg=305.85, stdev=187.63, samples=39 iops : min= 7, max= 54, avg=18.62, stdev=11.85, samples=39 write: IOPS=18, BW=299KiB/s (307kB/s)(6032KiB/20145msec); 0 zone resets clat (msec): min=8, max=131, avg=31.46, stdev=20.74 lat (msec): min=8, max=131, avg=31.47, stdev=20.74 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 12], 10.00th=[ 13], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 26], 60.00th=[ 31], | 70.00th=[ 40], 80.00th=[ 47], 90.00th=[ 59], 95.00th=[ 71], | 99.00th=[ 116], 99.50th=[ 121], 99.90th=[ 132], 99.95th=[ 132], | 99.99th=[ 132] bw ( KiB/s): min= 95, max= 731, per=23.37%, avg=287.49, stdev=148.22, samples=39 iops : min= 5, max= 45, avg=17.46, stdev= 9.39, samples=39 lat (msec) : 10=14.84%, 20=38.80%, 50=33.46%, 100=12.37%, 250=0.52% cpu : usr=0.08%, sys=10.87%, ctx=1125, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190446: Thu Jul 24 10:35:10 2025 read: IOPS=18, BW=298KiB/s (305kB/s)(6032KiB/20274msec) clat (usec): min=4572, max=94356, avg=21927.39, stdev=16644.76 lat (usec): min=4573, max=94359, avg=21929.21, stdev=16644.98 clat percentiles (usec): | 1.00th=[ 5473], 5.00th=[ 6652], 10.00th=[ 7439], 20.00th=[ 8717], | 30.00th=[10814], 40.00th=[13173], 50.00th=[16057], 60.00th=[19792], | 70.00th=[24511], 80.00th=[32375], 90.00th=[47973], 95.00th=[62129], | 99.00th=[73925], 99.50th=[85459], 99.90th=[93848], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 95, max= 800, per=25.07%, avg=293.10, stdev=173.70, samples=40 iops : min= 5, max= 50, avg=17.82, stdev=10.96, samples=40 write: IOPS=19, BW=309KiB/s (316kB/s)(6256KiB/20274msec); 0 zone resets clat (msec): min=6, max=176, avg=30.55, stdev=21.62 lat (msec): min=6, max=176, avg=30.55, stdev=21.62 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 13], 20.00th=[ 15], | 30.00th=[ 17], 40.00th=[ 20], 50.00th=[ 23], 60.00th=[ 29], | 70.00th=[ 36], 80.00th=[ 45], 90.00th=[ 60], 95.00th=[ 69], | 99.00th=[ 107], 99.50th=[ 148], 99.90th=[ 178], 99.95th=[ 178], | 99.99th=[ 178] bw ( KiB/s): min= 95, max= 896, per=24.85%, avg=305.62, stdev=172.94, samples=40 iops : min= 5, max= 56, avg=18.60, stdev=11.01, samples=40 lat (msec) : 10=14.97%, 20=35.94%, 50=36.46%, 100=12.11%, 250=0.52% cpu : usr=0.14%, sys=10.73%, ctx=1122, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1170KiB/s (1198kB/s), 280KiB/s-311KiB/s (287kB/s-318kB/s), io=23.4MiB (24.5MB), run=20145-20480msec WRITE: bw=1230KiB/s (1260kB/s), 299KiB/s-320KiB/s (307kB/s-327kB/s), io=24.6MiB (25.8MB), run=20145-20480msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=190462: Thu Jul 24 10:35:24 2025 read: IOPS=1, BW=1588KiB/s (1626kB/s)(7168KiB/4513msec) clat (usec): min=644, max=328441, avg=188271.09, stdev=112591.28 lat (usec): min=645, max=328443, avg=188272.82, stdev=112591.58 clat percentiles (usec): | 1.00th=[ 644], 5.00th=[ 644], 10.00th=[ 644], 20.00th=[ 95945], | 30.00th=[152044], 40.00th=[152044], 50.00th=[221250], 60.00th=[254804], | 70.00th=[254804], 80.00th=[265290], 90.00th=[329253], 95.00th=[329253], | 99.00th=[329253], 99.50th=[329253], 99.90th=[329253], 99.95th=[329253], | 99.99th=[329253] bw ( KiB/s): min= 1812, max= 4087, per=84.28%, avg=2972.50, stdev=1226.47, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 write: IOPS=1, BW=1135KiB/s (1162kB/s)(5120KiB/4513msec); 0 zone resets clat (msec): min=245, max=1384, avg=601.32, stdev=465.06 lat (msec): min=245, max=1384, avg=601.41, stdev=465.07 clat percentiles (msec): | 1.00th=[ 245], 5.00th=[ 245], 10.00th=[ 245], 20.00th=[ 245], | 30.00th=[ 284], 40.00th=[ 284], 50.00th=[ 456], 60.00th=[ 456], | 70.00th=[ 642], 80.00th=[ 642], 90.00th=[ 1385], 95.00th=[ 1385], | 99.00th=[ 1385], 99.50th=[ 1385], 99.90th=[ 1385], 99.95th=[ 1385], | 99.99th=[ 1385] bw ( KiB/s): min= 1961, max= 3923, per=53.13%, avg=2624.00, stdev=1125.05, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (usec) : 750=8.33% lat (msec) : 100=8.33%, 250=25.00%, 500=41.67%, 750=8.33%, 2000=8.33% cpu : usr=0.00%, sys=13.72%, ctx=157, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190463: Thu Jul 24 10:35:24 2025 read: IOPS=0, BW=353KiB/s (361kB/s)(2048KiB/5805msec) clat (msec): min=230, max=273, avg=252.16, stdev=30.63 lat (msec): min=230, max=273, avg=252.17, stdev=30.63 clat percentiles (msec): | 1.00th=[ 230], 5.00th=[ 230], 10.00th=[ 230], 20.00th=[ 230], | 30.00th=[ 230], 40.00th=[ 230], 50.00th=[ 230], 60.00th=[ 275], | 70.00th=[ 275], 80.00th=[ 275], 90.00th=[ 275], 95.00th=[ 275], | 99.00th=[ 275], 99.50th=[ 275], 99.90th=[ 275], 99.95th=[ 275], | 99.99th=[ 275] bw ( KiB/s): min= 1600, max= 2015, per=51.25%, avg=1807.50, stdev=293.45, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1764KiB/s (1806kB/s)(10.0MiB/5805msec); 0 zone resets clat (msec): min=153, max=1138, avg=501.93, stdev=291.58 lat (msec): min=154, max=1138, avg=502.04, stdev=291.56 clat percentiles (msec): | 1.00th=[ 155], 5.00th=[ 155], 10.00th=[ 155], 20.00th=[ 284], | 30.00th=[ 309], 40.00th=[ 368], 50.00th=[ 393], 60.00th=[ 409], | 70.00th=[ 523], 80.00th=[ 625], 90.00th=[ 818], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 1662, max= 4087, per=52.99%, avg=2617.17, stdev=1085.03, samples=6 iops : min= 1, max= 3, avg= 1.67, stdev= 1.03, samples=6 lat (msec) : 250=16.67%, 500=50.00%, 750=16.67%, 1000=8.33%, 2000=8.33% cpu : usr=0.00%, sys=14.46%, ctx=174, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190465: Thu Jul 24 10:35:24 2025 read: IOPS=1, BW=1586KiB/s (1624kB/s)(6144KiB/3873msec) clat (usec): min=547, max=349321, avg=238312.79, stdev=123633.07 lat (usec): min=548, max=349322, avg=238314.15, stdev=123633.12 clat percentiles (usec): | 1.00th=[ 545], 5.00th=[ 545], 10.00th=[ 545], 20.00th=[229639], | 30.00th=[229639], 40.00th=[254804], 50.00th=[254804], 60.00th=[291505], | 70.00th=[304088], 80.00th=[304088], 90.00th=[350225], 95.00th=[350225], | 99.00th=[350225], 99.50th=[350225], 99.90th=[350225], 99.95th=[350225], | 99.99th=[350225] bw ( KiB/s): min= 1906, max= 2027, per=55.82%, avg=1968.60, stdev=44.22, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 write: IOPS=1, BW=1586KiB/s (1624kB/s)(6144KiB/3873msec); 0 zone resets clat (msec): min=263, max=663, avg=392.55, stdev=172.29 lat (msec): min=263, max=664, avg=392.90, stdev=172.15 clat percentiles (msec): | 1.00th=[ 264], 5.00th=[ 264], 10.00th=[ 264], 20.00th=[ 268], | 30.00th=[ 268], 40.00th=[ 292], 50.00th=[ 292], 60.00th=[ 317], | 70.00th=[ 558], 80.00th=[ 558], 90.00th=[ 667], 95.00th=[ 667], | 99.00th=[ 667], 99.50th=[ 667], 99.90th=[ 667], 99.95th=[ 667], | 99.99th=[ 667] bw ( KiB/s): min= 1610, max= 2043, per=38.50%, avg=1901.60, stdev=170.32, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (usec) : 750=8.33% lat (msec) : 250=8.33%, 500=66.67%, 750=16.67% cpu : usr=0.08%, sys=14.90%, ctx=203, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190467: Thu Jul 24 10:35:24 2025 read: IOPS=0, BW=960KiB/s (983kB/s)(5120KiB/5334msec) clat (usec): min=1143, max=363276, avg=187375.04, stdev=132650.07 lat (usec): min=1144, max=363278, avg=187375.92, stdev=132650.15 clat percentiles (usec): | 1.00th=[ 1139], 5.00th=[ 1139], 10.00th=[ 1139], 20.00th=[ 1139], | 30.00th=[137364], 40.00th=[137364], 50.00th=[206570], 60.00th=[206570], | 70.00th=[229639], 80.00th=[229639], 90.00th=[362808], 95.00th=[362808], | 99.00th=[362808], 99.50th=[362808], 99.90th=[362808], 99.95th=[362808], | 99.99th=[362808] bw ( KiB/s): min= 4031, max= 4087, per=100.00%, avg=4059.00, stdev=39.60, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1344KiB/s (1376kB/s)(7168KiB/5334msec); 0 zone resets clat (msec): min=136, max=924, avg=577.71, stdev=326.57 lat (msec): min=136, max=924, avg=577.82, stdev=326.59 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 288], | 30.00th=[ 305], 40.00th=[ 305], 50.00th=[ 693], 60.00th=[ 835], | 70.00th=[ 835], 80.00th=[ 869], 90.00th=[ 927], 95.00th=[ 927], | 99.00th=[ 927], 99.50th=[ 927], 99.90th=[ 927], 99.95th=[ 927], | 99.99th=[ 927] bw ( KiB/s): min= 1662, max= 4087, per=45.78%, avg=2260.83, stdev=902.62, samples=6 iops : min= 1, max= 3, avg= 1.33, stdev= 0.82, samples=6 lat (msec) : 2=8.33%, 250=33.33%, 500=25.00%, 750=8.33%, 1000=25.00% cpu : usr=0.00%, sys=9.77%, ctx=193, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3528KiB/s (3613kB/s), 353KiB/s-1588KiB/s (361kB/s-1626kB/s), io=20.0MiB (20.0MB), run=3873-5805msec WRITE: bw=4939KiB/s (5058kB/s), 1135KiB/s-1764KiB/s (1162kB/s-1806kB/s), io=28.0MiB (29.4MB), run=3873-5805msec rand-rw: (groupid=0, jobs=1): err= 0: pid=190460: Thu Jul 24 10:35:25 2025 read: IOPS=1, BW=2003KiB/s (2051kB/s)(7168KiB/3579msec) clat (msec): min=99, max=354, avg=192.49, stdev=90.81 lat (msec): min=99, max=354, avg=192.49, stdev=90.81 clat percentiles (msec): | 1.00th=[ 100], 5.00th=[ 100], 10.00th=[ 100], 20.00th=[ 122], | 30.00th=[ 140], 40.00th=[ 140], 50.00th=[ 153], 60.00th=[ 230], | 70.00th=[ 230], 80.00th=[ 253], 90.00th=[ 355], 95.00th=[ 355], | 99.00th=[ 355], 99.50th=[ 355], 99.90th=[ 355], 99.95th=[ 355], | 99.99th=[ 355] bw ( KiB/s): min= 2019, max= 3968, per=96.46%, avg=2932.50, stdev=1055.23, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 write: IOPS=1, BW=1431KiB/s (1465kB/s)(5120KiB/3579msec); 0 zone resets clat (msec): min=134, max=853, avg=432.15, stdev=263.75 lat (msec): min=134, max=853, avg=432.30, stdev=263.73 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 136], | 30.00th=[ 342], 40.00th=[ 342], 50.00th=[ 376], 60.00th=[ 376], | 70.00th=[ 456], 80.00th=[ 456], 90.00th=[ 852], 95.00th=[ 852], | 99.00th=[ 852], 99.50th=[ 852], 99.90th=[ 852], 99.95th=[ 852], | 99.99th=[ 852] bw ( KiB/s): min= 2003, max= 4055, per=63.24%, avg=2692.33, stdev=1180.13, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 100=8.33%, 250=41.67%, 500=41.67%, 1000=8.33% cpu : usr=0.03%, sys=4.28%, ctx=62, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190461: Thu Jul 24 10:35:25 2025 read: IOPS=0, BW=304KiB/s (311kB/s)(2048KiB/6735msec) clat (msec): min=159, max=161, avg=160.69, stdev= 1.58 lat (msec): min=159, max=161, avg=160.69, stdev= 1.58 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 161], | 30.00th=[ 161], 40.00th=[ 161], 50.00th=[ 161], 60.00th=[ 163], | 70.00th=[ 163], 80.00th=[ 163], 90.00th=[ 163], 95.00th=[ 163], | 99.00th=[ 163], 99.50th=[ 163], 99.90th=[ 163], 99.95th=[ 163], | 99.99th=[ 163] bw ( KiB/s): min= 1643, max= 1996, per=59.85%, avg=1819.50, stdev=249.61, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1520KiB/s (1557kB/s)(10.0MiB/6735msec); 0 zone resets clat (msec): min=131, max=1520, avg=633.68, stdev=448.31 lat (msec): min=131, max=1520, avg=633.82, stdev=448.33 clat percentiles (msec): | 1.00th=[ 132], 5.00th=[ 132], 10.00th=[ 132], 20.00th=[ 161], | 30.00th=[ 288], 40.00th=[ 359], 50.00th=[ 542], 60.00th=[ 584], | 70.00th=[ 684], 80.00th=[ 944], 90.00th=[ 1116], 95.00th=[ 1519], | 99.00th=[ 1519], 99.50th=[ 1519], 99.90th=[ 1519], 99.95th=[ 1519], | 99.99th=[ 1519] bw ( KiB/s): min= 1790, max= 4015, per=57.22%, avg=2435.86, stdev=843.04, samples=7 iops : min= 1, max= 3, avg= 1.57, stdev= 0.98, samples=7 lat (msec) : 250=33.33%, 500=16.67%, 750=25.00%, 1000=8.33%, 2000=16.67% cpu : usr=0.00%, sys=3.55%, ctx=81, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190464: Thu Jul 24 10:35:25 2025 read: IOPS=0, BW=960KiB/s (983kB/s)(6144KiB/6401msec) clat (msec): min=134, max=776, avg=479.03, stdev=238.72 lat (msec): min=134, max=776, avg=479.04, stdev=238.72 clat percentiles (msec): | 1.00th=[ 134], 5.00th=[ 134], 10.00th=[ 134], 20.00th=[ 330], | 30.00th=[ 330], 40.00th=[ 376], 50.00th=[ 376], 60.00th=[ 625], | 70.00th=[ 634], 80.00th=[ 634], 90.00th=[ 776], 95.00th=[ 776], | 99.00th=[ 776], 99.50th=[ 776], 99.90th=[ 776], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 1984, max= 4055, per=88.10%, avg=2678.33, stdev=1192.24, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=0, BW=960KiB/s (983kB/s)(6144KiB/6401msec); 0 zone resets clat (msec): min=133, max=1023, avg=551.23, stdev=349.46 lat (msec): min=133, max=1024, avg=551.32, stdev=349.48 clat percentiles (msec): | 1.00th=[ 134], 5.00th=[ 134], 10.00th=[ 134], 20.00th=[ 247], | 30.00th=[ 247], 40.00th=[ 439], 50.00th=[ 439], 60.00th=[ 592], | 70.00th=[ 877], 80.00th=[ 877], 90.00th=[ 1028], 95.00th=[ 1028], | 99.00th=[ 1028], 99.50th=[ 1028], 99.90th=[ 1028], 99.95th=[ 1028], | 99.99th=[ 1028] bw ( KiB/s): min= 1790, max= 2019, per=45.43%, avg=1933.80, stdev=102.34, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (msec) : 250=25.00%, 500=25.00%, 750=25.00%, 1000=16.67%, 2000=8.33% cpu : usr=0.00%, sys=2.95%, ctx=74, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190466: Thu Jul 24 10:35:25 2025 read: IOPS=0, BW=781KiB/s (800kB/s)(5120KiB/6554msec) clat (msec): min=134, max=1011, avg=468.87, stdev=409.43 lat (msec): min=134, max=1011, avg=468.88, stdev=409.43 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 136], | 30.00th=[ 165], 40.00th=[ 165], 50.00th=[ 230], 60.00th=[ 230], | 70.00th=[ 810], 80.00th=[ 810], 90.00th=[ 1011], 95.00th=[ 1011], | 99.00th=[ 1011], 99.50th=[ 1011], 99.90th=[ 1011], 99.95th=[ 1011], | 99.99th=[ 1011] bw ( KiB/s): min= 1976, max= 4039, per=87.71%, avg=2666.33, stdev=1188.77, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=1, BW=1094KiB/s (1120kB/s)(7168KiB/6554msec); 0 zone resets clat (msec): min=114, max=828, avg=553.00, stdev=258.44 lat (msec): min=114, max=828, avg=553.13, stdev=258.45 clat percentiles (msec): | 1.00th=[ 115], 5.00th=[ 115], 10.00th=[ 115], 20.00th=[ 321], | 30.00th=[ 550], 40.00th=[ 550], 50.00th=[ 609], 60.00th=[ 625], | 70.00th=[ 625], 80.00th=[ 818], 90.00th=[ 827], 95.00th=[ 827], | 99.00th=[ 827], 99.50th=[ 827], 99.90th=[ 827], 99.95th=[ 827], | 99.99th=[ 827] bw ( KiB/s): min= 1605, max= 2011, per=44.16%, avg=1879.80, stdev=161.85, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (msec) : 250=33.33%, 500=8.33%, 750=25.00%, 1000=25.00%, 2000=8.33% cpu : usr=0.00%, sys=3.11%, ctx=79, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=3041KiB/s (3114kB/s), 304KiB/s-2003KiB/s (311kB/s-2051kB/s), io=20.0MiB (20.0MB), run=3579-6735msec WRITE: bw=4257KiB/s (4359kB/s), 960KiB/s-1520KiB/s (983kB/s-1557kB/s), io=28.0MiB (29.4MB), run=3579-6735msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=190474: Thu Jul 24 10:35:44 2025 read: IOPS=0, BW=2067KiB/s (2117kB/s)(12.0MiB/5944msec) clat (msec): min=748, max=3242, avg=1980.95, stdev=1247.38 lat (msec): min=748, max=3242, avg=1980.95, stdev=1247.38 clat percentiles (msec): | 1.00th=[ 751], 5.00th=[ 751], 10.00th=[ 751], 20.00th=[ 751], | 30.00th=[ 751], 40.00th=[ 1955], 50.00th=[ 1955], 60.00th=[ 1955], | 70.00th=[ 3239], 80.00th=[ 3239], 90.00th=[ 3239], 95.00th=[ 3239], | 99.00th=[ 3239], 99.50th=[ 3239], 99.90th=[ 3239], 99.95th=[ 3239], | 99.99th=[ 3239] bw ( KiB/s): min= 6942, max= 8062, per=100.00%, avg=7502.00, stdev=791.96, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.08%, sys=3.05%, ctx=46, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190475: Thu Jul 24 10:35:44 2025 read: IOPS=0, BW=670KiB/s (686kB/s)(4096KiB/6116msec) clat (nsec): min=1303.0M, max=1303.0M, avg=1303022959.00, stdev= 0.00 lat (nsec): min=1303.0M, max=1303.0M, avg=1303025334.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1301], 5.00th=[ 1301], 10.00th=[ 1301], 20.00th=[ 1301], | 30.00th=[ 1301], 40.00th=[ 1301], 50.00th=[ 1301], 60.00th=[ 1301], | 70.00th=[ 1301], 80.00th=[ 1301], 90.00th=[ 1301], 95.00th=[ 1301], | 99.00th=[ 1301], 99.50th=[ 1301], 99.90th=[ 1301], 99.95th=[ 1301], | 99.99th=[ 1301] bw ( KiB/s): min= 7433, max= 7433, per=100.00%, avg=7433.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1339KiB/s (1372kB/s)(8192KiB/6116msec); 0 zone resets clat (msec): min=1539, max=3165, avg=2352.63, stdev=1149.36 lat (msec): min=1540, max=3165, avg=2352.88, stdev=1149.25 clat percentiles (msec): | 1.00th=[ 1536], 5.00th=[ 1536], 10.00th=[ 1536], 20.00th=[ 1536], | 30.00th=[ 1536], 40.00th=[ 1536], 50.00th=[ 1536], 60.00th=[ 3171], | 70.00th=[ 3171], 80.00th=[ 3171], 90.00th=[ 3171], 95.00th=[ 3171], | 99.00th=[ 3171], 99.50th=[ 3171], 99.90th=[ 3171], 99.95th=[ 3171], | 99.99th=[ 3171] bw ( KiB/s): min= 8031, max= 8031, per=100.00%, avg=8031.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 2000=66.67%, >=2000=33.33% cpu : usr=0.11%, sys=4.37%, ctx=75, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190476: Thu Jul 24 10:35:44 2025 write: IOPS=0, BW=1936KiB/s (1982kB/s)(12.0MiB/6348msec); 0 zone resets clat (msec): min=661, max=4963, avg=2095.37, stdev=2483.65 lat (msec): min=661, max=4965, avg=2096.73, stdev=2484.68 clat percentiles (msec): | 1.00th=[ 659], 5.00th=[ 659], 10.00th=[ 659], 20.00th=[ 659], | 30.00th=[ 659], 40.00th=[ 659], 50.00th=[ 659], 60.00th=[ 659], | 70.00th=[ 4933], 80.00th=[ 4933], 90.00th=[ 4933], 95.00th=[ 4933], | 99.00th=[ 4933], 99.50th=[ 4933], 99.90th=[ 4933], 99.95th=[ 4933], | 99.99th=[ 4933] bw ( KiB/s): min= 7433, max= 8062, per=100.00%, avg=7747.50, stdev=444.77, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=66.67%, >=2000=33.33% cpu : usr=0.00%, sys=2.69%, ctx=52, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190477: Thu Jul 24 10:35:44 2025 write: IOPS=0, BW=2739KiB/s (2804kB/s)(12.0MiB/4487msec); 0 zone resets clat (msec): min=631, max=2474, avg=1467.19, stdev=933.02 lat (msec): min=637, max=2474, avg=1469.48, stdev=930.37 clat percentiles (msec): | 1.00th=[ 634], 5.00th=[ 634], 10.00th=[ 634], 20.00th=[ 634], | 30.00th=[ 634], 40.00th=[ 1301], 50.00th=[ 1301], 60.00th=[ 1301], | 70.00th=[ 2467], 80.00th=[ 2467], 90.00th=[ 2467], 95.00th=[ 2467], | 99.00th=[ 2467], 99.50th=[ 2467], 99.90th=[ 2467], 99.95th=[ 2467], | 99.99th=[ 2467] bw ( KiB/s): min= 7876, max= 8062, per=100.00%, avg=7969.00, stdev=131.52, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.16%, sys=3.72%, ctx=82, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2679KiB/s (2743kB/s), 670KiB/s-2067KiB/s (686kB/s-2117kB/s), io=16.0MiB (16.8MB), run=5944-6116msec WRITE: bw=5162KiB/s (5286kB/s), 1339KiB/s-2739KiB/s (1372kB/s-2804kB/s), io=32.0MiB (33.6MB), run=4487-6348msec rand-rw: (groupid=0, jobs=1): err= 0: pid=190479: Thu Jul 24 10:35:45 2025 read: IOPS=0, BW=3961KiB/s (4056kB/s)(12.0MiB/3102msec) clat (msec): min=943, max=1192, avg=1033.71, stdev=138.02 lat (msec): min=943, max=1192, avg=1033.71, stdev=138.02 clat percentiles (msec): | 1.00th=[ 944], 5.00th=[ 944], 10.00th=[ 944], 20.00th=[ 944], | 30.00th=[ 944], 40.00th=[ 969], 50.00th=[ 969], 60.00th=[ 969], | 70.00th=[ 1200], 80.00th=[ 1200], 90.00th=[ 1200], 95.00th=[ 1200], | 99.00th=[ 1200], 99.50th=[ 1200], 99.90th=[ 1200], 99.95th=[ 1200], | 99.99th=[ 1200] bw ( KiB/s): min= 7557, max= 8078, per=100.00%, avg=7817.50, stdev=368.40, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=66.67%, 2000=33.33% cpu : usr=0.00%, sys=9.32%, ctx=220, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190480: Thu Jul 24 10:35:45 2025 read: IOPS=0, BW=721KiB/s (739kB/s)(4096KiB/5679msec) clat (nsec): min=922204k, max=922204k, avg=922204304.00, stdev= 0.00 lat (nsec): min=922205k, max=922205k, avg=922205318.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 919], 5.00th=[ 919], 10.00th=[ 919], 20.00th=[ 919], | 30.00th=[ 919], 40.00th=[ 919], 50.00th=[ 919], 60.00th=[ 919], | 70.00th=[ 919], 80.00th=[ 919], 90.00th=[ 919], 95.00th=[ 919], | 99.00th=[ 919], 99.50th=[ 919], 99.90th=[ 919], 99.95th=[ 919], | 99.99th=[ 919] bw ( KiB/s): min= 7557, max= 7557, per=100.00%, avg=7557.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=1443KiB/s (1477kB/s)(8192KiB/5679msec); 0 zone resets clat (msec): min=1489, max=3263, avg=2376.52, stdev=1254.30 lat (msec): min=1489, max=3263, avg=2376.90, stdev=1254.44 clat percentiles (msec): | 1.00th=[ 1485], 5.00th=[ 1485], 10.00th=[ 1485], 20.00th=[ 1485], | 30.00th=[ 1485], 40.00th=[ 1485], 50.00th=[ 1485], 60.00th=[ 3272], | 70.00th=[ 3272], 80.00th=[ 3272], 90.00th=[ 3272], 95.00th=[ 3272], | 99.00th=[ 3272], 99.50th=[ 3272], 99.90th=[ 3272], 99.95th=[ 3272], | 99.99th=[ 3272] bw ( KiB/s): min= 7968, max= 7968, per=100.00%, avg=7968.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 1000=33.33%, 2000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=8.49%, ctx=147, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190481: Thu Jul 24 10:35:45 2025 write: IOPS=0, BW=2088KiB/s (2138kB/s)(12.0MiB/5884msec); 0 zone resets clat (msec): min=1508, max=2468, avg=1960.73, stdev=482.52 lat (msec): min=1508, max=2468, avg=1961.10, stdev=482.57 clat percentiles (msec): | 1.00th=[ 1502], 5.00th=[ 1502], 10.00th=[ 1502], 20.00th=[ 1502], | 30.00th=[ 1502], 40.00th=[ 1905], 50.00th=[ 1905], 60.00th=[ 1905], | 70.00th=[ 2467], 80.00th=[ 2467], 90.00th=[ 2467], 95.00th=[ 2467], | 99.00th=[ 2467], 99.50th=[ 2467], 99.90th=[ 2467], 99.95th=[ 2467], | 99.99th=[ 2467] bw ( KiB/s): min= 8015, max= 8078, per=100.00%, avg=8046.50, stdev=44.55, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 2000=66.67%, >=2000=33.33% cpu : usr=0.07%, sys=10.18%, ctx=71, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=190482: Thu Jul 24 10:35:45 2025 write: IOPS=0, BW=3048KiB/s (3121kB/s)(12.0MiB/4032msec); 0 zone resets clat (msec): min=791, max=1857, avg=1341.71, stdev=533.59 lat (msec): min=792, max=1857, avg=1342.14, stdev=533.39 clat percentiles (msec): | 1.00th=[ 793], 5.00th=[ 793], 10.00th=[ 793], 20.00th=[ 793], | 30.00th=[ 793], 40.00th=[ 1385], 50.00th=[ 1385], 60.00th=[ 1385], | 70.00th=[ 1854], 80.00th=[ 1854], 90.00th=[ 1854], 95.00th=[ 1854], | 99.00th=[ 1854], 99.50th=[ 1854], 99.90th=[ 1854], 99.95th=[ 1854], | 99.99th=[ 1854] bw ( KiB/s): min= 7907, max= 8110, per=100.00%, avg=8008.50, stdev=143.54, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.25%, sys=16.70%, ctx=83, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2885KiB/s (2954kB/s), 721KiB/s-3961KiB/s (739kB/s-4056kB/s), io=16.0MiB (16.8MB), run=3102-5679msec WRITE: bw=5569KiB/s (5703kB/s), 1443KiB/s-3048KiB/s (1477kB/s-3121kB/s), io=32.0MiB (33.6MB), run=4032-5884msec