== sanity test 398b: DIO and buffer IO race ============== 11:19:03 (1755098343) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.57213 s, 19.6 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189371: Wed Aug 13 11:19:59 2025 read: IOPS=31, BW=127KiB/s (130kB/s)(5992KiB/47316msec) clat (usec): min=251, max=833068, avg=17278.83, stdev=22673.74 lat (usec): min=251, max=833068, avg=17279.43, stdev=22673.75 clat percentiles (usec): | 1.00th=[ 424], 5.00th=[ 7635], 10.00th=[ 8586], 20.00th=[ 10290], | 30.00th=[ 11863], 40.00th=[ 13435], 50.00th=[ 15401], 60.00th=[ 17433], | 70.00th=[ 19530], 80.00th=[ 21890], 90.00th=[ 26346], 95.00th=[ 31327], | 99.00th=[ 44303], 99.50th=[ 49021], 99.90th=[ 89654], 99.95th=[834667], | 99.99th=[834667] bw ( KiB/s): min= 6, max= 215, per=24.95%, avg=126.52, stdev=32.44, samples=94 iops : min= 1, max= 53, avg=31.31, stdev= 8.09, samples=94 write: IOPS=33, BW=133KiB/s (136kB/s)(6296KiB/47316msec); 0 zone resets clat (usec): min=681, max=77505, avg=13480.70, stdev=10477.79 lat (usec): min=681, max=77505, avg=13481.46, stdev=10477.78 clat percentiles (usec): | 1.00th=[ 1156], 5.00th=[ 2573], 10.00th=[ 2966], 20.00th=[ 3589], | 30.00th=[ 5997], 40.00th=[ 8979], 50.00th=[11076], 60.00th=[13435], | 70.00th=[16909], 80.00th=[21365], 90.00th=[27919], 95.00th=[34866], | 99.00th=[46924], 99.50th=[50594], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 24, max= 223, per=25.28%, avg=134.26, stdev=37.22, samples=93 iops : min= 6, max= 55, avg=33.25, stdev= 9.29, samples=93 lat (usec) : 500=0.78%, 750=0.23%, 1000=0.20% lat (msec) : 2=0.85%, 4=10.64%, 10=18.72%, 20=43.39%, 50=24.67% lat (msec) : 100=0.49%, 1000=0.03% cpu : usr=0.10%, sys=26.15%, ctx=7977, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189372: Wed Aug 13 11:19:59 2025 read: IOPS=31, BW=126KiB/s (129kB/s)(5940KiB/47037msec) clat (usec): min=310, max=80435, avg=17079.73, stdev=8265.92 lat (usec): min=311, max=80435, avg=17080.41, stdev=8266.01 clat percentiles (usec): | 1.00th=[ 465], 5.00th=[ 7898], 10.00th=[ 8848], 20.00th=[10552], | 30.00th=[12125], 40.00th=[13960], 50.00th=[15795], 60.00th=[17433], | 70.00th=[19792], 80.00th=[22152], 90.00th=[26870], 95.00th=[31851], | 99.00th=[44827], 99.50th=[47973], 99.90th=[68682], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 6, max= 192, per=24.87%, avg=126.09, stdev=34.05, samples=93 iops : min= 1, max= 48, avg=31.20, stdev= 8.52, samples=93 write: IOPS=33, BW=135KiB/s (138kB/s)(6348KiB/47037msec); 0 zone resets clat (usec): min=691, max=831736, avg=13526.49, stdev=23069.82 lat (usec): min=691, max=831737, avg=13527.26, stdev=23069.83 clat percentiles (usec): | 1.00th=[ 1123], 5.00th=[ 2573], 10.00th=[ 2966], 20.00th=[ 3621], | 30.00th=[ 5276], 40.00th=[ 8455], 50.00th=[ 10814], 60.00th=[ 12780], | 70.00th=[ 15926], 80.00th=[ 20055], 90.00th=[ 27657], 95.00th=[ 33162], | 99.00th=[ 49546], 99.50th=[ 57934], 99.90th=[ 93848], 99.95th=[834667], | 99.99th=[834667] bw ( KiB/s): min= 55, max= 207, per=25.54%, avg=135.64, stdev=35.52, samples=92 iops : min= 13, max= 51, avg=33.60, stdev= 8.88, samples=92 lat (usec) : 500=0.62%, 750=0.20%, 1000=0.20% lat (msec) : 2=0.85%, 4=11.07%, 10=18.82%, 20=43.72%, 50=23.89% lat (msec) : 100=0.62%, 1000=0.03% cpu : usr=0.11%, sys=26.29%, ctx=8068, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189373: Wed Aug 13 11:19:59 2025 read: IOPS=31, BW=128KiB/s (131kB/s)(6028KiB/47152msec) clat (usec): min=101, max=84381, avg=17019.13, stdev=8497.80 lat (usec): min=102, max=84382, avg=17019.84, stdev=8497.84 clat percentiles (usec): | 1.00th=[ 449], 5.00th=[ 7767], 10.00th=[ 8848], 20.00th=[10421], | 30.00th=[11863], 40.00th=[13566], 50.00th=[15533], 60.00th=[17433], | 70.00th=[19792], 80.00th=[22414], 90.00th=[26346], 95.00th=[33817], | 99.00th=[45876], 99.50th=[47449], 99.90th=[60031], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 40, max= 199, per=25.44%, avg=128.96, stdev=29.06, samples=92 iops : min= 10, max= 49, avg=31.92, stdev= 7.23, samples=92 write: IOPS=33, BW=133KiB/s (136kB/s)(6260KiB/47152msec); 0 zone resets clat (usec): min=721, max=777245, avg=13611.20, stdev=21888.56 lat (usec): min=722, max=777247, avg=13612.16, stdev=21888.49 clat percentiles (usec): | 1.00th=[ 1004], 5.00th=[ 2442], 10.00th=[ 2933], 20.00th=[ 3654], | 30.00th=[ 5473], 40.00th=[ 8848], 50.00th=[ 11076], 60.00th=[ 13304], | 70.00th=[ 16188], 80.00th=[ 20579], 90.00th=[ 26608], 95.00th=[ 32637], | 99.00th=[ 49021], 99.50th=[ 55313], 99.90th=[ 63701], 99.95th=[775947], | 99.99th=[775947] bw ( KiB/s): min= 48, max= 240, per=25.14%, avg=133.48, stdev=33.98, samples=92 iops : min= 12, max= 60, avg=33.05, stdev= 8.53, samples=92 lat (usec) : 250=0.10%, 500=0.65%, 750=0.23%, 1000=0.42% lat (msec) : 2=1.14%, 4=10.35%, 10=18.91%, 20=43.36%, 50=24.19% lat (msec) : 100=0.62%, 1000=0.03% cpu : usr=0.08%, sys=26.24%, ctx=8060, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189374: Wed Aug 13 11:19:59 2025 read: IOPS=32, BW=130KiB/s (133kB/s)(6052KiB/46664msec) clat (usec): min=252, max=89850, avg=16843.25, stdev=8556.12 lat (usec): min=253, max=89851, avg=16843.89, stdev=8556.15 clat percentiles (usec): | 1.00th=[ 392], 5.00th=[ 7373], 10.00th=[ 8586], 20.00th=[10159], | 30.00th=[11731], 40.00th=[13566], 50.00th=[15795], 60.00th=[17433], | 70.00th=[19792], 80.00th=[22414], 90.00th=[26084], 95.00th=[30540], | 99.00th=[47973], 99.50th=[53740], 99.90th=[73925], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 64, max= 191, per=25.77%, avg=130.64, stdev=25.88, samples=91 iops : min= 16, max= 47, avg=32.34, stdev= 6.45, samples=91 write: IOPS=33, BW=134KiB/s (137kB/s)(6236KiB/46664msec); 0 zone resets clat (usec): min=673, max=760976, avg=13457.43, stdev=21305.37 lat (usec): min=673, max=760977, avg=13458.22, stdev=21305.36 clat percentiles (usec): | 1.00th=[ 1254], 5.00th=[ 2540], 10.00th=[ 2933], 20.00th=[ 3687], | 30.00th=[ 5669], 40.00th=[ 8848], 50.00th=[ 10814], 60.00th=[ 13042], | 70.00th=[ 16450], 80.00th=[ 20841], 90.00th=[ 26608], 95.00th=[ 31589], | 99.00th=[ 43254], 99.50th=[ 47973], 99.90th=[ 67634], 99.95th=[759170], | 99.99th=[759170] bw ( KiB/s): min= 20, max= 216, per=25.11%, avg=133.34, stdev=34.05, samples=92 iops : min= 5, max= 54, avg=33.02, stdev= 8.50, samples=92 lat (usec) : 500=0.88%, 750=0.23%, 1000=0.26% lat (msec) : 2=0.81%, 4=10.19%, 10=19.79%, 20=42.48%, 50=24.87% lat (msec) : 100=0.46%, 1000=0.03% cpu : usr=0.10%, sys=26.66%, ctx=8073, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=507KiB/s (520kB/s), 126KiB/s-130KiB/s (129kB/s-133kB/s), io=23.4MiB (24.6MB), run=46664-47316msec WRITE: bw=531KiB/s (544kB/s), 133KiB/s-135KiB/s (136kB/s-138kB/s), io=24.6MiB (25.7MB), run=46664-47316msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189375: Wed Aug 13 11:20:06 2025 read: IOPS=27, BW=111KiB/s (113kB/s)(5992KiB/54186msec) clat (msec): min=3, max=837, avg=15.91, stdev=24.49 lat (msec): min=3, max=837, avg=15.91, stdev=24.49 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 10], 50.00th=[ 11], 60.00th=[ 14], | 70.00th=[ 18], 80.00th=[ 25], 90.00th=[ 34], 95.00th=[ 40], | 99.00th=[ 57], 99.50th=[ 63], 99.90th=[ 104], 99.95th=[ 835], | 99.99th=[ 835] bw ( KiB/s): min= 32, max= 376, per=25.05%, avg=110.71, stdev=76.93, samples=106 iops : min= 8, max= 94, avg=27.39, stdev=19.35, samples=106 write: IOPS=29, BW=116KiB/s (119kB/s)(6296KiB/54186msec); 0 zone resets clat (msec): min=4, max=108, avg=19.17, stdev=14.11 lat (msec): min=4, max=108, avg=19.17, stdev=14.11 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 20], | 70.00th=[ 25], 80.00th=[ 30], 90.00th=[ 39], 95.00th=[ 46], | 99.00th=[ 70], 99.50th=[ 75], 99.90th=[ 102], 99.95th=[ 109], | 99.99th=[ 109] bw ( KiB/s): min= 8, max= 384, per=25.09%, avg=115.91, stdev=79.05, samples=106 iops : min= 2, max= 96, avg=28.69, stdev=19.86, samples=106 lat (msec) : 4=1.27%, 10=38.51%, 20=28.16%, 50=29.46%, 100=2.47% lat (msec) : 250=0.10%, 1000=0.03% cpu : usr=0.09%, sys=13.43%, ctx=3951, majf=0, minf=1510 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189376: Wed Aug 13 11:20:06 2025 read: IOPS=27, BW=109KiB/s (112kB/s)(5940KiB/54255msec) clat (msec): min=2, max=769, avg=15.90, stdev=23.11 lat (msec): min=2, max=769, avg=15.90, stdev=23.11 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 14], | 70.00th=[ 18], 80.00th=[ 25], 90.00th=[ 34], 95.00th=[ 41], | 99.00th=[ 56], 99.50th=[ 61], 99.90th=[ 96], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 8, max= 376, per=24.98%, avg=110.42, stdev=80.21, samples=107 iops : min= 2, max= 94, avg=27.32, stdev=20.16, samples=107 write: IOPS=29, BW=117KiB/s (120kB/s)(6348KiB/54255msec); 0 zone resets clat (msec): min=4, max=110, avg=19.19, stdev=14.07 lat (msec): min=4, max=110, avg=19.20, stdev=14.07 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 31], 90.00th=[ 39], 95.00th=[ 47], | 99.00th=[ 65], 99.50th=[ 77], 99.90th=[ 100], 99.95th=[ 110], | 99.99th=[ 110] bw ( KiB/s): min= 40, max= 360, per=25.57%, avg=118.11, stdev=79.92, samples=107 iops : min= 10, max= 90, avg=29.24, stdev=20.08, samples=107 lat (msec) : 4=1.53%, 10=38.54%, 20=27.57%, 50=29.56%, 100=2.73% lat (msec) : 250=0.03%, 1000=0.03% cpu : usr=0.09%, sys=13.30%, ctx=3954, majf=0, minf=1497 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189377: Wed Aug 13 11:20:06 2025 read: IOPS=27, BW=111KiB/s (114kB/s)(6028KiB/54170msec) clat (msec): min=2, max=104, avg=15.30, stdev=12.53 lat (msec): min=2, max=104, avg=15.30, stdev=12.53 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 23], 90.00th=[ 35], 95.00th=[ 43], | 99.00th=[ 57], 99.50th=[ 64], 99.90th=[ 70], 99.95th=[ 105], | 99.99th=[ 105] bw ( KiB/s): min= 23, max= 376, per=25.16%, avg=111.22, stdev=79.91, samples=106 iops : min= 5, max= 94, avg=27.52, stdev=20.08, samples=106 write: IOPS=28, BW=116KiB/s (118kB/s)(6260KiB/54170msec); 0 zone resets clat (msec): min=4, max=776, avg=19.76, stdev=23.88 lat (msec): min=4, max=776, avg=19.77, stdev=23.88 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 20], | 70.00th=[ 25], 80.00th=[ 31], 90.00th=[ 39], 95.00th=[ 48], | 99.00th=[ 69], 99.50th=[ 80], 99.90th=[ 109], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 31, max= 368, per=24.97%, avg=115.38, stdev=78.06, samples=106 iops : min= 7, max= 92, avg=28.56, stdev=19.63, samples=106 lat (msec) : 4=1.24%, 10=39.75%, 20=27.90%, 50=28.22%, 100=2.73% lat (msec) : 250=0.13%, 1000=0.03% cpu : usr=0.08%, sys=13.52%, ctx=3976, majf=0, minf=1519 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189378: Wed Aug 13 11:20:06 2025 read: IOPS=27, BW=111KiB/s (114kB/s)(6052KiB/54311msec) clat (usec): min=2630, max=78747, avg=15519.87, stdev=12458.56 lat (usec): min=2631, max=78748, avg=15520.47, stdev=12458.58 clat percentiles (usec): | 1.00th=[ 3228], 5.00th=[ 4228], 10.00th=[ 4752], 20.00th=[ 5604], | 30.00th=[ 6718], 40.00th=[ 8979], 50.00th=[11076], 60.00th=[13698], | 70.00th=[17957], 80.00th=[24511], 90.00th=[33817], 95.00th=[41157], | 99.00th=[57410], 99.50th=[64226], 99.90th=[70779], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 16, max= 408, per=25.35%, avg=112.06, stdev=79.63, samples=107 iops : min= 4, max= 102, avg=27.73, stdev=20.01, samples=107 write: IOPS=28, BW=115KiB/s (118kB/s)(6236KiB/54311msec); 0 zone resets clat (msec): min=3, max=772, avg=19.66, stdev=23.78 lat (msec): min=3, max=772, avg=19.66, stdev=23.78 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 15], 60.00th=[ 19], | 70.00th=[ 24], 80.00th=[ 31], 90.00th=[ 39], 95.00th=[ 47], | 99.00th=[ 68], 99.50th=[ 78], 99.90th=[ 114], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 32, max= 376, per=24.97%, avg=115.36, stdev=79.80, samples=107 iops : min= 8, max= 94, avg=28.55, stdev=20.05, samples=107 lat (msec) : 4=1.95%, 10=38.31%, 20=27.83%, 50=29.10%, 100=2.73% lat (msec) : 250=0.03%, 1000=0.03% cpu : usr=0.08%, sys=13.47%, ctx=3981, majf=0, minf=1525 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=442KiB/s (453kB/s), 109KiB/s-111KiB/s (112kB/s-114kB/s), io=23.4MiB (24.6MB), run=54170-54311msec WRITE: bw=463KiB/s (474kB/s), 115KiB/s-117KiB/s (118kB/s-120kB/s), io=24.6MiB (25.7MB), run=54170-54311msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189404: Wed Aug 13 11:20:28 2025 read: IOPS=24, BW=390KiB/s (399kB/s)(5920KiB/15180msec) clat (usec): min=334, max=68755, avg=21174.71, stdev=11588.20 lat (usec): min=335, max=68756, avg=21175.37, stdev=11588.23 clat percentiles (usec): | 1.00th=[ 371], 5.00th=[ 955], 10.00th=[ 9896], 20.00th=[12780], | 30.00th=[14877], 40.00th=[17171], 50.00th=[19006], 60.00th=[21365], | 70.00th=[24511], 80.00th=[28705], 90.00th=[36963], 95.00th=[43254], | 99.00th=[58983], 99.50th=[68682], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 223, max= 670, per=25.48%, avg=390.37, stdev=108.30, samples=30 iops : min= 13, max= 41, avg=23.90, stdev= 6.84, samples=30 write: IOPS=26, BW=419KiB/s (430kB/s)(6368KiB/15180msec); 0 zone resets clat (usec): min=926, max=70854, avg=18371.71, stdev=12730.46 lat (usec): min=927, max=70855, avg=18375.89, stdev=12730.91 clat percentiles (usec): | 1.00th=[ 2802], 5.00th=[ 4555], 10.00th=[ 5145], 20.00th=[ 6587], | 30.00th=[ 9896], 40.00th=[12387], 50.00th=[15533], 60.00th=[19792], | 70.00th=[23200], 80.00th=[27919], 90.00th=[35914], 95.00th=[43779], | 99.00th=[64750], 99.50th=[68682], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 95, max= 640, per=25.97%, avg=418.63, stdev=136.03, samples=30 iops : min= 5, max= 40, avg=25.70, stdev= 8.58, samples=30 lat (usec) : 500=1.43%, 750=0.65%, 1000=0.52% lat (msec) : 2=0.13%, 4=1.04%, 10=16.93%, 20=37.89%, 50=38.67% lat (msec) : 100=2.73% cpu : usr=0.05%, sys=29.29%, ctx=2786, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189405: Wed Aug 13 11:20:28 2025 read: IOPS=23, BW=372KiB/s (381kB/s)(5744KiB/15430msec) clat (usec): min=268, max=77477, avg=20801.64, stdev=10359.14 lat (usec): min=269, max=77477, avg=20802.28, stdev=10359.17 clat percentiles (usec): | 1.00th=[ 375], 5.00th=[ 816], 10.00th=[10290], 20.00th=[13042], | 30.00th=[14877], 40.00th=[17695], 50.00th=[19792], 60.00th=[22152], | 70.00th=[24773], 80.00th=[27657], 90.00th=[32900], 95.00th=[39584], | 99.00th=[49021], 99.50th=[62653], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 191, max= 605, per=24.17%, avg=370.27, stdev=106.05, samples=30 iops : min= 11, max= 37, avg=22.67, stdev= 6.66, samples=30 write: IOPS=26, BW=424KiB/s (434kB/s)(6544KiB/15430msec); 0 zone resets clat (usec): min=2861, max=69454, avg=19312.50, stdev=12757.90 lat (usec): min=2861, max=69455, avg=19313.58, stdev=12757.92 clat percentiles (usec): | 1.00th=[ 4228], 5.00th=[ 4948], 10.00th=[ 6325], 20.00th=[ 8586], | 30.00th=[11076], 40.00th=[13698], 50.00th=[16319], 60.00th=[19530], | 70.00th=[23462], 80.00th=[28181], 90.00th=[34866], 95.00th=[46400], | 99.00th=[61604], 99.50th=[63701], 99.90th=[69731], 99.95th=[69731], | 99.99th=[69731] bw ( KiB/s): min= 224, max= 670, per=25.71%, avg=414.40, stdev=115.29, samples=30 iops : min= 14, max= 41, avg=25.43, stdev= 7.22, samples=30 lat (usec) : 500=1.69%, 750=0.52%, 1000=0.13% lat (msec) : 4=0.52%, 10=15.23%, 20=38.67%, 50=41.28%, 100=1.95% cpu : usr=0.08%, sys=29.59%, ctx=2839, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189406: Wed Aug 13 11:20:28 2025 read: IOPS=25, BW=400KiB/s (410kB/s)(6256KiB/15627msec) clat (usec): min=202, max=68271, avg=21296.97, stdev=11504.95 lat (usec): min=203, max=68273, avg=21297.64, stdev=11505.00 clat percentiles (usec): | 1.00th=[ 371], 5.00th=[ 8455], 10.00th=[10421], 20.00th=[12387], | 30.00th=[14222], 40.00th=[16909], 50.00th=[19268], 60.00th=[21103], | 70.00th=[23987], 80.00th=[29230], 90.00th=[37487], 95.00th=[44303], | 99.00th=[60031], 99.50th=[62653], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 127, max= 603, per=25.88%, avg=396.43, stdev=140.61, samples=30 iops : min= 7, max= 37, avg=24.30, stdev= 8.86, samples=30 write: IOPS=24, BW=386KiB/s (395kB/s)(6032KiB/15627msec); 0 zone resets clat (usec): min=2517, max=80175, avg=19051.28, stdev=13724.19 lat (usec): min=2518, max=80176, avg=19052.38, stdev=13724.23 clat percentiles (usec): | 1.00th=[ 3064], 5.00th=[ 4555], 10.00th=[ 5473], 20.00th=[ 7570], | 30.00th=[ 9765], 40.00th=[12518], 50.00th=[15533], 60.00th=[18482], | 70.00th=[22414], 80.00th=[28443], 90.00th=[38536], 95.00th=[47449], | 99.00th=[68682], 99.50th=[77071], 99.90th=[80217], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 191, max= 638, per=23.06%, avg=371.67, stdev=121.56, samples=30 iops : min= 11, max= 39, avg=22.73, stdev= 7.63, samples=30 lat (usec) : 250=0.13%, 500=1.04%, 750=0.26% lat (msec) : 2=0.13%, 4=1.17%, 10=16.80%, 20=40.10%, 50=36.98% lat (msec) : 100=3.39% cpu : usr=0.07%, sys=29.03%, ctx=2764, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189407: Wed Aug 13 11:20:28 2025 read: IOPS=24, BW=396KiB/s (405kB/s)(6032KiB/15249msec) clat (usec): min=114, max=81149, avg=20625.50, stdev=10567.51 lat (usec): min=115, max=81150, avg=20626.25, stdev=10567.48 clat percentiles (usec): | 1.00th=[ 363], 5.00th=[ 1369], 10.00th=[10552], 20.00th=[13566], | 30.00th=[15270], 40.00th=[17171], 50.00th=[19268], 60.00th=[21365], | 70.00th=[23987], 80.00th=[27395], 90.00th=[33424], 95.00th=[39060], | 99.00th=[51119], 99.50th=[67634], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 223, max= 605, per=25.74%, avg=394.33, stdev=101.78, samples=30 iops : min= 13, max= 37, avg=24.17, stdev= 6.41, samples=30 write: IOPS=25, BW=410KiB/s (420kB/s)(6256KiB/15249msec); 0 zone resets clat (usec): min=1076, max=71892, avg=18934.79, stdev=13767.54 lat (usec): min=1077, max=71893, avg=18935.90, stdev=13767.66 clat percentiles (usec): | 1.00th=[ 2212], 5.00th=[ 4555], 10.00th=[ 5211], 20.00th=[ 7373], | 30.00th=[10028], 40.00th=[12387], 50.00th=[15139], 60.00th=[18220], | 70.00th=[22938], 80.00th=[28181], 90.00th=[40633], 95.00th=[49021], | 99.00th=[62653], 99.50th=[69731], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min= 223, max= 704, per=25.51%, avg=411.27, stdev=121.62, samples=30 iops : min= 13, max= 44, avg=25.23, stdev= 7.71, samples=30 lat (usec) : 250=0.13%, 500=1.43%, 750=0.52%, 1000=0.13% lat (msec) : 2=0.78%, 4=1.04%, 10=15.63%, 20=40.10%, 50=37.24% lat (msec) : 100=2.99% cpu : usr=0.08%, sys=29.72%, ctx=2839, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1533KiB/s (1570kB/s), 372KiB/s-400KiB/s (381kB/s-410kB/s), io=23.4MiB (24.5MB), run=15180-15627msec WRITE: bw=1613KiB/s (1651kB/s), 386KiB/s-424KiB/s (395kB/s-434kB/s), io=24.6MiB (25.8MB), run=15180-15627msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189399: Wed Aug 13 11:20:30 2025 read: IOPS=21, BW=345KiB/s (353kB/s)(5920KiB/17151msec) clat (usec): min=2575, max=81561, avg=19667.24, stdev=14980.45 lat (usec): min=2576, max=81562, avg=19667.87, stdev=14980.49 clat percentiles (usec): | 1.00th=[ 2704], 5.00th=[ 4555], 10.00th=[ 5800], 20.00th=[ 7373], | 30.00th=[ 9896], 40.00th=[12256], 50.00th=[14484], 60.00th=[17957], | 70.00th=[22676], 80.00th=[31065], 90.00th=[41681], 95.00th=[52167], | 99.00th=[68682], 99.50th=[70779], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 128, max= 992, per=23.00%, avg=321.12, stdev=180.10, samples=33 iops : min= 8, max= 62, avg=19.79, stdev=11.34, samples=33 write: IOPS=23, BW=371KiB/s (380kB/s)(6368KiB/17151msec); 0 zone resets clat (usec): min=3304, max=97589, avg=24758.30, stdev=17555.11 lat (usec): min=3305, max=97590, avg=24761.29, stdev=17553.44 clat percentiles (usec): | 1.00th=[ 3621], 5.00th=[ 6063], 10.00th=[ 7046], 20.00th=[ 9503], | 30.00th=[11863], 40.00th=[14222], 50.00th=[19530], 60.00th=[25822], | 70.00th=[32375], 80.00th=[38536], 90.00th=[49021], 95.00th=[60556], | 99.00th=[73925], 99.50th=[94897], 99.90th=[98042], 99.95th=[98042], | 99.99th=[98042] bw ( KiB/s): min= 64, max= 1280, per=23.97%, avg=352.09, stdev=224.05, samples=33 iops : min= 4, max= 80, avg=21.73, stdev=14.06, samples=33 lat (msec) : 4=2.34%, 10=23.44%, 20=31.64%, 50=34.90%, 100=7.68% cpu : usr=0.05%, sys=13.92%, ctx=1194, majf=0, minf=1493 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189400: Wed Aug 13 11:20:30 2025 read: IOPS=21, BW=337KiB/s (345kB/s)(5744KiB/17057msec) clat (usec): min=3892, max=75427, avg=20636.19, stdev=14661.41 lat (usec): min=3893, max=75427, avg=20636.82, stdev=14661.43 clat percentiles (usec): | 1.00th=[ 4752], 5.00th=[ 5473], 10.00th=[ 6325], 20.00th=[ 8455], | 30.00th=[10421], 40.00th=[13435], 50.00th=[15664], 60.00th=[20055], | 70.00th=[23200], 80.00th=[31327], 90.00th=[42730], 95.00th=[52167], | 99.00th=[60556], 99.50th=[73925], 99.90th=[74974], 99.95th=[74974], | 99.99th=[74974] bw ( KiB/s): min= 96, max= 960, per=23.51%, avg=328.24, stdev=175.31, samples=33 iops : min= 6, max= 60, avg=20.24, stdev=10.97, samples=33 write: IOPS=23, BW=384KiB/s (393kB/s)(6544KiB/17057msec); 0 zone resets clat (usec): min=4522, max=96427, avg=23502.44, stdev=16264.72 lat (usec): min=4523, max=96428, avg=23503.74, stdev=16264.83 clat percentiles (usec): | 1.00th=[ 5669], 5.00th=[ 6652], 10.00th=[ 7635], 20.00th=[ 9896], | 30.00th=[12649], 40.00th=[14877], 50.00th=[18220], 60.00th=[23200], | 70.00th=[29230], 80.00th=[35390], 90.00th=[44827], 95.00th=[59507], | 99.00th=[71828], 99.50th=[77071], 99.90th=[95945], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 160, max= 1344, per=25.02%, avg=367.61, stdev=228.17, samples=33 iops : min= 10, max= 84, avg=22.70, stdev=14.32, samples=33 lat (msec) : 4=0.13%, 10=23.57%, 20=33.59%, 50=35.68%, 100=7.03% cpu : usr=0.11%, sys=14.19%, ctx=1228, majf=0, minf=1449 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189401: Wed Aug 13 11:20:30 2025 read: IOPS=23, BW=370KiB/s (379kB/s)(6256KiB/16915msec) clat (usec): min=3198, max=92968, avg=19702.42, stdev=14944.35 lat (usec): min=3199, max=92969, avg=19703.21, stdev=14944.40 clat percentiles (usec): | 1.00th=[ 4817], 5.00th=[ 5473], 10.00th=[ 6325], 20.00th=[ 8225], | 30.00th=[10159], 40.00th=[12125], 50.00th=[14222], 60.00th=[17433], | 70.00th=[22676], 80.00th=[30016], 90.00th=[40109], 95.00th=[51643], | 99.00th=[68682], 99.50th=[83362], 99.90th=[92799], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 128, max= 1312, per=26.18%, avg=365.52, stdev=244.24, samples=33 iops : min= 8, max= 82, avg=22.55, stdev=15.34, samples=33 write: IOPS=22, BW=357KiB/s (365kB/s)(6032KiB/16915msec); 0 zone resets clat (msec): min=5, max=105, avg=24.30, stdev=16.62 lat (msec): min=5, max=105, avg=24.30, stdev=16.62 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 11], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 25], | 70.00th=[ 30], 80.00th=[ 36], 90.00th=[ 47], 95.00th=[ 56], | 99.00th=[ 78], 99.50th=[ 92], 99.90th=[ 106], 99.95th=[ 106], | 99.99th=[ 106] bw ( KiB/s): min= 159, max= 960, per=23.49%, avg=345.00, stdev=171.23, samples=33 iops : min= 9, max= 60, avg=21.27, stdev=10.76, samples=33 lat (msec) : 4=0.13%, 10=23.57%, 20=35.29%, 50=33.98%, 100=6.90% lat (msec) : 250=0.13% cpu : usr=0.05%, sys=14.43%, ctx=1231, majf=0, minf=1578 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189402: Wed Aug 13 11:20:30 2025 read: IOPS=22, BW=353KiB/s (361kB/s)(6032KiB/17087msec) clat (usec): min=3149, max=74190, avg=19549.00, stdev=13973.20 lat (usec): min=3150, max=74191, avg=19549.61, stdev=13973.24 clat percentiles (usec): | 1.00th=[ 3851], 5.00th=[ 5145], 10.00th=[ 5932], 20.00th=[ 7963], | 30.00th=[ 9896], 40.00th=[12256], 50.00th=[14746], 60.00th=[18220], | 70.00th=[23725], 80.00th=[29754], 90.00th=[41157], 95.00th=[49021], | 99.00th=[60556], 99.50th=[64750], 99.90th=[73925], 99.95th=[73925], | 99.99th=[73925] bw ( KiB/s): min= 159, max= 1216, per=24.14%, avg=336.97, stdev=200.36, samples=33 iops : min= 9, max= 76, avg=20.79, stdev=12.57, samples=33 write: IOPS=22, BW=366KiB/s (375kB/s)(6256KiB/17087msec); 0 zone resets clat (usec): min=4820, max=93950, avg=24750.92, stdev=16199.61 lat (usec): min=4821, max=93951, avg=24752.21, stdev=16199.70 clat percentiles (usec): | 1.00th=[ 5800], 5.00th=[ 6915], 10.00th=[ 8029], 20.00th=[10290], | 30.00th=[12780], 40.00th=[16450], 50.00th=[20055], 60.00th=[25560], | 70.00th=[31065], 80.00th=[37487], 90.00th=[47973], 95.00th=[56886], | 99.00th=[74974], 99.50th=[81265], 99.90th=[93848], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 190, max= 1056, per=24.17%, avg=355.03, stdev=183.54, samples=33 iops : min= 11, max= 66, avg=21.91, stdev=11.60, samples=33 lat (msec) : 4=0.65%, 10=24.09%, 20=31.64%, 50=36.46%, 100=7.16% cpu : usr=0.01%, sys=14.37%, ctx=1222, majf=0, minf=1519 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1397KiB/s (1430kB/s), 337KiB/s-370KiB/s (345kB/s-379kB/s), io=23.4MiB (24.5MB), run=16915-17151msec WRITE: bw=1469KiB/s (1505kB/s), 357KiB/s-384KiB/s (365kB/s-393kB/s), io=24.6MiB (25.8MB), run=16915-17151msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189414: Wed Aug 13 11:20:38 2025 read: IOPS=2, BW=2214KiB/s (2268kB/s)(7168KiB/3237msec) clat (msec): min=55, max=567, avg=205.13, stdev=191.28 lat (msec): min=55, max=567, avg=205.13, stdev=191.28 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 56], 10.00th=[ 56], 20.00th=[ 58], | 30.00th=[ 78], 40.00th=[ 78], 50.00th=[ 157], 60.00th=[ 163], | 70.00th=[ 163], 80.00th=[ 359], 90.00th=[ 567], 95.00th=[ 567], | 99.00th=[ 567], 99.50th=[ 567], 99.90th=[ 567], 99.95th=[ 567], | 99.99th=[ 567] bw ( KiB/s): min= 2043, max= 5044, per=45.56%, avg=2795.75, stdev=1498.84, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.26, samples=4 write: IOPS=1, BW=1582KiB/s (1620kB/s)(5120KiB/3237msec); 0 zone resets clat (msec): min=104, max=844, avg=349.64, stdev=294.37 lat (msec): min=104, max=844, avg=349.71, stdev=294.37 clat percentiles (msec): | 1.00th=[ 105], 5.00th=[ 105], 10.00th=[ 105], 20.00th=[ 105], | 30.00th=[ 174], 40.00th=[ 174], 50.00th=[ 251], 60.00th=[ 251], | 70.00th=[ 376], 80.00th=[ 376], 90.00th=[ 844], 95.00th=[ 844], | 99.00th=[ 844], 99.50th=[ 844], 99.90th=[ 844], 99.95th=[ 844], | 99.99th=[ 844] bw ( KiB/s): min= 2048, max= 4096, per=31.78%, avg=2730.67, stdev=1182.41, samples=3 iops : min= 2, max= 4, avg= 2.67, stdev= 1.15, samples=3 lat (msec) : 100=25.00%, 250=33.33%, 500=25.00%, 750=8.33%, 1000=8.33% cpu : usr=0.00%, sys=15.76%, ctx=103, majf=0, minf=1804 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189415: Wed Aug 13 11:20:38 2025 read: IOPS=0, BW=614KiB/s (628kB/s)(2048KiB/3337msec) clat (msec): min=177, max=241, avg=209.23, stdev=45.24 lat (msec): min=177, max=241, avg=209.23, stdev=45.24 clat percentiles (msec): | 1.00th=[ 178], 5.00th=[ 178], 10.00th=[ 178], 20.00th=[ 178], | 30.00th=[ 178], 40.00th=[ 178], 50.00th=[ 178], 60.00th=[ 243], | 70.00th=[ 243], 80.00th=[ 243], 90.00th=[ 243], 95.00th=[ 243], | 99.00th=[ 243], 99.50th=[ 243], 99.90th=[ 243], 99.95th=[ 243], | 99.99th=[ 243] bw ( KiB/s): min= 1686, max= 2048, per=30.42%, avg=1867.00, stdev=255.97, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=3069KiB/s (3142kB/s)(10.0MiB/3337msec); 0 zone resets clat (msec): min=141, max=742, avg=281.12, stdev=179.70 lat (msec): min=141, max=742, avg=281.19, stdev=179.70 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 146], | 30.00th=[ 159], 40.00th=[ 215], 50.00th=[ 224], 60.00th=[ 232], | 70.00th=[ 262], 80.00th=[ 296], 90.00th=[ 397], 95.00th=[ 743], | 99.00th=[ 743], 99.50th=[ 743], 99.90th=[ 743], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 2048, max= 6144, per=38.12%, avg=3275.00, stdev=1830.78, samples=5 iops : min= 2, max= 6, avg= 3.00, stdev= 1.73, samples=5 lat (msec) : 250=66.67%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=23.32%, ctx=149, majf=0, minf=524 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189416: Wed Aug 13 11:20:38 2025 read: IOPS=2, BW=2226KiB/s (2280kB/s)(6144KiB/2760msec) clat (msec): min=68, max=229, avg=120.78, stdev=60.96 lat (msec): min=68, max=229, avg=120.78, stdev=60.96 clat percentiles (msec): | 1.00th=[ 69], 5.00th=[ 69], 10.00th=[ 69], 20.00th=[ 69], | 30.00th=[ 69], 40.00th=[ 94], 50.00th=[ 94], 60.00th=[ 121], | 70.00th=[ 144], 80.00th=[ 144], 90.00th=[ 230], 95.00th=[ 230], | 99.00th=[ 230], 99.50th=[ 230], 99.90th=[ 230], 99.95th=[ 230], | 99.99th=[ 230] bw ( KiB/s): min= 2048, max= 4096, per=55.57%, avg=3410.33, stdev=1179.82, samples=3 iops : min= 2, max= 4, avg= 3.00, stdev= 1.00, samples=3 write: IOPS=2, BW=2226KiB/s (2280kB/s)(6144KiB/2760msec); 0 zone resets clat (msec): min=43, max=745, avg=337.24, stdev=227.95 lat (msec): min=43, max=745, avg=337.31, stdev=227.93 clat percentiles (msec): | 1.00th=[ 44], 5.00th=[ 44], 10.00th=[ 44], 20.00th=[ 284], | 30.00th=[ 284], 40.00th=[ 292], 50.00th=[ 292], 60.00th=[ 296], | 70.00th=[ 363], 80.00th=[ 363], 90.00th=[ 743], 95.00th=[ 743], | 99.00th=[ 743], 99.50th=[ 743], 99.90th=[ 743], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 2043, max= 3368, per=27.66%, avg=2376.75, stdev=660.84, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (msec) : 50=8.33%, 100=25.00%, 250=25.00%, 500=33.33%, 750=8.33% cpu : usr=0.00%, sys=15.51%, ctx=74, majf=0, minf=1547 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189417: Wed Aug 13 11:20:38 2025 read: IOPS=1, BW=1835KiB/s (1879kB/s)(5120KiB/2790msec) clat (msec): min=73, max=261, avg=151.82, stdev=72.43 lat (msec): min=73, max=261, avg=151.82, stdev=72.43 clat percentiles (msec): | 1.00th=[ 73], 5.00th=[ 73], 10.00th=[ 73], 20.00th=[ 73], | 30.00th=[ 104], 40.00th=[ 104], 50.00th=[ 150], 60.00th=[ 150], | 70.00th=[ 171], 80.00th=[ 171], 90.00th=[ 262], 95.00th=[ 262], | 99.00th=[ 262], 99.50th=[ 262], 99.90th=[ 262], 99.95th=[ 262], | 99.99th=[ 262] bw ( KiB/s): min= 4087, max= 4096, per=66.67%, avg=4091.50, stdev= 6.36, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2569KiB/s (2631kB/s)(7168KiB/2790msec); 0 zone resets clat (msec): min=63, max=564, avg=283.58, stdev=175.32 lat (msec): min=63, max=564, avg=283.64, stdev=175.32 clat percentiles (msec): | 1.00th=[ 64], 5.00th=[ 64], 10.00th=[ 64], 20.00th=[ 113], | 30.00th=[ 215], 40.00th=[ 215], 50.00th=[ 271], 60.00th=[ 330], | 70.00th=[ 330], 80.00th=[ 430], 90.00th=[ 567], 95.00th=[ 567], | 99.00th=[ 567], 99.50th=[ 567], 99.90th=[ 567], 99.95th=[ 567], | 99.99th=[ 567] bw ( KiB/s): min= 1684, max= 4087, per=28.71%, avg=2466.75, stdev=1093.71, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (msec) : 100=16.67%, 250=41.67%, 500=33.33%, 750=8.33% cpu : usr=0.00%, sys=24.31%, ctx=141, majf=0, minf=1292 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6137KiB/s (6285kB/s), 614KiB/s-2226KiB/s (628kB/s-2280kB/s), io=20.0MiB (20.0MB), run=2760-3337msec WRITE: bw=8592KiB/s (8798kB/s), 1582KiB/s-3069KiB/s (1620kB/s-3142kB/s), io=28.0MiB (29.4MB), run=2760-3337msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189419: Wed Aug 13 11:20:38 2025 read: IOPS=2, BW=2677KiB/s (2741kB/s)(7168KiB/2678msec) clat (usec): min=913, max=291073, avg=152942.13, stdev=95196.89 lat (usec): min=914, max=291075, avg=152943.30, stdev=95196.96 clat percentiles (usec): | 1.00th=[ 914], 5.00th=[ 914], 10.00th=[ 914], 20.00th=[101188], | 30.00th=[135267], 40.00th=[135267], 50.00th=[145753], 60.00th=[147850], | 70.00th=[147850], 80.00th=[248513], 90.00th=[291505], 95.00th=[291505], | 99.00th=[291505], 99.50th=[291505], 99.90th=[291505], 99.95th=[291505], | 99.99th=[291505] bw ( KiB/s): min= 1868, max= 5086, per=44.76%, avg=2744.50, stdev=1562.75, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 write: IOPS=1, BW=1912KiB/s (1958kB/s)(5120KiB/2678msec); 0 zone resets clat (msec): min=170, max=608, avg=306.17, stdev=175.60 lat (msec): min=170, max=609, avg=306.23, stdev=175.58 clat percentiles (msec): | 1.00th=[ 171], 5.00th=[ 171], 10.00th=[ 171], 20.00th=[ 171], | 30.00th=[ 203], 40.00th=[ 203], 50.00th=[ 257], 60.00th=[ 257], | 70.00th=[ 292], 80.00th=[ 292], 90.00th=[ 609], 95.00th=[ 609], | 99.00th=[ 609], 99.50th=[ 609], 99.90th=[ 609], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 1868, max= 4096, per=31.11%, avg=2670.67, stdev=1237.65, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 lat (usec) : 1000=8.33% lat (msec) : 250=58.33%, 500=25.00%, 750=8.33% cpu : usr=0.04%, sys=50.39%, ctx=695, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189420: Wed Aug 13 11:20:38 2025 read: IOPS=0, BW=613KiB/s (628kB/s)(2048KiB/3340msec) clat (msec): min=180, max=375, avg=277.74, stdev=138.14 lat (msec): min=180, max=375, avg=277.74, stdev=138.14 clat percentiles (msec): | 1.00th=[ 180], 5.00th=[ 180], 10.00th=[ 180], 20.00th=[ 180], | 30.00th=[ 180], 40.00th=[ 180], 50.00th=[ 180], 60.00th=[ 376], | 70.00th=[ 376], 80.00th=[ 376], 90.00th=[ 376], 95.00th=[ 376], | 99.00th=[ 376], 99.50th=[ 376], 99.90th=[ 376], 99.95th=[ 376], | 99.99th=[ 376] bw ( KiB/s): min= 1684, max= 2048, per=30.44%, avg=1866.00, stdev=257.39, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=3066KiB/s (3139kB/s)(10.0MiB/3340msec); 0 zone resets clat (msec): min=50, max=498, avg=269.65, stdev=125.29 lat (msec): min=50, max=498, avg=269.71, stdev=125.28 clat percentiles (msec): | 1.00th=[ 51], 5.00th=[ 51], 10.00th=[ 51], 20.00th=[ 163], | 30.00th=[ 188], 40.00th=[ 234], 50.00th=[ 239], 60.00th=[ 284], | 70.00th=[ 313], 80.00th=[ 342], 90.00th=[ 388], 95.00th=[ 498], | 99.00th=[ 498], 99.50th=[ 498], 99.90th=[ 498], 99.95th=[ 498], | 99.99th=[ 498] bw ( KiB/s): min= 1868, max= 4096, per=37.59%, avg=3226.40, stdev=1191.36, samples=5 iops : min= 1, max= 4, avg= 2.80, stdev= 1.64, samples=5 lat (msec) : 100=8.33%, 250=41.67%, 500=50.00% cpu : usr=0.00%, sys=43.58%, ctx=236, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189421: Wed Aug 13 11:20:38 2025 read: IOPS=2, BW=2124KiB/s (2175kB/s)(6144KiB/2892msec) clat (usec): min=717, max=223637, avg=128518.64, stdev=96308.63 lat (usec): min=717, max=223638, avg=128519.53, stdev=96308.66 clat percentiles (usec): | 1.00th=[ 717], 5.00th=[ 717], 10.00th=[ 717], 20.00th=[ 20055], | 30.00th=[ 20055], 40.00th=[137364], 50.00th=[137364], 60.00th=[179307], | 70.00th=[210764], 80.00th=[210764], 90.00th=[223347], 95.00th=[223347], | 99.00th=[223347], 99.50th=[223347], 99.90th=[223347], 99.95th=[223347], | 99.99th=[223347] bw ( KiB/s): min= 4096, max= 5930, per=81.76%, avg=5013.00, stdev=1296.83, samples=2 iops : min= 4, max= 5, avg= 4.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2124KiB/s (2175kB/s)(6144KiB/2892msec); 0 zone resets clat (msec): min=141, max=623, avg=348.31, stdev=197.73 lat (msec): min=141, max=623, avg=348.37, stdev=197.73 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 192], | 30.00th=[ 192], 40.00th=[ 218], 50.00th=[ 218], 60.00th=[ 384], | 70.00th=[ 531], 80.00th=[ 531], 90.00th=[ 625], 95.00th=[ 625], | 99.00th=[ 625], 99.50th=[ 625], 99.90th=[ 625], 99.95th=[ 625], | 99.99th=[ 625] bw ( KiB/s): min= 1976, max= 3346, per=27.43%, avg=2354.50, stdev=661.87, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (usec) : 750=8.33% lat (msec) : 50=8.33%, 250=58.33%, 500=8.33%, 750=16.67% cpu : usr=0.00%, sys=40.64%, ctx=566, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189422: Wed Aug 13 11:20:38 2025 read: IOPS=1, BW=1760KiB/s (1802kB/s)(5120KiB/2909msec) clat (usec): min=686, max=137233, avg=54233.50, stdev=73281.18 lat (usec): min=686, max=137233, avg=54234.28, stdev=73281.19 clat percentiles (usec): | 1.00th=[ 685], 5.00th=[ 685], 10.00th=[ 685], 20.00th=[ 685], | 30.00th=[ 693], 40.00th=[ 693], 50.00th=[ 824], 60.00th=[ 824], | 70.00th=[131597], 80.00th=[131597], 90.00th=[137364], 95.00th=[137364], | 99.00th=[137364], 99.50th=[137364], 99.90th=[137364], 99.95th=[137364], | 99.99th=[137364] bw ( KiB/s): min= 3737, max= 4096, per=63.88%, avg=3916.50, stdev=253.85, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2464KiB/s (2523kB/s)(7168KiB/2909msec); 0 zone resets clat (msec): min=150, max=663, avg=369.07, stdev=205.27 lat (msec): min=150, max=663, avg=369.13, stdev=205.27 clat percentiles (msec): | 1.00th=[ 150], 5.00th=[ 150], 10.00th=[ 150], 20.00th=[ 155], | 30.00th=[ 271], 40.00th=[ 271], 50.00th=[ 284], 60.00th=[ 506], | 70.00th=[ 506], 80.00th=[ 558], 90.00th=[ 667], 95.00th=[ 667], | 99.00th=[ 667], 99.50th=[ 667], 99.90th=[ 667], 99.95th=[ 667], | 99.99th=[ 667] bw ( KiB/s): min= 1689, max= 3953, per=27.84%, avg=2389.50, stdev=1052.59, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (usec) : 750=16.67%, 1000=8.33% lat (msec) : 250=33.33%, 500=16.67%, 750=25.00% cpu : usr=0.00%, sys=40.92%, ctx=535, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6132KiB/s (6279kB/s), 613KiB/s-2677KiB/s (628kB/s-2741kB/s), io=20.0MiB (20.0MB), run=2678-3340msec WRITE: bw=8584KiB/s (8790kB/s), 1912KiB/s-3066KiB/s (1958kB/s-3139kB/s), io=28.0MiB (29.4MB), run=2678-3340msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=189428: Wed Aug 13 11:20:48 2025 read: IOPS=1, BW=5491KiB/s (5622kB/s)(12.0MiB/2238msec) clat (msec): min=266, max=1058, avg=699.12, stdev=401.35 lat (msec): min=266, max=1058, avg=699.12, stdev=401.35 clat percentiles (msec): | 1.00th=[ 266], 5.00th=[ 266], 10.00th=[ 266], 20.00th=[ 266], | 30.00th=[ 266], 40.00th=[ 776], 50.00th=[ 776], 60.00th=[ 776], | 70.00th=[ 1062], 80.00th=[ 1062], 90.00th=[ 1062], 95.00th=[ 1062], | 99.00th=[ 1062], 99.50th=[ 1062], 99.90th=[ 1062], 99.95th=[ 1062], | 99.99th=[ 1062] bw ( KiB/s): min= 7937, max= 8192, per=100.00%, avg=8064.50, stdev=180.31, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=18.55%, ctx=255, majf=0, minf=3081 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189429: Wed Aug 13 11:20:48 2025 read: IOPS=0, BW=1103KiB/s (1130kB/s)(4096KiB/3713msec) clat (nsec): min=562236k, max=562236k, avg=562235784.00, stdev= 0.00 lat (nsec): min=562238k, max=562238k, avg=562238039.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 567], 5.00th=[ 567], 10.00th=[ 567], 20.00th=[ 567], | 30.00th=[ 567], 40.00th=[ 567], 50.00th=[ 567], 60.00th=[ 567], | 70.00th=[ 567], 80.00th=[ 567], 90.00th=[ 567], 95.00th=[ 567], | 99.00th=[ 567], 99.50th=[ 567], 99.90th=[ 567], 99.95th=[ 567], | 99.99th=[ 567] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2206KiB/s (2259kB/s)(8192KiB/3713msec); 0 zone resets clat (msec): min=1296, max=1714, avg=1505.56, stdev=296.16 lat (msec): min=1296, max=1715, avg=1505.84, stdev=296.16 clat percentiles (msec): | 1.00th=[ 1301], 5.00th=[ 1301], 10.00th=[ 1301], 20.00th=[ 1301], | 30.00th=[ 1301], 40.00th=[ 1301], 50.00th=[ 1301], 60.00th=[ 1720], | 70.00th=[ 1720], 80.00th=[ 1720], 90.00th=[ 1720], 95.00th=[ 1720], | 99.00th=[ 1720], 99.50th=[ 1720], 99.90th=[ 1720], 99.95th=[ 1720], | 99.99th=[ 1720] bw ( KiB/s): min= 7937, max= 7937, per=89.94%, avg=7937.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=26.00%, ctx=108, majf=0, minf=1035 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189430: Wed Aug 13 11:20:48 2025 write: IOPS=1, BW=5084KiB/s (5206kB/s)(12.0MiB/2417msec); 0 zone resets clat (msec): min=260, max=1138, avg=759.05, stdev=450.94 lat (msec): min=260, max=1138, avg=759.34, stdev=450.97 clat percentiles (msec): | 1.00th=[ 262], 5.00th=[ 262], 10.00th=[ 262], 20.00th=[ 262], | 30.00th=[ 262], 40.00th=[ 877], 50.00th=[ 877], 60.00th=[ 877], | 70.00th=[ 1133], 80.00th=[ 1133], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 8192, max= 8192, per=92.83%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.04%, sys=24.05%, ctx=160, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189432: Wed Aug 13 11:20:48 2025 write: IOPS=0, BW=3974KiB/s (4070kB/s)(12.0MiB/3092msec); 0 zone resets clat (msec): min=120, max=2599, avg=982.93, stdev=1401.36 lat (msec): min=121, max=2600, avg=983.25, stdev=1401.35 clat percentiles (msec): | 1.00th=[ 122], 5.00th=[ 122], 10.00th=[ 122], 20.00th=[ 122], | 30.00th=[ 122], 40.00th=[ 228], 50.00th=[ 228], 60.00th=[ 228], | 70.00th=[ 2601], 80.00th=[ 2601], 90.00th=[ 2601], 95.00th=[ 2601], | 99.00th=[ 2601], 99.50th=[ 2601], 99.90th=[ 2601], 99.95th=[ 2601], | 99.99th=[ 2601] bw ( KiB/s): min=15753, max=15753, per=100.00%, avg=15753.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, >=2000=33.33% cpu : usr=0.03%, sys=22.45%, ctx=138, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4413KiB/s (4519kB/s), 1103KiB/s-5491KiB/s (1130kB/s-5622kB/s), io=16.0MiB (16.8MB), run=2238-3713msec WRITE: bw=8825KiB/s (9037kB/s), 2206KiB/s-5084KiB/s (2259kB/s-5206kB/s), io=32.0MiB (33.6MB), run=2417-3713msec rand-rw: (groupid=0, jobs=1): err= 0: pid=189433: Wed Aug 13 11:20:48 2025 read: IOPS=2, BW=10.5MiB/s (10.0MB/s)(12.0MiB/1147msec) clat (msec): min=2, max=613, avg=373.00, stdev=325.78 lat (msec): min=2, max=613, avg=373.00, stdev=325.78 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 3], 20.00th=[ 3], | 30.00th=[ 3], 40.00th=[ 506], 50.00th=[ 506], 60.00th=[ 506], | 70.00th=[ 617], 80.00th=[ 617], 90.00th=[ 617], 95.00th=[ 617], | 99.00th=[ 617], 99.50th=[ 617], 99.90th=[ 617], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 5649, max= 5649, per=89.27%, avg=5649.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 4=33.33%, 750=66.67% cpu : usr=0.00%, sys=58.64%, ctx=1550, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189434: Wed Aug 13 11:20:48 2025 read: IOPS=0, BW=1582KiB/s (1620kB/s)(4096KiB/2589msec) clat (nsec): min=474840k, max=474840k, avg=474840148.00, stdev= 0.00 lat (nsec): min=474842k, max=474842k, avg=474842203.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 477], 5.00th=[ 477], 10.00th=[ 477], 20.00th=[ 477], | 30.00th=[ 477], 40.00th=[ 477], 50.00th=[ 477], 60.00th=[ 477], | 70.00th=[ 477], 80.00th=[ 477], 90.00th=[ 477], 95.00th=[ 477], | 99.00th=[ 477], 99.50th=[ 477], 99.90th=[ 477], 99.95th=[ 477], | 99.99th=[ 477] bw ( KiB/s): min= 5696, max= 5696, per=90.01%, avg=5696.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3164KiB/s (3240kB/s)(8192KiB/2589msec); 0 zone resets clat (msec): min=651, max=1411, avg=1031.75, stdev=537.31 lat (msec): min=652, max=1411, avg=1031.98, stdev=537.18 clat percentiles (msec): | 1.00th=[ 651], 5.00th=[ 651], 10.00th=[ 651], 20.00th=[ 651], | 30.00th=[ 651], 40.00th=[ 651], 50.00th=[ 651], 60.00th=[ 1418], | 70.00th=[ 1418], 80.00th=[ 1418], 90.00th=[ 1418], 95.00th=[ 1418], | 99.00th=[ 1418], 99.50th=[ 1418], 99.90th=[ 1418], 99.95th=[ 1418], | 99.99th=[ 1418] bw ( KiB/s): min= 8192, max= 8192, per=89.10%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.08%, sys=43.16%, ctx=982, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189435: Wed Aug 13 11:20:48 2025 write: IOPS=0, BW=3448KiB/s (3531kB/s)(12.0MiB/3564msec); 0 zone resets clat (msec): min=593, max=1707, avg=1168.52, stdev=558.15 lat (msec): min=593, max=1708, avg=1168.80, stdev=558.15 clat percentiles (msec): | 1.00th=[ 592], 5.00th=[ 592], 10.00th=[ 592], 20.00th=[ 592], | 30.00th=[ 592], 40.00th=[ 1200], 50.00th=[ 1200], 60.00th=[ 1200], | 70.00th=[ 1703], 80.00th=[ 1703], 90.00th=[ 1703], 95.00th=[ 1703], | 99.00th=[ 1703], 99.50th=[ 1703], 99.90th=[ 1703], 99.95th=[ 1703], | 99.99th=[ 1703] bw ( KiB/s): min= 8175, max= 8192, per=89.01%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=43.00%, ctx=48, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=189436: Wed Aug 13 11:20:48 2025 write: IOPS=1, BW=4452KiB/s (4559kB/s)(12.0MiB/2760msec); 0 zone resets clat (msec): min=586, max=1373, avg=897.35, stdev=418.58 lat (msec): min=586, max=1373, avg=897.66, stdev=418.53 clat percentiles (msec): | 1.00th=[ 584], 5.00th=[ 584], 10.00th=[ 584], 20.00th=[ 584], | 30.00th=[ 584], 40.00th=[ 735], 50.00th=[ 735], 60.00th=[ 735], | 70.00th=[ 1368], 80.00th=[ 1368], 90.00th=[ 1368], 95.00th=[ 1368], | 99.00th=[ 1368], 99.50th=[ 1368], 99.90th=[ 1368], 99.95th=[ 1368], | 99.99th=[ 1368] bw ( KiB/s): min= 5649, max= 8192, per=75.27%, avg=6920.50, stdev=1798.17, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=51.90%, ctx=179, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6328KiB/s (6480kB/s), 1582KiB/s-10.5MiB/s (1620kB/s-10.0MB/s), io=16.0MiB (16.8MB), run=1147-2589msec WRITE: bw=9194KiB/s (9415kB/s), 3164KiB/s-4452KiB/s (3240kB/s-4559kB/s), io=32.0MiB (33.6MB), run=2589-3564msec