== sanity test 398b: DIO and buffer IO race ============== 03:37:44 (1753429064) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.91768 s, 17.3 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=194635: Fri Jul 25 03:38:41 2025 read: IOPS=31, BW=126KiB/s (129kB/s)(5992KiB/47486msec) clat (usec): min=98, max=769936, avg=17464.64, stdev=21531.37 lat (usec): min=98, max=769936, avg=17465.81, stdev=21531.33 clat percentiles (usec): | 1.00th=[ 351], 5.00th=[ 6456], 10.00th=[ 7898], 20.00th=[ 9765], | 30.00th=[ 11338], 40.00th=[ 13173], 50.00th=[ 15401], 60.00th=[ 17433], | 70.00th=[ 19792], 80.00th=[ 23462], 90.00th=[ 28967], 95.00th=[ 34866], | 99.00th=[ 46400], 99.50th=[ 53740], 99.90th=[ 76022], 99.95th=[767558], | 99.99th=[767558] bw ( KiB/s): min= 13, max= 208, per=25.24%, avg=126.44, stdev=37.42, samples=94 iops : min= 3, max= 52, avg=31.26, stdev= 9.41, samples=94 write: IOPS=33, BW=133KiB/s (136kB/s)(6296KiB/47486msec); 0 zone resets clat (usec): min=656, max=66710, avg=13401.98, stdev=10262.96 lat (usec): min=657, max=66711, avg=13402.90, stdev=10263.03 clat percentiles (usec): | 1.00th=[ 1188], 5.00th=[ 2409], 10.00th=[ 2802], 20.00th=[ 3720], | 30.00th=[ 6718], 40.00th=[ 9110], 50.00th=[11076], 60.00th=[13698], | 70.00th=[16581], 80.00th=[20841], 90.00th=[27132], 95.00th=[33162], | 99.00th=[47449], 99.50th=[53740], 99.90th=[62653], 99.95th=[66847], | 99.99th=[66847] bw ( KiB/s): min= 24, max= 247, per=25.56%, avg=134.17, stdev=42.74, samples=93 iops : min= 6, max= 61, avg=33.19, stdev=10.67, samples=93 lat (usec) : 100=0.03%, 250=0.20%, 500=0.72%, 750=0.39%, 1000=0.23% lat (msec) : 2=1.11%, 4=9.51%, 10=21.71%, 20=40.36%, 50=24.93% lat (msec) : 100=0.78%, 1000=0.03% cpu : usr=0.11%, sys=27.14%, ctx=8395, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194636: Fri Jul 25 03:38:41 2025 read: IOPS=31, BW=124KiB/s (127kB/s)(5940KiB/47788msec) clat (usec): min=107, max=80045, avg=17024.56, stdev=8731.05 lat (usec): min=107, max=80047, avg=17025.33, stdev=8731.17 clat percentiles (usec): | 1.00th=[ 371], 5.00th=[ 6915], 10.00th=[ 8225], 20.00th=[ 9896], | 30.00th=[11863], 40.00th=[13829], 50.00th=[15664], 60.00th=[17695], | 70.00th=[19530], 80.00th=[22152], 90.00th=[28705], 95.00th=[33424], | 99.00th=[45351], 99.50th=[50594], 99.90th=[58983], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 6, max= 200, per=24.81%, avg=124.28, stdev=38.59, samples=95 iops : min= 1, max= 50, avg=30.72, stdev= 9.69, samples=95 write: IOPS=33, BW=133KiB/s (136kB/s)(6348KiB/47788msec); 0 zone resets clat (usec): min=701, max=828480, avg=14038.99, stdev=23093.17 lat (usec): min=701, max=828481, avg=14040.90, stdev=23093.01 clat percentiles (usec): | 1.00th=[ 1139], 5.00th=[ 2343], 10.00th=[ 2737], 20.00th=[ 3621], | 30.00th=[ 5735], 40.00th=[ 8848], 50.00th=[ 10814], 60.00th=[ 13435], | 70.00th=[ 16909], 80.00th=[ 21365], 90.00th=[ 28967], 95.00th=[ 34341], | 99.00th=[ 48497], 99.50th=[ 54789], 99.90th=[ 72877], 99.95th=[826278], | 99.99th=[826278] bw ( KiB/s): min= 19, max= 272, per=25.32%, avg=132.91, stdev=42.14, samples=95 iops : min= 4, max= 68, avg=32.87, stdev=10.61, samples=95 lat (usec) : 250=0.16%, 500=0.68%, 750=0.20%, 1000=0.23% lat (msec) : 2=1.40%, 4=10.03%, 10=21.03%, 20=40.59%, 50=25.03% lat (msec) : 100=0.62%, 1000=0.03% cpu : usr=0.11%, sys=27.31%, ctx=8356, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194637: Fri Jul 25 03:38:41 2025 read: IOPS=31, BW=127KiB/s (130kB/s)(6028KiB/47347msec) clat (usec): min=98, max=777120, avg=17391.89, stdev=21621.16 lat (usec): min=99, max=777120, avg=17393.20, stdev=21621.13 clat percentiles (usec): | 1.00th=[ 318], 5.00th=[ 6849], 10.00th=[ 8094], 20.00th=[ 9765], | 30.00th=[ 11600], 40.00th=[ 13173], 50.00th=[ 15139], 60.00th=[ 17171], | 70.00th=[ 19268], 80.00th=[ 22938], 90.00th=[ 28705], 95.00th=[ 33817], | 99.00th=[ 46924], 99.50th=[ 52691], 99.90th=[ 82314], 99.95th=[775947], | 99.99th=[775947] bw ( KiB/s): min= 6, max= 207, per=25.42%, avg=127.37, stdev=37.11, samples=94 iops : min= 1, max= 51, avg=31.49, stdev= 9.28, samples=94 write: IOPS=33, BW=132KiB/s (135kB/s)(6260KiB/47347msec); 0 zone resets clat (usec): min=652, max=74346, avg=13356.98, stdev=10428.48 lat (usec): min=652, max=74348, avg=13357.94, stdev=10428.58 clat percentiles (usec): | 1.00th=[ 1156], 5.00th=[ 2442], 10.00th=[ 2769], 20.00th=[ 3785], | 30.00th=[ 5997], 40.00th=[ 8848], 50.00th=[10814], 60.00th=[13173], | 70.00th=[16581], 80.00th=[21103], 90.00th=[27919], 95.00th=[34866], | 99.00th=[46400], 99.50th=[49546], 99.90th=[58983], 99.95th=[73925], | 99.99th=[73925] bw ( KiB/s): min= 19, max= 208, per=25.18%, avg=132.19, stdev=38.56, samples=94 iops : min= 4, max= 52, avg=32.69, stdev= 9.67, samples=94 lat (usec) : 100=0.03%, 250=0.16%, 500=0.78%, 750=0.36%, 1000=0.46% lat (msec) : 2=0.94%, 4=9.73%, 10=21.58%, 20=41.05%, 50=24.32% lat (msec) : 100=0.55%, 1000=0.03% cpu : usr=0.10%, sys=27.18%, ctx=8533, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194638: Fri Jul 25 03:38:41 2025 read: IOPS=31, BW=126KiB/s (129kB/s)(6052KiB/47867msec) clat (usec): min=86, max=64824, avg=16783.19, stdev=9098.28 lat (usec): min=87, max=64826, avg=16784.00, stdev=9098.31 clat percentiles (usec): | 1.00th=[ 334], 5.00th=[ 5735], 10.00th=[ 7635], 20.00th=[ 9634], | 30.00th=[11469], 40.00th=[13435], 50.00th=[15270], 60.00th=[17433], | 70.00th=[19530], 80.00th=[22676], 90.00th=[28705], 95.00th=[33817], | 99.00th=[46400], 99.50th=[55313], 99.90th=[59507], 99.95th=[64750], | 99.99th=[64750] bw ( KiB/s): min= 55, max= 216, per=25.79%, avg=129.20, stdev=32.90, samples=93 iops : min= 13, max= 54, avg=31.95, stdev= 8.30, samples=93 write: IOPS=32, BW=130KiB/s (133kB/s)(6236KiB/47867msec); 0 zone resets clat (usec): min=601, max=856917, avg=14272.74, stdev=23783.22 lat (usec): min=601, max=856918, avg=14273.61, stdev=23783.22 clat percentiles (usec): | 1.00th=[ 1139], 5.00th=[ 2409], 10.00th=[ 2802], 20.00th=[ 3785], | 30.00th=[ 6390], 40.00th=[ 9241], 50.00th=[ 11207], 60.00th=[ 14091], | 70.00th=[ 17695], 80.00th=[ 21890], 90.00th=[ 27919], 95.00th=[ 33817], | 99.00th=[ 47973], 99.50th=[ 52691], 99.90th=[ 73925], 99.95th=[859833], | 99.99th=[859833] bw ( KiB/s): min= 24, max= 231, per=25.04%, avg=131.48, stdev=35.86, samples=94 iops : min= 6, max= 57, avg=32.52, stdev= 8.97, samples=94 lat (usec) : 100=0.03%, 250=0.07%, 500=1.27%, 750=0.23%, 1000=0.23% lat (msec) : 2=1.30%, 4=9.34%, 10=20.51%, 20=40.66%, 50=25.59% lat (msec) : 100=0.75%, 1000=0.03% cpu : usr=0.08%, sys=26.85%, ctx=8536, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=502KiB/s (514kB/s), 124KiB/s-127KiB/s (127kB/s-130kB/s), io=23.4MiB (24.6MB), run=47347-47867msec WRITE: bw=525KiB/s (538kB/s), 130KiB/s-133KiB/s (133kB/s-136kB/s), io=24.6MiB (25.7MB), run=47347-47867msec rand-rw: (groupid=0, jobs=1): err= 0: pid=194639: Fri Jul 25 03:38:49 2025 read: IOPS=27, BW=109KiB/s (111kB/s)(5992KiB/55037msec) clat (msec): min=2, max=783, avg=16.09, stdev=23.52 lat (msec): min=2, max=783, avg=16.09, stdev=23.52 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 18], 80.00th=[ 23], 90.00th=[ 34], 95.00th=[ 42], | 99.00th=[ 61], 99.50th=[ 67], 99.90th=[ 112], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min= 32, max= 432, per=25.17%, avg=109.48, stdev=73.75, samples=108 iops : min= 8, max= 108, avg=27.04, stdev=18.56, samples=108 write: IOPS=28, BW=114KiB/s (117kB/s)(6296KiB/55037msec); 0 zone resets clat (usec): min=4119, max=95668, avg=19527.80, stdev=13903.68 lat (usec): min=4120, max=95669, avg=19528.71, stdev=13903.83 clat percentiles (usec): | 1.00th=[ 4883], 5.00th=[ 5669], 10.00th=[ 6521], 20.00th=[ 7767], | 30.00th=[ 9634], 40.00th=[11994], 50.00th=[15401], 60.00th=[19268], | 70.00th=[23987], 80.00th=[29492], 90.00th=[38011], 95.00th=[46400], | 99.00th=[68682], 99.50th=[79168], 99.90th=[94897], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 16, max= 392, per=25.27%, avg=114.96, stdev=74.30, samples=108 iops : min= 4, max= 98, avg=28.41, stdev=18.68, samples=108 lat (msec) : 4=1.69%, 10=36.56%, 20=30.18%, 50=28.39%, 100=3.12% lat (msec) : 250=0.03%, 1000=0.03% cpu : usr=0.09%, sys=13.81%, ctx=4192, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194640: Fri Jul 25 03:38:49 2025 read: IOPS=27, BW=108KiB/s (111kB/s)(5940KiB/54935msec) clat (msec): min=2, max=847, avg=15.90, stdev=24.71 lat (msec): min=2, max=847, avg=15.90, stdev=24.71 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 15], | 70.00th=[ 18], 80.00th=[ 23], 90.00th=[ 33], 95.00th=[ 39], | 99.00th=[ 59], 99.50th=[ 67], 99.90th=[ 92], 99.95th=[ 852], | 99.99th=[ 852] bw ( KiB/s): min= 8, max= 472, per=25.08%, avg=109.11, stdev=73.70, samples=108 iops : min= 2, max= 118, avg=26.94, stdev=18.54, samples=108 write: IOPS=28, BW=116KiB/s (118kB/s)(6348KiB/54935msec); 0 zone resets clat (msec): min=3, max=116, avg=19.62, stdev=14.09 lat (msec): min=3, max=116, avg=19.62, stdev=14.09 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 13], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 30], 90.00th=[ 39], 95.00th=[ 47], | 99.00th=[ 67], 99.50th=[ 74], 99.90th=[ 93], 99.95th=[ 116], | 99.99th=[ 116] bw ( KiB/s): min= 23, max= 368, per=25.56%, avg=116.29, stdev=73.20, samples=108 iops : min= 5, max= 92, avg=28.74, stdev=18.41, samples=108 lat (msec) : 4=1.92%, 10=35.35%, 20=30.76%, 50=28.87%, 100=3.03% lat (msec) : 250=0.03%, 1000=0.03% cpu : usr=0.10%, sys=13.87%, ctx=4191, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194641: Fri Jul 25 03:38:49 2025 read: IOPS=27, BW=109KiB/s (112kB/s)(6028KiB/55105msec) clat (usec): min=2489, max=86598, avg=15852.28, stdev=12429.64 lat (usec): min=2489, max=86599, avg=15853.03, stdev=12429.68 clat percentiles (usec): | 1.00th=[ 3425], 5.00th=[ 4228], 10.00th=[ 4883], 20.00th=[ 6128], | 30.00th=[ 7635], 40.00th=[ 9372], 50.00th=[11338], 60.00th=[14091], | 70.00th=[18482], 80.00th=[24773], 90.00th=[33817], 95.00th=[40633], | 99.00th=[57410], 99.50th=[66323], 99.90th=[81265], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 16, max= 400, per=25.25%, avg=109.85, stdev=74.62, samples=108 iops : min= 4, max= 100, avg=27.13, stdev=18.77, samples=108 write: IOPS=28, BW=114KiB/s (116kB/s)(6260KiB/55105msec); 0 zone resets clat (msec): min=3, max=852, avg=19.83, stdev=25.20 lat (msec): min=3, max=852, avg=19.83, stdev=25.20 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 30], 90.00th=[ 39], 95.00th=[ 47], | 99.00th=[ 67], 99.50th=[ 73], 99.90th=[ 111], 99.95th=[ 852], | 99.99th=[ 852] bw ( KiB/s): min= 23, max= 368, per=24.97%, avg=113.62, stdev=73.17, samples=108 iops : min= 5, max= 92, avg=28.07, stdev=18.41, samples=108 lat (msec) : 4=1.66%, 10=35.84%, 20=30.18%, 50=29.33%, 100=2.93% lat (msec) : 250=0.03%, 1000=0.03% cpu : usr=0.09%, sys=13.77%, ctx=4138, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194642: Fri Jul 25 03:38:49 2025 read: IOPS=27, BW=110KiB/s (112kB/s)(6052KiB/55155msec) clat (usec): min=2558, max=83075, avg=15708.87, stdev=12316.08 lat (usec): min=2559, max=83076, avg=15710.36, stdev=12315.74 clat percentiles (usec): | 1.00th=[ 3359], 5.00th=[ 4146], 10.00th=[ 4817], 20.00th=[ 6063], | 30.00th=[ 7767], 40.00th=[ 9372], 50.00th=[11469], 60.00th=[13960], | 70.00th=[17433], 80.00th=[24249], 90.00th=[33817], 95.00th=[41681], | 99.00th=[55313], 99.50th=[66847], 99.90th=[77071], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 23, max= 384, per=25.42%, avg=110.58, stdev=71.81, samples=107 iops : min= 5, max= 96, avg=27.31, stdev=18.09, samples=107 write: IOPS=28, BW=113KiB/s (116kB/s)(6236KiB/55155msec); 0 zone resets clat (msec): min=3, max=840, avg=20.01, stdev=25.13 lat (msec): min=3, max=840, avg=20.01, stdev=25.13 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 10], 40.00th=[ 12], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 30], 90.00th=[ 39], 95.00th=[ 48], | 99.00th=[ 67], 99.50th=[ 74], 99.90th=[ 111], 99.95th=[ 844], | 99.99th=[ 844] bw ( KiB/s): min= 31, max= 376, per=24.89%, avg=113.26, stdev=73.94, samples=108 iops : min= 7, max= 94, avg=27.98, stdev=18.60, samples=108 lat (msec) : 4=2.18%, 10=35.42%, 20=30.63%, 50=28.74%, 100=2.96% lat (msec) : 250=0.03%, 1000=0.03% cpu : usr=0.11%, sys=13.71%, ctx=4110, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=435KiB/s (446kB/s), 108KiB/s-110KiB/s (111kB/s-112kB/s), io=23.4MiB (24.6MB), run=54935-55155msec WRITE: bw=456KiB/s (467kB/s), 113KiB/s-116KiB/s (116kB/s-118kB/s), io=24.6MiB (25.7MB), run=54935-55155msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=194662: Fri Jul 25 03:39:10 2025 read: IOPS=23, BW=369KiB/s (378kB/s)(5920KiB/16051msec) clat (usec): min=130, max=88277, avg=21568.92, stdev=12808.51 lat (usec): min=131, max=88278, avg=21569.69, stdev=12808.52 clat percentiles (usec): | 1.00th=[ 147], 5.00th=[ 363], 10.00th=[ 8225], 20.00th=[11863], | 30.00th=[14484], 40.00th=[16712], 50.00th=[19530], 60.00th=[22938], | 70.00th=[26870], 80.00th=[31851], 90.00th=[38011], 95.00th=[44303], | 99.00th=[56361], 99.50th=[66847], 99.90th=[88605], 99.95th=[88605], | 99.99th=[88605] bw ( KiB/s): min= 160, max= 640, per=24.90%, avg=367.84, stdev=112.37, samples=31 iops : min= 10, max= 40, avg=22.52, stdev= 7.08, samples=31 write: IOPS=24, BW=397KiB/s (406kB/s)(6368KiB/16051msec); 0 zone resets clat (usec): min=1341, max=93685, avg=20133.57, stdev=17089.42 lat (usec): min=1342, max=93687, avg=20135.11, stdev=17089.44 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 2024], 10.00th=[ 3097], 20.00th=[ 4752], | 30.00th=[ 7373], 40.00th=[11731], 50.00th=[14615], 60.00th=[20055], | 70.00th=[25822], 80.00th=[34341], 90.00th=[45351], 95.00th=[53216], | 99.00th=[73925], 99.50th=[88605], 99.90th=[93848], 99.95th=[93848], | 99.99th=[93848] bw ( KiB/s): min= 256, max= 698, per=25.63%, avg=398.26, stdev=101.61, samples=31 iops : min= 16, max= 43, avg=24.42, stdev= 6.30, samples=31 lat (usec) : 250=1.43%, 500=2.08%, 1000=0.13% lat (msec) : 2=2.47%, 4=5.47%, 10=13.54%, 20=30.73%, 50=39.58% lat (msec) : 100=4.56% cpu : usr=0.12%, sys=16.41%, ctx=1975, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194663: Fri Jul 25 03:39:10 2025 read: IOPS=22, BW=354KiB/s (363kB/s)(5744KiB/16211msec) clat (usec): min=120, max=103428, avg=23997.08, stdev=15386.62 lat (usec): min=120, max=103429, avg=23997.85, stdev=15386.64 clat percentiles (usec): | 1.00th=[ 131], 5.00th=[ 334], 10.00th=[ 9765], 20.00th=[ 13698], | 30.00th=[ 16188], 40.00th=[ 18220], 50.00th=[ 21365], 60.00th=[ 23725], | 70.00th=[ 29230], 80.00th=[ 34341], 90.00th=[ 40109], 95.00th=[ 49546], | 99.00th=[ 91751], 99.50th=[101188], 99.90th=[103285], 99.95th=[103285], | 99.99th=[103285] bw ( KiB/s): min= 128, max= 539, per=24.05%, avg=355.16, stdev=101.46, samples=32 iops : min= 8, max= 33, avg=21.72, stdev= 6.32, samples=32 write: IOPS=25, BW=404KiB/s (413kB/s)(6544KiB/16211msec); 0 zone resets clat (usec): min=1358, max=125568, avg=18420.11, stdev=16202.63 lat (usec): min=1360, max=125569, avg=18421.62, stdev=16202.64 clat percentiles (usec): | 1.00th=[ 1598], 5.00th=[ 2278], 10.00th=[ 3490], 20.00th=[ 4883], | 30.00th=[ 7111], 40.00th=[ 10552], 50.00th=[ 13960], 60.00th=[ 18482], | 70.00th=[ 21627], 80.00th=[ 29230], 90.00th=[ 41157], 95.00th=[ 49546], | 99.00th=[ 69731], 99.50th=[ 72877], 99.90th=[125305], 99.95th=[125305], | 99.99th=[125305] bw ( KiB/s): min= 127, max= 638, per=25.87%, avg=402.00, stdev=121.93, samples=32 iops : min= 7, max= 39, avg=24.66, stdev= 7.72, samples=32 lat (usec) : 250=1.30%, 500=1.04%, 750=0.26%, 1000=0.26% lat (msec) : 2=1.82%, 4=6.77%, 10=13.93%, 20=30.60%, 50=39.32% lat (msec) : 100=4.30%, 250=0.39% cpu : usr=0.09%, sys=16.30%, ctx=2079, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194664: Fri Jul 25 03:39:10 2025 read: IOPS=24, BW=386KiB/s (396kB/s)(6256KiB/16190msec) clat (usec): min=124, max=178607, avg=23230.90, stdev=17021.03 lat (usec): min=124, max=178607, avg=23231.72, stdev=17021.01 clat percentiles (usec): | 1.00th=[ 145], 5.00th=[ 231], 10.00th=[ 4686], 20.00th=[ 12125], | 30.00th=[ 15139], 40.00th=[ 18482], 50.00th=[ 20841], 60.00th=[ 24249], | 70.00th=[ 27657], 80.00th=[ 31327], 90.00th=[ 41157], 95.00th=[ 49546], | 99.00th=[ 74974], 99.50th=[116917], 99.90th=[179307], 99.95th=[179307], | 99.99th=[179307] bw ( KiB/s): min= 224, max= 539, per=26.14%, avg=386.03, stdev=102.50, samples=32 iops : min= 14, max= 33, avg=23.66, stdev= 6.43, samples=32 write: IOPS=23, BW=373KiB/s (382kB/s)(6032KiB/16190msec); 0 zone resets clat (usec): min=1247, max=101063, avg=18691.17, stdev=16617.70 lat (usec): min=1249, max=101065, avg=18692.60, stdev=16617.70 clat percentiles (usec): | 1.00th=[ 1483], 5.00th=[ 2343], 10.00th=[ 3097], 20.00th=[ 4817], | 30.00th=[ 7439], 40.00th=[ 11469], 50.00th=[ 15139], 60.00th=[ 17957], | 70.00th=[ 22676], 80.00th=[ 28705], 90.00th=[ 39060], 95.00th=[ 53216], | 99.00th=[ 77071], 99.50th=[ 93848], 99.90th=[101188], 99.95th=[101188], | 99.99th=[101188] bw ( KiB/s): min= 160, max= 620, per=23.80%, avg=369.81, stdev=127.90, samples=32 iops : min= 10, max= 38, avg=22.62, stdev= 8.06, samples=32 lat (usec) : 250=2.86%, 500=2.21% lat (msec) : 2=2.21%, 4=4.69%, 10=12.76%, 20=31.25%, 50=38.67% lat (msec) : 100=4.95%, 250=0.39% cpu : usr=0.09%, sys=16.09%, ctx=2036, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194665: Fri Jul 25 03:39:10 2025 read: IOPS=24, BW=392KiB/s (401kB/s)(6032KiB/15398msec) clat (usec): min=126, max=79125, avg=21123.40, stdev=13589.45 lat (usec): min=126, max=79126, avg=21124.17, stdev=13589.47 clat percentiles (usec): | 1.00th=[ 131], 5.00th=[ 245], 10.00th=[ 404], 20.00th=[11469], | 30.00th=[14484], 40.00th=[16909], 50.00th=[19530], 60.00th=[22938], | 70.00th=[26346], 80.00th=[30540], 90.00th=[38536], 95.00th=[44303], | 99.00th=[68682], 99.50th=[70779], 99.90th=[79168], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 192, max= 728, per=26.28%, avg=388.10, stdev=113.33, samples=30 iops : min= 12, max= 45, avg=23.77, stdev= 7.12, samples=30 write: IOPS=25, BW=406KiB/s (416kB/s)(6256KiB/15398msec); 0 zone resets clat (usec): min=1447, max=103957, avg=18925.13, stdev=16595.17 lat (usec): min=1447, max=103959, avg=18926.57, stdev=16595.30 clat percentiles (usec): | 1.00th=[ 1565], 5.00th=[ 2245], 10.00th=[ 2835], 20.00th=[ 4883], | 30.00th=[ 7308], 40.00th=[ 10552], 50.00th=[ 14484], 60.00th=[ 19268], | 70.00th=[ 24249], 80.00th=[ 30016], 90.00th=[ 40109], 95.00th=[ 51119], | 99.00th=[ 79168], 99.50th=[ 86508], 99.90th=[104334], 99.95th=[104334], | 99.99th=[104334] bw ( KiB/s): min= 190, max= 649, per=25.84%, avg=401.60, stdev=113.98, samples=30 iops : min= 11, max= 40, avg=24.60, stdev= 7.14, samples=30 lat (usec) : 250=2.73%, 500=2.99%, 750=0.26%, 1000=0.13% lat (msec) : 2=1.82%, 4=5.86%, 10=13.80%, 20=30.08%, 50=38.28% lat (msec) : 100=3.91%, 250=0.13% cpu : usr=0.07%, sys=16.56%, ctx=1899, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1478KiB/s (1513kB/s), 354KiB/s-392KiB/s (363kB/s-401kB/s), io=23.4MiB (24.5MB), run=15398-16211msec WRITE: bw=1555KiB/s (1592kB/s), 373KiB/s-406KiB/s (382kB/s-416kB/s), io=24.6MiB (25.8MB), run=15398-16211msec rand-rw: (groupid=0, jobs=1): err= 0: pid=194667: Fri Jul 25 03:39:17 2025 read: IOPS=15, BW=256KiB/s (262kB/s)(5920KiB/23162msec) clat (msec): min=4, max=150, avg=26.04, stdev=18.75 lat (msec): min=4, max=150, avg=26.04, stdev=18.75 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 22], 60.00th=[ 26], | 70.00th=[ 30], 80.00th=[ 36], 90.00th=[ 52], 95.00th=[ 63], | 99.00th=[ 92], 99.50th=[ 118], 99.90th=[ 150], 99.95th=[ 150], | 99.99th=[ 150] bw ( KiB/s): min= 62, max= 480, per=23.91%, avg=247.27, stdev=110.97, samples=45 iops : min= 3, max= 30, avg=14.76, stdev= 7.12, samples=45 write: IOPS=17, BW=275KiB/s (282kB/s)(6368KiB/23162msec); 0 zone resets clat (msec): min=8, max=180, avg=33.89, stdev=22.53 lat (msec): min=8, max=180, avg=33.89, stdev=22.53 clat percentiles (msec): | 1.00th=[ 11], 5.00th=[ 14], 10.00th=[ 16], 20.00th=[ 19], | 30.00th=[ 21], 40.00th=[ 23], 50.00th=[ 28], 60.00th=[ 32], | 70.00th=[ 37], 80.00th=[ 47], 90.00th=[ 63], 95.00th=[ 81], | 99.00th=[ 107], 99.50th=[ 167], 99.90th=[ 180], 99.95th=[ 180], | 99.99th=[ 180] bw ( KiB/s): min= 63, max= 640, per=24.93%, avg=270.96, stdev=119.97, samples=45 iops : min= 3, max= 40, avg=16.24, stdev= 7.53, samples=45 lat (msec) : 10=7.94%, 20=29.04%, 50=48.96%, 100=13.02%, 250=1.04% cpu : usr=0.11%, sys=6.17%, ctx=1145, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194668: Fri Jul 25 03:39:17 2025 read: IOPS=15, BW=252KiB/s (258kB/s)(5744KiB/22768msec) clat (msec): min=5, max=150, avg=25.31, stdev=17.57 lat (msec): min=5, max=150, avg=25.31, stdev=17.57 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 16], 50.00th=[ 21], 60.00th=[ 26], | 70.00th=[ 31], 80.00th=[ 37], 90.00th=[ 47], 95.00th=[ 58], | 99.00th=[ 86], 99.50th=[ 109], 99.90th=[ 150], 99.95th=[ 150], | 99.99th=[ 150] bw ( KiB/s): min= 31, max= 640, per=24.36%, avg=251.84, stdev=128.88, samples=45 iops : min= 1, max= 40, avg=15.02, stdev= 8.18, samples=45 write: IOPS=17, BW=287KiB/s (294kB/s)(6544KiB/22768msec); 0 zone resets clat (msec): min=8, max=195, avg=33.36, stdev=21.10 lat (msec): min=8, max=195, avg=33.36, stdev=21.10 clat percentiles (msec): | 1.00th=[ 12], 5.00th=[ 13], 10.00th=[ 15], 20.00th=[ 17], | 30.00th=[ 20], 40.00th=[ 23], 50.00th=[ 27], 60.00th=[ 32], | 70.00th=[ 40], 80.00th=[ 50], 90.00th=[ 63], 95.00th=[ 71], | 99.00th=[ 104], 99.50th=[ 109], 99.90th=[ 197], 99.95th=[ 197], | 99.99th=[ 197] bw ( KiB/s): min= 127, max= 540, per=26.37%, avg=286.62, stdev=121.96, samples=45 iops : min= 7, max= 33, avg=17.22, stdev= 7.66, samples=45 lat (msec) : 10=6.90%, 20=32.29%, 50=47.01%, 100=12.89%, 250=0.91% cpu : usr=0.10%, sys=6.27%, ctx=1144, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194669: Fri Jul 25 03:39:17 2025 read: IOPS=17, BW=273KiB/s (279kB/s)(6256KiB/22939msec) clat (msec): min=5, max=100, avg=25.82, stdev=17.23 lat (msec): min=5, max=100, avg=25.82, stdev=17.23 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 12], | 30.00th=[ 14], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 26], | 70.00th=[ 31], 80.00th=[ 40], 90.00th=[ 48], 95.00th=[ 62], | 99.00th=[ 81], 99.50th=[ 90], 99.90th=[ 101], 99.95th=[ 101], | 99.99th=[ 101] bw ( KiB/s): min= 31, max= 731, per=26.47%, avg=273.69, stdev=136.14, samples=45 iops : min= 1, max= 45, avg=16.42, stdev= 8.58, samples=45 write: IOPS=16, BW=263KiB/s (269kB/s)(6032KiB/22939msec); 0 zone resets clat (msec): min=7, max=165, avg=33.99, stdev=22.89 lat (msec): min=7, max=165, avg=33.99, stdev=22.89 clat percentiles (msec): | 1.00th=[ 11], 5.00th=[ 14], 10.00th=[ 16], 20.00th=[ 18], | 30.00th=[ 20], 40.00th=[ 22], 50.00th=[ 26], 60.00th=[ 32], | 70.00th=[ 39], 80.00th=[ 48], 90.00th=[ 62], 95.00th=[ 84], | 99.00th=[ 114], 99.50th=[ 159], 99.90th=[ 165], 99.95th=[ 165], | 99.99th=[ 165] bw ( KiB/s): min= 63, max= 640, per=23.88%, avg=259.58, stdev=115.28, samples=45 iops : min= 3, max= 40, avg=15.51, stdev= 7.25, samples=45 lat (msec) : 10=8.46%, 20=31.77%, 50=45.96%, 100=12.37%, 250=1.43% cpu : usr=0.07%, sys=6.31%, ctx=1160, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194670: Fri Jul 25 03:39:17 2025 read: IOPS=16, BW=263KiB/s (270kB/s)(6032KiB/22898msec) clat (msec): min=5, max=113, avg=26.82, stdev=18.34 lat (msec): min=5, max=113, avg=26.82, stdev=18.34 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 12], | 30.00th=[ 15], 40.00th=[ 18], 50.00th=[ 23], 60.00th=[ 28], | 70.00th=[ 32], 80.00th=[ 38], 90.00th=[ 52], 95.00th=[ 69], | 99.00th=[ 86], 99.50th=[ 106], 99.90th=[ 113], 99.95th=[ 113], | 99.99th=[ 113] bw ( KiB/s): min= 94, max= 608, per=25.26%, avg=261.20, stdev=115.86, samples=45 iops : min= 5, max= 38, avg=15.62, stdev= 7.31, samples=45 write: IOPS=17, BW=273KiB/s (280kB/s)(6256KiB/22898msec); 0 zone resets clat (msec): min=9, max=156, avg=32.65, stdev=20.09 lat (msec): min=9, max=156, avg=32.65, stdev=20.09 clat percentiles (msec): | 1.00th=[ 11], 5.00th=[ 13], 10.00th=[ 15], 20.00th=[ 18], | 30.00th=[ 20], 40.00th=[ 23], 50.00th=[ 27], 60.00th=[ 32], | 70.00th=[ 37], 80.00th=[ 46], 90.00th=[ 61], 95.00th=[ 74], | 99.00th=[ 102], 99.50th=[ 105], 99.90th=[ 157], 99.95th=[ 157], | 99.99th=[ 157] bw ( KiB/s): min= 95, max= 604, per=25.04%, avg=272.22, stdev=118.88, samples=45 iops : min= 5, max= 37, avg=16.31, stdev= 7.52, samples=45 lat (msec) : 10=6.51%, 20=32.81%, 50=46.09%, 100=13.67%, 250=0.91% cpu : usr=0.09%, sys=6.13%, ctx=1108, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1034KiB/s (1059kB/s), 252KiB/s-273KiB/s (258kB/s-279kB/s), io=23.4MiB (24.5MB), run=22768-23162msec WRITE: bw=1088KiB/s (1114kB/s), 263KiB/s-287KiB/s (269kB/s-294kB/s), io=24.6MiB (25.8MB), run=22768-23162msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=194682: Fri Jul 25 03:39:26 2025 read: IOPS=2, BW=2807KiB/s (2874kB/s)(7168KiB/2554msec) clat (msec): min=77, max=189, avg=141.39, stdev=39.47 lat (msec): min=77, max=189, avg=141.39, stdev=39.47 clat percentiles (msec): | 1.00th=[ 79], 5.00th=[ 79], 10.00th=[ 79], 20.00th=[ 108], | 30.00th=[ 127], 40.00th=[ 127], 50.00th=[ 155], 60.00th=[ 165], | 70.00th=[ 165], 80.00th=[ 171], 90.00th=[ 190], 95.00th=[ 190], | 99.00th=[ 190], 99.50th=[ 190], 99.90th=[ 190], 99.95th=[ 190], | 99.99th=[ 190] bw ( KiB/s): min= 4096, max= 5485, per=76.40%, avg=4790.50, stdev=982.17, samples=2 iops : min= 4, max= 5, avg= 4.50, stdev= 0.71, samples=2 write: IOPS=1, BW=2005KiB/s (2053kB/s)(5120KiB/2554msec); 0 zone resets clat (msec): min=121, max=590, avg=303.31, stdev=189.38 lat (msec): min=121, max=590, avg=303.37, stdev=189.38 clat percentiles (msec): | 1.00th=[ 122], 5.00th=[ 122], 10.00th=[ 122], 20.00th=[ 122], | 30.00th=[ 146], 40.00th=[ 146], 50.00th=[ 300], 60.00th=[ 300], | 70.00th=[ 359], 80.00th=[ 359], 90.00th=[ 592], 95.00th=[ 592], | 99.00th=[ 592], 99.50th=[ 592], 99.90th=[ 592], 99.95th=[ 592], | 99.99th=[ 592] bw ( KiB/s): min= 2023, max= 4087, per=30.98%, avg=2719.33, stdev=1184.50, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 lat (msec) : 100=8.33%, 250=66.67%, 500=16.67%, 750=8.33% cpu : usr=0.04%, sys=39.21%, ctx=711, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194683: Fri Jul 25 03:39:26 2025 read: IOPS=0, BW=627KiB/s (642kB/s)(2048KiB/3266msec) clat (msec): min=107, max=128, avg=117.99, stdev=15.09 lat (msec): min=107, max=128, avg=118.00, stdev=15.09 clat percentiles (msec): | 1.00th=[ 108], 5.00th=[ 108], 10.00th=[ 108], 20.00th=[ 108], | 30.00th=[ 108], 40.00th=[ 108], 50.00th=[ 108], 60.00th=[ 129], | 70.00th=[ 129], 80.00th=[ 129], 90.00th=[ 129], 95.00th=[ 129], | 99.00th=[ 129], 99.50th=[ 129], 99.90th=[ 129], 99.95th=[ 129], | 99.99th=[ 129] bw ( KiB/s): min= 1802, max= 2043, per=30.66%, avg=1922.50, stdev=170.41, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3135KiB/s (3211kB/s)(10.0MiB/3266msec); 0 zone resets clat (msec): min=46, max=582, avg=297.56, stdev=174.10 lat (msec): min=46, max=582, avg=297.63, stdev=174.09 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 46], 10.00th=[ 46], 20.00th=[ 144], | 30.00th=[ 148], 40.00th=[ 159], 50.00th=[ 259], 60.00th=[ 355], | 70.00th=[ 359], 80.00th=[ 456], 90.00th=[ 468], 95.00th=[ 584], | 99.00th=[ 584], 99.50th=[ 584], 99.90th=[ 584], 99.95th=[ 584], | 99.99th=[ 584] bw ( KiB/s): min= 1835, max= 6144, per=41.43%, avg=3637.00, stdev=1771.33, samples=5 iops : min= 1, max= 6, avg= 3.00, stdev= 2.12, samples=5 lat (msec) : 50=8.33%, 250=41.67%, 500=41.67%, 750=8.33% cpu : usr=0.03%, sys=33.57%, ctx=120, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194684: Fri Jul 25 03:39:26 2025 read: IOPS=2, BW=2306KiB/s (2362kB/s)(6144KiB/2664msec) clat (usec): min=850, max=307527, avg=123749.35, stdev=111711.85 lat (usec): min=851, max=307528, avg=123750.34, stdev=111711.80 clat percentiles (usec): | 1.00th=[ 848], 5.00th=[ 848], 10.00th=[ 848], 20.00th=[ 18744], | 30.00th=[ 18744], 40.00th=[103285], 50.00th=[103285], 60.00th=[154141], | 70.00th=[158335], 80.00th=[158335], 90.00th=[308282], 95.00th=[308282], | 99.00th=[308282], 99.50th=[308282], 99.90th=[308282], 99.95th=[308282], | 99.99th=[308282] bw ( KiB/s): min= 1835, max= 4096, per=53.05%, avg=3326.00, stdev=1291.48, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=2, BW=2306KiB/s (2362kB/s)(6144KiB/2664msec); 0 zone resets clat (msec): min=130, max=618, avg=316.92, stdev=181.07 lat (msec): min=130, max=618, avg=316.98, stdev=181.07 clat percentiles (msec): | 1.00th=[ 131], 5.00th=[ 131], 10.00th=[ 131], 20.00th=[ 169], | 30.00th=[ 169], 40.00th=[ 226], 50.00th=[ 226], 60.00th=[ 376], | 70.00th=[ 384], 80.00th=[ 384], 90.00th=[ 617], 95.00th=[ 617], | 99.00th=[ 617], 99.50th=[ 617], 99.90th=[ 617], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 1835, max= 3618, per=26.35%, avg=2313.40, stdev=734.63, samples=5 iops : min= 1, max= 3, avg= 1.60, stdev= 0.89, samples=5 lat (usec) : 1000=8.33% lat (msec) : 20=8.33%, 250=50.00%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=31.06%, ctx=399, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194685: Fri Jul 25 03:39:26 2025 read: IOPS=2, BW=2165KiB/s (2217kB/s)(5120KiB/2365msec) clat (usec): min=395, max=203682, avg=41203.54, stdev=90828.77 lat (usec): min=395, max=203683, avg=41204.31, stdev=90828.88 clat percentiles (usec): | 1.00th=[ 396], 5.00th=[ 396], 10.00th=[ 396], 20.00th=[ 396], | 30.00th=[ 474], 40.00th=[ 474], 50.00th=[ 660], 60.00th=[ 660], | 70.00th=[ 807], 80.00th=[ 807], 90.00th=[204473], 95.00th=[204473], | 99.00th=[204473], 99.50th=[204473], 99.90th=[204473], 99.95th=[204473], | 99.99th=[204473] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=2, BW=3031KiB/s (3104kB/s)(7168KiB/2365msec); 0 zone resets clat (msec): min=184, max=514, avg=301.73, stdev=133.41 lat (msec): min=184, max=514, avg=301.79, stdev=133.40 clat percentiles (msec): | 1.00th=[ 186], 5.00th=[ 186], 10.00th=[ 186], 20.00th=[ 188], | 30.00th=[ 230], 40.00th=[ 230], 50.00th=[ 266], 60.00th=[ 266], | 70.00th=[ 266], 80.00th=[ 468], 90.00th=[ 514], 95.00th=[ 514], | 99.00th=[ 514], 99.50th=[ 514], 99.90th=[ 514], 99.95th=[ 514], | 99.99th=[ 514] bw ( KiB/s): min= 1828, max= 4096, per=28.45%, avg=2497.50, stdev=1070.07, samples=4 iops : min= 1, max= 4, avg= 1.75, stdev= 1.50, samples=4 lat (usec) : 500=16.67%, 750=8.33%, 1000=8.33% lat (msec) : 250=33.33%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=35.58%, ctx=173, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6271KiB/s (6421kB/s), 627KiB/s-2807KiB/s (642kB/s-2874kB/s), io=20.0MiB (20.0MB), run=2365-3266msec WRITE: bw=8779KiB/s (8990kB/s), 2005KiB/s-3135KiB/s (2053kB/s-3211kB/s), io=28.0MiB (29.4MB), run=2365-3266msec rand-rw: (groupid=0, jobs=1): err= 0: pid=194687: Fri Jul 25 03:39:26 2025 read: IOPS=2, BW=2462KiB/s (2521kB/s)(7168KiB/2912msec) clat (msec): min=68, max=704, avg=207.44, stdev=230.97 lat (msec): min=68, max=704, avg=207.44, stdev=230.97 clat percentiles (msec): | 1.00th=[ 69], 5.00th=[ 69], 10.00th=[ 69], 20.00th=[ 74], | 30.00th=[ 78], 40.00th=[ 78], 50.00th=[ 99], 60.00th=[ 157], | 70.00th=[ 157], 80.00th=[ 275], 90.00th=[ 701], 95.00th=[ 701], | 99.00th=[ 701], 99.50th=[ 701], 99.90th=[ 701], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 1957, max= 4055, per=39.60%, avg=2525.75, stdev=1020.36, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 write: IOPS=1, BW=1758KiB/s (1800kB/s)(5120KiB/2912msec); 0 zone resets clat (msec): min=185, max=337, avg=267.84, stdev=68.86 lat (msec): min=185, max=337, avg=267.92, stdev=68.87 clat percentiles (msec): | 1.00th=[ 186], 5.00th=[ 186], 10.00th=[ 186], 20.00th=[ 186], | 30.00th=[ 211], 40.00th=[ 211], 50.00th=[ 275], 60.00th=[ 275], | 70.00th=[ 330], 80.00th=[ 330], 90.00th=[ 338], 95.00th=[ 338], | 99.00th=[ 338], 99.50th=[ 338], 99.90th=[ 338], 99.95th=[ 338], | 99.99th=[ 338] bw ( KiB/s): min= 2027, max= 4087, per=30.45%, avg=2719.00, stdev=1184.75, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 100=33.33%, 250=25.00%, 500=33.33%, 750=8.33% cpu : usr=0.00%, sys=15.56%, ctx=196, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194688: Fri Jul 25 03:39:26 2025 read: IOPS=0, BW=638KiB/s (653kB/s)(2048KiB/3211msec) clat (msec): min=220, max=243, avg=232.05, stdev=16.46 lat (msec): min=220, max=243, avg=232.05, stdev=16.46 clat percentiles (msec): | 1.00th=[ 222], 5.00th=[ 222], 10.00th=[ 222], 20.00th=[ 222], | 30.00th=[ 222], 40.00th=[ 222], 50.00th=[ 222], 60.00th=[ 245], | 70.00th=[ 245], 80.00th=[ 245], 90.00th=[ 245], 95.00th=[ 245], | 99.00th=[ 245], 99.50th=[ 245], 99.90th=[ 245], 99.95th=[ 245], | 99.99th=[ 245] bw ( KiB/s): min= 1946, max= 2043, per=31.27%, avg=1994.50, stdev=68.59, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3189KiB/s (3266kB/s)(10.0MiB/3211msec); 0 zone resets clat (msec): min=95, max=645, avg=263.50, stdev=157.31 lat (msec): min=95, max=645, avg=263.57, stdev=157.30 clat percentiles (msec): | 1.00th=[ 95], 5.00th=[ 95], 10.00th=[ 95], 20.00th=[ 109], | 30.00th=[ 176], 40.00th=[ 197], 50.00th=[ 201], 60.00th=[ 284], | 70.00th=[ 292], 80.00th=[ 309], 90.00th=[ 330], 95.00th=[ 642], | 99.00th=[ 642], 99.50th=[ 642], 99.90th=[ 642], 99.95th=[ 642], | 99.99th=[ 642] bw ( KiB/s): min= 2043, max= 4087, per=36.56%, avg=3264.00, stdev=1112.41, samples=5 iops : min= 1, max= 3, avg= 2.40, stdev= 0.89, samples=5 lat (msec) : 100=8.33%, 250=50.00%, 500=33.33%, 750=8.33% cpu : usr=0.00%, sys=17.88%, ctx=164, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194689: Fri Jul 25 03:39:26 2025 read: IOPS=1, BW=1947KiB/s (1993kB/s)(6144KiB/3156msec) clat (msec): min=76, max=313, avg=171.98, stdev=103.89 lat (msec): min=76, max=313, avg=171.98, stdev=103.89 clat percentiles (msec): | 1.00th=[ 78], 5.00th=[ 78], 10.00th=[ 78], 20.00th=[ 101], | 30.00th=[ 101], 40.00th=[ 122], 50.00th=[ 122], 60.00th=[ 127], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 313], 95.00th=[ 313], | 99.00th=[ 313], 99.50th=[ 313], 99.90th=[ 313], 99.95th=[ 313], | 99.99th=[ 313] bw ( KiB/s): min= 4096, max= 4096, per=64.22%, avg=4096.00, stdev= 0.00, samples=2 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1947KiB/s (1993kB/s)(6144KiB/3156msec); 0 zone resets clat (msec): min=74, max=673, avg=346.94, stdev=243.41 lat (msec): min=74, max=673, avg=347.02, stdev=243.41 clat percentiles (msec): | 1.00th=[ 75], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 205], | 30.00th=[ 205], 40.00th=[ 215], 50.00th=[ 215], 60.00th=[ 296], | 70.00th=[ 617], 80.00th=[ 617], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 2027, max= 2896, per=26.02%, avg=2323.67, stdev=495.77, samples=3 iops : min= 1, max= 2, avg= 1.67, stdev= 0.58, samples=3 lat (msec) : 100=25.00%, 250=33.33%, 500=25.00%, 750=16.67% cpu : usr=0.03%, sys=13.28%, ctx=92, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194690: Fri Jul 25 03:39:26 2025 read: IOPS=1, BW=1645KiB/s (1684kB/s)(5120KiB/3113msec) clat (msec): min=58, max=329, avg=166.38, stdev=125.62 lat (msec): min=58, max=329, avg=166.38, stdev=125.62 clat percentiles (msec): | 1.00th=[ 58], 5.00th=[ 58], 10.00th=[ 58], 20.00th=[ 58], | 30.00th=[ 77], 40.00th=[ 77], 50.00th=[ 96], 60.00th=[ 96], | 70.00th=[ 275], 80.00th=[ 275], 90.00th=[ 330], 95.00th=[ 330], | 99.00th=[ 330], 99.50th=[ 330], 99.90th=[ 330], 99.95th=[ 330], | 99.99th=[ 330] bw ( KiB/s): min= 4087, max= 4087, per=64.08%, avg=4087.00, stdev= 0.00, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2303KiB/s (2358kB/s)(7168KiB/3113msec); 0 zone resets clat (msec): min=95, max=774, avg=310.31, stdev=243.95 lat (msec): min=95, max=775, avg=310.39, stdev=243.96 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 96], 10.00th=[ 96], 20.00th=[ 102], | 30.00th=[ 220], 40.00th=[ 220], 50.00th=[ 222], 60.00th=[ 266], | 70.00th=[ 266], 80.00th=[ 493], 90.00th=[ 776], 95.00th=[ 776], | 99.00th=[ 776], 99.50th=[ 776], 99.90th=[ 776], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 1957, max= 6131, per=39.72%, avg=3546.50, stdev=1976.97, samples=4 iops : min= 1, max= 5, avg= 2.50, stdev= 1.91, samples=4 lat (msec) : 100=33.33%, 250=25.00%, 500=33.33%, 1000=8.33% cpu : usr=0.00%, sys=14.56%, ctx=103, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6378KiB/s (6531kB/s), 638KiB/s-2462KiB/s (653kB/s-2521kB/s), io=20.0MiB (20.0MB), run=2912-3211msec WRITE: bw=8929KiB/s (9144kB/s), 1758KiB/s-3189KiB/s (1800kB/s-3266kB/s), io=28.0MiB (29.4MB), run=2912-3211msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=194695: Fri Jul 25 03:39:35 2025 read: IOPS=0, BW=3944KiB/s (4038kB/s)(12.0MiB/3116msec) clat (msec): min=782, max=1223, avg=1018.18, stdev=221.99 lat (msec): min=782, max=1223, avg=1018.18, stdev=221.99 clat percentiles (msec): | 1.00th=[ 785], 5.00th=[ 785], 10.00th=[ 785], 20.00th=[ 785], | 30.00th=[ 785], 40.00th=[ 1053], 50.00th=[ 1053], 60.00th=[ 1053], | 70.00th=[ 1217], 80.00th=[ 1217], 90.00th=[ 1217], 95.00th=[ 1217], | 99.00th=[ 1217], 99.50th=[ 1217], 99.90th=[ 1217], 99.95th=[ 1217], | 99.99th=[ 1217] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=11.17%, ctx=77, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194696: Fri Jul 25 03:39:35 2025 read: IOPS=0, BW=1205KiB/s (1234kB/s)(4096KiB/3399msec) clat (nsec): min=392547k, max=392547k, avg=392547486.00, stdev= 0.00 lat (nsec): min=392549k, max=392549k, avg=392549244.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 393], 5.00th=[ 393], 10.00th=[ 393], 20.00th=[ 393], | 30.00th=[ 393], 40.00th=[ 393], 50.00th=[ 393], 60.00th=[ 393], | 70.00th=[ 393], 80.00th=[ 393], 90.00th=[ 393], 95.00th=[ 393], | 99.00th=[ 393], 99.50th=[ 393], 99.90th=[ 393], 99.95th=[ 393], | 99.99th=[ 393] bw ( KiB/s): min= 7728, max= 7728, per=100.00%, avg=7728.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2410KiB/s (2468kB/s)(8192KiB/3399msec); 0 zone resets clat (msec): min=1244, max=1728, avg=1486.81, stdev=342.06 lat (msec): min=1245, max=1728, avg=1487.01, stdev=341.98 clat percentiles (msec): | 1.00th=[ 1250], 5.00th=[ 1250], 10.00th=[ 1250], 20.00th=[ 1250], | 30.00th=[ 1250], 40.00th=[ 1250], 50.00th=[ 1250], 60.00th=[ 1737], | 70.00th=[ 1737], 80.00th=[ 1737], 90.00th=[ 1737], 95.00th=[ 1737], | 99.00th=[ 1737], 99.50th=[ 1737], 99.90th=[ 1737], 99.95th=[ 1737], | 99.99th=[ 1737] bw ( KiB/s): min= 8192, max= 8192, per=84.98%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.00%, sys=20.34%, ctx=100, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194697: Fri Jul 25 03:39:35 2025 write: IOPS=1, BW=4627KiB/s (4738kB/s)(12.0MiB/2656msec); 0 zone resets clat (msec): min=643, max=1297, avg=873.05, stdev=368.28 lat (msec): min=643, max=1298, avg=873.39, stdev=368.25 clat percentiles (msec): | 1.00th=[ 642], 5.00th=[ 642], 10.00th=[ 642], 20.00th=[ 642], | 30.00th=[ 642], 40.00th=[ 676], 50.00th=[ 676], 60.00th=[ 676], | 70.00th=[ 1301], 80.00th=[ 1301], 90.00th=[ 1301], 95.00th=[ 1301], | 99.00th=[ 1301], 99.50th=[ 1301], 99.90th=[ 1301], 99.95th=[ 1301], | 99.99th=[ 1301] bw ( KiB/s): min= 8175, max= 8192, per=84.89%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.08%, sys=32.05%, ctx=131, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194698: Fri Jul 25 03:39:35 2025 write: IOPS=1, BW=6120KiB/s (6266kB/s)(12.0MiB/2008msec); 0 zone resets clat (msec): min=168, max=1495, avg=658.62, stdev=728.64 lat (msec): min=169, max=1496, avg=658.94, stdev=728.66 clat percentiles (msec): | 1.00th=[ 169], 5.00th=[ 169], 10.00th=[ 169], 20.00th=[ 169], | 30.00th=[ 169], 40.00th=[ 313], 50.00th=[ 313], 60.00th=[ 313], | 70.00th=[ 1502], 80.00th=[ 1502], 90.00th=[ 1502], 95.00th=[ 1502], | 99.00th=[ 1502], 99.50th=[ 1502], 99.90th=[ 1502], 99.95th=[ 1502], | 99.99th=[ 1502] bw ( KiB/s): min=15485, max=15485, per=100.00%, avg=15485.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=23.52%, ctx=86, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4820KiB/s (4936kB/s), 1205KiB/s-3944KiB/s (1234kB/s-4038kB/s), io=16.0MiB (16.8MB), run=3116-3399msec WRITE: bw=9640KiB/s (9872kB/s), 2410KiB/s-6120KiB/s (2468kB/s-6266kB/s), io=32.0MiB (33.6MB), run=2008-3399msec rand-rw: (groupid=0, jobs=1): err= 0: pid=194700: Fri Jul 25 03:39:35 2025 read: IOPS=1, BW=7115KiB/s (7286kB/s)(12.0MiB/1727msec) clat (msec): min=467, max=653, avg=568.90, stdev=94.37 lat (msec): min=467, max=653, avg=568.90, stdev=94.37 clat percentiles (msec): | 1.00th=[ 468], 5.00th=[ 468], 10.00th=[ 468], 20.00th=[ 468], | 30.00th=[ 468], 40.00th=[ 584], 50.00th=[ 584], 60.00th=[ 584], | 70.00th=[ 651], 80.00th=[ 651], 90.00th=[ 651], 95.00th=[ 651], | 99.00th=[ 651], 99.50th=[ 651], 99.90th=[ 651], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 7787, max= 8062, per=100.00%, avg=7924.50, stdev=194.45, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.00%, sys=47.62%, ctx=1914, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194701: Fri Jul 25 03:39:35 2025 read: IOPS=0, BW=1129KiB/s (1156kB/s)(4096KiB/3627msec) clat (nsec): min=411763k, max=411763k, avg=411762824.00, stdev= 0.00 lat (nsec): min=411765k, max=411765k, avg=411764744.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 414], 5.00th=[ 414], 10.00th=[ 414], 20.00th=[ 414], | 30.00th=[ 414], 40.00th=[ 414], 50.00th=[ 414], 60.00th=[ 414], | 70.00th=[ 414], 80.00th=[ 414], 90.00th=[ 414], 95.00th=[ 414], | 99.00th=[ 414], 99.50th=[ 414], 99.90th=[ 414], 99.95th=[ 414], | 99.99th=[ 414] bw ( KiB/s): min= 5468, max= 5468, per=100.00%, avg=5468.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2259KiB/s (2313kB/s)(8192KiB/3627msec); 0 zone resets clat (msec): min=1560, max=1607, avg=1584.35, stdev=33.05 lat (msec): min=1561, max=1608, avg=1584.62, stdev=33.07 clat percentiles (msec): | 1.00th=[ 1569], 5.00th=[ 1569], 10.00th=[ 1569], 20.00th=[ 1569], | 30.00th=[ 1569], 40.00th=[ 1569], 50.00th=[ 1569], 60.00th=[ 1603], | 70.00th=[ 1603], 80.00th=[ 1603], 90.00th=[ 1603], 95.00th=[ 1603], | 99.00th=[ 1603], 99.50th=[ 1603], 99.90th=[ 1603], 99.95th=[ 1603], | 99.99th=[ 1603] bw ( KiB/s): min= 8192, max= 8192, per=90.68%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.00%, sys=25.76%, ctx=773, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194702: Fri Jul 25 03:39:35 2025 write: IOPS=0, BW=3502KiB/s (3586kB/s)(12.0MiB/3509msec); 0 zone resets clat (msec): min=799, max=1365, avg=1163.20, stdev=316.02 lat (msec): min=799, max=1365, avg=1163.50, stdev=316.05 clat percentiles (msec): | 1.00th=[ 802], 5.00th=[ 802], 10.00th=[ 802], 20.00th=[ 802], | 30.00th=[ 802], 40.00th=[ 1318], 50.00th=[ 1318], 60.00th=[ 1318], | 70.00th=[ 1368], 80.00th=[ 1368], 90.00th=[ 1368], 95.00th=[ 1368], | 99.00th=[ 1368], 99.50th=[ 1368], 99.90th=[ 1368], 99.95th=[ 1368], | 99.99th=[ 1368] bw ( KiB/s): min= 8159, max= 8175, per=90.40%, avg=8167.00, stdev=11.31, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=42.82%, ctx=82, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=194703: Fri Jul 25 03:39:35 2025 write: IOPS=1, BW=4261KiB/s (4363kB/s)(12.0MiB/2884msec); 0 zone resets clat (msec): min=645, max=1415, avg=952.15, stdev=408.07 lat (msec): min=645, max=1415, avg=952.54, stdev=408.05 clat percentiles (msec): | 1.00th=[ 642], 5.00th=[ 642], 10.00th=[ 642], 20.00th=[ 642], | 30.00th=[ 642], 40.00th=[ 793], 50.00th=[ 793], 60.00th=[ 793], | 70.00th=[ 1418], 80.00th=[ 1418], 90.00th=[ 1418], 95.00th=[ 1418], | 99.00th=[ 1418], 99.50th=[ 1418], 99.90th=[ 1418], 99.95th=[ 1418], | 99.99th=[ 1418] bw ( KiB/s): min= 8192, max= 8192, per=90.68%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=53.00%, ctx=51, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4517KiB/s (4626kB/s), 1129KiB/s-7115KiB/s (1156kB/s-7286kB/s), io=16.0MiB (16.8MB), run=1727-3627msec WRITE: bw=9034KiB/s (9251kB/s), 2259KiB/s-4261KiB/s (2313kB/s-4363kB/s), io=32.0MiB (33.6MB), run=2884-3627msec