== sanity test 398b: DIO and buffer IO race ============== 10:17:05 (1753366625) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.16335 s, 15.9 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=154283: Thu Jul 24 10:18:31 2025 read: IOPS=19, BW=79.5KiB/s (81.4kB/s)(5992KiB/75385msec) clat (usec): min=110, max=696315, avg=24392.61, stdev=25669.70 lat (usec): min=111, max=696318, avg=24393.81, stdev=25669.79 clat percentiles (usec): | 1.00th=[ 157], 5.00th=[ 322], 10.00th=[ 6194], 20.00th=[ 11600], | 30.00th=[ 14484], 40.00th=[ 17433], 50.00th=[ 20579], 60.00th=[ 23987], | 70.00th=[ 28705], 80.00th=[ 34341], 90.00th=[ 44303], 95.00th=[ 54264], | 99.00th=[ 91751], 99.50th=[123208], 99.90th=[242222], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 15, max= 176, per=25.00%, avg=79.24, stdev=34.16, samples=148 iops : min= 3, max= 44, avg=19.31, stdev= 8.62, samples=148 write: IOPS=20, BW=83.5KiB/s (85.5kB/s)(6296KiB/75385msec); 0 zone resets clat (usec): min=925, max=346772, avg=24478.14, stdev=25712.88 lat (usec): min=926, max=346773, avg=24484.00, stdev=25720.04 clat percentiles (usec): | 1.00th=[ 1139], 5.00th=[ 1565], 10.00th=[ 2474], 20.00th=[ 4424], | 30.00th=[ 7898], 40.00th=[ 13173], 50.00th=[ 18220], 60.00th=[ 23725], | 70.00th=[ 30540], 80.00th=[ 40109], 90.00th=[ 52691], 95.00th=[ 68682], | 99.00th=[106431], 99.50th=[122160], 99.90th=[278922], 99.95th=[346031], | 99.99th=[346031] bw ( KiB/s): min= 23, max= 175, per=25.09%, avg=83.30, stdev=32.53, samples=148 iops : min= 5, max= 43, avg=20.32, stdev= 8.24, samples=148 lat (usec) : 250=1.73%, 500=1.79%, 750=0.62%, 1000=0.36% lat (msec) : 2=3.48%, 4=6.32%, 10=10.42%, 20=26.37%, 50=39.65% lat (msec) : 100=8.17%, 250=1.01%, 500=0.07%, 750=0.03% cpu : usr=0.12%, sys=13.64%, ctx=7039, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154284: Thu Jul 24 10:18:31 2025 read: IOPS=19, BW=78.6KiB/s (80.5kB/s)(5940KiB/75558msec) clat (usec): min=120, max=732518, avg=24922.89, stdev=26239.33 lat (usec): min=121, max=732521, avg=24924.22, stdev=26239.29 clat percentiles (usec): | 1.00th=[ 163], 5.00th=[ 367], 10.00th=[ 7439], 20.00th=[ 11338], | 30.00th=[ 13960], 40.00th=[ 17171], 50.00th=[ 20841], 60.00th=[ 24773], | 70.00th=[ 29492], 80.00th=[ 34866], 90.00th=[ 45876], 95.00th=[ 57934], | 99.00th=[ 89654], 99.50th=[113771], 99.90th=[250610], 99.95th=[734004], | 99.99th=[734004] bw ( KiB/s): min= 7, max= 176, per=25.05%, avg=79.41, stdev=34.94, samples=148 iops : min= 1, max= 44, avg=19.35, stdev= 8.83, samples=148 write: IOPS=21, BW=84.0KiB/s (86.0kB/s)(6348KiB/75558msec); 0 zone resets clat (usec): min=904, max=256576, avg=24098.00, stdev=25076.26 lat (usec): min=906, max=256578, avg=24099.42, stdev=25076.30 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 1369], 10.00th=[ 2212], 20.00th=[ 4015], | 30.00th=[ 7439], 40.00th=[ 12649], 50.00th=[ 17957], 60.00th=[ 23462], | 70.00th=[ 30016], 80.00th=[ 38536], 90.00th=[ 54789], 95.00th=[ 69731], | 99.00th=[110625], 99.50th=[127402], 99.90th=[242222], 99.95th=[256902], | 99.99th=[256902] bw ( KiB/s): min= 15, max= 303, per=25.58%, avg=84.93, stdev=38.71, samples=148 iops : min= 3, max= 75, avg=20.73, stdev= 9.73, samples=148 lat (usec) : 250=1.30%, 500=1.69%, 750=0.65%, 1000=0.62% lat (msec) : 2=4.46%, 4=5.83%, 10=11.33%, 20=25.42%, 50=38.96% lat (msec) : 100=8.66%, 250=0.98%, 500=0.07%, 750=0.03% cpu : usr=0.11%, sys=13.56%, ctx=6944, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154285: Thu Jul 24 10:18:31 2025 read: IOPS=20, BW=80.4KiB/s (82.3kB/s)(6028KiB/75018msec) clat (usec): min=114, max=365343, avg=24301.94, stdev=19628.72 lat (usec): min=115, max=365344, avg=24304.93, stdev=19628.74 clat percentiles (usec): | 1.00th=[ 161], 5.00th=[ 334], 10.00th=[ 7898], 20.00th=[ 11600], | 30.00th=[ 14091], 40.00th=[ 17433], 50.00th=[ 21103], 60.00th=[ 24511], | 70.00th=[ 28705], 80.00th=[ 34341], 90.00th=[ 43254], 95.00th=[ 55837], | 99.00th=[ 88605], 99.50th=[109577], 99.90th=[189793], 99.95th=[367002], | 99.99th=[367002] bw ( KiB/s): min= 15, max= 152, per=25.47%, avg=80.74, stdev=30.64, samples=146 iops : min= 3, max= 38, avg=19.68, stdev= 7.76, samples=146 write: IOPS=20, BW=83.4KiB/s (85.4kB/s)(6260KiB/75018msec); 0 zone resets clat (usec): min=864, max=888392, avg=24331.78, stdev=32707.73 lat (usec): min=865, max=888396, avg=24334.60, stdev=32707.76 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 1500], 10.00th=[ 2507], 20.00th=[ 4490], | 30.00th=[ 7767], 40.00th=[ 12780], 50.00th=[ 17695], 60.00th=[ 22938], | 70.00th=[ 30016], 80.00th=[ 38536], 90.00th=[ 52691], 95.00th=[ 68682], | 99.00th=[114820], 99.50th=[162530], 99.90th=[242222], 99.95th=[884999], | 99.99th=[884999] bw ( KiB/s): min= 8, max= 176, per=25.19%, avg=83.62, stdev=32.84, samples=146 iops : min= 2, max= 44, avg=20.41, stdev= 8.28, samples=146 lat (usec) : 250=1.89%, 500=1.20%, 750=0.36%, 1000=0.36% lat (msec) : 2=3.81%, 4=5.21%, 10=12.04%, 20=25.85%, 50=40.49% lat (msec) : 100=7.71%, 250=1.01%, 500=0.03%, 1000=0.03% cpu : usr=0.14%, sys=13.78%, ctx=7065, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154286: Thu Jul 24 10:18:31 2025 read: IOPS=20, BW=80.4KiB/s (82.3kB/s)(6052KiB/75279msec) clat (usec): min=91, max=219986, avg=24233.63, stdev=18827.61 lat (usec): min=92, max=219989, avg=24235.79, stdev=18827.72 clat percentiles (usec): | 1.00th=[ 145], 5.00th=[ 306], 10.00th=[ 6718], 20.00th=[ 11600], | 30.00th=[ 13960], 40.00th=[ 16909], 50.00th=[ 20579], 60.00th=[ 23987], | 70.00th=[ 28705], 80.00th=[ 35390], 90.00th=[ 47973], 95.00th=[ 56361], | 99.00th=[ 78119], 99.50th=[ 98042], 99.90th=[210764], 99.95th=[219153], | 99.99th=[219153] bw ( KiB/s): min= 7, max= 183, per=25.61%, avg=81.20, stdev=34.35, samples=147 iops : min= 1, max= 45, avg=19.80, stdev= 8.69, samples=147 write: IOPS=20, BW=82.8KiB/s (84.8kB/s)(6236KiB/75279msec); 0 zone resets clat (usec): min=843, max=897223, avg=24584.88, stdev=32685.33 lat (usec): min=844, max=897225, avg=24587.61, stdev=32685.12 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 1369], 10.00th=[ 2278], 20.00th=[ 4015], | 30.00th=[ 7439], 40.00th=[ 13042], 50.00th=[ 18482], 60.00th=[ 24249], | 70.00th=[ 30540], 80.00th=[ 38011], 90.00th=[ 53216], 95.00th=[ 66323], | 99.00th=[109577], 99.50th=[152044], 99.90th=[263193], 99.95th=[893387], | 99.99th=[893387] bw ( KiB/s): min= 15, max= 184, per=25.14%, avg=83.46, stdev=35.22, samples=147 iops : min= 3, max= 46, avg=20.36, stdev= 8.90, samples=147 lat (usec) : 100=0.10%, 250=1.92%, 500=1.89%, 750=0.23%, 1000=0.49% lat (msec) : 2=4.17%, 4=5.96%, 10=10.45%, 20=25.36%, 50=39.52% lat (msec) : 100=9.05%, 250=0.81%, 500=0.03%, 1000=0.03% cpu : usr=0.14%, sys=13.54%, ctx=6900, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=318KiB/s (325kB/s), 78.6KiB/s-80.4KiB/s (80.5kB/s-82.3kB/s), io=23.4MiB (24.6MB), run=75018-75558msec WRITE: bw=333KiB/s (341kB/s), 82.8KiB/s-84.0KiB/s (84.8kB/s-86.0kB/s), io=24.6MiB (25.7MB), run=75018-75558msec rand-rw: (groupid=0, jobs=1): err= 0: pid=154279: Thu Jul 24 10:19:28 2025 read: IOPS=11, BW=45.4KiB/s (46.5kB/s)(5992KiB/132062msec) clat (msec): min=3, max=722, avg=21.29, stdev=28.87 lat (msec): min=3, max=722, avg=21.29, stdev=28.87 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 14], 60.00th=[ 18], | 70.00th=[ 23], 80.00th=[ 29], 90.00th=[ 44], 95.00th=[ 61], | 99.00th=[ 101], 99.50th=[ 155], 99.90th=[ 359], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 7, max= 144, per=25.43%, avg=46.04, stdev=31.11, samples=255 iops : min= 1, max= 36, avg=11.02, stdev= 7.89, samples=255 write: IOPS=11, BW=47.7KiB/s (48.8kB/s)(6296KiB/132062msec); 0 zone resets clat (msec): min=19, max=337, avg=63.46, stdev=40.67 lat (msec): min=19, max=337, avg=63.47, stdev=40.67 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 28], 10.00th=[ 31], 20.00th=[ 36], | 30.00th=[ 41], 40.00th=[ 45], 50.00th=[ 50], 60.00th=[ 57], | 70.00th=[ 67], 80.00th=[ 83], 90.00th=[ 118], 95.00th=[ 153], | 99.00th=[ 222], 99.50th=[ 241], 99.90th=[ 317], 99.95th=[ 338], | 99.99th=[ 338] bw ( KiB/s): min= 7, max= 104, per=25.15%, avg=47.53, stdev=25.30, samples=260 iops : min= 1, max= 26, avg=11.39, stdev= 6.45, samples=260 lat (msec) : 4=0.03%, 10=17.97%, 20=13.77%, 50=39.68%, 100=21.00% lat (msec) : 250=7.26%, 500=0.26%, 750=0.03% cpu : usr=0.08%, sys=4.20%, ctx=4152, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154280: Thu Jul 24 10:19:28 2025 read: IOPS=11, BW=44.8KiB/s (45.9kB/s)(5940KiB/132522msec) clat (msec): min=3, max=773, avg=20.54, stdev=29.08 lat (msec): min=3, max=773, avg=20.54, stdev=29.08 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 14], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 28], 90.00th=[ 42], 95.00th=[ 55], | 99.00th=[ 109], 99.50th=[ 134], 99.90th=[ 351], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 7, max= 152, per=25.88%, avg=46.84, stdev=31.05, samples=249 iops : min= 1, max= 38, avg=11.22, stdev= 7.86, samples=249 write: IOPS=11, BW=47.9KiB/s (49.1kB/s)(6348KiB/132522msec); 0 zone resets clat (msec): min=19, max=285, avg=64.11, stdev=41.29 lat (msec): min=19, max=285, avg=64.12, stdev=41.30 clat percentiles (msec): | 1.00th=[ 23], 5.00th=[ 28], 10.00th=[ 31], 20.00th=[ 36], | 30.00th=[ 41], 40.00th=[ 45], 50.00th=[ 50], 60.00th=[ 57], | 70.00th=[ 67], 80.00th=[ 85], 90.00th=[ 126], 95.00th=[ 163], | 99.00th=[ 211], 99.50th=[ 222], 99.90th=[ 271], 99.95th=[ 288], | 99.99th=[ 288] bw ( KiB/s): min= 7, max= 119, per=25.21%, avg=47.66, stdev=25.41, samples=261 iops : min= 1, max= 29, avg=11.43, stdev= 6.48, samples=261 lat (msec) : 4=0.07%, 10=18.20%, 20=13.96%, 50=39.36%, 100=20.35% lat (msec) : 250=7.88%, 500=0.16%, 1000=0.03% cpu : usr=0.09%, sys=4.02%, ctx=4101, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154281: Thu Jul 24 10:19:28 2025 read: IOPS=11, BW=45.6KiB/s (46.7kB/s)(6028KiB/132136msec) clat (msec): min=4, max=210, avg=20.73, stdev=19.09 lat (msec): min=4, max=210, avg=20.73, stdev=19.09 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 16], 60.00th=[ 20], | 70.00th=[ 24], 80.00th=[ 30], 90.00th=[ 42], 95.00th=[ 56], | 99.00th=[ 96], 99.50th=[ 115], 99.90th=[ 197], 99.95th=[ 211], | 99.99th=[ 211] bw ( KiB/s): min= 7, max= 151, per=25.62%, avg=46.37, stdev=29.06, samples=255 iops : min= 1, max= 37, avg=11.11, stdev= 7.34, samples=255 write: IOPS=11, BW=47.4KiB/s (48.5kB/s)(6260KiB/132136msec); 0 zone resets clat (msec): min=20, max=881, avg=64.30, stdev=46.22 lat (msec): min=20, max=881, avg=64.30, stdev=46.22 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 28], 10.00th=[ 31], 20.00th=[ 36], | 30.00th=[ 41], 40.00th=[ 45], 50.00th=[ 50], 60.00th=[ 57], | 70.00th=[ 67], 80.00th=[ 84], 90.00th=[ 122], 95.00th=[ 159], | 99.00th=[ 213], 99.50th=[ 243], 99.90th=[ 401], 99.95th=[ 885], | 99.99th=[ 885] bw ( KiB/s): min= 7, max= 112, per=24.95%, avg=47.16, stdev=25.58, samples=260 iops : min= 1, max= 28, avg=11.30, stdev= 6.53, samples=260 lat (msec) : 10=17.02%, 20=13.64%, 50=41.50%, 100=20.05%, 250=7.58% lat (msec) : 500=0.16%, 1000=0.03% cpu : usr=0.07%, sys=4.14%, ctx=4124, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154282: Thu Jul 24 10:19:28 2025 read: IOPS=11, BW=46.2KiB/s (47.3kB/s)(6052KiB/131066msec) clat (msec): min=4, max=196, avg=19.70, stdev=18.78 lat (msec): min=4, max=196, avg=19.70, stdev=18.78 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 14], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 28], 90.00th=[ 39], 95.00th=[ 56], | 99.00th=[ 89], 99.50th=[ 120], 99.90th=[ 186], 99.95th=[ 197], | 99.99th=[ 197] bw ( KiB/s): min= 7, max= 144, per=26.09%, avg=47.22, stdev=29.96, samples=252 iops : min= 1, max= 36, avg=11.31, stdev= 7.60, samples=252 write: IOPS=11, BW=47.6KiB/s (48.7kB/s)(6236KiB/131066msec); 0 zone resets clat (msec): min=18, max=879, avg=64.78, stdev=47.73 lat (msec): min=18, max=879, avg=64.78, stdev=47.73 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 28], 10.00th=[ 32], 20.00th=[ 36], | 30.00th=[ 40], 40.00th=[ 45], 50.00th=[ 51], 60.00th=[ 57], | 70.00th=[ 67], 80.00th=[ 84], 90.00th=[ 117], 95.00th=[ 150], | 99.00th=[ 243], 99.50th=[ 279], 99.90th=[ 393], 99.95th=[ 877], | 99.99th=[ 877] bw ( KiB/s): min= 7, max= 112, per=25.11%, avg=47.47, stdev=24.91, samples=258 iops : min= 1, max= 28, avg=11.37, stdev= 6.36, samples=258 lat (msec) : 10=18.33%, 20=14.13%, 50=38.96%, 100=21.00%, 250=7.19% lat (msec) : 500=0.36%, 1000=0.03% cpu : usr=0.08%, sys=4.11%, ctx=4125, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=181KiB/s (186kB/s), 44.8KiB/s-46.2KiB/s (45.9kB/s-47.3kB/s), io=23.4MiB (24.6MB), run=131066-132522msec WRITE: bw=190KiB/s (194kB/s), 47.4KiB/s-47.9KiB/s (48.5kB/s-49.1kB/s), io=24.6MiB (25.7MB), run=131066-132522msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=154321: Thu Jul 24 10:19:50 2025 read: IOPS=24, BW=387KiB/s (396kB/s)(5920KiB/15314msec) clat (usec): min=109, max=91253, avg=21356.37, stdev=13520.74 lat (usec): min=110, max=91255, avg=21357.52, stdev=13520.76 clat percentiles (usec): | 1.00th=[ 233], 5.00th=[ 482], 10.00th=[ 619], 20.00th=[12256], | 30.00th=[15139], 40.00th=[17957], 50.00th=[20579], 60.00th=[23200], | 70.00th=[25822], 80.00th=[30540], 90.00th=[37487], 95.00th=[44827], | 99.00th=[63701], 99.50th=[80217], 99.90th=[91751], 99.95th=[91751], | 99.99th=[91751] bw ( KiB/s): min= 254, max= 633, per=25.78%, avg=387.80, stdev=102.52, samples=30 iops : min= 15, max= 39, avg=23.57, stdev= 6.43, samples=30 write: IOPS=25, BW=416KiB/s (426kB/s)(6368KiB/15314msec); 0 zone resets clat (usec): min=1365, max=72123, avg=18529.58, stdev=13492.42 lat (usec): min=1367, max=72125, avg=18531.79, stdev=13492.57 clat percentiles (usec): | 1.00th=[ 1909], 5.00th=[ 3785], 10.00th=[ 4555], 20.00th=[ 6652], | 30.00th=[ 8717], 40.00th=[11731], 50.00th=[14746], 60.00th=[19006], | 70.00th=[23725], 80.00th=[29492], 90.00th=[36963], 95.00th=[44827], | 99.00th=[60556], 99.50th=[70779], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min= 221, max= 636, per=26.41%, avg=417.73, stdev=96.69, samples=30 iops : min= 13, max= 39, avg=25.43, stdev= 6.08, samples=30 lat (usec) : 250=0.78%, 500=1.69%, 750=2.73%, 1000=0.65% lat (msec) : 2=1.30%, 4=2.60%, 10=14.97%, 20=29.95%, 50=42.06% lat (msec) : 100=3.26% cpu : usr=0.12%, sys=30.09%, ctx=2226, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154322: Thu Jul 24 10:19:50 2025 read: IOPS=22, BW=361KiB/s (369kB/s)(5744KiB/15922msec) clat (usec): min=156, max=174334, avg=22420.92, stdev=15837.26 lat (usec): min=157, max=174335, avg=22422.05, stdev=15837.24 clat percentiles (usec): | 1.00th=[ 188], 5.00th=[ 429], 10.00th=[ 660], 20.00th=[ 13173], | 30.00th=[ 15533], 40.00th=[ 18744], 50.00th=[ 21103], 60.00th=[ 25297], | 70.00th=[ 28181], 80.00th=[ 31065], 90.00th=[ 36963], 95.00th=[ 42730], | 99.00th=[ 67634], 99.50th=[103285], 99.90th=[175113], 99.95th=[175113], | 99.99th=[175113] bw ( KiB/s): min= 84, max= 574, per=23.81%, avg=358.03, stdev=114.78, samples=31 iops : min= 5, max= 35, avg=21.71, stdev= 7.16, samples=31 write: IOPS=25, BW=411KiB/s (421kB/s)(6544KiB/15922msec); 0 zone resets clat (usec): min=1441, max=98652, avg=19053.56, stdev=14109.74 lat (usec): min=1443, max=98655, avg=19058.75, stdev=14108.82 clat percentiles (usec): | 1.00th=[ 1647], 5.00th=[ 3458], 10.00th=[ 4883], 20.00th=[ 6652], | 30.00th=[ 8717], 40.00th=[12125], 50.00th=[16188], 60.00th=[20055], | 70.00th=[23725], 80.00th=[29492], 90.00th=[38011], 95.00th=[45876], | 99.00th=[56361], 99.50th=[81265], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 160, max= 574, per=25.30%, avg=400.32, stdev=104.85, samples=31 iops : min= 10, max= 35, avg=24.35, stdev= 6.56, samples=31 lat (usec) : 250=0.65%, 500=2.21%, 750=2.34%, 1000=0.26% lat (msec) : 2=1.56%, 4=2.34%, 10=15.49%, 20=27.99%, 50=43.75% lat (msec) : 100=3.12%, 250=0.26% cpu : usr=0.09%, sys=29.74%, ctx=2326, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154323: Thu Jul 24 10:19:50 2025 read: IOPS=24, BW=394KiB/s (403kB/s)(6256KiB/15885msec) clat (usec): min=138, max=77879, avg=22386.40, stdev=13713.82 lat (usec): min=138, max=77881, avg=22387.73, stdev=13713.83 clat percentiles (usec): | 1.00th=[ 167], 5.00th=[ 498], 10.00th=[ 758], 20.00th=[12387], | 30.00th=[16188], 40.00th=[18744], 50.00th=[21890], 60.00th=[23987], | 70.00th=[27132], 80.00th=[31327], 90.00th=[38011], 95.00th=[47973], | 99.00th=[68682], 99.50th=[74974], 99.90th=[78119], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 159, max= 699, per=26.19%, avg=393.90, stdev=102.82, samples=31 iops : min= 9, max= 43, avg=23.94, stdev= 6.45, samples=31 write: IOPS=23, BW=380KiB/s (389kB/s)(6032KiB/15885msec); 0 zone resets clat (usec): min=1465, max=71592, avg=18663.27, stdev=13013.31 lat (usec): min=1467, max=71594, avg=18665.41, stdev=13013.32 clat percentiles (usec): | 1.00th=[ 1860], 5.00th=[ 3490], 10.00th=[ 4686], 20.00th=[ 6587], | 30.00th=[ 8848], 40.00th=[12518], 50.00th=[16188], 60.00th=[19530], | 70.00th=[23200], 80.00th=[29230], 90.00th=[36963], 95.00th=[44303], | 99.00th=[53740], 99.50th=[69731], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min= 191, max= 508, per=23.58%, avg=372.97, stdev=94.63, samples=31 iops : min= 11, max= 31, avg=22.61, stdev= 5.94, samples=31 lat (usec) : 250=0.91%, 500=1.69%, 750=2.47%, 1000=0.52% lat (msec) : 2=0.91%, 4=3.26%, 10=13.54%, 20=29.04%, 50=44.66% lat (msec) : 100=2.99% cpu : usr=0.16%, sys=28.76%, ctx=2286, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154324: Thu Jul 24 10:19:50 2025 read: IOPS=23, BW=384KiB/s (393kB/s)(6032KiB/15723msec) clat (usec): min=146, max=89988, avg=21384.19, stdev=13618.77 lat (usec): min=148, max=89989, avg=21385.44, stdev=13618.78 clat percentiles (usec): | 1.00th=[ 176], 5.00th=[ 404], 10.00th=[ 594], 20.00th=[11863], | 30.00th=[15795], 40.00th=[18744], 50.00th=[21103], 60.00th=[24511], | 70.00th=[27132], 80.00th=[30802], 90.00th=[37487], 95.00th=[44827], | 99.00th=[61080], 99.50th=[71828], 99.90th=[89654], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 169, max= 544, per=25.37%, avg=381.52, stdev=93.54, samples=31 iops : min= 10, max= 34, avg=23.16, stdev= 5.96, samples=31 write: IOPS=24, BW=398KiB/s (407kB/s)(6256KiB/15723msec); 0 zone resets clat (usec): min=1492, max=103645, avg=19216.34, stdev=14189.29 lat (usec): min=1495, max=103647, avg=19218.67, stdev=14189.40 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 13], 50.00th=[ 17], 60.00th=[ 21], | 70.00th=[ 25], 80.00th=[ 30], 90.00th=[ 39], 95.00th=[ 44], | 99.00th=[ 59], 99.50th=[ 104], 99.90th=[ 104], 99.95th=[ 104], | 99.99th=[ 104] bw ( KiB/s): min= 159, max= 670, per=25.21%, avg=398.84, stdev=116.99, samples=31 iops : min= 9, max= 41, avg=24.26, stdev= 7.28, samples=31 lat (usec) : 250=0.78%, 500=2.99%, 750=2.73%, 1000=0.91% lat (msec) : 2=0.78%, 4=2.47%, 10=15.63%, 20=27.08%, 50=44.14% lat (msec) : 100=2.21%, 250=0.26% cpu : usr=0.18%, sys=28.74%, ctx=2327, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1504KiB/s (1540kB/s), 361KiB/s-394KiB/s (369kB/s-403kB/s), io=23.4MiB (24.5MB), run=15314-15922msec WRITE: bw=1583KiB/s (1621kB/s), 380KiB/s-416KiB/s (389kB/s-426kB/s), io=24.6MiB (25.8MB), run=15314-15922msec rand-rw: (groupid=0, jobs=1): err= 0: pid=154325: Thu Jul 24 10:20:02 2025 read: IOPS=13, BW=217KiB/s (222kB/s)(5920KiB/27296msec) clat (msec): min=4, max=125, avg=17.92, stdev=16.85 lat (msec): min=4, max=125, avg=17.92, stdev=16.85 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 15], | 70.00th=[ 18], 80.00th=[ 25], 90.00th=[ 41], 95.00th=[ 51], | 99.00th=[ 87], 99.50th=[ 97], 99.90th=[ 126], 99.95th=[ 126], | 99.99th=[ 126] bw ( KiB/s): min= 31, max= 544, per=25.24%, avg=216.83, stdev=112.24, samples=54 iops : min= 1, max= 34, avg=13.09, stdev= 7.19, samples=54 write: IOPS=14, BW=233KiB/s (239kB/s)(6368KiB/27296msec); 0 zone resets clat (msec): min=19, max=146, avg=51.62, stdev=22.22 lat (msec): min=19, max=146, avg=51.62, stdev=22.22 clat percentiles (msec): | 1.00th=[ 22], 5.00th=[ 27], 10.00th=[ 30], 20.00th=[ 33], | 30.00th=[ 37], 40.00th=[ 42], 50.00th=[ 46], 60.00th=[ 52], | 70.00th=[ 60], 80.00th=[ 68], 90.00th=[ 83], 95.00th=[ 96], | 99.00th=[ 129], 99.50th=[ 140], 99.90th=[ 146], 99.95th=[ 146], | 99.99th=[ 146] bw ( KiB/s): min= 95, max= 448, per=25.80%, avg=233.00, stdev=96.93, samples=54 iops : min= 5, max= 28, avg=14.09, stdev= 6.29, samples=54 lat (msec) : 10=21.35%, 20=14.97%, 50=39.45%, 100=22.14%, 250=2.08% cpu : usr=0.13%, sys=7.56%, ctx=1103, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154326: Thu Jul 24 10:20:02 2025 read: IOPS=12, BW=206KiB/s (211kB/s)(5744KiB/27879msec) clat (msec): min=4, max=145, avg=17.85, stdev=19.34 lat (msec): min=4, max=145, avg=17.85, stdev=19.34 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 13], | 70.00th=[ 17], 80.00th=[ 23], 90.00th=[ 41], 95.00th=[ 61], | 99.00th=[ 96], 99.50th=[ 136], 99.90th=[ 146], 99.95th=[ 146], | 99.99th=[ 146] bw ( KiB/s): min= 56, max= 448, per=24.32%, avg=208.93, stdev=116.02, samples=54 iops : min= 3, max= 28, avg=12.59, stdev= 7.48, samples=54 write: IOPS=14, BW=235KiB/s (240kB/s)(6544KiB/27879msec); 0 zone resets clat (msec): min=20, max=203, avg=52.37, stdev=23.44 lat (msec): min=20, max=203, avg=52.37, stdev=23.44 clat percentiles (msec): | 1.00th=[ 23], 5.00th=[ 28], 10.00th=[ 32], 20.00th=[ 35], | 30.00th=[ 39], 40.00th=[ 42], 50.00th=[ 46], 60.00th=[ 50], | 70.00th=[ 58], 80.00th=[ 68], 90.00th=[ 86], 95.00th=[ 100], | 99.00th=[ 131], 99.50th=[ 138], 99.90th=[ 205], 99.95th=[ 205], | 99.99th=[ 205] bw ( KiB/s): min= 96, max= 480, per=25.91%, avg=233.93, stdev=94.66, samples=55 iops : min= 6, max= 30, avg=14.16, stdev= 6.12, samples=55 lat (msec) : 10=22.01%, 20=12.76%, 50=41.02%, 100=21.35%, 250=2.86% cpu : usr=0.11%, sys=7.60%, ctx=1172, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154327: Thu Jul 24 10:20:02 2025 read: IOPS=14, BW=230KiB/s (235kB/s)(6256KiB/27248msec) clat (msec): min=4, max=103, avg=18.83, stdev=17.78 lat (msec): min=4, max=103, avg=18.83, stdev=17.78 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 14], | 70.00th=[ 20], 80.00th=[ 31], 90.00th=[ 41], 95.00th=[ 62], | 99.00th=[ 85], 99.50th=[ 92], 99.90th=[ 104], 99.95th=[ 104], | 99.99th=[ 104] bw ( KiB/s): min= 63, max= 544, per=26.84%, avg=230.54, stdev=127.32, samples=54 iops : min= 3, max= 34, avg=13.94, stdev= 8.13, samples=54 write: IOPS=13, BW=221KiB/s (227kB/s)(6032KiB/27248msec); 0 zone resets clat (msec): min=18, max=159, avg=52.41, stdev=23.26 lat (msec): min=18, max=159, avg=52.41, stdev=23.26 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 28], 10.00th=[ 31], 20.00th=[ 35], | 30.00th=[ 38], 40.00th=[ 42], 50.00th=[ 47], 60.00th=[ 52], | 70.00th=[ 58], 80.00th=[ 67], 90.00th=[ 81], 95.00th=[ 103], | 99.00th=[ 136], 99.50th=[ 146], 99.90th=[ 159], 99.95th=[ 159], | 99.99th=[ 159] bw ( KiB/s): min= 63, max= 416, per=24.49%, avg=221.13, stdev=97.44, samples=54 iops : min= 3, max= 26, avg=13.35, stdev= 6.30, samples=54 lat (msec) : 10=23.83%, 20=12.37%, 50=39.32%, 100=21.74%, 250=2.73% cpu : usr=0.10%, sys=7.80%, ctx=1135, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154328: Thu Jul 24 10:20:02 2025 read: IOPS=13, BW=220KiB/s (226kB/s)(6032KiB/27378msec) clat (msec): min=4, max=153, avg=18.76, stdev=18.88 lat (msec): min=4, max=153, avg=18.76, stdev=18.88 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 15], | 70.00th=[ 19], 80.00th=[ 27], 90.00th=[ 42], 95.00th=[ 65], | 99.00th=[ 93], 99.50th=[ 101], 99.90th=[ 155], 99.95th=[ 155], | 99.99th=[ 155] bw ( KiB/s): min= 63, max= 480, per=25.41%, avg=218.30, stdev=107.46, samples=54 iops : min= 3, max= 30, avg=13.19, stdev= 6.88, samples=54 write: IOPS=14, BW=229KiB/s (234kB/s)(6256KiB/27378msec); 0 zone resets clat (msec): min=19, max=204, avg=51.64, stdev=22.89 lat (msec): min=19, max=204, avg=51.64, stdev=22.89 clat percentiles (msec): | 1.00th=[ 23], 5.00th=[ 27], 10.00th=[ 30], 20.00th=[ 35], | 30.00th=[ 39], 40.00th=[ 43], 50.00th=[ 47], 60.00th=[ 51], | 70.00th=[ 57], 80.00th=[ 66], 90.00th=[ 81], 95.00th=[ 97], | 99.00th=[ 138], 99.50th=[ 169], 99.90th=[ 205], 99.95th=[ 205], | 99.99th=[ 205] bw ( KiB/s): min= 32, max= 416, per=25.22%, avg=227.74, stdev=97.10, samples=54 iops : min= 2, max= 26, avg=13.78, stdev= 6.27, samples=54 lat (msec) : 10=22.27%, 20=13.41%, 50=40.49%, 100=21.48%, 250=2.34% cpu : usr=0.09%, sys=7.70%, ctx=1096, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=859KiB/s (880kB/s), 206KiB/s-230KiB/s (211kB/s-235kB/s), io=23.4MiB (24.5MB), run=27248-27879msec WRITE: bw=904KiB/s (926kB/s), 221KiB/s-235KiB/s (227kB/s-240kB/s), io=24.6MiB (25.8MB), run=27248-27879msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=154338: Thu Jul 24 10:20:13 2025 read: IOPS=3, BW=3183KiB/s (3259kB/s)(7168KiB/2252msec) clat (msec): min=102, max=314, avg=166.33, stdev=74.06 lat (msec): min=102, max=314, avg=166.34, stdev=74.06 clat percentiles (msec): | 1.00th=[ 103], 5.00th=[ 103], 10.00th=[ 103], 20.00th=[ 104], | 30.00th=[ 138], 40.00th=[ 138], 50.00th=[ 146], 60.00th=[ 157], | 70.00th=[ 157], 80.00th=[ 207], 90.00th=[ 313], 95.00th=[ 313], | 99.00th=[ 313], 99.50th=[ 313], 99.90th=[ 313], 99.95th=[ 313], | 99.99th=[ 313] bw ( KiB/s): min= 2019, max= 5595, per=91.08%, avg=3807.00, stdev=2528.61, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=2, BW=2274KiB/s (2328kB/s)(5120KiB/2252msec); 0 zone resets clat (msec): min=137, max=346, avg=212.93, stdev=81.17 lat (msec): min=137, max=346, avg=213.02, stdev=81.20 clat percentiles (msec): | 1.00th=[ 138], 5.00th=[ 138], 10.00th=[ 138], 20.00th=[ 138], | 30.00th=[ 169], 40.00th=[ 169], 50.00th=[ 186], 60.00th=[ 186], | 70.00th=[ 226], 80.00th=[ 226], 90.00th=[ 347], 95.00th=[ 347], | 99.00th=[ 347], 99.50th=[ 347], 99.90th=[ 347], 99.95th=[ 347], | 99.99th=[ 347] bw ( KiB/s): min= 1398, max= 3670, per=40.37%, avg=2362.33, stdev=1174.27, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 250=83.33%, 500=16.67% cpu : usr=0.00%, sys=59.53%, ctx=589, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154339: Thu Jul 24 10:20:13 2025 read: IOPS=0, BW=459KiB/s (470kB/s)(2048KiB/4464msec) clat (msec): min=98, max=102, avg=100.21, stdev= 3.11 lat (msec): min=98, max=102, avg=100.21, stdev= 3.11 clat percentiles (msec): | 1.00th=[ 99], 5.00th=[ 99], 10.00th=[ 99], 20.00th=[ 99], | 30.00th=[ 99], 40.00th=[ 99], 50.00th=[ 99], 60.00th=[ 103], | 70.00th=[ 103], 80.00th=[ 103], 90.00th=[ 103], 95.00th=[ 103], | 99.00th=[ 103], 99.50th=[ 103], 99.90th=[ 103], 99.95th=[ 103], | 99.99th=[ 103] bw ( KiB/s): min= 2027, max= 2048, per=48.74%, avg=2037.50, stdev=14.85, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2294KiB/s (2349kB/s)(10.0MiB/4464msec); 0 zone resets clat (msec): min=110, max=942, avg=424.88, stdev=306.00 lat (msec): min=110, max=942, avg=424.97, stdev=306.00 clat percentiles (msec): | 1.00th=[ 110], 5.00th=[ 110], 10.00th=[ 110], 20.00th=[ 127], | 30.00th=[ 169], 40.00th=[ 184], 50.00th=[ 228], 60.00th=[ 502], | 70.00th=[ 567], 80.00th=[ 584], 90.00th=[ 844], 95.00th=[ 944], | 99.00th=[ 944], 99.50th=[ 944], 99.90th=[ 944], 99.95th=[ 944], | 99.99th=[ 944] bw ( KiB/s): min= 1892, max= 8192, per=61.89%, avg=3622.00, stdev=2720.36, samples=5 iops : min= 1, max= 8, avg= 3.00, stdev= 3.08, samples=5 lat (msec) : 100=8.33%, 250=50.00%, 750=25.00%, 1000=16.67% cpu : usr=0.02%, sys=30.97%, ctx=291, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154341: Thu Jul 24 10:20:13 2025 read: IOPS=1, BW=1921KiB/s (1967kB/s)(6144KiB/3198msec) clat (usec): min=836, max=242840, avg=119241.02, stdev=97459.45 lat (usec): min=837, max=242841, avg=119242.21, stdev=97459.34 clat percentiles (usec): | 1.00th=[ 840], 5.00th=[ 840], 10.00th=[ 840], 20.00th=[ 6194], | 30.00th=[ 6194], 40.00th=[132645], 50.00th=[132645], 60.00th=[147850], | 70.00th=[185598], 80.00th=[185598], 90.00th=[242222], 95.00th=[242222], | 99.00th=[242222], 99.50th=[242222], 99.90th=[242222], 99.95th=[242222], | 99.99th=[242222] bw ( KiB/s): min= 1835, max= 4055, per=62.85%, avg=2627.33, stdev=1238.89, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=1, BW=1921KiB/s (1967kB/s)(6144KiB/3198msec); 0 zone resets clat (msec): min=152, max=1084, avg=407.33, stdev=352.45 lat (msec): min=152, max=1084, avg=407.43, stdev=352.44 clat percentiles (msec): | 1.00th=[ 153], 5.00th=[ 153], 10.00th=[ 153], 20.00th=[ 186], | 30.00th=[ 186], 40.00th=[ 197], 50.00th=[ 197], 60.00th=[ 372], | 70.00th=[ 456], 80.00th=[ 456], 90.00th=[ 1083], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 1835, max= 4055, per=49.48%, avg=2895.67, stdev=1113.28, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 lat (usec) : 1000=8.33% lat (msec) : 10=8.33%, 250=58.33%, 500=16.67%, 2000=8.33% cpu : usr=0.03%, sys=34.03%, ctx=655, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154343: Thu Jul 24 10:20:13 2025 read: IOPS=1, BW=1045KiB/s (1070kB/s)(5120KiB/4899msec) clat (usec): min=837, max=1693.9k, avg=392226.15, stdev=729536.84 lat (usec): min=837, max=1693.9k, avg=392227.21, stdev=729536.49 clat percentiles (usec): | 1.00th=[ 840], 5.00th=[ 840], 10.00th=[ 840], | 20.00th=[ 840], 30.00th=[ 39584], 40.00th=[ 39584], | 50.00th=[ 86508], 60.00th=[ 86508], 70.00th=[ 141558], | 80.00th=[ 141558], 90.00th=[1686111], 95.00th=[1686111], | 99.00th=[1686111], 99.50th=[1686111], 99.90th=[1686111], | 99.95th=[1686111], 99.99th=[1686111] bw ( KiB/s): min= 2027, max= 4096, per=65.16%, avg=2723.67, stdev=1188.52, samples=3 iops : min= 1, max= 4, avg= 2.33, stdev= 1.53, samples=3 write: IOPS=1, BW=1463KiB/s (1498kB/s)(7168KiB/4899msec); 0 zone resets clat (msec): min=85, max=896, avg=413.47, stdev=344.48 lat (msec): min=85, max=896, avg=413.59, stdev=344.49 clat percentiles (msec): | 1.00th=[ 86], 5.00th=[ 86], 10.00th=[ 86], 20.00th=[ 117], | 30.00th=[ 176], 40.00th=[ 176], 50.00th=[ 241], 60.00th=[ 542], | 70.00th=[ 542], 80.00th=[ 835], 90.00th=[ 894], 95.00th=[ 894], | 99.00th=[ 894], 99.50th=[ 894], 99.90th=[ 894], 99.95th=[ 894], | 99.99th=[ 894] bw ( KiB/s): min= 1416, max= 6144, per=57.46%, avg=3362.50, stdev=2186.20, samples=4 iops : min= 1, max= 6, avg= 2.75, stdev= 2.36, samples=4 lat (usec) : 1000=8.33% lat (msec) : 50=8.33%, 100=16.67%, 250=33.33%, 750=8.33%, 1000=16.67% lat (msec) : 2000=8.33% cpu : usr=0.04%, sys=27.15%, ctx=534, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4180KiB/s (4281kB/s), 459KiB/s-3183KiB/s (470kB/s-3259kB/s), io=20.0MiB (20.0MB), run=2252-4899msec WRITE: bw=5853KiB/s (5993kB/s), 1463KiB/s-2294KiB/s (1498kB/s-2349kB/s), io=28.0MiB (29.4MB), run=2252-4899msec rand-rw: (groupid=0, jobs=1): err= 0: pid=154340: Thu Jul 24 10:20:13 2025 read: IOPS=1, BW=1611KiB/s (1650kB/s)(7168KiB/4449msec) clat (msec): min=84, max=258, avg=158.69, stdev=70.29 lat (msec): min=84, max=258, avg=158.69, stdev=70.29 clat percentiles (msec): | 1.00th=[ 85], 5.00th=[ 85], 10.00th=[ 85], 20.00th=[ 89], | 30.00th=[ 125], 40.00th=[ 125], 50.00th=[ 146], 60.00th=[ 163], | 70.00th=[ 163], 80.00th=[ 249], 90.00th=[ 259], 95.00th=[ 259], | 99.00th=[ 259], 99.50th=[ 259], 99.90th=[ 259], 99.95th=[ 259], | 99.99th=[ 259] bw ( KiB/s): min= 2015, max= 4213, per=63.51%, avg=2573.50, stdev=1093.06, samples=4 iops : min= 1, max= 4, avg= 1.75, stdev= 1.50, samples=4 write: IOPS=1, BW=1151KiB/s (1178kB/s)(5120KiB/4449msec); 0 zone resets clat (msec): min=218, max=1942, avg=637.82, stdev=733.95 lat (msec): min=218, max=1942, avg=637.92, stdev=733.96 clat percentiles (msec): | 1.00th=[ 220], 5.00th=[ 220], 10.00th=[ 220], 20.00th=[ 220], | 30.00th=[ 241], 40.00th=[ 241], 50.00th=[ 393], 60.00th=[ 393], | 70.00th=[ 397], 80.00th=[ 397], 90.00th=[ 1938], 95.00th=[ 1938], | 99.00th=[ 1938], 99.50th=[ 1938], 99.90th=[ 1938], 99.95th=[ 1938], | 99.99th=[ 1938] bw ( KiB/s): min= 2015, max= 3716, per=45.56%, avg=2584.67, stdev=979.77, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 100=16.67%, 250=50.00%, 500=25.00%, 2000=8.33% cpu : usr=0.02%, sys=13.04%, ctx=137, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154342: Thu Jul 24 10:20:13 2025 read: IOPS=0, BW=438KiB/s (448kB/s)(2048KiB/4680msec) clat (msec): min=133, max=354, avg=244.00, stdev=156.15 lat (msec): min=133, max=354, avg=244.01, stdev=156.15 clat percentiles (msec): | 1.00th=[ 134], 5.00th=[ 134], 10.00th=[ 134], 20.00th=[ 134], | 30.00th=[ 134], 40.00th=[ 134], 50.00th=[ 134], 60.00th=[ 355], | 70.00th=[ 355], 80.00th=[ 355], 90.00th=[ 355], 95.00th=[ 355], | 99.00th=[ 355], 99.50th=[ 355], 99.90th=[ 355], 99.95th=[ 355], | 99.99th=[ 355] bw ( KiB/s): min= 1374, max= 2043, per=42.16%, avg=1708.50, stdev=473.05, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2188KiB/s (2241kB/s)(10.0MiB/4680msec); 0 zone resets clat (msec): min=139, max=813, avg=417.00, stdev=257.10 lat (msec): min=139, max=813, avg=417.13, stdev=257.08 clat percentiles (msec): | 1.00th=[ 140], 5.00th=[ 140], 10.00th=[ 140], 20.00th=[ 161], | 30.00th=[ 176], 40.00th=[ 266], 50.00th=[ 279], 60.00th=[ 376], | 70.00th=[ 531], 80.00th=[ 693], 90.00th=[ 743], 95.00th=[ 810], | 99.00th=[ 810], 99.50th=[ 810], 99.90th=[ 810], 99.95th=[ 810], | 99.99th=[ 810] bw ( KiB/s): min= 1858, max= 4031, per=48.49%, avg=2750.80, stdev=1068.34, samples=5 iops : min= 1, max= 3, avg= 1.80, stdev= 1.10, samples=5 lat (msec) : 250=33.33%, 500=33.33%, 750=25.00%, 1000=8.33% cpu : usr=0.04%, sys=17.05%, ctx=252, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154344: Thu Jul 24 10:20:13 2025 read: IOPS=1, BW=1216KiB/s (1245kB/s)(6144KiB/5054msec) clat (msec): min=44, max=1541, avg=341.32, stdev=589.58 lat (msec): min=44, max=1541, avg=341.32, stdev=589.58 clat percentiles (msec): | 1.00th=[ 45], 5.00th=[ 45], 10.00th=[ 45], 20.00th=[ 83], | 30.00th=[ 83], 40.00th=[ 86], 50.00th=[ 86], 60.00th=[ 146], | 70.00th=[ 150], 80.00th=[ 150], 90.00th=[ 1536], 95.00th=[ 1536], | 99.00th=[ 1536], 99.50th=[ 1536], 99.90th=[ 1536], 99.95th=[ 1536], | 99.99th=[ 1536] bw ( KiB/s): min= 2015, max= 3716, per=63.95%, avg=2591.33, stdev=974.09, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=1, BW=1216KiB/s (1245kB/s)(6144KiB/5054msec); 0 zone resets clat (msec): min=57, max=1104, avg=475.78, stdev=377.15 lat (msec): min=57, max=1104, avg=475.87, stdev=377.13 clat percentiles (msec): | 1.00th=[ 58], 5.00th=[ 58], 10.00th=[ 58], 20.00th=[ 275], | 30.00th=[ 275], 40.00th=[ 321], 50.00th=[ 321], 60.00th=[ 368], | 70.00th=[ 735], 80.00th=[ 735], 90.00th=[ 1099], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 1400, max= 2043, per=32.99%, avg=1871.80, stdev=274.73, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 lat (msec) : 50=8.33%, 100=25.00%, 250=16.67%, 500=25.00%, 750=8.33% lat (msec) : 2000=16.67% cpu : usr=0.00%, sys=10.96%, ctx=132, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154345: Thu Jul 24 10:20:13 2025 read: IOPS=1, BW=1724KiB/s (1765kB/s)(5120KiB/2970msec) clat (msec): min=81, max=458, avg=218.27, stdev=162.68 lat (msec): min=81, max=458, avg=218.27, stdev=162.68 clat percentiles (msec): | 1.00th=[ 82], 5.00th=[ 82], 10.00th=[ 82], 20.00th=[ 82], | 30.00th=[ 86], 40.00th=[ 86], 50.00th=[ 159], 60.00th=[ 159], | 70.00th=[ 309], 80.00th=[ 309], 90.00th=[ 460], 95.00th=[ 460], | 99.00th=[ 460], 99.50th=[ 460], 99.90th=[ 460], 99.95th=[ 460], | 99.99th=[ 460] bw ( KiB/s): min= 2023, max= 3716, per=64.02%, avg=2594.00, stdev=971.73, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 write: IOPS=2, BW=2413KiB/s (2471kB/s)(7168KiB/2970msec); 0 zone resets clat (msec): min=145, max=496, avg=261.31, stdev=136.90 lat (msec): min=145, max=496, avg=261.41, stdev=136.90 clat percentiles (msec): | 1.00th=[ 146], 5.00th=[ 146], 10.00th=[ 146], 20.00th=[ 148], | 30.00th=[ 150], 40.00th=[ 150], 50.00th=[ 197], 60.00th=[ 347], | 70.00th=[ 347], 80.00th=[ 347], 90.00th=[ 498], 95.00th=[ 498], | 99.00th=[ 498], 99.50th=[ 498], 99.90th=[ 498], 99.95th=[ 498], | 99.99th=[ 498] bw ( KiB/s): min= 1393, max= 4087, per=50.83%, avg=2883.50, stdev=1381.69, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 lat (msec) : 100=16.67%, 250=41.67%, 500=41.67% cpu : usr=0.07%, sys=13.94%, ctx=109, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4052KiB/s (4149kB/s), 438KiB/s-1724KiB/s (448kB/s-1765kB/s), io=20.0MiB (20.0MB), run=2970-5054msec WRITE: bw=5673KiB/s (5809kB/s), 1151KiB/s-2413KiB/s (1178kB/s-2471kB/s), io=28.0MiB (29.4MB), run=2970-5054msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=154355: Thu Jul 24 10:20:24 2025 read: IOPS=1, BW=6086KiB/s (6232kB/s)(12.0MiB/2019msec) clat (msec): min=180, max=1068, avg=630.02, stdev=444.24 lat (msec): min=180, max=1068, avg=630.03, stdev=444.24 clat percentiles (msec): | 1.00th=[ 180], 5.00th=[ 180], 10.00th=[ 180], 20.00th=[ 180], | 30.00th=[ 180], 40.00th=[ 642], 50.00th=[ 642], 60.00th=[ 642], | 70.00th=[ 1070], 80.00th=[ 1070], 90.00th=[ 1070], 95.00th=[ 1070], | 99.00th=[ 1070], 99.50th=[ 1070], 99.90th=[ 1070], 99.95th=[ 1070], | 99.99th=[ 1070] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=29.29%, ctx=1082, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154356: Thu Jul 24 10:20:24 2025 read: IOPS=0, BW=1141KiB/s (1169kB/s)(4096KiB/3589msec) clat (nsec): min=679246k, max=679246k, avg=679246048.00, stdev= 0.00 lat (nsec): min=679249k, max=679249k, avg=679248756.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 676], 5.00th=[ 676], 10.00th=[ 676], 20.00th=[ 676], | 30.00th=[ 676], 40.00th=[ 676], 50.00th=[ 676], 60.00th=[ 676], | 70.00th=[ 676], 80.00th=[ 676], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2283KiB/s (2337kB/s)(8192KiB/3589msec); 0 zone resets clat (msec): min=1364, max=1433, avg=1398.80, stdev=48.47 lat (msec): min=1364, max=1433, avg=1399.12, stdev=48.64 clat percentiles (msec): | 1.00th=[ 1368], 5.00th=[ 1368], 10.00th=[ 1368], 20.00th=[ 1368], | 30.00th=[ 1368], 40.00th=[ 1368], 50.00th=[ 1368], 60.00th=[ 1435], | 70.00th=[ 1435], 80.00th=[ 1435], 90.00th=[ 1435], 95.00th=[ 1435], | 99.00th=[ 1435], 99.50th=[ 1435], 99.90th=[ 1435], 99.95th=[ 1435], | 99.99th=[ 1435] bw ( KiB/s): min= 8175, max= 8175, per=91.07%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.03%, sys=30.57%, ctx=965, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154357: Thu Jul 24 10:20:24 2025 write: IOPS=0, BW=3367KiB/s (3447kB/s)(12.0MiB/3650msec); 0 zone resets clat (msec): min=560, max=1990, avg=1198.89, stdev=727.13 lat (msec): min=560, max=1990, avg=1199.23, stdev=727.14 clat percentiles (msec): | 1.00th=[ 558], 5.00th=[ 558], 10.00th=[ 558], 20.00th=[ 558], | 30.00th=[ 558], 40.00th=[ 1045], 50.00th=[ 1045], 60.00th=[ 1045], | 70.00th=[ 1989], 80.00th=[ 1989], 90.00th=[ 1989], 95.00th=[ 1989], | 99.00th=[ 1989], 99.50th=[ 1989], 99.90th=[ 1989], 99.95th=[ 1989], | 99.99th=[ 1989] bw ( KiB/s): min= 8175, max= 8175, per=91.07%, avg=8175.00, stdev= 0.00, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.00%, sys=40.81%, ctx=102, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154358: Thu Jul 24 10:20:24 2025 write: IOPS=0, BW=3991KiB/s (4087kB/s)(12.0MiB/3079msec); 0 zone resets clat (msec): min=858, max=1134, avg=981.72, stdev=140.53 lat (msec): min=859, max=1135, avg=982.09, stdev=140.65 clat percentiles (msec): | 1.00th=[ 860], 5.00th=[ 860], 10.00th=[ 860], 20.00th=[ 860], | 30.00th=[ 860], 40.00th=[ 953], 50.00th=[ 953], 60.00th=[ 953], | 70.00th=[ 1133], 80.00th=[ 1133], 90.00th=[ 1133], 95.00th=[ 1133], | 99.00th=[ 1133], 99.50th=[ 1133], 99.90th=[ 1133], 99.95th=[ 1133], | 99.99th=[ 1133] bw ( KiB/s): min= 8175, max= 8192, per=91.16%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 1000=66.67%, 2000=33.33% cpu : usr=0.00%, sys=46.30%, ctx=134, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4565KiB/s (4675kB/s), 1141KiB/s-6086KiB/s (1169kB/s-6232kB/s), io=16.0MiB (16.8MB), run=2019-3589msec WRITE: bw=8978KiB/s (9193kB/s), 2283KiB/s-3991KiB/s (2337kB/s-4087kB/s), io=32.0MiB (33.6MB), run=3079-3650msec rand-rw: (groupid=0, jobs=1): err= 0: pid=154350: Thu Jul 24 10:20:24 2025 read: IOPS=1, BW=7842KiB/s (8030kB/s)(12.0MiB/1567msec) clat (msec): min=159, max=734, avg=504.51, stdev=304.29 lat (msec): min=159, max=734, avg=504.52, stdev=304.28 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 161], | 30.00th=[ 161], 40.00th=[ 617], 50.00th=[ 617], 60.00th=[ 617], | 70.00th=[ 735], 80.00th=[ 735], 90.00th=[ 735], 95.00th=[ 735], | 99.00th=[ 735], 99.50th=[ 735], 99.90th=[ 735], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 8175, max= 8175, per=100.00%, avg=8175.00, stdev= 0.00, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 750=66.67% cpu : usr=0.06%, sys=23.75%, ctx=72, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154351: Thu Jul 24 10:20:24 2025 read: IOPS=0, BW=1030KiB/s (1055kB/s)(4096KiB/3977msec) clat (nsec): min=879245k, max=879245k, avg=879244737.00, stdev= 0.00 lat (nsec): min=879247k, max=879247k, avg=879247408.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 877], 5.00th=[ 877], 10.00th=[ 877], 20.00th=[ 877], | 30.00th=[ 877], 40.00th=[ 877], 50.00th=[ 877], 60.00th=[ 877], | 70.00th=[ 877], 80.00th=[ 877], 90.00th=[ 877], 95.00th=[ 877], | 99.00th=[ 877], 99.50th=[ 877], 99.90th=[ 877], 99.95th=[ 877], | 99.99th=[ 877] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2060KiB/s (2109kB/s)(8192KiB/3977msec); 0 zone resets clat (msec): min=1399, max=1645, avg=1522.39, stdev=173.95 lat (msec): min=1399, max=1645, avg=1522.83, stdev=174.10 clat percentiles (msec): | 1.00th=[ 1401], 5.00th=[ 1401], 10.00th=[ 1401], 20.00th=[ 1401], | 30.00th=[ 1401], 40.00th=[ 1401], 50.00th=[ 1401], 60.00th=[ 1653], | 70.00th=[ 1653], 80.00th=[ 1653], 90.00th=[ 1653], 95.00th=[ 1653], | 99.00th=[ 1653], 99.50th=[ 1653], 99.90th=[ 1653], 99.95th=[ 1653], | 99.99th=[ 1653] bw ( KiB/s): min= 8175, max= 8175, per=99.22%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.03%, sys=11.54%, ctx=101, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154353: Thu Jul 24 10:20:24 2025 write: IOPS=0, BW=3970KiB/s (4066kB/s)(12.0MiB/3095msec); 0 zone resets clat (msec): min=506, max=1341, avg=1012.93, stdev=445.12 lat (msec): min=506, max=1342, avg=1013.25, stdev=445.09 clat percentiles (msec): | 1.00th=[ 506], 5.00th=[ 506], 10.00th=[ 506], 20.00th=[ 506], | 30.00th=[ 506], 40.00th=[ 1183], 50.00th=[ 1183], 60.00th=[ 1183], | 70.00th=[ 1334], 80.00th=[ 1334], 90.00th=[ 1334], 95.00th=[ 1334], | 99.00th=[ 1334], 99.50th=[ 1334], 99.90th=[ 1334], 99.95th=[ 1334], | 99.99th=[ 1334] bw ( KiB/s): min= 8175, max= 8192, per=99.33%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.03%, sys=16.22%, ctx=101, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=154354: Thu Jul 24 10:20:24 2025 write: IOPS=0, BW=3665KiB/s (3753kB/s)(12.0MiB/3353msec); 0 zone resets clat (msec): min=185, max=2311, avg=1109.76, stdev=1089.69 lat (msec): min=186, max=2311, avg=1110.09, stdev=1089.64 clat percentiles (msec): | 1.00th=[ 186], 5.00th=[ 186], 10.00th=[ 186], 20.00th=[ 186], | 30.00th=[ 186], 40.00th=[ 835], 50.00th=[ 835], 60.00th=[ 835], | 70.00th=[ 2299], 80.00th=[ 2299], 90.00th=[ 2299], 95.00th=[ 2299], | 99.00th=[ 2299], 99.50th=[ 2299], 99.90th=[ 2299], 99.95th=[ 2299], | 99.99th=[ 2299] bw ( KiB/s): min= 5634, max= 8159, per=83.71%, avg=6896.50, stdev=1785.44, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 1000=33.33%, >=2000=33.33% cpu : usr=0.00%, sys=12.86%, ctx=101, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4120KiB/s (4219kB/s), 1030KiB/s-7842KiB/s (1055kB/s-8030kB/s), io=16.0MiB (16.8MB), run=1567-3977msec WRITE: bw=8239KiB/s (8437kB/s), 2060KiB/s-3970KiB/s (2109kB/s-4066kB/s), io=32.0MiB (33.6MB), run=3095-3977msec