== sanity test 398b: DIO and buffer IO race ============== 21:51:35 (1749606695) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB) copied, 1.00196 s, 50.2 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=18746: Tue Jun 10 21:51:52 2025 read: IOPS=105, BW=421KiB/s (431kB/s)(5992KiB/14236msec) clat (usec): min=51, max=167094, avg=4188.77, stdev=5451.31 lat (usec): min=51, max=167095, avg=4190.27, stdev=5451.03 clat percentiles (usec): | 1.00th=[ 65], 5.00th=[ 72], 10.00th=[ 92], 20.00th=[ 302], | 30.00th=[ 2442], 40.00th=[ 2868], 50.00th=[ 3359], 60.00th=[ 3916], | 70.00th=[ 5014], 80.00th=[ 6587], 90.00th=[ 8717], 95.00th=[ 10421], | 99.00th=[ 13829], 99.50th=[ 15664], 99.90th=[ 34866], 99.95th=[166724], | 99.99th=[166724] bw ( KiB/s): min= 232, max= 608, per=24.76%, avg=416.79, stdev=84.40, samples=28 iops : min= 58, max= 152, avg=104.14, stdev=21.11, samples=28 write: IOPS=110, BW=442KiB/s (453kB/s)(6296KiB/14236msec) clat (usec): min=485, max=38695, avg=4995.92, stdev=4694.55 lat (usec): min=486, max=38695, avg=4996.69, stdev=4694.55 clat percentiles (usec): | 1.00th=[ 519], 5.00th=[ 562], 10.00th=[ 611], 20.00th=[ 824], | 30.00th=[ 1303], 40.00th=[ 2802], 50.00th=[ 3982], 60.00th=[ 5014], | 70.00th=[ 6259], 80.00th=[ 8225], 90.00th=[11076], 95.00th=[14091], | 99.00th=[21103], 99.50th=[23725], 99.90th=[33817], 99.95th=[38536], | 99.99th=[38536] bw ( KiB/s): min= 296, max= 584, per=24.85%, avg=437.93, stdev=65.34, samples=28 iops : min= 74, max= 146, avg=109.43, stdev=16.31, samples=28 lat (usec) : 100=5.73%, 250=3.97%, 500=0.36%, 750=8.33%, 1000=5.11% lat (msec) : 2=5.01%, 4=27.12%, 10=34.47%, 20=9.11%, 50=0.75% lat (msec) : 250=0.03% cpu : usr=0.18%, sys=22.63%, ctx=5703, majf=0, minf=36 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18747: Tue Jun 10 21:51:52 2025 read: IOPS=104, BW=418KiB/s (428kB/s)(5940KiB/14224msec) clat (usec): min=53, max=191854, avg=4209.65, stdev=5814.32 lat (usec): min=54, max=191857, avg=4210.68, stdev=5814.12 clat percentiles (usec): | 1.00th=[ 65], 5.00th=[ 72], 10.00th=[ 90], 20.00th=[ 1860], | 30.00th=[ 2474], 40.00th=[ 2900], 50.00th=[ 3392], 60.00th=[ 3982], | 70.00th=[ 5080], 80.00th=[ 6587], 90.00th=[ 8717], 95.00th=[ 10159], | 99.00th=[ 13304], 99.50th=[ 13829], 99.90th=[ 17433], 99.95th=[191890], | 99.99th=[191890] bw ( KiB/s): min= 136, max= 664, per=24.83%, avg=417.89, stdev=95.10, samples=28 iops : min= 34, max= 166, avg=104.39, stdev=23.80, samples=28 write: IOPS=111, BW=446KiB/s (457kB/s)(6348KiB/14224msec) clat (usec): min=462, max=52065, avg=4950.63, stdev=4755.69 lat (usec): min=463, max=52066, avg=4952.21, stdev=4755.52 clat percentiles (usec): | 1.00th=[ 506], 5.00th=[ 545], 10.00th=[ 594], 20.00th=[ 799], | 30.00th=[ 1106], 40.00th=[ 2802], 50.00th=[ 3982], 60.00th=[ 5014], | 70.00th=[ 6456], 80.00th=[ 8291], 90.00th=[11207], 95.00th=[13173], | 99.00th=[19530], 99.50th=[24511], 99.90th=[43254], 99.95th=[52167], | 99.99th=[52167] bw ( KiB/s): min= 216, max= 688, per=25.09%, avg=442.14, stdev=90.24, samples=28 iops : min= 54, max= 172, avg=110.46, stdev=22.53, samples=28 lat (usec) : 100=5.73%, 250=3.06%, 500=0.55%, 750=9.02%, 1000=5.37% lat (msec) : 2=5.31%, 4=25.98%, 10=35.38%, 20=9.15%, 50=0.39% lat (msec) : 100=0.03%, 250=0.03% cpu : usr=0.17%, sys=22.64%, ctx=5632, majf=0, minf=35 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18748: Tue Jun 10 21:51:52 2025 read: IOPS=105, BW=423KiB/s (433kB/s)(6028KiB/14261msec) clat (usec): min=51, max=42560, avg=4001.52, stdev=3366.26 lat (usec): min=51, max=42561, avg=4003.24, stdev=3365.75 clat percentiles (usec): | 1.00th=[ 64], 5.00th=[ 76], 10.00th=[ 92], 20.00th=[ 200], | 30.00th=[ 2442], 40.00th=[ 2835], 50.00th=[ 3261], 60.00th=[ 3982], | 70.00th=[ 5014], 80.00th=[ 6652], 90.00th=[ 8356], 95.00th=[ 9765], | 99.00th=[12649], 99.50th=[15008], 99.90th=[28181], 99.95th=[42730], | 99.99th=[42730] bw ( KiB/s): min= 232, max= 560, per=24.93%, avg=419.64, stdev=82.04, samples=28 iops : min= 58, max= 140, avg=104.86, stdev=20.50, samples=28 write: IOPS=109, BW=439KiB/s (449kB/s)(6260KiB/14261msec) clat (usec): min=461, max=167834, avg=5199.93, stdev=6219.60 lat (usec): min=462, max=167835, avg=5201.21, stdev=6219.48 clat percentiles (usec): | 1.00th=[ 515], 5.00th=[ 553], 10.00th=[ 611], 20.00th=[ 824], | 30.00th=[ 1123], 40.00th=[ 2638], 50.00th=[ 4178], 60.00th=[ 5407], | 70.00th=[ 6849], 80.00th=[ 8848], 90.00th=[ 11338], 95.00th=[ 13960], | 99.00th=[ 20841], 99.50th=[ 24773], 99.90th=[ 32637], 99.95th=[168821], | 99.99th=[168821] bw ( KiB/s): min= 312, max= 576, per=24.69%, avg=435.04, stdev=66.93, samples=28 iops : min= 78, max= 144, avg=108.71, stdev=16.69, samples=28 lat (usec) : 100=5.34%, 250=4.52%, 500=0.52%, 750=8.33%, 1000=5.47% lat (msec) : 2=5.37%, 4=24.84%, 10=35.68%, 20=9.15%, 50=0.75% lat (msec) : 250=0.03% cpu : usr=0.18%, sys=22.83%, ctx=5769, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18749: Tue Jun 10 21:51:52 2025 read: IOPS=107, BW=431KiB/s (442kB/s)(6052KiB/14027msec) clat (usec): min=57, max=204240, avg=4167.65, stdev=6095.66 lat (usec): min=58, max=204241, avg=4168.55, stdev=6095.68 clat percentiles (usec): | 1.00th=[ 66], 5.00th=[ 78], 10.00th=[ 93], 20.00th=[ 1975], | 30.00th=[ 2540], 40.00th=[ 2900], 50.00th=[ 3326], 60.00th=[ 3982], | 70.00th=[ 4883], 80.00th=[ 6259], 90.00th=[ 8455], 95.00th=[ 10159], | 99.00th=[ 13698], 99.50th=[ 17433], 99.90th=[ 31327], 99.95th=[204473], | 99.99th=[204473] bw ( KiB/s): min= 264, max= 584, per=25.65%, avg=431.64, stdev=73.93, samples=28 iops : min= 66, max= 146, avg=107.86, stdev=18.47, samples=28 write: IOPS=111, BW=445KiB/s (455kB/s)(6236KiB/14027msec) clat (usec): min=485, max=53606, avg=4893.90, stdev=4673.96 lat (usec): min=485, max=53606, avg=4895.67, stdev=4675.83 clat percentiles (usec): | 1.00th=[ 510], 5.00th=[ 545], 10.00th=[ 594], 20.00th=[ 791], | 30.00th=[ 1188], 40.00th=[ 2507], 50.00th=[ 3916], 60.00th=[ 5014], | 70.00th=[ 6456], 80.00th=[ 8225], 90.00th=[11076], 95.00th=[13304], | 99.00th=[19006], 99.50th=[22152], 99.90th=[45876], 99.95th=[53740], | 99.99th=[53740] bw ( KiB/s): min= 336, max= 600, per=25.21%, avg=444.18, stdev=60.98, samples=28 iops : min= 84, max= 150, avg=111.00, stdev=15.22, samples=28 lat (usec) : 100=5.44%, 250=3.55%, 500=0.46%, 750=9.15%, 1000=4.79% lat (msec) : 2=5.11%, 4=27.15%, 10=35.12%, 20=8.79%, 50=0.39% lat (msec) : 100=0.03%, 250=0.03% cpu : usr=0.21%, sys=22.55%, ctx=5651, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1684KiB/s (1724kB/s), 418KiB/s-431KiB/s (428kB/s-442kB/s), io=23.4MiB (24.6MB), run=14027-14261msec WRITE: bw=1763KiB/s (1805kB/s), 439KiB/s-446KiB/s (449kB/s-457kB/s), io=24.6MiB (25.7MB), run=14027-14261msec rand-rw: (groupid=0, jobs=1): err= 0: pid=18741: Tue Jun 10 21:52:11 2025 read: IOPS=45, BW=181KiB/s (185kB/s)(5992KiB/33185msec) clat (usec): min=1651, max=170317, avg=4303.53, stdev=5769.45 lat (usec): min=1652, max=170318, avg=4304.30, stdev=5769.46 clat percentiles (usec): | 1.00th=[ 1778], 5.00th=[ 1926], 10.00th=[ 2024], 20.00th=[ 2180], | 30.00th=[ 2343], 40.00th=[ 2474], 50.00th=[ 2704], 60.00th=[ 3163], | 70.00th=[ 3851], 80.00th=[ 5145], 90.00th=[ 8848], 95.00th=[ 11207], | 99.00th=[ 17171], 99.50th=[ 31065], 99.90th=[ 51643], 99.95th=[170918], | 99.99th=[170918] bw ( KiB/s): min= 56, max= 328, per=24.97%, avg=180.26, stdev=76.63, samples=66 iops : min= 14, max= 82, avg=44.98, stdev=19.21, samples=66 write: IOPS=47, BW=190KiB/s (194kB/s)(6296KiB/33185msec) clat (usec): min=7854, max=66021, avg=16914.49, stdev=7150.83 lat (usec): min=7855, max=66022, avg=16915.46, stdev=7150.83 clat percentiles (usec): | 1.00th=[ 8717], 5.00th=[ 9503], 10.00th=[10159], 20.00th=[11207], | 30.00th=[12387], 40.00th=[13566], 50.00th=[14877], 60.00th=[16319], | 70.00th=[17957], 80.00th=[21890], 90.00th=[27395], 95.00th=[32113], | 99.00th=[40109], 99.50th=[42730], 99.90th=[54789], 99.95th=[65799], | 99.99th=[65799] bw ( KiB/s): min= 48, max= 288, per=25.03%, avg=189.20, stdev=67.49, samples=66 iops : min= 12, max= 72, avg=47.23, stdev=16.93, samples=66 lat (msec) : 2=4.26%, 4=30.44%, 10=14.58%, 20=38.25%, 50=12.30% lat (msec) : 100=0.13%, 250=0.03% cpu : usr=0.12%, sys=5.96%, ctx=3554, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18742: Tue Jun 10 21:52:11 2025 read: IOPS=44, BW=179KiB/s (183kB/s)(5940KiB/33226msec) clat (usec): min=1528, max=192231, avg=4116.69, stdev=6113.68 lat (usec): min=1529, max=192232, avg=4117.50, stdev=6113.72 clat percentiles (usec): | 1.00th=[ 1778], 5.00th=[ 1926], 10.00th=[ 2040], 20.00th=[ 2180], | 30.00th=[ 2311], 40.00th=[ 2474], 50.00th=[ 2671], 60.00th=[ 3032], | 70.00th=[ 3720], 80.00th=[ 4817], 90.00th=[ 8029], 95.00th=[ 10421], | 99.00th=[ 16057], 99.50th=[ 22414], 99.90th=[ 66323], 99.95th=[191890], | 99.99th=[191890] bw ( KiB/s): min= 15, max= 368, per=24.75%, avg=178.71, stdev=84.10, samples=66 iops : min= 3, max= 92, avg=44.62, stdev=21.08, samples=66 write: IOPS=47, BW=191KiB/s (196kB/s)(6348KiB/33226msec) clat (usec): min=7718, max=59886, avg=17009.50, stdev=7010.47 lat (usec): min=7719, max=59887, avg=17010.72, stdev=7010.37 clat percentiles (usec): | 1.00th=[ 8717], 5.00th=[ 9765], 10.00th=[10421], 20.00th=[11469], | 30.00th=[12518], 40.00th=[13698], 50.00th=[15008], 60.00th=[16319], | 70.00th=[18220], 80.00th=[21627], 90.00th=[27395], 95.00th=[31327], | 99.00th=[40633], 99.50th=[42730], 99.90th=[46400], 99.95th=[60031], | 99.99th=[60031] bw ( KiB/s): min= 63, max= 280, per=25.18%, avg=190.35, stdev=64.17, samples=66 iops : min= 15, max= 70, avg=47.53, stdev=16.09, samples=66 lat (msec) : 2=3.97%, 4=31.32%, 10=13.51%, 20=38.48%, 50=12.63% lat (msec) : 100=0.07%, 250=0.03% cpu : usr=0.13%, sys=5.92%, ctx=3524, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18743: Tue Jun 10 21:52:11 2025 read: IOPS=45, BW=182KiB/s (186kB/s)(6028KiB/33154msec) clat (usec): min=1680, max=66199, avg=4200.81, stdev=4216.82 lat (usec): min=1680, max=66200, avg=4201.65, stdev=4216.81 clat percentiles (usec): | 1.00th=[ 1778], 5.00th=[ 1909], 10.00th=[ 2024], 20.00th=[ 2180], | 30.00th=[ 2343], 40.00th=[ 2474], 50.00th=[ 2638], 60.00th=[ 2999], | 70.00th=[ 3752], 80.00th=[ 5145], 90.00th=[ 8979], 95.00th=[11469], | 99.00th=[17957], 99.50th=[27657], 99.90th=[54264], 99.95th=[66323], | 99.99th=[66323] bw ( KiB/s): min= 55, max= 352, per=25.16%, avg=181.64, stdev=71.13, samples=66 iops : min= 13, max= 88, avg=45.36, stdev=17.82, samples=66 write: IOPS=47, BW=189KiB/s (193kB/s)(6260KiB/33154msec) clat (msec): min=8, max=210, avg=17.06, stdev= 8.60 lat (msec): min=8, max=210, avg=17.06, stdev= 8.60 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 12], | 30.00th=[ 13], 40.00th=[ 14], 50.00th=[ 15], 60.00th=[ 17], | 70.00th=[ 18], 80.00th=[ 22], 90.00th=[ 28], 95.00th=[ 32], | 99.00th=[ 41], 99.50th=[ 44], 99.90th=[ 55], 99.95th=[ 211], | 99.99th=[ 211] bw ( KiB/s): min= 71, max= 296, per=24.89%, avg=188.18, stdev=69.12, samples=66 iops : min= 17, max= 74, avg=47.00, stdev=17.31, samples=66 lat (msec) : 2=4.43%, 4=31.35%, 10=13.87%, 20=37.99%, 50=12.21% lat (msec) : 100=0.13%, 250=0.03% cpu : usr=0.13%, sys=5.99%, ctx=3557, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18744: Tue Jun 10 21:52:11 2025 read: IOPS=45, BW=184KiB/s (188kB/s)(6052KiB/32933msec) clat (usec): min=1636, max=54518, avg=4127.98, stdev=4088.11 lat (usec): min=1637, max=54519, avg=4128.80, stdev=4088.09 clat percentiles (usec): | 1.00th=[ 1745], 5.00th=[ 1909], 10.00th=[ 2008], 20.00th=[ 2180], | 30.00th=[ 2311], 40.00th=[ 2442], 50.00th=[ 2638], 60.00th=[ 2999], | 70.00th=[ 3720], 80.00th=[ 4817], 90.00th=[ 8586], 95.00th=[11207], | 99.00th=[17695], 99.50th=[33817], 99.90th=[48497], 99.95th=[54264], | 99.99th=[54264] bw ( KiB/s): min= 48, max= 328, per=25.30%, avg=182.69, stdev=75.37, samples=65 iops : min= 12, max= 82, avg=45.62, stdev=18.86, samples=65 write: IOPS=47, BW=189KiB/s (194kB/s)(6236KiB/32933msec) clat (msec): min=7, max=211, avg=17.04, stdev= 8.81 lat (msec): min=7, max=211, avg=17.05, stdev= 8.81 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 10], 10.00th=[ 11], 20.00th=[ 12], | 30.00th=[ 13], 40.00th=[ 14], 50.00th=[ 15], 60.00th=[ 17], | 70.00th=[ 18], 80.00th=[ 22], 90.00th=[ 28], 95.00th=[ 33], | 99.00th=[ 43], 99.50th=[ 45], 99.90th=[ 62], 99.95th=[ 211], | 99.99th=[ 211] bw ( KiB/s): min= 64, max= 288, per=24.98%, avg=188.85, stdev=68.99, samples=65 iops : min= 16, max= 72, avg=47.15, stdev=17.27, samples=65 lat (msec) : 2=4.72%, 4=31.77%, 10=14.13%, 20=37.30%, 50=11.91% lat (msec) : 100=0.13%, 250=0.03% cpu : usr=0.11%, sys=6.00%, ctx=3555, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=723KiB/s (740kB/s), 179KiB/s-184KiB/s (183kB/s-188kB/s), io=23.4MiB (24.6MB), run=32933-33226msec WRITE: bw=757KiB/s (775kB/s), 189KiB/s-191KiB/s (193kB/s-196kB/s), io=24.6MiB (25.7MB), run=32933-33226msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=18788: Tue Jun 10 21:52:14 2025 read: IOPS=131, BW=2107KiB/s (2157kB/s)(5920KiB/2810msec) clat (usec): min=60, max=24533, avg=3719.21, stdev=3318.72 lat (usec): min=61, max=24534, avg=3719.88, stdev=3318.75 clat percentiles (usec): | 1.00th=[ 67], 5.00th=[ 74], 10.00th=[ 86], 20.00th=[ 120], | 30.00th=[ 2540], 40.00th=[ 2966], 50.00th=[ 3425], 60.00th=[ 3916], | 70.00th=[ 4686], 80.00th=[ 5735], 90.00th=[ 7308], 95.00th=[ 8586], | 99.00th=[17695], 99.50th=[21627], 99.90th=[24511], 99.95th=[24511], | 99.99th=[24511] bw ( KiB/s): min= 1856, max= 2336, per=24.99%, avg=2092.00, stdev=216.04, samples=5 iops : min= 116, max= 146, avg=130.60, stdev=13.63, samples=5 write: IOPS=141, BW=2266KiB/s (2321kB/s)(6368KiB/2810msec) clat (usec): min=691, max=35592, avg=3571.36, stdev=3930.09 lat (usec): min=692, max=35593, avg=3572.36, stdev=3930.11 clat percentiles (usec): | 1.00th=[ 725], 5.00th=[ 791], 10.00th=[ 824], 20.00th=[ 947], | 30.00th=[ 1123], 40.00th=[ 1352], 50.00th=[ 1942], 60.00th=[ 3195], | 70.00th=[ 4359], 80.00th=[ 5800], 90.00th=[ 8356], 95.00th=[10159], | 99.00th=[22152], 99.50th=[31589], 99.90th=[35390], 99.95th=[35390], | 99.99th=[35390] bw ( KiB/s): min= 1952, max= 2688, per=26.07%, avg=2296.60, stdev=342.25, samples=5 iops : min= 122, max= 168, avg=143.40, stdev=21.54, samples=5 lat (usec) : 100=6.90%, 250=5.21%, 500=0.39%, 750=1.17%, 1000=11.20% lat (msec) : 2=14.19%, 4=25.52%, 10=31.25%, 20=3.39%, 50=0.78% cpu : usr=0.25%, sys=28.94%, ctx=1460, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18789: Tue Jun 10 21:52:14 2025 read: IOPS=130, BW=2093KiB/s (2144kB/s)(5744KiB/2744msec) clat (usec): min=58, max=15682, avg=3477.09, stdev=2826.40 lat (usec): min=58, max=15683, avg=3479.17, stdev=2826.45 clat percentiles (usec): | 1.00th=[ 66], 5.00th=[ 74], 10.00th=[ 83], 20.00th=[ 120], | 30.00th=[ 2278], 40.00th=[ 2802], 50.00th=[ 3326], 60.00th=[ 3916], | 70.00th=[ 4686], 80.00th=[ 5866], 90.00th=[ 7242], 95.00th=[ 8586], | 99.00th=[10945], 99.50th=[12780], 99.90th=[15664], 99.95th=[15664], | 99.99th=[15664] bw ( KiB/s): min= 1692, max= 2304, per=25.22%, avg=2111.20, stdev=262.55, samples=5 iops : min= 105, max= 144, avg=131.80, stdev=16.71, samples=5 write: IOPS=149, BW=2385KiB/s (2442kB/s)(6544KiB/2744msec) clat (usec): min=635, max=19937, avg=3567.15, stdev=3371.13 lat (usec): min=635, max=19938, avg=3569.03, stdev=3371.00 clat percentiles (usec): | 1.00th=[ 693], 5.00th=[ 783], 10.00th=[ 848], 20.00th=[ 988], | 30.00th=[ 1139], 40.00th=[ 1385], 50.00th=[ 2114], 60.00th=[ 3163], | 70.00th=[ 4228], 80.00th=[ 6259], 90.00th=[ 8717], 95.00th=[10290], | 99.00th=[13698], 99.50th=[16188], 99.90th=[20055], 99.95th=[20055], | 99.99th=[20055] bw ( KiB/s): min= 1948, max= 2944, per=26.80%, avg=2360.80, stdev=373.87, samples=5 iops : min= 121, max= 184, avg=147.40, stdev=23.58, samples=5 lat (usec) : 100=7.16%, 250=5.86%, 750=1.82%, 1000=9.64% lat (msec) : 2=14.84%, 4=25.39%, 10=30.73%, 20=4.56% cpu : usr=0.18%, sys=29.35%, ctx=1428, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18790: Tue Jun 10 21:52:14 2025 read: IOPS=136, BW=2187KiB/s (2239kB/s)(6256KiB/2861msec) clat (usec): min=59, max=22916, avg=3608.34, stdev=3492.59 lat (usec): min=59, max=22917, avg=3609.01, stdev=3492.61 clat percentiles (usec): | 1.00th=[ 65], 5.00th=[ 73], 10.00th=[ 84], 20.00th=[ 110], | 30.00th=[ 137], 40.00th=[ 2802], 50.00th=[ 3261], 60.00th=[ 3916], | 70.00th=[ 4948], 80.00th=[ 6063], 90.00th=[ 7701], 95.00th=[ 9372], | 99.00th=[17957], 99.50th=[22152], 99.90th=[22938], 99.95th=[22938], | 99.99th=[22938] bw ( KiB/s): min= 1596, max= 2400, per=25.07%, avg=2098.40, stdev=309.06, samples=5 iops : min= 99, max= 150, avg=131.00, stdev=19.62, samples=5 write: IOPS=131, BW=2108KiB/s (2159kB/s)(6032KiB/2861msec) clat (usec): min=641, max=30136, avg=3781.84, stdev=3830.91 lat (usec): min=642, max=30137, avg=3783.09, stdev=3830.92 clat percentiles (usec): | 1.00th=[ 701], 5.00th=[ 775], 10.00th=[ 848], 20.00th=[ 971], | 30.00th=[ 1156], 40.00th=[ 1483], 50.00th=[ 2114], 60.00th=[ 3425], | 70.00th=[ 4490], 80.00th=[ 5997], 90.00th=[ 8717], 95.00th=[11994], | 99.00th=[17433], 99.50th=[18744], 99.90th=[30016], 99.95th=[30016], | 99.99th=[30016] bw ( KiB/s): min= 1980, max= 2432, per=23.68%, avg=2085.60, stdev=194.39, samples=5 iops : min= 123, max= 152, avg=130.20, stdev=12.26, samples=5 lat (usec) : 100=8.72%, 250=7.68%, 500=0.13%, 750=1.95%, 1000=9.11% lat (msec) : 2=13.02%, 4=22.27%, 10=31.90%, 20=4.82%, 50=0.39% cpu : usr=0.17%, sys=28.08%, ctx=1497, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18791: Tue Jun 10 21:52:14 2025 read: IOPS=138, BW=2222KiB/s (2275kB/s)(6032KiB/2715msec) clat (usec): min=60, max=18246, avg=3413.84, stdev=3090.72 lat (usec): min=60, max=18247, avg=3414.50, stdev=3090.74 clat percentiles (usec): | 1.00th=[ 62], 5.00th=[ 72], 10.00th=[ 80], 20.00th=[ 116], | 30.00th=[ 151], 40.00th=[ 2737], 50.00th=[ 3163], 60.00th=[ 4015], | 70.00th=[ 4817], 80.00th=[ 5669], 90.00th=[ 7373], 95.00th=[ 8717], | 99.00th=[13435], 99.50th=[15795], 99.90th=[18220], 99.95th=[18220], | 99.99th=[18220] bw ( KiB/s): min= 1884, max= 2656, per=26.37%, avg=2207.20, stdev=291.76, samples=5 iops : min= 117, max= 166, avg=137.80, stdev=18.44, samples=5 write: IOPS=144, BW=2304KiB/s (2360kB/s)(6256KiB/2715msec) clat (usec): min=674, max=32131, avg=3543.76, stdev=3716.67 lat (usec): min=675, max=32132, avg=3544.75, stdev=3716.68 clat percentiles (usec): | 1.00th=[ 701], 5.00th=[ 758], 10.00th=[ 799], 20.00th=[ 889], | 30.00th=[ 1074], 40.00th=[ 1319], 50.00th=[ 1729], 60.00th=[ 2966], | 70.00th=[ 4146], 80.00th=[ 6259], 90.00th=[ 8356], 95.00th=[10290], | 99.00th=[18744], 99.50th=[20317], 99.90th=[32113], 99.95th=[32113], | 99.99th=[32113] bw ( KiB/s): min= 1980, max= 2720, per=26.37%, avg=2322.40, stdev=296.70, samples=5 iops : min= 123, max= 170, avg=145.00, stdev=18.76, samples=5 lat (usec) : 100=7.42%, 250=8.07%, 500=0.65%, 750=2.47%, 1000=11.20% lat (msec) : 2=13.80%, 4=19.92%, 10=31.90%, 20=4.30%, 50=0.26% cpu : usr=0.18%, sys=28.22%, ctx=1401, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8372KiB/s (8573kB/s), 2093KiB/s-2222KiB/s (2144kB/s-2275kB/s), io=23.4MiB (24.5MB), run=2715-2861msec WRITE: bw=8808KiB/s (9020kB/s), 2108KiB/s-2385KiB/s (2159kB/s-2442kB/s), io=24.6MiB (25.8MB), run=2715-2861msec rand-rw: (groupid=0, jobs=1): err= 0: pid=18792: Tue Jun 10 21:52:20 2025 read: IOPS=44, BW=713KiB/s (730kB/s)(5920KiB/8300msec) clat (usec): min=1920, max=24227, avg=4028.43, stdev=3181.51 lat (usec): min=1920, max=24227, avg=4029.22, stdev=3181.52 clat percentiles (usec): | 1.00th=[ 2008], 5.00th=[ 2147], 10.00th=[ 2278], 20.00th=[ 2409], | 30.00th=[ 2507], 40.00th=[ 2638], 50.00th=[ 2802], 60.00th=[ 3032], | 70.00th=[ 3326], 80.00th=[ 4948], 90.00th=[ 7308], 95.00th=[ 9896], | 99.00th=[20317], 99.50th=[23200], 99.90th=[24249], 99.95th=[24249], | 99.99th=[24249] bw ( KiB/s): min= 352, max= 1248, per=24.34%, avg=697.81, stdev=252.28, samples=16 iops : min= 22, max= 78, avg=43.50, stdev=15.79, samples=16 write: IOPS=47, BW=767KiB/s (786kB/s)(6368KiB/8300msec) clat (usec): min=8759, max=49480, avg=16822.93, stdev=6149.77 lat (usec): min=8760, max=49481, avg=16824.52, stdev=6149.74 clat percentiles (usec): | 1.00th=[ 8979], 5.00th=[10028], 10.00th=[10814], 20.00th=[12125], | 30.00th=[13042], 40.00th=[14484], 50.00th=[15664], 60.00th=[16909], | 70.00th=[17957], 80.00th=[20055], 90.00th=[25035], 95.00th=[29492], | 99.00th=[40109], 99.50th=[41157], 99.90th=[49546], 99.95th=[49546], | 99.99th=[49546] bw ( KiB/s): min= 415, max= 1056, per=25.06%, avg=755.81, stdev=219.22, samples=16 iops : min= 25, max= 66, avg=47.13, stdev=13.77, samples=16 lat (msec) : 2=0.52%, 4=36.72%, 10=10.94%, 20=40.89%, 50=10.94% cpu : usr=0.08%, sys=7.15%, ctx=903, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18793: Tue Jun 10 21:52:20 2025 read: IOPS=42, BW=688KiB/s (704kB/s)(5744KiB/8354msec) clat (usec): min=1899, max=24997, avg=4061.03, stdev=3461.68 lat (usec): min=1899, max=24998, avg=4061.87, stdev=3461.66 clat percentiles (usec): | 1.00th=[ 1958], 5.00th=[ 2114], 10.00th=[ 2180], 20.00th=[ 2343], | 30.00th=[ 2507], 40.00th=[ 2638], 50.00th=[ 2769], 60.00th=[ 2999], | 70.00th=[ 3294], 80.00th=[ 4555], 90.00th=[ 7701], 95.00th=[11600], | 99.00th=[19792], 99.50th=[20317], 99.90th=[25035], 99.95th=[25035], | 99.99th=[25035] bw ( KiB/s): min= 256, max= 1184, per=24.06%, avg=689.88, stdev=274.19, samples=16 iops : min= 16, max= 74, avg=43.06, stdev=17.10, samples=16 write: IOPS=48, BW=783KiB/s (802kB/s)(6544KiB/8354msec) clat (usec): min=8111, max=53697, avg=16566.17, stdev=5821.68 lat (usec): min=8113, max=53698, avg=16567.80, stdev=5821.69 clat percentiles (usec): | 1.00th=[ 8848], 5.00th=[10028], 10.00th=[10683], 20.00th=[11731], | 30.00th=[12911], 40.00th=[14222], 50.00th=[15533], 60.00th=[16712], | 70.00th=[18220], 80.00th=[20317], 90.00th=[23987], 95.00th=[28181], | 99.00th=[33817], 99.50th=[40109], 99.90th=[53740], 99.95th=[53740], | 99.99th=[53740] bw ( KiB/s): min= 448, max= 1024, per=25.59%, avg=771.87, stdev=225.24, samples=16 iops : min= 28, max= 64, avg=48.19, stdev=14.04, samples=16 lat (msec) : 2=0.52%, 4=35.68%, 10=9.90%, 20=42.45%, 50=11.33% lat (msec) : 100=0.13% cpu : usr=0.12%, sys=6.88%, ctx=900, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18794: Tue Jun 10 21:52:20 2025 read: IOPS=49, BW=790KiB/s (808kB/s)(6256KiB/7924msec) clat (usec): min=1944, max=24550, avg=3833.32, stdev=3046.35 lat (usec): min=1945, max=24551, avg=3834.17, stdev=3046.33 clat percentiles (usec): | 1.00th=[ 2008], 5.00th=[ 2180], 10.00th=[ 2245], 20.00th=[ 2442], | 30.00th=[ 2540], 40.00th=[ 2638], 50.00th=[ 2769], 60.00th=[ 2933], | 70.00th=[ 3261], 80.00th=[ 4490], 90.00th=[ 6128], 95.00th=[ 8848], | 99.00th=[21365], 99.50th=[23200], 99.90th=[24511], 99.95th=[24511], | 99.99th=[24511] bw ( KiB/s): min= 352, max= 1504, per=27.90%, avg=799.80, stdev=372.01, samples=15 iops : min= 22, max= 94, avg=49.87, stdev=23.29, samples=15 write: IOPS=47, BW=761KiB/s (780kB/s)(6032KiB/7924msec) clat (usec): min=8544, max=68936, avg=16943.94, stdev=6815.45 lat (usec): min=8545, max=68939, avg=16945.41, stdev=6815.52 clat percentiles (usec): | 1.00th=[ 9241], 5.00th=[10290], 10.00th=[10814], 20.00th=[11863], | 30.00th=[12911], 40.00th=[13960], 50.00th=[15270], 60.00th=[16581], | 70.00th=[18220], 80.00th=[20317], 90.00th=[25560], 95.00th=[31065], | 99.00th=[40109], 99.50th=[44303], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 447, max= 1024, per=24.75%, avg=746.47, stdev=212.88, samples=15 iops : min= 27, max= 64, avg=46.53, stdev=13.38, samples=15 lat (msec) : 2=0.26%, 4=38.54%, 10=11.72%, 20=38.28%, 50=11.07% lat (msec) : 100=0.13% cpu : usr=0.08%, sys=7.52%, ctx=911, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18795: Tue Jun 10 21:52:20 2025 read: IOPS=45, BW=727KiB/s (744kB/s)(6032KiB/8302msec) clat (usec): min=1875, max=24728, avg=3989.97, stdev=3385.89 lat (usec): min=1876, max=24729, avg=3990.73, stdev=3385.89 clat percentiles (usec): | 1.00th=[ 1991], 5.00th=[ 2114], 10.00th=[ 2212], 20.00th=[ 2343], | 30.00th=[ 2540], 40.00th=[ 2638], 50.00th=[ 2769], 60.00th=[ 2999], | 70.00th=[ 3359], 80.00th=[ 4359], 90.00th=[ 7046], 95.00th=[10814], | 99.00th=[20317], 99.50th=[23462], 99.90th=[24773], 99.95th=[24773], | 99.99th=[24773] bw ( KiB/s): min= 288, max= 1248, per=24.83%, avg=712.00, stdev=247.18, samples=16 iops : min= 18, max= 78, avg=44.50, stdev=15.45, samples=16 write: IOPS=47, BW=754KiB/s (772kB/s)(6256KiB/8302msec) clat (usec): min=8617, max=46831, avg=17034.27, stdev=6049.71 lat (usec): min=8619, max=46833, avg=17035.76, stdev=6049.71 clat percentiles (usec): | 1.00th=[ 9110], 5.00th=[10290], 10.00th=[11207], 20.00th=[11994], | 30.00th=[13173], 40.00th=[14484], 50.00th=[15795], 60.00th=[16909], | 70.00th=[18482], 80.00th=[20841], 90.00th=[24511], 95.00th=[29492], | 99.00th=[37487], 99.50th=[45351], 99.90th=[46924], 99.95th=[46924], | 99.99th=[46924] bw ( KiB/s): min= 384, max= 1024, per=24.67%, avg=744.00, stdev=226.12, samples=16 iops : min= 24, max= 64, avg=46.50, stdev=14.13, samples=16 lat (msec) : 2=0.52%, 4=38.28%, 10=9.51%, 20=39.71%, 50=11.98% cpu : usr=0.11%, sys=7.04%, ctx=904, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2867KiB/s (2936kB/s), 688KiB/s-790KiB/s (704kB/s-808kB/s), io=23.4MiB (24.5MB), run=7924-8354msec WRITE: bw=3017KiB/s (3089kB/s), 754KiB/s-783KiB/s (772kB/s-802kB/s), io=24.6MiB (25.8MB), run=7924-8354msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=18811: Tue Jun 10 21:52:22 2025 read: IOPS=6, BW=6952KiB/s (7119kB/s)(7168KiB/1031msec) clat (usec): min=469, max=56906, avg=25281.80, stdev=19767.27 lat (usec): min=470, max=56906, avg=25282.81, stdev=19767.26 clat percentiles (usec): | 1.00th=[ 469], 5.00th=[ 469], 10.00th=[ 469], 20.00th=[ 553], | 30.00th=[25560], 40.00th=[25560], 50.00th=[28181], 60.00th=[32375], | 70.00th=[32375], 80.00th=[32900], 90.00th=[56886], 95.00th=[56886], | 99.00th=[56886], 99.50th=[56886], 99.90th=[56886], 99.95th=[56886], | 99.99th=[56886] bw ( KiB/s): min= 6071, max= 6144, per=44.70%, avg=6107.50, stdev=51.62, samples=2 iops : min= 5, max= 6, avg= 5.50, stdev= 0.71, samples=2 write: IOPS=4, BW=4966KiB/s (5085kB/s)(5120KiB/1031msec) clat (msec): min=60, max=453, avg=167.36, stdev=163.42 lat (msec): min=61, max=453, avg=167.43, stdev=163.40 clat percentiles (msec): | 1.00th=[ 62], 5.00th=[ 62], 10.00th=[ 62], 20.00th=[ 62], | 30.00th=[ 67], 40.00th=[ 67], 50.00th=[ 116], 60.00th=[ 116], | 70.00th=[ 140], 80.00th=[ 140], 90.00th=[ 456], 95.00th=[ 456], | 99.00th=[ 456], 99.50th=[ 456], 99.90th=[ 456], 99.95th=[ 456], | 99.99th=[ 456] bw ( KiB/s): min= 8094, max= 8094, per=42.32%, avg=8094.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 lat (usec) : 500=8.33%, 750=8.33% lat (msec) : 50=33.33%, 100=25.00%, 250=16.67%, 500=8.33% cpu : usr=0.00%, sys=16.41%, ctx=1173, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18813: Tue Jun 10 21:52:22 2025 read: IOPS=1, BW=1900KiB/s (1945kB/s)(2048KiB/1078msec) clat (usec): min=462, max=508914, avg=254688.52, stdev=359529.28 lat (usec): min=463, max=508915, avg=254689.42, stdev=359529.53 clat percentiles (usec): | 1.00th=[ 465], 5.00th=[ 465], 10.00th=[ 465], 20.00th=[ 465], | 30.00th=[ 465], 40.00th=[ 465], 50.00th=[ 465], 60.00th=[509608], | 70.00th=[509608], 80.00th=[509608], 90.00th=[509608], 95.00th=[509608], | 99.00th=[509608], 99.50th=[509608], 99.90th=[509608], 99.95th=[509608], | 99.99th=[509608] bw ( KiB/s): min= 4031, max= 4031, per=29.51%, avg=4031.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 write: IOPS=9, BW=9499KiB/s (9727kB/s)(10.0MiB/1078msec) clat (msec): min=25, max=155, avg=55.76, stdev=40.62 lat (msec): min=25, max=155, avg=55.82, stdev=40.62 clat percentiles (msec): | 1.00th=[ 26], 5.00th=[ 26], 10.00th=[ 26], 20.00th=[ 28], | 30.00th=[ 28], 40.00th=[ 29], 50.00th=[ 42], 60.00th=[ 52], | 70.00th=[ 55], 80.00th=[ 55], 90.00th=[ 94], 95.00th=[ 157], | 99.00th=[ 157], 99.50th=[ 157], 99.90th=[ 157], 99.95th=[ 157], | 99.99th=[ 157] bw ( KiB/s): min=16125, max=16125, per=84.30%, avg=16125.00, stdev= 0.00, samples=1 iops : min= 15, max= 15, avg=15.00, stdev= 0.00, samples=1 lat (usec) : 500=8.33% lat (msec) : 50=41.67%, 100=33.33%, 250=8.33%, 750=8.33% cpu : usr=0.00%, sys=25.26%, ctx=381, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18815: Tue Jun 10 21:52:22 2025 read: IOPS=7, BW=7245KiB/s (7419kB/s)(6144KiB/848msec) clat (msec): min=27, max=309, avg=79.33, stdev=113.04 lat (msec): min=27, max=309, avg=79.33, stdev=113.05 clat percentiles (msec): | 1.00th=[ 28], 5.00th=[ 28], 10.00th=[ 28], 20.00th=[ 29], | 30.00th=[ 29], 40.00th=[ 32], 50.00th=[ 32], 60.00th=[ 33], | 70.00th=[ 47], 80.00th=[ 47], 90.00th=[ 309], 95.00th=[ 309], | 99.00th=[ 309], 99.50th=[ 309], 99.90th=[ 309], 99.95th=[ 309], | 99.99th=[ 309] bw ( KiB/s): min= 4087, max= 4087, per=29.92%, avg=4087.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 write: IOPS=7, BW=7245KiB/s (7419kB/s)(6144KiB/848msec) clat (msec): min=23, max=106, avg=60.26, stdev=32.99 lat (msec): min=23, max=106, avg=60.32, stdev=32.98 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 24], 10.00th=[ 24], 20.00th=[ 35], | 30.00th=[ 35], 40.00th=[ 43], 50.00th=[ 43], 60.00th=[ 63], | 70.00th=[ 92], 80.00th=[ 92], 90.00th=[ 107], 95.00th=[ 107], | 99.00th=[ 107], 99.50th=[ 107], 99.90th=[ 107], 99.95th=[ 107], | 99.99th=[ 107] bw ( KiB/s): min= 8175, max= 8175, per=42.74%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 lat (msec) : 50=66.67%, 100=16.67%, 250=8.33%, 500=8.33% cpu : usr=0.00%, sys=25.38%, ctx=840, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18817: Tue Jun 10 21:52:22 2025 read: IOPS=3, BW=3416KiB/s (3498kB/s)(5120KiB/1499msec) clat (usec): min=509, max=401581, avg=108354.11, stdev=165302.98 lat (usec): min=510, max=401582, avg=108355.21, stdev=165302.75 clat percentiles (usec): | 1.00th=[ 510], 5.00th=[ 510], 10.00th=[ 510], 20.00th=[ 510], | 30.00th=[ 35914], 40.00th=[ 35914], 50.00th=[ 46400], 60.00th=[ 46400], | 70.00th=[ 57410], 80.00th=[ 57410], 90.00th=[400557], 95.00th=[400557], | 99.00th=[400557], 99.50th=[400557], 99.90th=[400557], 99.95th=[400557], | 99.99th=[400557] bw ( KiB/s): min= 8159, max= 8159, per=59.72%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=4, BW=4782KiB/s (4897kB/s)(7168KiB/1499msec) clat (msec): min=35, max=529, avg=126.76, stdev=178.50 lat (msec): min=35, max=530, avg=126.82, stdev=178.51 clat percentiles (msec): | 1.00th=[ 36], 5.00th=[ 36], 10.00th=[ 36], 20.00th=[ 44], | 30.00th=[ 58], 40.00th=[ 58], 50.00th=[ 68], 60.00th=[ 74], | 70.00th=[ 74], 80.00th=[ 80], 90.00th=[ 531], 95.00th=[ 531], | 99.00th=[ 531], 99.50th=[ 531], 99.90th=[ 531], 99.95th=[ 531], | 99.99th=[ 531] bw ( KiB/s): min=10199, max=10199, per=53.32%, avg=10199.00, stdev= 0.00, samples=1 iops : min= 9, max= 9, avg= 9.00, stdev= 0.00, samples=1 lat (usec) : 750=8.33% lat (msec) : 50=33.33%, 100=41.67%, 500=8.33%, 750=8.33% cpu : usr=0.00%, sys=17.02%, ctx=842, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=13.3MiB/s (13.0MB/s), 1900KiB/s-7245KiB/s (1945kB/s-7419kB/s), io=20.0MiB (20.0MB), run=848-1499msec WRITE: bw=18.7MiB/s (19.6MB/s), 4782KiB/s-9499KiB/s (4897kB/s-9727kB/s), io=28.0MiB (29.4MB), run=848-1499msec rand-rw: (groupid=0, jobs=1): err= 0: pid=18812: Tue Jun 10 21:52:22 2025 read: IOPS=6, BW=6238KiB/s (6388kB/s)(7168KiB/1149msec) clat (msec): min=19, max=126, avg=58.47, stdev=41.11 lat (msec): min=19, max=126, avg=58.47, stdev=41.11 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 20], 10.00th=[ 20], 20.00th=[ 28], | 30.00th=[ 31], 40.00th=[ 31], 50.00th=[ 36], 60.00th=[ 72], | 70.00th=[ 72], 80.00th=[ 99], 90.00th=[ 127], 95.00th=[ 127], | 99.00th=[ 127], 99.50th=[ 127], 99.90th=[ 127], 99.95th=[ 127], | 99.99th=[ 127] bw ( KiB/s): min= 4087, max= 6131, per=38.92%, avg=5109.00, stdev=1445.33, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=4, BW=4456KiB/s (4563kB/s)(5120KiB/1149msec) clat (msec): min=24, max=409, avg=136.61, stdev=154.81 lat (msec): min=24, max=409, avg=136.66, stdev=154.81 clat percentiles (msec): | 1.00th=[ 25], 5.00th=[ 25], 10.00th=[ 25], 20.00th=[ 25], | 30.00th=[ 75], 40.00th=[ 75], 50.00th=[ 87], 60.00th=[ 87], | 70.00th=[ 88], 80.00th=[ 88], 90.00th=[ 409], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 6131, max= 6131, per=33.36%, avg=6131.00, stdev= 0.00, samples=1 iops : min= 5, max= 5, avg= 5.00, stdev= 0.00, samples=1 lat (msec) : 20=8.33%, 50=33.33%, 100=41.67%, 250=8.33%, 500=8.33% cpu : usr=0.00%, sys=9.23%, ctx=58, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18814: Tue Jun 10 21:52:22 2025 read: IOPS=1, BW=1315KiB/s (1347kB/s)(2048KiB/1557msec) clat (msec): min=49, max=534, avg=291.81, stdev=343.33 lat (msec): min=49, max=534, avg=291.81, stdev=343.33 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 50], 20.00th=[ 50], | 30.00th=[ 50], 40.00th=[ 50], 50.00th=[ 50], 60.00th=[ 535], | 70.00th=[ 535], 80.00th=[ 535], 90.00th=[ 535], 95.00th=[ 535], | 99.00th=[ 535], 99.50th=[ 535], 99.90th=[ 535], 99.95th=[ 535], | 99.99th=[ 535] bw ( KiB/s): min= 4096, max= 4096, per=31.20%, avg=4096.00, stdev= 0.00, samples=1 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=1 write: IOPS=6, BW=6577KiB/s (6735kB/s)(10.0MiB/1557msec) clat (msec): min=21, max=454, avg=96.63, stdev=128.07 lat (msec): min=21, max=454, avg=96.70, stdev=128.08 clat percentiles (msec): | 1.00th=[ 22], 5.00th=[ 22], 10.00th=[ 22], 20.00th=[ 34], | 30.00th=[ 39], 40.00th=[ 46], 50.00th=[ 56], 60.00th=[ 64], | 70.00th=[ 80], 80.00th=[ 80], 90.00th=[ 104], 95.00th=[ 456], | 99.00th=[ 456], 99.50th=[ 456], 99.90th=[ 456], 99.95th=[ 456], | 99.99th=[ 456] bw ( KiB/s): min= 4063, max=12288, per=44.48%, avg=8175.50, stdev=5815.95, samples=2 iops : min= 3, max= 12, avg= 7.50, stdev= 6.36, samples=2 lat (msec) : 50=41.67%, 100=33.33%, 250=8.33%, 500=8.33%, 750=8.33% cpu : usr=0.00%, sys=7.71%, ctx=47, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18816: Tue Jun 10 21:52:22 2025 read: IOPS=5, BW=5684KiB/s (5820kB/s)(6144KiB/1081msec) clat (usec): min=15282, max=89324, avg=47772.12, stdev=24358.36 lat (usec): min=15283, max=89326, avg=47773.41, stdev=24358.43 clat percentiles (usec): | 1.00th=[15270], 5.00th=[15270], 10.00th=[15270], 20.00th=[36963], | 30.00th=[36963], 40.00th=[42730], 50.00th=[42730], 60.00th=[47973], | 70.00th=[54264], 80.00th=[54264], 90.00th=[89654], 95.00th=[89654], | 99.00th=[89654], 99.50th=[89654], 99.90th=[89654], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 8192, max= 8192, per=62.40%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=5, BW=5684KiB/s (5820kB/s)(6144KiB/1081msec) clat (msec): min=36, max=490, avg=123.52, stdev=180.26 lat (msec): min=36, max=490, avg=123.57, stdev=180.27 clat percentiles (msec): | 1.00th=[ 37], 5.00th=[ 37], 10.00th=[ 37], 20.00th=[ 44], | 30.00th=[ 44], 40.00th=[ 45], 50.00th=[ 45], 60.00th=[ 62], | 70.00th=[ 65], 80.00th=[ 65], 90.00th=[ 493], 95.00th=[ 493], | 99.00th=[ 493], 99.50th=[ 493], 99.90th=[ 493], 99.95th=[ 493], | 99.99th=[ 493] bw ( KiB/s): min=10240, max=10240, per=55.72%, avg=10240.00, stdev= 0.00, samples=1 iops : min= 10, max= 10, avg=10.00, stdev= 0.00, samples=1 lat (msec) : 20=8.33%, 50=50.00%, 100=33.33%, 500=8.33% cpu : usr=0.00%, sys=8.89%, ctx=60, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18818: Tue Jun 10 21:52:22 2025 read: IOPS=3, BW=3282KiB/s (3361kB/s)(5120KiB/1560msec) clat (msec): min=19, max=120, avg=45.75, stdev=43.32 lat (msec): min=19, max=120, avg=45.75, stdev=43.32 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 20], 10.00th=[ 20], 20.00th=[ 20], | 30.00th=[ 21], 40.00th=[ 21], 50.00th=[ 23], 60.00th=[ 23], | 70.00th=[ 46], 80.00th=[ 46], 90.00th=[ 122], 95.00th=[ 122], | 99.00th=[ 122], 99.50th=[ 122], 99.90th=[ 122], 99.95th=[ 122], | 99.99th=[ 122] bw ( KiB/s): min= 8192, max= 8192, per=62.40%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=4, BW=4595KiB/s (4705kB/s)(7168KiB/1560msec) clat (msec): min=33, max=524, avg=184.63, stdev=210.18 lat (msec): min=33, max=524, avg=184.69, stdev=210.19 clat percentiles (msec): | 1.00th=[ 34], 5.00th=[ 34], 10.00th=[ 34], 20.00th=[ 36], | 30.00th=[ 66], 40.00th=[ 66], 50.00th=[ 67], 60.00th=[ 114], | 70.00th=[ 114], 80.00th=[ 451], 90.00th=[ 527], 95.00th=[ 527], | 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527], | 99.99th=[ 527] bw ( KiB/s): min= 4096, max= 8192, per=33.43%, avg=6144.00, stdev=2896.31, samples=2 iops : min= 4, max= 8, avg= 6.00, stdev= 2.83, samples=2 lat (msec) : 20=8.33%, 50=41.67%, 100=16.67%, 250=16.67%, 500=8.33% lat (msec) : 750=8.33% cpu : usr=0.06%, sys=6.67%, ctx=58, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=12.8MiB/s (13.4MB/s), 1315KiB/s-6238KiB/s (1347kB/s-6388kB/s), io=20.0MiB (20.0MB), run=1081-1560msec WRITE: bw=17.9MiB/s (18.8MB/s), 4456KiB/s-6577KiB/s (4563kB/s-6735kB/s), io=28.0MiB (29.4MB), run=1081-1560msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=18825: Tue Jun 10 21:52:24 2025 read: IOPS=10, BW=40.1MiB/s (42.1MB/s)(12.0MiB/299msec) clat (msec): min=9, max=232, avg=90.92, stdev=123.34 lat (msec): min=9, max=232, avg=90.92, stdev=123.34 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 10], 10.00th=[ 10], 20.00th=[ 10], | 30.00th=[ 10], 40.00th=[ 31], 50.00th=[ 31], 60.00th=[ 31], | 70.00th=[ 234], 80.00th=[ 234], 90.00th=[ 234], 95.00th=[ 234], | 99.00th=[ 234], 99.50th=[ 234], 99.90th=[ 234], 99.95th=[ 234], | 99.99th=[ 234] lat (msec) : 10=33.33%, 50=33.33%, 250=33.33% cpu : usr=0.00%, sys=23.15%, ctx=499, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18826: Tue Jun 10 21:52:24 2025 read: IOPS=2, BW=8208KiB/s (8405kB/s)(4096KiB/499msec) clat (nsec): min=135648k, max=135648k, avg=135648112.00, stdev= 0.00 lat (nsec): min=135650k, max=135650k, avg=135650453.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 136], | 30.00th=[ 136], 40.00th=[ 136], 50.00th=[ 136], 60.00th=[ 136], | 70.00th=[ 136], 80.00th=[ 136], 90.00th=[ 136], 95.00th=[ 136], | 99.00th=[ 136], 99.50th=[ 136], 99.90th=[ 136], 99.95th=[ 136], | 99.99th=[ 136] write: IOPS=4, BW=16.0MiB/s (16.8MB/s)(8192KiB/499msec) clat (msec): min=152, max=201, avg=177.07, stdev=34.95 lat (msec): min=152, max=202, avg=177.38, stdev=34.87 clat percentiles (msec): | 1.00th=[ 153], 5.00th=[ 153], 10.00th=[ 153], 20.00th=[ 153], | 30.00th=[ 153], 40.00th=[ 153], 50.00th=[ 153], 60.00th=[ 203], | 70.00th=[ 203], 80.00th=[ 203], 90.00th=[ 203], 95.00th=[ 203], | 99.00th=[ 203], 99.50th=[ 203], 99.90th=[ 203], 99.95th=[ 203], | 99.99th=[ 203] lat (msec) : 250=100.00% cpu : usr=0.00%, sys=34.14%, ctx=287, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18827: Tue Jun 10 21:52:24 2025 write: IOPS=4, BW=19.9MiB/s (20.9MB/s)(12.0MiB/603msec) clat (msec): min=115, max=298, avg=196.24, stdev=93.49 lat (msec): min=116, max=299, avg=196.51, stdev=93.53 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 116], | 30.00th=[ 116], 40.00th=[ 174], 50.00th=[ 174], 60.00th=[ 174], | 70.00th=[ 300], 80.00th=[ 300], 90.00th=[ 300], 95.00th=[ 300], | 99.00th=[ 300], 99.50th=[ 300], 99.90th=[ 300], 99.95th=[ 300], | 99.99th=[ 300] bw ( KiB/s): min=16384, max=16384, per=30.15%, avg=16384.00, stdev= 0.00, samples=1 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, 500=33.33% cpu : usr=0.00%, sys=43.19%, ctx=129, majf=0, minf=29 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18828: Tue Jun 10 21:52:24 2025 write: IOPS=5, BW=20.5MiB/s (21.5MB/s)(12.0MiB/585msec) clat (msec): min=98, max=319, avg=182.49, stdev=119.82 lat (msec): min=98, max=319, avg=182.75, stdev=119.84 clat percentiles (msec): | 1.00th=[ 99], 5.00th=[ 99], 10.00th=[ 99], 20.00th=[ 99], | 30.00th=[ 99], 40.00th=[ 130], 50.00th=[ 130], 60.00th=[ 130], | 70.00th=[ 321], 80.00th=[ 321], 90.00th=[ 321], 95.00th=[ 321], | 99.00th=[ 321], 99.50th=[ 321], 99.90th=[ 321], 99.95th=[ 321], | 99.99th=[ 321] bw ( KiB/s): min=16384, max=16384, per=30.15%, avg=16384.00, stdev= 0.00, samples=1 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=1 lat (msec) : 100=33.33%, 250=33.33%, 500=33.33% cpu : usr=0.00%, sys=46.23%, ctx=83, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=32.1MiB/s (33.6MB/s), 8208KiB/s-40.1MiB/s (8405kB/s-42.1MB/s), io=16.0MiB (16.8MB), run=299-499msec WRITE: bw=53.1MiB/s (55.6MB/s), 16.0MiB/s-20.5MiB/s (16.8MB/s-21.5MB/s), io=32.0MiB (33.6MB), run=499-603msec rand-rw: (groupid=0, jobs=1): err= 0: pid=18829: Tue Jun 10 21:52:24 2025 read: IOPS=4, BW=16.6MiB/s (17.5MB/s)(12.0MiB/721msec) clat (msec): min=86, max=287, avg=210.50, stdev=108.48 lat (msec): min=86, max=287, avg=210.51, stdev=108.48 clat percentiles (msec): | 1.00th=[ 87], 5.00th=[ 87], 10.00th=[ 87], 20.00th=[ 87], | 30.00th=[ 87], 40.00th=[ 257], 50.00th=[ 257], 60.00th=[ 257], | 70.00th=[ 288], 80.00th=[ 288], 90.00th=[ 288], 95.00th=[ 288], | 99.00th=[ 288], 99.50th=[ 288], 99.90th=[ 288], 99.95th=[ 288], | 99.99th=[ 288] bw ( KiB/s): min= 8192, max= 8192, per=36.05%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 100=33.33%, 500=66.67% cpu : usr=0.00%, sys=13.04%, ctx=66, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18830: Tue Jun 10 21:52:24 2025 read: IOPS=1, BW=5902KiB/s (6044kB/s)(4096KiB/694msec) clat (nsec): min=337726k, max=337726k, avg=337726292.00, stdev= 0.00 lat (nsec): min=337728k, max=337728k, avg=337727610.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 338], 5.00th=[ 338], 10.00th=[ 338], 20.00th=[ 338], | 30.00th=[ 338], 40.00th=[ 338], 50.00th=[ 338], 60.00th=[ 338], | 70.00th=[ 338], 80.00th=[ 338], 90.00th=[ 338], 95.00th=[ 338], | 99.00th=[ 338], 99.50th=[ 338], 99.90th=[ 338], 99.95th=[ 338], | 99.99th=[ 338] bw ( KiB/s): min= 8192, max= 8192, per=36.05%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=2, BW=11.5MiB/s (12.1MB/s)(8192KiB/694msec) clat (msec): min=88, max=252, avg=170.41, stdev=115.93 lat (msec): min=88, max=252, avg=170.67, stdev=115.92 clat percentiles (msec): | 1.00th=[ 89], 5.00th=[ 89], 10.00th=[ 89], 20.00th=[ 89], | 30.00th=[ 89], 40.00th=[ 89], 50.00th=[ 89], 60.00th=[ 253], | 70.00th=[ 253], 80.00th=[ 253], 90.00th=[ 253], 95.00th=[ 253], | 99.00th=[ 253], 99.50th=[ 253], 99.90th=[ 253], 99.95th=[ 253], | 99.99th=[ 253] lat (msec) : 100=33.33%, 500=66.67% cpu : usr=0.00%, sys=14.00%, ctx=56, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18831: Tue Jun 10 21:52:24 2025 write: IOPS=4, BW=16.9MiB/s (17.7MB/s)(12.0MiB/712msec) clat (msec): min=169, max=303, avg=224.98, stdev=70.08 lat (msec): min=169, max=304, avg=225.26, stdev=70.07 clat percentiles (msec): | 1.00th=[ 169], 5.00th=[ 169], 10.00th=[ 169], 20.00th=[ 169], | 30.00th=[ 169], 40.00th=[ 203], 50.00th=[ 203], 60.00th=[ 203], | 70.00th=[ 305], 80.00th=[ 305], 90.00th=[ 305], 95.00th=[ 305], | 99.00th=[ 305], 99.50th=[ 305], 99.90th=[ 305], 99.95th=[ 305], | 99.99th=[ 305] bw ( KiB/s): min= 8175, max= 8175, per=17.76%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, 500=33.33% cpu : usr=0.14%, sys=13.92%, ctx=41, majf=0, minf=29 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=18832: Tue Jun 10 21:52:24 2025 write: IOPS=4, BW=18.0MiB/s (18.9MB/s)(12.0MiB/665msec) clat (msec): min=110, max=308, avg=212.29, stdev=99.35 lat (msec): min=110, max=309, avg=212.58, stdev=99.37 clat percentiles (msec): | 1.00th=[ 111], 5.00th=[ 111], 10.00th=[ 111], 20.00th=[ 111], | 30.00th=[ 111], 40.00th=[ 218], 50.00th=[ 218], 60.00th=[ 218], | 70.00th=[ 309], 80.00th=[ 309], 90.00th=[ 309], 95.00th=[ 309], | 99.00th=[ 309], 99.50th=[ 309], 99.90th=[ 309], 99.95th=[ 309], | 99.99th=[ 309] bw ( KiB/s): min= 8192, max= 8192, per=17.80%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67%, 500=33.33% cpu : usr=0.45%, sys=15.51%, ctx=115, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=22.2MiB/s (23.3MB/s), 5902KiB/s-16.6MiB/s (6044kB/s-17.5MB/s), io=16.0MiB (16.8MB), run=694-721msec WRITE: bw=44.9MiB/s (47.1MB/s), 11.5MiB/s-18.0MiB/s (12.1MB/s-18.9MB/s), io=32.0MiB (33.6MB), run=665-712msec