== sanity test 398b: DIO and buffer IO race ============== 10:31:24 (1746801084) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB) copied, 1.18315 s, 42.5 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=26912: Fri May 9 10:31:38 2025 read: IOPS=136, BW=546KiB/s (559kB/s)(5992KiB/10969msec) clat (usec): min=52, max=160179, avg=4009.86, stdev=4568.02 lat (usec): min=53, max=160181, avg=4010.47, stdev=4568.05 clat percentiles (usec): | 1.00th=[ 67], 5.00th=[ 1614], 10.00th=[ 1975], 20.00th=[ 2311], | 30.00th=[ 2671], 40.00th=[ 3097], 50.00th=[ 3556], 60.00th=[ 4047], | 70.00th=[ 4621], 80.00th=[ 5342], 90.00th=[ 6456], 95.00th=[ 7504], | 99.00th=[ 11731], 99.50th=[ 13173], 99.90th=[ 25822], 99.95th=[160433], | 99.99th=[160433] bw ( KiB/s): min= 280, max= 632, per=24.62%, avg=533.10, stdev=76.43, samples=21 iops : min= 70, max= 158, avg=133.19, stdev=19.12, samples=21 write: IOPS=143, BW=574KiB/s (588kB/s)(6296KiB/10969msec) clat (usec): min=395, max=19980, avg=3094.04, stdev=2770.80 lat (usec): min=395, max=19980, avg=3094.77, stdev=2770.77 clat percentiles (usec): | 1.00th=[ 482], 5.00th=[ 510], 10.00th=[ 537], 20.00th=[ 578], | 30.00th=[ 734], 40.00th=[ 1860], 50.00th=[ 2507], 60.00th=[ 3032], | 70.00th=[ 3982], 80.00th=[ 5145], 90.00th=[ 7046], 95.00th=[ 8455], | 99.00th=[11994], 99.50th=[13960], 99.90th=[16057], 99.95th=[20055], | 99.99th=[20055] bw ( KiB/s): min= 360, max= 680, per=24.74%, avg=560.86, stdev=74.26, samples=21 iops : min= 90, max= 170, avg=140.14, stdev=18.57, samples=21 lat (usec) : 100=2.02%, 250=0.20%, 500=1.50%, 750=14.29%, 1000=2.28% lat (msec) : 2=6.54%, 4=37.66%, 10=33.50%, 20=1.95%, 50=0.03% lat (msec) : 250=0.03% cpu : usr=0.24%, sys=22.37%, ctx=5752, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26914: Fri May 9 10:31:38 2025 read: IOPS=135, BW=543KiB/s (556kB/s)(5940KiB/10943msec) clat (usec): min=52, max=19840, avg=3986.74, stdev=2234.57 lat (usec): min=52, max=19841, avg=3988.52, stdev=2233.47 clat percentiles (usec): | 1.00th=[ 67], 5.00th=[ 141], 10.00th=[ 1958], 20.00th=[ 2343], | 30.00th=[ 2671], 40.00th=[ 3130], 50.00th=[ 3621], 60.00th=[ 4146], | 70.00th=[ 4752], 80.00th=[ 5407], 90.00th=[ 6652], 95.00th=[ 7832], | 99.00th=[11994], 99.50th=[13173], 99.90th=[18744], 99.95th=[19792], | 99.99th=[19792] bw ( KiB/s): min= 327, max= 640, per=24.75%, avg=535.81, stdev=68.44, samples=21 iops : min= 81, max= 160, avg=133.86, stdev=17.21, samples=21 write: IOPS=145, BW=580KiB/s (594kB/s)(6348KiB/10943msec) clat (usec): min=411, max=158774, avg=3109.39, stdev=4663.04 lat (usec): min=412, max=158775, avg=3110.15, stdev=4663.06 clat percentiles (usec): | 1.00th=[ 486], 5.00th=[ 515], 10.00th=[ 537], 20.00th=[ 594], | 30.00th=[ 807], 40.00th=[ 1926], 50.00th=[ 2474], 60.00th=[ 3064], | 70.00th=[ 3851], 80.00th=[ 4948], 90.00th=[ 6652], 95.00th=[ 7963], | 99.00th=[ 11469], 99.50th=[ 12256], 99.90th=[ 16909], 99.95th=[158335], | 99.99th=[158335] bw ( KiB/s): min= 359, max= 702, per=24.99%, avg=566.62, stdev=70.64, samples=21 iops : min= 89, max= 175, avg=141.57, stdev=17.70, samples=21 lat (usec) : 100=2.25%, 250=0.26%, 500=1.30%, 750=13.51%, 1000=2.38% lat (msec) : 2=6.54%, 4=38.38%, 10=33.85%, 20=1.50%, 250=0.03% cpu : usr=0.19%, sys=22.77%, ctx=5764, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26916: Fri May 9 10:31:38 2025 read: IOPS=135, BW=544KiB/s (557kB/s)(6028KiB/11087msec) clat (usec): min=57, max=20654, avg=4085.16, stdev=2333.16 lat (usec): min=57, max=20654, avg=4086.37, stdev=2333.45 clat percentiles (usec): | 1.00th=[ 67], 5.00th=[ 1565], 10.00th=[ 1942], 20.00th=[ 2311], | 30.00th=[ 2671], 40.00th=[ 3097], 50.00th=[ 3654], 60.00th=[ 4228], | 70.00th=[ 4817], 80.00th=[ 5669], 90.00th=[ 6915], 95.00th=[ 8029], | 99.00th=[12387], 99.50th=[14222], 99.90th=[17957], 99.95th=[20579], | 99.99th=[20579] bw ( KiB/s): min= 256, max= 744, per=25.00%, avg=541.27, stdev=93.41, samples=22 iops : min= 64, max= 186, avg=135.27, stdev=23.34, samples=22 write: IOPS=141, BW=565KiB/s (578kB/s)(6260KiB/11087msec) clat (usec): min=426, max=152878, avg=3084.69, stdev=4631.47 lat (usec): min=426, max=152879, avg=3085.59, stdev=4631.69 clat percentiles (usec): | 1.00th=[ 482], 5.00th=[ 510], 10.00th=[ 537], 20.00th=[ 570], | 30.00th=[ 660], 40.00th=[ 1467], 50.00th=[ 2474], 60.00th=[ 3097], | 70.00th=[ 3916], 80.00th=[ 5145], 90.00th=[ 6652], 95.00th=[ 7963], | 99.00th=[ 11469], 99.50th=[ 13042], 99.90th=[ 22152], 99.95th=[152044], | 99.99th=[152044] bw ( KiB/s): min= 320, max= 736, per=24.73%, avg=560.59, stdev=93.99, samples=22 iops : min= 80, max= 184, avg=140.09, stdev=23.54, samples=22 lat (usec) : 100=2.02%, 250=0.23%, 500=1.46%, 750=15.62%, 1000=1.43% lat (msec) : 2=6.71%, 4=35.97%, 10=34.64%, 20=1.79%, 50=0.10% lat (msec) : 250=0.03% cpu : usr=0.18%, sys=22.18%, ctx=5578, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26917: Fri May 9 10:31:38 2025 read: IOPS=138, BW=555KiB/s (568kB/s)(6052KiB/10908msec) clat (usec): min=55, max=18449, avg=3952.71, stdev=2134.00 lat (usec): min=55, max=18449, avg=3953.94, stdev=2134.60 clat percentiles (usec): | 1.00th=[ 69], 5.00th=[ 1729], 10.00th=[ 1975], 20.00th=[ 2343], | 30.00th=[ 2671], 40.00th=[ 3064], 50.00th=[ 3523], 60.00th=[ 4080], | 70.00th=[ 4686], 80.00th=[ 5342], 90.00th=[ 6587], 95.00th=[ 7701], | 99.00th=[10683], 99.50th=[13960], 99.90th=[16057], 99.95th=[18482], | 99.99th=[18482] bw ( KiB/s): min= 248, max= 672, per=25.52%, avg=552.57, stdev=88.23, samples=21 iops : min= 62, max= 168, avg=138.10, stdev=22.06, samples=21 write: IOPS=142, BW=572KiB/s (585kB/s)(6236KiB/10908msec) clat (usec): min=425, max=152591, avg=3093.42, stdev=4602.36 lat (usec): min=425, max=152593, avg=3094.18, stdev=4602.45 clat percentiles (usec): | 1.00th=[ 486], 5.00th=[ 515], 10.00th=[ 537], 20.00th=[ 586], | 30.00th=[ 734], 40.00th=[ 1811], 50.00th=[ 2474], 60.00th=[ 2999], | 70.00th=[ 3818], 80.00th=[ 5014], 90.00th=[ 6718], 95.00th=[ 8356], | 99.00th=[ 11076], 99.50th=[ 12518], 99.90th=[ 16909], 99.95th=[152044], | 99.99th=[152044] bw ( KiB/s): min= 304, max= 720, per=25.18%, avg=570.86, stdev=94.99, samples=21 iops : min= 76, max= 180, avg=142.67, stdev=23.75, samples=21 lat (usec) : 100=1.56%, 250=0.10%, 500=1.20%, 750=14.29%, 1000=2.80% lat (msec) : 2=6.45%, 4=38.83%, 10=33.11%, 20=1.63%, 250=0.03% cpu : usr=0.17%, sys=22.47%, ctx=5675, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2166KiB/s (2218kB/s), 543KiB/s-555KiB/s (556kB/s-568kB/s), io=23.4MiB (24.6MB), run=10908-11087msec WRITE: bw=2268KiB/s (2322kB/s), 565KiB/s-580KiB/s (578kB/s-594kB/s), io=24.6MiB (25.7MB), run=10908-11087msec rand-rw: (groupid=0, jobs=1): err= 0: pid=26910: Fri May 9 10:31:45 2025 read: IOPS=85, BW=344KiB/s (352kB/s)(5992KiB/17421msec) clat (usec): min=1465, max=162539, avg=4718.80, stdev=4855.96 lat (usec): min=1466, max=162540, avg=4719.67, stdev=4855.95 clat percentiles (usec): | 1.00th=[ 1778], 5.00th=[ 2024], 10.00th=[ 2245], 20.00th=[ 2704], | 30.00th=[ 3294], 40.00th=[ 3785], 50.00th=[ 3982], 60.00th=[ 4178], | 70.00th=[ 4555], 80.00th=[ 5669], 90.00th=[ 8225], 95.00th=[ 10028], | 99.00th=[ 15270], 99.50th=[ 16581], 99.90th=[ 23462], 99.95th=[162530], | 99.99th=[162530] bw ( KiB/s): min= 152, max= 592, per=24.91%, avg=343.21, stdev=70.48, samples=34 iops : min= 38, max= 148, avg=85.74, stdev=17.63, samples=34 write: IOPS=90, BW=361KiB/s (370kB/s)(6296KiB/17421msec) clat (usec): min=2246, max=24454, avg=6517.39, stdev=2834.45 lat (usec): min=2247, max=24455, avg=6518.58, stdev=2834.30 clat percentiles (usec): | 1.00th=[ 2573], 5.00th=[ 3097], 10.00th=[ 3458], 20.00th=[ 4146], | 30.00th=[ 5145], 40.00th=[ 5800], 50.00th=[ 6259], 60.00th=[ 6652], | 70.00th=[ 6980], 80.00th=[ 7963], 90.00th=[ 9896], 95.00th=[11600], | 99.00th=[17171], 99.50th=[20317], 99.90th=[23987], 99.95th=[24511], | 99.99th=[24511] bw ( KiB/s): min= 168, max= 656, per=24.97%, avg=360.38, stdev=73.50, samples=34 iops : min= 42, max= 164, avg=90.03, stdev=18.37, samples=34 lat (msec) : 2=2.15%, 4=32.06%, 10=58.46%, 20=6.93%, 50=0.36% lat (msec) : 250=0.03% cpu : usr=0.26%, sys=12.61%, ctx=3595, majf=0, minf=35 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26911: Fri May 9 10:31:45 2025 read: IOPS=85, BW=343KiB/s (351kB/s)(5940KiB/17335msec) clat (usec): min=1575, max=161848, avg=4727.80, stdev=4891.31 lat (usec): min=1576, max=161850, avg=4728.64, stdev=4891.32 clat percentiles (usec): | 1.00th=[ 1778], 5.00th=[ 2008], 10.00th=[ 2245], 20.00th=[ 2704], | 30.00th=[ 3195], 40.00th=[ 3687], 50.00th=[ 3949], 60.00th=[ 4178], | 70.00th=[ 4686], 80.00th=[ 5735], 90.00th=[ 8225], 95.00th=[ 10552], | 99.00th=[ 15008], 99.50th=[ 16581], 99.90th=[ 23987], 99.95th=[162530], | 99.99th=[162530] bw ( KiB/s): min= 168, max= 592, per=24.97%, avg=344.15, stdev=70.98, samples=34 iops : min= 42, max= 148, avg=85.97, stdev=17.73, samples=34 write: IOPS=91, BW=366KiB/s (375kB/s)(6348KiB/17335msec) clat (usec): min=2244, max=23688, avg=6440.27, stdev=2627.36 lat (usec): min=2244, max=23688, avg=6441.29, stdev=2627.45 clat percentiles (usec): | 1.00th=[ 2606], 5.00th=[ 3097], 10.00th=[ 3425], 20.00th=[ 4146], | 30.00th=[ 5080], 40.00th=[ 5800], 50.00th=[ 6259], 60.00th=[ 6652], | 70.00th=[ 7046], 80.00th=[ 7963], 90.00th=[ 9634], 95.00th=[11076], | 99.00th=[15664], 99.50th=[18220], 99.90th=[22938], 99.95th=[23725], | 99.99th=[23725] bw ( KiB/s): min= 224, max= 648, per=25.24%, avg=364.15, stdev=66.68, samples=34 iops : min= 56, max= 162, avg=90.97, stdev=16.65, samples=34 lat (msec) : 2=2.41%, 4=32.45%, 10=57.94%, 20=6.97%, 50=0.20% lat (msec) : 250=0.03% cpu : usr=0.21%, sys=12.86%, ctx=3604, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26913: Fri May 9 10:31:45 2025 read: IOPS=87, BW=349KiB/s (358kB/s)(6028KiB/17252msec) clat (usec): min=1526, max=23435, avg=4560.31, stdev=2606.34 lat (usec): min=1527, max=23435, avg=4561.39, stdev=2606.33 clat percentiles (usec): | 1.00th=[ 1745], 5.00th=[ 2024], 10.00th=[ 2212], 20.00th=[ 2671], | 30.00th=[ 3228], 40.00th=[ 3687], 50.00th=[ 3949], 60.00th=[ 4146], | 70.00th=[ 4490], 80.00th=[ 5669], 90.00th=[ 8225], 95.00th=[10159], | 99.00th=[14484], 99.50th=[15664], 99.90th=[19530], 99.95th=[23462], | 99.99th=[23462] bw ( KiB/s): min= 144, max= 600, per=25.39%, avg=349.82, stdev=70.60, samples=34 iops : min= 36, max= 150, avg=87.41, stdev=17.69, samples=34 write: IOPS=90, BW=363KiB/s (372kB/s)(6260KiB/17252msec) clat (msec): min=2, max=150, avg= 6.56, stdev= 4.58 lat (msec): min=2, max=150, avg= 6.56, stdev= 4.58 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 6], 50.00th=[ 7], 60.00th=[ 7], | 70.00th=[ 8], 80.00th=[ 8], 90.00th=[ 10], 95.00th=[ 12], | 99.00th=[ 18], 99.50th=[ 21], 99.90th=[ 25], 99.95th=[ 150], | 99.99th=[ 150] bw ( KiB/s): min= 176, max= 672, per=25.07%, avg=361.82, stdev=82.23, samples=34 iops : min= 44, max= 168, avg=90.41, stdev=20.54, samples=34 lat (msec) : 2=2.31%, 4=32.58%, 10=57.91%, 20=6.90%, 50=0.26% lat (msec) : 250=0.03% cpu : usr=0.28%, sys=12.49%, ctx=3596, majf=0, minf=35 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26915: Fri May 9 10:31:45 2025 read: IOPS=86, BW=348KiB/s (356kB/s)(6052KiB/17406msec) clat (usec): min=1547, max=23743, avg=4643.05, stdev=2658.12 lat (usec): min=1548, max=23744, avg=4643.99, stdev=2658.04 clat percentiles (usec): | 1.00th=[ 1778], 5.00th=[ 1991], 10.00th=[ 2245], 20.00th=[ 2704], | 30.00th=[ 3294], 40.00th=[ 3720], 50.00th=[ 3949], 60.00th=[ 4178], | 70.00th=[ 4490], 80.00th=[ 5735], 90.00th=[ 8586], 95.00th=[10290], | 99.00th=[13829], 99.50th=[16188], 99.90th=[21103], 99.95th=[23725], | 99.99th=[23725] bw ( KiB/s): min= 152, max= 616, per=25.18%, avg=346.97, stdev=79.14, samples=34 iops : min= 38, max= 154, avg=86.68, stdev=19.78, samples=34 write: IOPS=89, BW=358KiB/s (367kB/s)(6236KiB/17406msec) clat (msec): min=2, max=165, avg= 6.60, stdev= 4.83 lat (msec): min=2, max=165, avg= 6.60, stdev= 4.83 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 6], 50.00th=[ 7], 60.00th=[ 7], | 70.00th=[ 8], 80.00th=[ 8], 90.00th=[ 10], 95.00th=[ 12], | 99.00th=[ 17], 99.50th=[ 20], 99.90th=[ 25], 99.95th=[ 165], | 99.99th=[ 165] bw ( KiB/s): min= 184, max= 640, per=24.76%, avg=357.32, stdev=70.15, samples=34 iops : min= 46, max= 160, avg=89.26, stdev=17.54, samples=34 lat (msec) : 2=2.51%, 4=31.38%, 10=58.63%, 20=7.19%, 50=0.26% lat (msec) : 250=0.03% cpu : usr=0.22%, sys=12.69%, ctx=3597, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1378KiB/s (1411kB/s), 343KiB/s-349KiB/s (351kB/s-358kB/s), io=23.4MiB (24.6MB), run=17252-17421msec WRITE: bw=1443KiB/s (1478kB/s), 358KiB/s-366KiB/s (367kB/s-375kB/s), io=24.6MiB (25.7MB), run=17252-17421msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=26950: Fri May 9 10:31:49 2025 read: IOPS=116, BW=1863KiB/s (1908kB/s)(5920KiB/3178msec) clat (usec): min=60, max=17615, avg=4428.59, stdev=2734.46 lat (usec): min=60, max=17617, avg=4429.16, stdev=2734.48 clat percentiles (usec): | 1.00th=[ 70], 5.00th=[ 82], 10.00th=[ 2024], 20.00th=[ 2573], | 30.00th=[ 2999], 40.00th=[ 3490], 50.00th=[ 3916], 60.00th=[ 4555], | 70.00th=[ 5145], 80.00th=[ 6063], 90.00th=[ 7504], 95.00th=[ 9503], | 99.00th=[13566], 99.50th=[16450], 99.90th=[17695], 99.95th=[17695], | 99.99th=[17695] bw ( KiB/s): min= 1024, max= 2144, per=24.12%, avg=1818.00, stdev=412.37, samples=6 iops : min= 64, max= 134, avg=113.50, stdev=25.75, samples=6 write: IOPS=125, BW=2004KiB/s (2052kB/s)(6368KiB/3178msec) clat (usec): min=635, max=17250, avg=3660.47, stdev=3092.05 lat (usec): min=635, max=17251, avg=3661.38, stdev=3092.09 clat percentiles (usec): | 1.00th=[ 676], 5.00th=[ 758], 10.00th=[ 783], 20.00th=[ 857], | 30.00th=[ 1156], 40.00th=[ 1991], 50.00th=[ 2704], 60.00th=[ 3654], | 70.00th=[ 4883], 80.00th=[ 6194], 90.00th=[ 8094], 95.00th=[ 9896], | 99.00th=[13435], 99.50th=[16450], 99.90th=[17171], 99.95th=[17171], | 99.99th=[17171] bw ( KiB/s): min= 1184, max= 2432, per=24.95%, avg=1978.00, stdev=438.77, samples=6 iops : min= 74, max= 152, avg=123.50, stdev=27.48, samples=6 lat (usec) : 100=3.52%, 250=0.26%, 500=0.13%, 750=2.34%, 1000=11.98% lat (msec) : 2=7.29%, 4=31.51%, 10=38.80%, 20=4.17% cpu : usr=0.19%, sys=23.70%, ctx=1549, majf=0, minf=35 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26951: Fri May 9 10:31:49 2025 read: IOPS=122, BW=1966KiB/s (2013kB/s)(5744KiB/2922msec) clat (usec): min=58, max=15530, avg=4074.49, stdev=2446.39 lat (usec): min=58, max=15531, avg=4075.14, stdev=2446.35 clat percentiles (usec): | 1.00th=[ 68], 5.00th=[ 78], 10.00th=[ 1958], 20.00th=[ 2442], | 30.00th=[ 2835], 40.00th=[ 3195], 50.00th=[ 3752], 60.00th=[ 4228], | 70.00th=[ 4883], 80.00th=[ 5800], 90.00th=[ 6783], 95.00th=[ 8455], | 99.00th=[13304], 99.50th=[14353], 99.90th=[15533], 99.95th=[15533], | 99.99th=[15533] bw ( KiB/s): min= 1408, max= 2368, per=25.13%, avg=1893.60, stdev=359.15, samples=5 iops : min= 88, max= 148, avg=118.20, stdev=22.52, samples=5 write: IOPS=139, BW=2240KiB/s (2293kB/s)(6544KiB/2922msec) clat (usec): min=667, max=16586, avg=3383.64, stdev=2964.73 lat (usec): min=668, max=16587, avg=3384.50, stdev=2964.72 clat percentiles (usec): | 1.00th=[ 717], 5.00th=[ 775], 10.00th=[ 807], 20.00th=[ 881], | 30.00th=[ 1090], 40.00th=[ 1844], 50.00th=[ 2606], 60.00th=[ 3163], | 70.00th=[ 4047], 80.00th=[ 5604], 90.00th=[ 7635], 95.00th=[ 8848], | 99.00th=[14615], 99.50th=[16319], 99.90th=[16581], 99.95th=[16581], | 99.99th=[16581] bw ( KiB/s): min= 1536, max= 2496, per=26.46%, avg=2098.20, stdev=368.88, samples=5 iops : min= 96, max= 156, avg=131.00, stdev=22.98, samples=5 lat (usec) : 100=3.91%, 250=0.52%, 750=1.17%, 1000=12.50% lat (msec) : 2=8.85%, 4=35.81%, 10=34.77%, 20=2.47% cpu : usr=0.27%, sys=24.85%, ctx=1569, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26952: Fri May 9 10:31:49 2025 read: IOPS=126, BW=2027KiB/s (2075kB/s)(6256KiB/3087msec) clat (usec): min=59, max=18544, avg=4235.53, stdev=2535.21 lat (usec): min=59, max=18545, avg=4236.10, stdev=2535.24 clat percentiles (usec): | 1.00th=[ 67], 5.00th=[ 78], 10.00th=[ 1926], 20.00th=[ 2474], | 30.00th=[ 2966], 40.00th=[ 3458], 50.00th=[ 4015], 60.00th=[ 4621], | 70.00th=[ 5080], 80.00th=[ 5800], 90.00th=[ 7242], 95.00th=[ 8586], | 99.00th=[12780], 99.50th=[14746], 99.90th=[18482], 99.95th=[18482], | 99.99th=[18482] bw ( KiB/s): min= 1440, max= 2592, per=26.88%, avg=2026.00, stdev=377.12, samples=6 iops : min= 90, max= 162, avg=126.50, stdev=23.63, samples=6 write: IOPS=122, BW=1954KiB/s (2001kB/s)(6032KiB/3087msec) clat (usec): min=614, max=16928, avg=3726.89, stdev=3040.97 lat (usec): min=614, max=16929, avg=3727.74, stdev=3040.99 clat percentiles (usec): | 1.00th=[ 685], 5.00th=[ 750], 10.00th=[ 783], 20.00th=[ 881], | 30.00th=[ 1205], 40.00th=[ 2147], 50.00th=[ 2933], 60.00th=[ 3884], | 70.00th=[ 5080], 80.00th=[ 5932], 90.00th=[ 8291], 95.00th=[ 9634], | 99.00th=[13435], 99.50th=[14353], 99.90th=[16909], 99.95th=[16909], | 99.99th=[16909] bw ( KiB/s): min= 1564, max= 2272, per=24.27%, avg=1924.67, stdev=242.71, samples=6 iops : min= 97, max= 142, avg=120.17, stdev=15.39, samples=6 lat (usec) : 100=4.43%, 250=0.65%, 750=2.34%, 1000=10.03% lat (msec) : 2=6.77%, 4=31.25%, 10=41.15%, 20=3.39% cpu : usr=0.19%, sys=24.11%, ctx=1649, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26953: Fri May 9 10:31:49 2025 read: IOPS=121, BW=1950KiB/s (1997kB/s)(6032KiB/3093msec) clat (usec): min=56, max=20068, avg=4414.60, stdev=2727.42 lat (usec): min=56, max=20068, avg=4415.28, stdev=2727.42 clat percentiles (usec): | 1.00th=[ 67], 5.00th=[ 81], 10.00th=[ 2040], 20.00th=[ 2671], | 30.00th=[ 3032], 40.00th=[ 3458], 50.00th=[ 3916], 60.00th=[ 4621], | 70.00th=[ 5276], 80.00th=[ 5866], 90.00th=[ 7570], 95.00th=[ 8979], | 99.00th=[14615], 99.50th=[15664], 99.90th=[20055], 99.95th=[20055], | 99.99th=[20055] bw ( KiB/s): min= 1280, max= 2176, per=25.40%, avg=1913.83, stdev=335.39, samples=6 iops : min= 80, max= 136, avg=119.50, stdev=20.90, samples=6 write: IOPS=126, BW=2023KiB/s (2071kB/s)(6256KiB/3093msec) clat (usec): min=626, max=14928, avg=3423.68, stdev=2793.82 lat (usec): min=627, max=14929, avg=3425.92, stdev=2795.14 clat percentiles (usec): | 1.00th=[ 717], 5.00th=[ 766], 10.00th=[ 799], 20.00th=[ 857], | 30.00th=[ 988], 40.00th=[ 2073], 50.00th=[ 2868], 60.00th=[ 3392], | 70.00th=[ 4293], 80.00th=[ 5735], 90.00th=[ 7635], 95.00th=[ 8848], | 99.00th=[11863], 99.50th=[14484], 99.90th=[14877], 99.95th=[14877], | 99.99th=[14877] bw ( KiB/s): min= 1632, max= 2522, per=25.28%, avg=2004.33, stdev=304.00, samples=6 iops : min= 102, max= 157, avg=125.17, stdev=18.79, samples=6 lat (usec) : 100=3.52%, 250=0.52%, 750=1.30%, 1000=14.58% lat (msec) : 2=5.08%, 4=33.85%, 10=38.28%, 20=2.73%, 50=0.13% cpu : usr=0.10%, sys=23.84%, ctx=1565, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7537KiB/s (7718kB/s), 1863KiB/s-2027KiB/s (1908kB/s-2075kB/s), io=23.4MiB (24.5MB), run=2922-3178msec WRITE: bw=7930KiB/s (8120kB/s), 1954KiB/s-2240KiB/s (2001kB/s-2293kB/s), io=24.6MiB (25.8MB), run=2922-3178msec rand-rw: (groupid=0, jobs=1): err= 0: pid=26946: Fri May 9 10:31:50 2025 read: IOPS=94, BW=1512KiB/s (1548kB/s)(5920KiB/3916msec) clat (usec): min=1791, max=25800, avg=4195.24, stdev=3218.94 lat (usec): min=1791, max=25801, avg=4195.88, stdev=3218.92 clat percentiles (usec): | 1.00th=[ 1860], 5.00th=[ 1942], 10.00th=[ 2008], 20.00th=[ 2212], | 30.00th=[ 2409], 40.00th=[ 2606], 50.00th=[ 2868], 60.00th=[ 3359], | 70.00th=[ 4113], 80.00th=[ 5342], 90.00th=[ 8094], 95.00th=[10945], | 99.00th=[17171], 99.50th=[20317], 99.90th=[25822], 99.95th=[25822], | 99.99th=[25822] bw ( KiB/s): min= 928, max= 2272, per=23.54%, avg=1374.57, stdev=440.20, samples=7 iops : min= 58, max= 142, avg=85.57, stdev=27.61, samples=7 write: IOPS=101, BW=1626KiB/s (1665kB/s)(6368KiB/3916msec) clat (usec): min=2491, max=21982, avg=5906.86, stdev=3350.14 lat (usec): min=2492, max=21983, avg=5907.96, stdev=3350.10 clat percentiles (usec): | 1.00th=[ 2540], 5.00th=[ 2835], 10.00th=[ 3064], 20.00th=[ 3359], | 30.00th=[ 3654], 40.00th=[ 3884], 50.00th=[ 4424], 60.00th=[ 5473], | 70.00th=[ 6652], 80.00th=[ 8717], 90.00th=[10945], 95.00th=[12125], | 99.00th=[17433], 99.50th=[21103], 99.90th=[21890], 99.95th=[21890], | 99.99th=[21890] bw ( KiB/s): min= 1056, max= 2400, per=24.46%, avg=1502.57, stdev=433.83, samples=7 iops : min= 66, max= 150, avg=93.57, stdev=27.17, samples=7 lat (msec) : 2=4.17%, 4=50.78%, 10=34.11%, 20=10.42%, 50=0.52% cpu : usr=0.28%, sys=11.57%, ctx=893, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26947: Fri May 9 10:31:50 2025 read: IOPS=89, BW=1437KiB/s (1471kB/s)(5744KiB/3998msec) clat (usec): min=1713, max=20125, avg=4142.60, stdev=3117.49 lat (usec): min=1713, max=20125, avg=4143.19, stdev=3117.49 clat percentiles (usec): | 1.00th=[ 1844], 5.00th=[ 1958], 10.00th=[ 2057], 20.00th=[ 2212], | 30.00th=[ 2343], 40.00th=[ 2573], 50.00th=[ 2835], 60.00th=[ 3163], | 70.00th=[ 3949], 80.00th=[ 5932], 90.00th=[ 8455], 95.00th=[10945], | 99.00th=[18744], 99.50th=[19530], 99.90th=[20055], 99.95th=[20055], | 99.99th=[20055] bw ( KiB/s): min= 864, max= 2432, per=23.01%, avg=1343.71, stdev=513.91, samples=7 iops : min= 54, max= 152, avg=83.86, stdev=32.23, samples=7 write: IOPS=102, BW=1637KiB/s (1676kB/s)(6544KiB/3998msec) clat (usec): min=2335, max=21910, avg=5948.28, stdev=3474.57 lat (usec): min=2336, max=21911, avg=5949.43, stdev=3474.52 clat percentiles (usec): | 1.00th=[ 2540], 5.00th=[ 2900], 10.00th=[ 3064], 20.00th=[ 3359], | 30.00th=[ 3621], 40.00th=[ 3982], 50.00th=[ 4424], 60.00th=[ 5473], | 70.00th=[ 6849], 80.00th=[ 8455], 90.00th=[10814], 95.00th=[12518], | 99.00th=[18220], 99.50th=[19006], 99.90th=[21890], 99.95th=[21890], | 99.99th=[21890] bw ( KiB/s): min= 1056, max= 2240, per=23.88%, avg=1467.00, stdev=373.83, samples=7 iops : min= 66, max= 140, avg=91.57, stdev=23.37, samples=7 lat (msec) : 2=3.12%, 4=51.30%, 10=36.20%, 20=8.98%, 50=0.39% cpu : usr=0.10%, sys=11.73%, ctx=932, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26948: Fri May 9 10:31:50 2025 read: IOPS=97, BW=1560KiB/s (1597kB/s)(6256KiB/4011msec) clat (usec): min=1729, max=19297, avg=4285.22, stdev=3107.69 lat (usec): min=1730, max=19298, avg=4285.81, stdev=3107.69 clat percentiles (usec): | 1.00th=[ 1827], 5.00th=[ 2008], 10.00th=[ 2089], 20.00th=[ 2278], | 30.00th=[ 2442], 40.00th=[ 2638], 50.00th=[ 2900], 60.00th=[ 3359], | 70.00th=[ 4293], 80.00th=[ 5932], 90.00th=[ 8848], 95.00th=[11076], | 99.00th=[16581], 99.50th=[18482], 99.90th=[19268], 99.95th=[19268], | 99.99th=[19268] bw ( KiB/s): min= 798, max= 2880, per=26.77%, avg=1563.38, stdev=655.82, samples=8 iops : min= 49, max= 180, avg=97.50, stdev=41.20, samples=8 write: IOPS=93, BW=1504KiB/s (1540kB/s)(6032KiB/4011msec) clat (usec): min=2290, max=22104, avg=6119.87, stdev=3788.28 lat (usec): min=2291, max=22106, avg=6121.61, stdev=3788.38 clat percentiles (usec): | 1.00th=[ 2606], 5.00th=[ 2966], 10.00th=[ 3130], 20.00th=[ 3326], | 30.00th=[ 3589], 40.00th=[ 3884], 50.00th=[ 4359], 60.00th=[ 5473], | 70.00th=[ 6783], 80.00th=[ 8717], 90.00th=[11994], 95.00th=[13435], | 99.00th=[18744], 99.50th=[21890], 99.90th=[22152], 99.95th=[22152], | 99.99th=[22152] bw ( KiB/s): min= 1056, max= 2400, per=24.40%, avg=1499.25, stdev=553.71, samples=8 iops : min= 66, max= 150, avg=93.50, stdev=34.72, samples=8 lat (msec) : 2=2.47%, 4=52.47%, 10=34.11%, 20=10.68%, 50=0.26% cpu : usr=0.12%, sys=11.85%, ctx=942, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26949: Fri May 9 10:31:50 2025 read: IOPS=91, BW=1471KiB/s (1506kB/s)(6032KiB/4101msec) clat (usec): min=1743, max=16027, avg=4282.88, stdev=2917.19 lat (usec): min=1744, max=16028, avg=4283.50, stdev=2917.19 clat percentiles (usec): | 1.00th=[ 1860], 5.00th=[ 1991], 10.00th=[ 2114], 20.00th=[ 2278], | 30.00th=[ 2442], 40.00th=[ 2671], 50.00th=[ 2999], 60.00th=[ 3556], | 70.00th=[ 4293], 80.00th=[ 5735], 90.00th=[ 9110], 95.00th=[11207], | 99.00th=[14877], 99.50th=[15270], 99.90th=[16057], 99.95th=[16057], | 99.99th=[16057] bw ( KiB/s): min= 832, max= 2688, per=25.27%, avg=1475.62, stdev=609.06, samples=8 iops : min= 52, max= 168, avg=92.12, stdev=38.10, samples=8 write: IOPS=95, BW=1525KiB/s (1562kB/s)(6256KiB/4101msec) clat (usec): min=2299, max=21403, avg=6204.08, stdev=3824.08 lat (usec): min=2300, max=21404, avg=6205.24, stdev=3824.07 clat percentiles (usec): | 1.00th=[ 2474], 5.00th=[ 2933], 10.00th=[ 3130], 20.00th=[ 3392], | 30.00th=[ 3621], 40.00th=[ 4047], 50.00th=[ 4621], 60.00th=[ 5473], | 70.00th=[ 6915], 80.00th=[ 8717], 90.00th=[11863], 95.00th=[13960], | 99.00th=[21103], 99.50th=[21103], 99.90th=[21365], 99.95th=[21365], | 99.99th=[21365] bw ( KiB/s): min= 960, max= 2528, per=24.86%, avg=1527.62, stdev=595.22, samples=8 iops : min= 60, max= 158, avg=95.38, stdev=37.28, samples=8 lat (msec) : 2=2.47%, 4=50.39%, 10=35.16%, 20=11.46%, 50=0.52% cpu : usr=0.15%, sys=11.76%, ctx=931, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5841KiB/s (5981kB/s), 1437KiB/s-1560KiB/s (1471kB/s-1597kB/s), io=23.4MiB (24.5MB), run=3916-4101msec WRITE: bw=6145KiB/s (6292kB/s), 1504KiB/s-1637KiB/s (1540kB/s-1676kB/s), io=24.6MiB (25.8MB), run=3916-4101msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=26961: Fri May 9 10:31:52 2025 read: IOPS=6, BW=6440KiB/s (6595kB/s)(7168KiB/1113msec) clat (usec): min=279, max=713728, avg=121773.23, stdev=261572.05 lat (usec): min=280, max=713729, avg=121774.32, stdev=261572.35 clat percentiles (usec): | 1.00th=[ 281], 5.00th=[ 281], 10.00th=[ 281], 20.00th=[ 367], | 30.00th=[ 28181], 40.00th=[ 28181], 50.00th=[ 28705], 60.00th=[ 40109], | 70.00th=[ 40109], 80.00th=[ 40633], 90.00th=[717226], 95.00th=[717226], | 99.00th=[717226], 99.50th=[717226], 99.90th=[717226], 99.95th=[717226], | 99.99th=[717226] bw ( KiB/s): min= 4096, max= 6144, per=27.83%, avg=5120.00, stdev=1448.15, samples=2 iops : min= 4, max= 6, avg= 5.00, stdev= 1.41, samples=2 write: IOPS=4, BW=4600KiB/s (4711kB/s)(5120KiB/1113msec) clat (usec): min=29875, max=70709, avg=47812.78, stdev=15942.73 lat (usec): min=29931, max=70727, avg=47850.02, stdev=15926.35 clat percentiles (usec): | 1.00th=[29754], 5.00th=[29754], 10.00th=[29754], 20.00th=[29754], | 30.00th=[40109], 40.00th=[40109], 50.00th=[41681], 60.00th=[41681], | 70.00th=[56361], 80.00th=[56361], 90.00th=[70779], 95.00th=[70779], | 99.00th=[70779], 99.50th=[70779], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 4096, max= 4096, per=15.90%, avg=4096.00, stdev= 0.00, samples=1 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=1 lat (usec) : 500=16.67% lat (msec) : 50=58.33%, 100=16.67%, 750=8.33% cpu : usr=0.00%, sys=25.09%, ctx=927, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26962: Fri May 9 10:31:52 2025 read: IOPS=1, BW=2010KiB/s (2058kB/s)(2048KiB/1019msec) clat (usec): min=25435, max=30558, avg=27996.75, stdev=3622.51 lat (usec): min=25435, max=30560, avg=27998.24, stdev=3623.75 clat percentiles (usec): | 1.00th=[25560], 5.00th=[25560], 10.00th=[25560], 20.00th=[25560], | 30.00th=[25560], 40.00th=[25560], 50.00th=[25560], 60.00th=[30540], | 70.00th=[30540], 80.00th=[30540], 90.00th=[30540], 95.00th=[30540], | 99.00th=[30540], 99.50th=[30540], 99.90th=[30540], 99.95th=[30540], | 99.99th=[30540] bw ( KiB/s): min= 4096, max= 4096, per=22.26%, avg=4096.00, stdev= 0.00, samples=1 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=1 write: IOPS=9, BW=9.81MiB/s (10.3MB/s)(10.0MiB/1019msec) clat (msec): min=23, max=562, avg=95.46, stdev=167.21 lat (msec): min=23, max=562, avg=95.49, stdev=167.20 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 24], 10.00th=[ 24], 20.00th=[ 25], | 30.00th=[ 26], 40.00th=[ 26], 50.00th=[ 29], 60.00th=[ 40], | 70.00th=[ 50], 80.00th=[ 50], 90.00th=[ 131], 95.00th=[ 567], | 99.00th=[ 567], 99.50th=[ 567], 99.90th=[ 567], 99.95th=[ 567], | 99.99th=[ 567] bw ( KiB/s): min= 8192, max=10240, per=35.78%, avg=9216.00, stdev=1448.15, samples=2 iops : min= 8, max= 10, avg= 9.00, stdev= 1.41, samples=2 lat (msec) : 50=83.33%, 250=8.33%, 750=8.33% cpu : usr=0.00%, sys=27.11%, ctx=544, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26963: Fri May 9 10:31:52 2025 read: IOPS=5, BW=6012KiB/s (6156kB/s)(6144KiB/1022msec) clat (usec): min=342, max=32953, avg=8489.31, stdev=13593.04 lat (usec): min=342, max=32954, avg=8490.05, stdev=13593.11 clat percentiles (usec): | 1.00th=[ 343], 5.00th=[ 343], 10.00th=[ 343], 20.00th=[ 383], | 30.00th=[ 383], 40.00th=[ 383], 50.00th=[ 383], 60.00th=[ 453], | 70.00th=[16450], 80.00th=[16450], 90.00th=[32900], 95.00th=[32900], | 99.00th=[32900], 99.50th=[32900], 99.90th=[32900], 99.95th=[32900], | 99.99th=[32900] bw ( KiB/s): min=10240, max=10240, per=55.65%, avg=10240.00, stdev= 0.00, samples=1 iops : min= 10, max= 10, avg=10.00, stdev= 0.00, samples=1 write: IOPS=5, BW=6012KiB/s (6156kB/s)(6144KiB/1022msec) clat (msec): min=18, max=736, avg=156.76, stdev=285.13 lat (msec): min=18, max=736, avg=156.79, stdev=285.14 clat percentiles (msec): | 1.00th=[ 19], 5.00th=[ 19], 10.00th=[ 19], 20.00th=[ 27], | 30.00th=[ 27], 40.00th=[ 31], 50.00th=[ 31], 60.00th=[ 39], | 70.00th=[ 91], 80.00th=[ 91], 90.00th=[ 735], 95.00th=[ 735], | 99.00th=[ 735], 99.50th=[ 735], 99.90th=[ 735], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 4096, max= 6144, per=19.88%, avg=5120.00, stdev=1448.15, samples=2 iops : min= 4, max= 6, avg= 5.00, stdev= 1.41, samples=2 lat (usec) : 500=33.33% lat (msec) : 20=16.67%, 50=33.33%, 100=8.33%, 750=8.33% cpu : usr=0.00%, sys=11.95%, ctx=418, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26964: Fri May 9 10:31:52 2025 read: IOPS=4, BW=4672KiB/s (4784kB/s)(5120KiB/1096msec) clat (usec): min=387, max=28372, avg=15734.84, stdev=14172.64 lat (usec): min=387, max=28373, avg=15735.47, stdev=14172.72 clat percentiles (usec): | 1.00th=[ 388], 5.00th=[ 388], 10.00th=[ 388], 20.00th=[ 388], | 30.00th=[ 453], 40.00th=[ 453], 50.00th=[22152], 60.00th=[22152], | 70.00th=[27132], 80.00th=[27132], 90.00th=[28443], 95.00th=[28443], | 99.00th=[28443], 99.50th=[28443], 99.90th=[28443], 99.95th=[28443], | 99.99th=[28443] bw ( KiB/s): min= 8192, max= 8192, per=44.52%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=6, BW=6540KiB/s (6697kB/s)(7168KiB/1096msec) clat (msec): min=17, max=777, avg=139.62, stdev=281.90 lat (msec): min=17, max=777, avg=139.67, stdev=281.93 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 18], 10.00th=[ 18], 20.00th=[ 20], | 30.00th=[ 21], 40.00th=[ 21], 50.00th=[ 26], 60.00th=[ 53], | 70.00th=[ 53], 80.00th=[ 65], 90.00th=[ 776], 95.00th=[ 776], | 99.00th=[ 776], 99.50th=[ 776], 99.90th=[ 776], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 2048, max= 6144, per=15.90%, avg=4096.00, stdev=2896.31, samples=2 iops : min= 2, max= 6, avg= 4.00, stdev= 2.83, samples=2 lat (usec) : 500=16.67% lat (msec) : 20=16.67%, 50=41.67%, 100=16.67%, 1000=8.33% cpu : usr=0.09%, sys=20.09%, ctx=512, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=17.0MiB/s (18.8MB/s), 2010KiB/s-6440KiB/s (2058kB/s-6595kB/s), io=20.0MiB (20.0MB), run=1019-1113msec WRITE: bw=25.2MiB/s (26.4MB/s), 4600KiB/s-9.81MiB/s (4711kB/s-10.3MB/s), io=28.0MiB (29.4MB), run=1019-1113msec rand-rw: (groupid=0, jobs=1): err= 0: pid=26966: Fri May 9 10:31:52 2025 read: IOPS=5, BW=5561KiB/s (5694kB/s)(7168KiB/1289msec) clat (msec): min=14, max=875, avg=146.92, stdev=321.39 lat (msec): min=14, max=875, avg=146.92, stdev=321.39 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 15], 10.00th=[ 15], 20.00th=[ 16], | 30.00th=[ 17], 40.00th=[ 17], 50.00th=[ 26], 60.00th=[ 35], | 70.00th=[ 35], 80.00th=[ 47], 90.00th=[ 877], 95.00th=[ 877], | 99.00th=[ 877], 99.50th=[ 877], 99.90th=[ 877], 99.95th=[ 877], | 99.99th=[ 877] bw ( KiB/s): min= 2048, max= 4096, per=20.61%, avg=3072.00, stdev=1448.15, samples=2 iops : min= 2, max= 4, avg= 3.00, stdev= 1.41, samples=2 write: IOPS=3, BW=3972KiB/s (4067kB/s)(5120KiB/1289msec) clat (usec): min=19960, max=81782, avg=36217.71, stdev=25839.91 lat (usec): min=20057, max=81858, avg=36290.53, stdev=25840.85 clat percentiles (usec): | 1.00th=[20055], 5.00th=[20055], 10.00th=[20055], 20.00th=[20055], | 30.00th=[21627], 40.00th=[21627], 50.00th=[26870], 60.00th=[26870], | 70.00th=[30802], 80.00th=[30802], 90.00th=[81265], 95.00th=[81265], | 99.00th=[81265], 99.50th=[81265], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] lat (msec) : 20=33.33%, 50=50.00%, 100=8.33%, 1000=8.33% cpu : usr=0.00%, sys=5.75%, ctx=53, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26967: Fri May 9 10:31:52 2025 read: IOPS=1, BW=1640KiB/s (1679kB/s)(2048KiB/1249msec) clat (msec): min=25, max=732, avg=378.94, stdev=499.74 lat (msec): min=25, max=732, avg=378.94, stdev=499.74 clat percentiles (msec): | 1.00th=[ 26], 5.00th=[ 26], 10.00th=[ 26], 20.00th=[ 26], | 30.00th=[ 26], 40.00th=[ 26], 50.00th=[ 26], 60.00th=[ 735], | 70.00th=[ 735], 80.00th=[ 735], 90.00th=[ 735], 95.00th=[ 735], | 99.00th=[ 735], 99.50th=[ 735], 99.90th=[ 735], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 2048, max= 2048, per=13.74%, avg=2048.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=8, BW=8199KiB/s (8395kB/s)(10.0MiB/1249msec) clat (usec): min=22014, max=71517, avg=42927.54, stdev=16166.29 lat (usec): min=22100, max=71550, avg=43009.00, stdev=16154.19 clat percentiles (usec): | 1.00th=[21890], 5.00th=[21890], 10.00th=[21890], 20.00th=[28705], | 30.00th=[31851], 40.00th=[35914], 50.00th=[36963], 60.00th=[36963], | 70.00th=[60556], 80.00th=[60556], 90.00th=[61604], 95.00th=[71828], | 99.00th=[71828], 99.50th=[71828], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min=10240, max=10240, per=49.07%, avg=10240.00, stdev= 0.00, samples=1 iops : min= 10, max= 10, avg=10.00, stdev= 0.00, samples=1 lat (msec) : 50=66.67%, 100=25.00%, 750=8.33% cpu : usr=0.24%, sys=12.02%, ctx=71, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26968: Fri May 9 10:31:52 2025 read: IOPS=4, BW=4679KiB/s (4792kB/s)(6144KiB/1313msec) clat (usec): min=17150, max=51936, avg=36272.01, stdev=15219.07 lat (usec): min=17151, max=51936, avg=36272.82, stdev=15219.12 clat percentiles (usec): | 1.00th=[17171], 5.00th=[17171], 10.00th=[17171], 20.00th=[23987], | 30.00th=[23987], 40.00th=[27657], 50.00th=[27657], 60.00th=[45351], | 70.00th=[51643], 80.00th=[51643], 90.00th=[52167], 95.00th=[52167], | 99.00th=[52167], 99.50th=[52167], 99.90th=[52167], 99.95th=[52167], | 99.99th=[52167] write: IOPS=4, BW=4679KiB/s (4792kB/s)(6144KiB/1313msec) clat (msec): min=15, max=837, avg=169.26, stdev=327.96 lat (msec): min=15, max=837, avg=169.32, stdev=327.97 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 16], 10.00th=[ 16], 20.00th=[ 24], | 30.00th=[ 24], 40.00th=[ 27], 50.00th=[ 27], 60.00th=[ 49], | 70.00th=[ 65], 80.00th=[ 65], 90.00th=[ 835], 95.00th=[ 835], | 99.00th=[ 835], 99.50th=[ 835], 99.90th=[ 835], 99.95th=[ 835], | 99.99th=[ 835] bw ( KiB/s): min= 2048, max= 4096, per=14.72%, avg=3072.00, stdev=1448.15, samples=2 iops : min= 2, max= 4, avg= 3.00, stdev= 1.41, samples=2 lat (msec) : 20=16.67%, 50=50.00%, 100=25.00%, 1000=8.33% cpu : usr=0.00%, sys=5.18%, ctx=52, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26969: Fri May 9 10:31:52 2025 read: IOPS=3, BW=3726KiB/s (3816kB/s)(5120KiB/1374msec) clat (usec): min=19931, max=67017, avg=37131.69, stdev=22070.82 lat (usec): min=19932, max=67018, avg=37132.73, stdev=22070.51 clat percentiles (usec): | 1.00th=[20055], 5.00th=[20055], 10.00th=[20055], 20.00th=[20055], | 30.00th=[21365], 40.00th=[21365], 50.00th=[22676], 60.00th=[22676], | 70.00th=[54789], 80.00th=[54789], 90.00th=[66847], 95.00th=[66847], | 99.00th=[66847], 99.50th=[66847], 99.90th=[66847], 99.95th=[66847], | 99.99th=[66847] write: IOPS=5, BW=5217KiB/s (5342kB/s)(7168KiB/1374msec) clat (msec): min=24, max=859, avg=166.61, stdev=306.97 lat (msec): min=24, max=859, avg=166.70, stdev=306.97 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 24], 10.00th=[ 24], 20.00th=[ 25], | 30.00th=[ 25], 40.00th=[ 25], 50.00th=[ 65], 60.00th=[ 77], | 70.00th=[ 77], 80.00th=[ 93], 90.00th=[ 860], 95.00th=[ 860], | 99.00th=[ 860], 99.50th=[ 860], 99.90th=[ 860], 99.95th=[ 860], | 99.99th=[ 860] bw ( KiB/s): min= 2043, max= 2048, per=9.80%, avg=2045.50, stdev= 3.54, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 20=8.33%, 50=41.67%, 100=41.67%, 1000=8.33% cpu : usr=0.22%, sys=8.52%, ctx=71, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=14.6MiB/s (15.3MB/s), 1640KiB/s-5561KiB/s (1679kB/s-5694kB/s), io=20.0MiB (20.0MB), run=1249-1374msec WRITE: bw=20.4MiB/s (21.4MB/s), 3972KiB/s-8199KiB/s (4067kB/s-8395kB/s), io=28.0MiB (29.4MB), run=1249-1374msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=26975: Fri May 9 10:31:55 2025 read: IOPS=2, BW=9768KiB/s (10.0MB/s)(12.0MiB/1258msec) clat (msec): min=98, max=1012, avg=411.70, stdev=520.73 lat (msec): min=98, max=1012, avg=411.71, stdev=520.73 clat percentiles (msec): | 1.00th=[ 99], 5.00th=[ 99], 10.00th=[ 99], 20.00th=[ 99], | 30.00th=[ 99], 40.00th=[ 125], 50.00th=[ 125], 60.00th=[ 125], | 70.00th=[ 1011], 80.00th=[ 1011], 90.00th=[ 1011], 95.00th=[ 1011], | 99.00th=[ 1011], 99.50th=[ 1011], 99.90th=[ 1011], 99.95th=[ 1011], | 99.99th=[ 1011] bw ( KiB/s): min=16384, max=16384, per=100.00%, avg=16384.00, stdev= 0.00, samples=1 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=1 lat (msec) : 100=33.33%, 250=33.33% cpu : usr=0.08%, sys=22.35%, ctx=2311, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26976: Fri May 9 10:31:55 2025 read: IOPS=0, BW=2325KiB/s (2380kB/s)(4096KiB/1762msec) clat (nsec): min=140774k, max=140774k, avg=140773564.00, stdev= 0.00 lat (nsec): min=140776k, max=140776k, avg=140775840.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 142], | 30.00th=[ 142], 40.00th=[ 142], 50.00th=[ 142], 60.00th=[ 142], | 70.00th=[ 142], 80.00th=[ 142], 90.00th=[ 142], 95.00th=[ 142], | 99.00th=[ 142], 99.50th=[ 142], 99.90th=[ 142], 99.95th=[ 142], | 99.99th=[ 142] bw ( KiB/s): min= 8192, max= 8192, per=88.10%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4649KiB/s (4761kB/s)(8192KiB/1762msec) clat (msec): min=328, max=1284, avg=806.43, stdev=675.87 lat (msec): min=328, max=1284, avg=806.74, stdev=675.74 clat percentiles (msec): | 1.00th=[ 330], 5.00th=[ 330], 10.00th=[ 330], 20.00th=[ 330], | 30.00th=[ 330], 40.00th=[ 330], 50.00th=[ 330], 60.00th=[ 1284], | 70.00th=[ 1284], 80.00th=[ 1284], 90.00th=[ 1284], 95.00th=[ 1284], | 99.00th=[ 1284], 99.50th=[ 1284], 99.90th=[ 1284], 99.95th=[ 1284], | 99.99th=[ 1284] bw ( KiB/s): min= 8192, max= 8192, per=44.05%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33% cpu : usr=0.00%, sys=16.70%, ctx=591, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26977: Fri May 9 10:31:55 2025 write: IOPS=1, BW=7928KiB/s (8118kB/s)(12.0MiB/1550msec) clat (msec): min=87, max=1280, avg=510.66, stdev=667.90 lat (msec): min=87, max=1280, avg=510.82, stdev=667.88 clat percentiles (msec): | 1.00th=[ 88], 5.00th=[ 88], 10.00th=[ 88], 20.00th=[ 88], | 30.00th=[ 88], 40.00th=[ 165], 50.00th=[ 165], 60.00th=[ 165], | 70.00th=[ 1284], 80.00th=[ 1284], 90.00th=[ 1284], 95.00th=[ 1284], | 99.00th=[ 1284], 99.50th=[ 1284], 99.90th=[ 1284], 99.95th=[ 1284], | 99.99th=[ 1284] bw ( KiB/s): min=16384, max=16384, per=88.10%, avg=16384.00, stdev= 0.00, samples=1 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=1 lat (msec) : 100=33.33%, 250=33.33% cpu : usr=0.00%, sys=13.94%, ctx=55, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26978: Fri May 9 10:31:55 2025 write: IOPS=2, BW=8275KiB/s (8473kB/s)(12.0MiB/1485msec) clat (msec): min=122, max=1200, avg=485.44, stdev=619.25 lat (msec): min=122, max=1200, avg=485.63, stdev=619.27 clat percentiles (msec): | 1.00th=[ 123], 5.00th=[ 123], 10.00th=[ 123], 20.00th=[ 123], | 30.00th=[ 123], 40.00th=[ 134], 50.00th=[ 134], 60.00th=[ 134], | 70.00th=[ 1200], 80.00th=[ 1200], 90.00th=[ 1200], 95.00th=[ 1200], | 99.00th=[ 1200], 99.50th=[ 1200], 99.90th=[ 1200], 99.95th=[ 1200], | 99.99th=[ 1200] bw ( KiB/s): min= 8192, max= 8192, per=44.05%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67% cpu : usr=0.00%, sys=21.70%, ctx=55, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=9299KiB/s (9522kB/s), 2325KiB/s-9768KiB/s (2380kB/s-10.0MB/s), io=16.0MiB (16.8MB), run=1258-1762msec WRITE: bw=18.2MiB/s (19.0MB/s), 4649KiB/s-8275KiB/s (4761kB/s-8473kB/s), io=32.0MiB (33.6MB), run=1485-1762msec rand-rw: (groupid=0, jobs=1): err= 0: pid=26980: Fri May 9 10:31:55 2025 read: IOPS=1, BW=7224KiB/s (7397kB/s)(12.0MiB/1701msec) clat (msec): min=226, max=1184, avg=558.70, stdev=542.26 lat (msec): min=226, max=1184, avg=558.70, stdev=542.26 clat percentiles (msec): | 1.00th=[ 226], 5.00th=[ 226], 10.00th=[ 226], 20.00th=[ 226], | 30.00th=[ 226], 40.00th=[ 266], 50.00th=[ 266], 60.00th=[ 266], | 70.00th=[ 1183], 80.00th=[ 1183], 90.00th=[ 1183], 95.00th=[ 1183], | 99.00th=[ 1183], 99.50th=[ 1183], 99.90th=[ 1183], 99.95th=[ 1183], | 99.99th=[ 1183] bw ( KiB/s): min= 8175, max= 8192, per=91.56%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 500=33.33% cpu : usr=0.00%, sys=5.59%, ctx=37, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26981: Fri May 9 10:31:55 2025 read: IOPS=0, BW=2235KiB/s (2288kB/s)(4096KiB/1833msec) clat (nsec): min=196134k, max=196134k, avg=196133757.00, stdev= 0.00 lat (nsec): min=196137k, max=196137k, avg=196136525.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 197], 5.00th=[ 197], 10.00th=[ 197], 20.00th=[ 197], | 30.00th=[ 197], 40.00th=[ 197], 50.00th=[ 197], 60.00th=[ 197], | 70.00th=[ 197], 80.00th=[ 197], 90.00th=[ 197], 95.00th=[ 197], | 99.00th=[ 197], 99.50th=[ 197], 99.90th=[ 197], 99.95th=[ 197], | 99.99th=[ 197] bw ( KiB/s): min= 8192, max= 8192, per=91.65%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4469KiB/s (4576kB/s)(8192KiB/1833msec) clat (msec): min=227, max=1389, avg=808.49, stdev=821.80 lat (msec): min=227, max=1389, avg=808.79, stdev=821.72 clat percentiles (msec): | 1.00th=[ 228], 5.00th=[ 228], 10.00th=[ 228], 20.00th=[ 228], | 30.00th=[ 228], 40.00th=[ 228], 50.00th=[ 228], 60.00th=[ 1385], | 70.00th=[ 1385], 80.00th=[ 1385], 90.00th=[ 1385], 95.00th=[ 1385], | 99.00th=[ 1385], 99.50th=[ 1385], 99.90th=[ 1385], 99.95th=[ 1385], | 99.99th=[ 1385] lat (msec) : 250=66.67% cpu : usr=0.11%, sys=8.41%, ctx=251, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26982: Fri May 9 10:31:55 2025 write: IOPS=1, BW=7157KiB/s (7328kB/s)(12.0MiB/1717msec) clat (msec): min=172, max=1271, avg=561.19, stdev=616.18 lat (msec): min=172, max=1271, avg=561.43, stdev=616.24 clat percentiles (msec): | 1.00th=[ 174], 5.00th=[ 174], 10.00th=[ 174], 20.00th=[ 174], | 30.00th=[ 174], 40.00th=[ 241], 50.00th=[ 241], 60.00th=[ 241], | 70.00th=[ 1267], 80.00th=[ 1267], 90.00th=[ 1267], 95.00th=[ 1267], | 99.00th=[ 1267], 99.50th=[ 1267], 99.90th=[ 1267], 99.95th=[ 1267], | 99.99th=[ 1267] bw ( KiB/s): min= 8192, max= 8192, per=47.23%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=66.67% cpu : usr=0.06%, sys=3.85%, ctx=32, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26983: Fri May 9 10:31:55 2025 write: IOPS=1, BW=6505KiB/s (6661kB/s)(12.0MiB/1889msec) clat (msec): min=61, max=1666, avg=627.14, stdev=901.18 lat (msec): min=61, max=1666, avg=627.36, stdev=901.21 clat percentiles (msec): | 1.00th=[ 62], 5.00th=[ 62], 10.00th=[ 62], 20.00th=[ 62], | 30.00th=[ 62], 40.00th=[ 155], 50.00th=[ 155], 60.00th=[ 155], | 70.00th=[ 1670], 80.00th=[ 1670], 90.00th=[ 1670], 95.00th=[ 1670], | 99.00th=[ 1670], 99.50th=[ 1670], 99.90th=[ 1670], 99.95th=[ 1670], | 99.99th=[ 1670] lat (msec) : 100=33.33%, 250=33.33% cpu : usr=0.05%, sys=5.51%, ctx=95, majf=0, minf=28 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8938KiB/s (9153kB/s), 2235KiB/s-7224KiB/s (2288kB/s-7397kB/s), io=16.0MiB (16.8MB), run=1701-1833msec WRITE: bw=16.9MiB/s (17.8MB/s), 4469KiB/s-7157KiB/s (4576kB/s-7328kB/s), io=32.0MiB (33.6MB), run=1717-1889msec