== sanity test 398b: DIO and buffer IO race ============== 22:25:16 (1746152716) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB) copied, 2.91585 s, 17.3 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=26715: Thu May 1 22:26:16 2025 read: IOPS=28, BW=114KiB/s (117kB/s)(5992KiB/52345msec) clat (usec): min=78, max=140109, avg=19881.46, stdev=13424.51 lat (usec): min=79, max=140110, avg=19883.85, stdev=13423.48 clat percentiles (usec): | 1.00th=[ 95], 5.00th=[ 128], 10.00th=[ 7111], 20.00th=[ 10421], | 30.00th=[ 12518], 40.00th=[ 14746], 50.00th=[ 17171], 60.00th=[ 20579], | 70.00th=[ 24249], 80.00th=[ 28705], 90.00th=[ 34866], 95.00th=[ 42730], | 99.00th=[ 64750], 99.50th=[ 76022], 99.90th=[113771], 99.95th=[139461], | 99.99th=[139461] bw ( KiB/s): min= 23, max= 208, per=25.29%, avg=113.28, stdev=32.16, samples=104 iops : min= 5, max= 52, avg=27.91, stdev= 8.07, samples=104 write: IOPS=30, BW=120KiB/s (123kB/s)(6296KiB/52345msec) clat (usec): min=549, max=592135, avg=14201.54, stdev=21195.48 lat (usec): min=550, max=592137, avg=14203.04, stdev=21195.45 clat percentiles (usec): | 1.00th=[ 693], 5.00th=[ 742], 10.00th=[ 783], 20.00th=[ 881], | 30.00th=[ 1991], 40.00th=[ 4293], 50.00th=[ 9765], 60.00th=[ 13435], | 70.00th=[ 17695], 80.00th=[ 25297], 90.00th=[ 34341], 95.00th=[ 42730], | 99.00th=[ 68682], 99.50th=[ 79168], 99.90th=[120062], 99.95th=[591397], | 99.99th=[591397] bw ( KiB/s): min= 40, max= 224, per=25.58%, avg=119.96, stdev=38.21, samples=103 iops : min= 10, max= 56, avg=29.59, stdev= 9.56, samples=103 lat (usec) : 100=0.75%, 250=2.25%, 500=0.07%, 750=3.32%, 1000=8.66% lat (msec) : 2=3.61%, 4=4.85%, 10=11.26%, 20=31.84%, 50=30.18% lat (msec) : 100=2.99%, 250=0.20%, 750=0.03% cpu : usr=0.11%, sys=8.98%, ctx=4943, majf=0, minf=35 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26716: Thu May 1 22:26:16 2025 read: IOPS=28, BW=113KiB/s (115kB/s)(5940KiB/52746msec) clat (usec): min=81, max=619355, avg=19558.05, stdev=19967.44 lat (usec): min=82, max=619357, avg=19563.33, stdev=19966.75 clat percentiles (usec): | 1.00th=[ 98], 5.00th=[ 127], 10.00th=[ 7504], 20.00th=[ 10421], | 30.00th=[ 12780], 40.00th=[ 14877], 50.00th=[ 16909], 60.00th=[ 19792], | 70.00th=[ 23200], 80.00th=[ 26870], 90.00th=[ 33162], 95.00th=[ 38536], | 99.00th=[ 62653], 99.50th=[ 67634], 99.90th=[196084], 99.95th=[616563], | 99.99th=[616563] bw ( KiB/s): min= 7, max= 216, per=25.15%, avg=112.67, stdev=33.78, samples=104 iops : min= 1, max= 54, avg=27.73, stdev= 8.48, samples=104 write: IOPS=30, BW=120KiB/s (123kB/s)(6348KiB/52746msec) clat (usec): min=619, max=118930, avg=14778.09, stdev=16033.51 lat (usec): min=619, max=118931, avg=14782.17, stdev=16033.51 clat percentiles (usec): | 1.00th=[ 693], 5.00th=[ 750], 10.00th=[ 791], 20.00th=[ 938], | 30.00th=[ 2507], 40.00th=[ 5800], 50.00th=[ 10290], 60.00th=[ 13829], | 70.00th=[ 18744], 80.00th=[ 26870], 90.00th=[ 36439], 95.00th=[ 44827], | 99.00th=[ 71828], 99.50th=[ 84411], 99.90th=[101188], 99.95th=[119014], | 99.99th=[119014] bw ( KiB/s): min= 15, max= 270, per=25.59%, avg=120.03, stdev=39.25, samples=104 iops : min= 3, max= 67, avg=29.58, stdev= 9.88, samples=104 lat (usec) : 100=0.72%, 250=2.21%, 500=0.07%, 750=2.64%, 1000=8.24% lat (msec) : 2=3.19%, 4=4.59%, 10=12.47%, 20=32.45%, 50=30.53% lat (msec) : 100=2.73%, 250=0.13%, 750=0.03% cpu : usr=0.12%, sys=8.92%, ctx=4955, majf=0, minf=35 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26717: Thu May 1 22:26:16 2025 read: IOPS=28, BW=116KiB/s (118kB/s)(6028KiB/52106msec) clat (usec): min=68, max=147962, avg=19254.28, stdev=12937.48 lat (usec): min=68, max=147963, avg=19259.91, stdev=12938.80 clat percentiles (usec): | 1.00th=[ 95], 5.00th=[ 131], 10.00th=[ 7504], 20.00th=[ 10159], | 30.00th=[ 11994], 40.00th=[ 14222], 50.00th=[ 16909], 60.00th=[ 19792], | 70.00th=[ 23462], 80.00th=[ 27395], 90.00th=[ 34341], 95.00th=[ 41157], | 99.00th=[ 64750], 99.50th=[ 72877], 99.90th=[109577], 99.95th=[147850], | 99.99th=[147850] bw ( KiB/s): min= 15, max= 208, per=25.87%, avg=115.91, stdev=32.06, samples=103 iops : min= 3, max= 52, avg=28.61, stdev= 8.06, samples=103 write: IOPS=30, BW=120KiB/s (123kB/s)(6260KiB/52106msec) clat (usec): min=604, max=582030, avg=14549.86, stdev=21589.71 lat (usec): min=605, max=582033, avg=14555.82, stdev=21588.37 clat percentiles (usec): | 1.00th=[ 693], 5.00th=[ 742], 10.00th=[ 791], 20.00th=[ 938], | 30.00th=[ 2311], 40.00th=[ 4817], 50.00th=[ 9765], 60.00th=[ 13304], | 70.00th=[ 17957], 80.00th=[ 24249], 90.00th=[ 35914], 95.00th=[ 44827], | 99.00th=[ 72877], 99.50th=[ 87557], 99.90th=[125305], 99.95th=[583009], | 99.99th=[583009] bw ( KiB/s): min= 23, max= 280, per=25.62%, avg=120.15, stdev=41.88, samples=103 iops : min= 5, max= 70, avg=29.67, stdev=10.47, samples=103 lat (usec) : 100=0.94%, 250=2.18%, 500=0.10%, 750=2.83%, 1000=8.04% lat (msec) : 2=3.65%, 4=5.18%, 10=12.43%, 20=32.00%, 50=29.62% lat (msec) : 100=2.73%, 250=0.26%, 750=0.03% cpu : usr=0.16%, sys=8.93%, ctx=4841, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26718: Thu May 1 22:26:16 2025 read: IOPS=28, BW=113KiB/s (116kB/s)(6052KiB/53590msec) clat (usec): min=82, max=128727, avg=19706.89, stdev=12929.78 lat (usec): min=82, max=128728, avg=19708.12, stdev=12929.92 clat percentiles (usec): | 1.00th=[ 101], 5.00th=[ 143], 10.00th=[ 7635], 20.00th=[ 10290], | 30.00th=[ 12387], 40.00th=[ 15008], 50.00th=[ 17433], 60.00th=[ 20579], | 70.00th=[ 23725], 80.00th=[ 27657], 90.00th=[ 33424], 95.00th=[ 41157], | 99.00th=[ 63177], 99.50th=[ 72877], 99.90th=[122160], 99.95th=[128451], | 99.99th=[128451] bw ( KiB/s): min= 23, max= 215, per=25.28%, avg=113.25, stdev=34.91, samples=106 iops : min= 5, max= 53, avg=27.94, stdev= 8.73, samples=106 write: IOPS=29, BW=116KiB/s (119kB/s)(6236KiB/53590msec) clat (usec): min=608, max=510644, avg=15023.30, stdev=20132.78 lat (usec): min=608, max=510647, avg=15029.28, stdev=20130.72 clat percentiles (usec): | 1.00th=[ 685], 5.00th=[ 758], 10.00th=[ 799], 20.00th=[ 938], | 30.00th=[ 2606], 40.00th=[ 5407], 50.00th=[ 10945], 60.00th=[ 15008], | 70.00th=[ 19006], 80.00th=[ 25822], 90.00th=[ 35914], 95.00th=[ 43254], | 99.00th=[ 73925], 99.50th=[ 85459], 99.90th=[121111], 99.95th=[509608], | 99.99th=[509608] bw ( KiB/s): min= 31, max= 223, per=24.85%, avg=116.57, stdev=38.21, samples=106 iops : min= 7, max= 55, avg=28.77, stdev= 9.62, samples=106 lat (usec) : 100=0.46%, 250=2.21%, 750=2.25%, 1000=8.50% lat (msec) : 2=3.19%, 4=4.69%, 10=11.95%, 20=31.74%, 50=31.97% lat (msec) : 100=2.86%, 250=0.16%, 750=0.03% cpu : usr=0.10%, sys=8.99%, ctx=5058, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=448KiB/s (459kB/s), 113KiB/s-116KiB/s (115kB/s-118kB/s), io=23.4MiB (24.6MB), run=52106-53590msec WRITE: bw=469KiB/s (480kB/s), 116KiB/s-120KiB/s (119kB/s-123kB/s), io=24.6MiB (25.7MB), run=52106-53590msec rand-rw: (groupid=0, jobs=1): err= 0: pid=26711: Thu May 1 22:26:35 2025 read: IOPS=20, BW=83.2KiB/s (85.2kB/s)(5992KiB/72013msec) clat (msec): min=3, max=532, avg=17.57, stdev=20.80 lat (msec): min=3, max=532, avg=17.57, stdev=20.79 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 25], 90.00th=[ 39], 95.00th=[ 53], | 99.00th=[ 80], 99.50th=[ 96], 99.90th=[ 130], 99.95th=[ 531], | 99.99th=[ 531] bw ( KiB/s): min= 8, max= 256, per=25.33%, avg=83.85, stdev=52.99, samples=142 iops : min= 2, max= 64, avg=20.56, stdev=13.31, samples=142 write: IOPS=21, BW=87.4KiB/s (89.5kB/s)(6296KiB/72013msec) clat (msec): min=6, max=194, avg=28.76, stdev=21.34 lat (msec): min=6, max=194, avg=28.77, stdev=21.34 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 33], 80.00th=[ 42], 90.00th=[ 56], 95.00th=[ 69], | 99.00th=[ 106], 99.50th=[ 144], 99.90th=[ 155], 99.95th=[ 194], | 99.99th=[ 194] bw ( KiB/s): min= 16, max= 215, per=25.38%, avg=88.08, stdev=51.75, samples=142 iops : min= 4, max= 53, avg=21.62, stdev=12.99, samples=142 lat (msec) : 4=0.33%, 10=22.59%, 20=38.54%, 50=28.65%, 100=9.05% lat (msec) : 250=0.81%, 750=0.03% cpu : usr=0.13%, sys=4.29%, ctx=3420, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26712: Thu May 1 22:26:35 2025 read: IOPS=20, BW=82.0KiB/s (84.0kB/s)(5940KiB/72409msec) clat (msec): min=3, max=652, avg=18.29, stdev=23.76 lat (msec): min=3, max=652, avg=18.29, stdev=23.76 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 18], 80.00th=[ 25], 90.00th=[ 43], 95.00th=[ 56], | 99.00th=[ 91], 99.50th=[ 99], 99.90th=[ 122], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 8, max= 247, per=24.66%, avg=81.63, stdev=54.07, samples=143 iops : min= 2, max= 61, avg=20.02, stdev=13.52, samples=143 write: IOPS=21, BW=87.7KiB/s (89.8kB/s)(6348KiB/72409msec) clat (msec): min=5, max=163, avg=28.36, stdev=20.60 lat (msec): min=5, max=163, avg=28.36, stdev=20.60 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 34], 80.00th=[ 43], 90.00th=[ 56], 95.00th=[ 67], | 99.00th=[ 100], 99.50th=[ 138], 99.90th=[ 157], 99.95th=[ 165], | 99.99th=[ 165] bw ( KiB/s): min= 16, max= 231, per=25.04%, avg=86.90, stdev=53.58, samples=143 iops : min= 4, max= 57, avg=21.34, stdev=13.43, samples=143 lat (msec) : 4=0.42%, 10=23.37%, 20=37.63%, 50=28.32%, 100=9.64% lat (msec) : 250=0.59%, 750=0.03% cpu : usr=0.09%, sys=4.34%, ctx=3461, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26713: Thu May 1 22:26:35 2025 read: IOPS=20, BW=83.5KiB/s (85.5kB/s)(6028KiB/72172msec) clat (msec): min=3, max=173, avg=17.63, stdev=16.86 lat (msec): min=3, max=173, avg=17.64, stdev=16.86 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 18], 80.00th=[ 26], 90.00th=[ 40], 95.00th=[ 54], | 99.00th=[ 75], 99.50th=[ 96], 99.90th=[ 171], 99.95th=[ 174], | 99.99th=[ 174] bw ( KiB/s): min= 15, max= 247, per=25.33%, avg=83.85, stdev=53.30, samples=142 iops : min= 3, max= 61, avg=20.54, stdev=13.33, samples=142 write: IOPS=21, BW=86.7KiB/s (88.8kB/s)(6260KiB/72172msec) clat (msec): min=5, max=669, avg=28.98, stdev=26.14 lat (msec): min=5, max=669, avg=28.98, stdev=26.14 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 26], | 70.00th=[ 33], 80.00th=[ 44], 90.00th=[ 56], 95.00th=[ 69], | 99.00th=[ 105], 99.50th=[ 123], 99.90th=[ 150], 99.95th=[ 667], | 99.99th=[ 667] bw ( KiB/s): min= 23, max= 216, per=24.88%, avg=86.34, stdev=54.33, samples=143 iops : min= 5, max= 54, avg=21.16, stdev=13.64, samples=143 lat (msec) : 4=0.36%, 10=23.70%, 20=37.24%, 50=28.32%, 100=9.64% lat (msec) : 250=0.72%, 750=0.03% cpu : usr=0.09%, sys=4.34%, ctx=3398, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26714: Thu May 1 22:26:35 2025 read: IOPS=20, BW=83.6KiB/s (85.6kB/s)(6052KiB/72388msec) clat (msec): min=3, max=171, avg=18.36, stdev=18.81 lat (msec): min=3, max=171, avg=18.36, stdev=18.81 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 18], 80.00th=[ 26], 90.00th=[ 43], 95.00th=[ 58], | 99.00th=[ 93], 99.50th=[ 118], 99.90th=[ 171], 99.95th=[ 174], | 99.99th=[ 174] bw ( KiB/s): min= 8, max= 256, per=25.42%, avg=84.15, stdev=53.73, samples=141 iops : min= 2, max= 64, avg=20.67, stdev=13.52, samples=141 write: IOPS=21, BW=86.1KiB/s (88.2kB/s)(6236KiB/72388msec) clat (msec): min=5, max=532, avg=28.37, stdev=23.74 lat (msec): min=5, max=532, avg=28.37, stdev=23.74 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 14], | 30.00th=[ 16], 40.00th=[ 18], 50.00th=[ 21], 60.00th=[ 25], | 70.00th=[ 32], 80.00th=[ 42], 90.00th=[ 57], 95.00th=[ 68], | 99.00th=[ 96], 99.50th=[ 129], 99.90th=[ 157], 99.95th=[ 531], | 99.99th=[ 531] bw ( KiB/s): min= 15, max= 223, per=24.68%, avg=85.64, stdev=52.85, samples=143 iops : min= 3, max= 55, avg=21.05, stdev=13.27, samples=143 lat (msec) : 4=0.46%, 10=23.50%, 20=37.76%, 50=27.51%, 100=9.96% lat (msec) : 250=0.78%, 750=0.03% cpu : usr=0.11%, sys=4.32%, ctx=3434, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=332KiB/s (340kB/s), 82.0KiB/s-83.6KiB/s (84.0kB/s-85.6kB/s), io=23.4MiB (24.6MB), run=72013-72409msec WRITE: bw=347KiB/s (356kB/s), 86.1KiB/s-87.7KiB/s (88.2kB/s-89.8kB/s), io=24.6MiB (25.7MB), run=72013-72409msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=26793: Thu May 1 22:26:47 2025 read: IOPS=33, BW=538KiB/s (551kB/s)(5920KiB/10997msec) clat (usec): min=77, max=66312, avg=16495.20, stdev=11850.36 lat (usec): min=77, max=66313, avg=16496.33, stdev=11850.48 clat percentiles (usec): | 1.00th=[ 93], 5.00th=[ 109], 10.00th=[ 128], 20.00th=[ 7898], | 30.00th=[10814], 40.00th=[13042], 50.00th=[14746], 60.00th=[16909], | 70.00th=[20055], 80.00th=[23725], 90.00th=[30016], 95.00th=[36963], | 99.00th=[62129], 99.50th=[66323], 99.90th=[66323], 99.95th=[66323], | 99.99th=[66323] bw ( KiB/s): min= 381, max= 736, per=24.59%, avg=526.86, stdev=106.49, samples=21 iops : min= 23, max= 46, avg=32.43, stdev= 6.82, samples=21 write: IOPS=36, BW=579KiB/s (593kB/s)(6368KiB/10997msec) clat (usec): min=686, max=99990, avg=12180.58, stdev=14167.66 lat (usec): min=687, max=99992, avg=12182.36, stdev=14167.81 clat percentiles (usec): | 1.00th=[ 898], 5.00th=[ 1090], 10.00th=[ 1188], 20.00th=[ 1385], | 30.00th=[ 2474], 40.00th=[ 4080], 50.00th=[ 6652], 60.00th=[ 10028], | 70.00th=[ 14746], 80.00th=[ 20317], 90.00th=[ 32900], 95.00th=[ 40633], | 99.00th=[ 66323], 99.50th=[ 72877], 99.90th=[100140], 99.95th=[100140], | 99.99th=[100140] bw ( KiB/s): min= 287, max= 768, per=25.05%, avg=564.81, stdev=127.91, samples=21 iops : min= 17, max= 48, avg=34.81, stdev= 8.20, samples=21 lat (usec) : 100=0.65%, 250=5.60%, 750=0.13%, 1000=0.78% lat (msec) : 2=12.89%, 4=6.64%, 10=17.71%, 20=29.82%, 50=23.70% lat (msec) : 100=2.08% cpu : usr=0.11%, sys=11.20%, ctx=1353, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26794: Thu May 1 22:26:47 2025 read: IOPS=32, BW=527KiB/s (540kB/s)(5744KiB/10901msec) clat (usec): min=87, max=80509, avg=15434.37, stdev=12647.28 lat (usec): min=88, max=80510, avg=15437.73, stdev=12648.92 clat percentiles (usec): | 1.00th=[ 90], 5.00th=[ 103], 10.00th=[ 119], 20.00th=[ 6128], | 30.00th=[ 9503], 40.00th=[11600], 50.00th=[13435], 60.00th=[15926], | 70.00th=[19792], 80.00th=[23462], 90.00th=[30278], 95.00th=[36963], | 99.00th=[68682], 99.50th=[76022], 99.90th=[80217], 99.95th=[80217], | 99.99th=[80217] bw ( KiB/s): min= 159, max= 764, per=24.32%, avg=521.10, stdev=140.00, samples=21 iops : min= 9, max= 47, avg=32.19, stdev= 8.79, samples=21 write: IOPS=37, BW=600KiB/s (615kB/s)(6544KiB/10901msec) clat (usec): min=896, max=61903, avg=12621.23, stdev=12572.31 lat (usec): min=897, max=61905, avg=12622.89, stdev=12572.27 clat percentiles (usec): | 1.00th=[ 996], 5.00th=[ 1074], 10.00th=[ 1156], 20.00th=[ 1631], | 30.00th=[ 3294], 40.00th=[ 5014], 50.00th=[ 8586], 60.00th=[12387], | 70.00th=[15926], 80.00th=[22152], 90.00th=[31589], 95.00th=[37487], | 99.00th=[53740], 99.50th=[58983], 99.90th=[62129], 99.95th=[62129], | 99.99th=[62129] bw ( KiB/s): min= 319, max= 992, per=25.89%, avg=583.86, stdev=168.59, samples=21 iops : min= 19, max= 62, avg=36.10, stdev=10.77, samples=21 lat (usec) : 100=1.69%, 250=5.73%, 500=0.52%, 750=0.26%, 1000=0.65% lat (msec) : 2=10.94%, 4=7.68%, 10=16.80%, 20=29.30%, 50=24.35% lat (msec) : 100=2.08% cpu : usr=0.15%, sys=11.44%, ctx=1199, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26795: Thu May 1 22:26:47 2025 read: IOPS=36, BW=586KiB/s (600kB/s)(6256KiB/10677msec) clat (usec): min=92, max=69836, avg=14862.28, stdev=10500.06 lat (usec): min=92, max=69837, avg=14863.32, stdev=10500.03 clat percentiles (usec): | 1.00th=[ 95], 5.00th=[ 110], 10.00th=[ 130], 20.00th=[ 7504], | 30.00th=[ 9765], 40.00th=[12125], 50.00th=[13960], 60.00th=[16057], | 70.00th=[18220], 80.00th=[22152], 90.00th=[27395], 95.00th=[32637], | 99.00th=[52691], 99.50th=[69731], 99.90th=[69731], 99.95th=[69731], | 99.99th=[69731] bw ( KiB/s): min= 223, max= 990, per=27.10%, avg=580.67, stdev=173.72, samples=21 iops : min= 13, max= 61, avg=35.76, stdev=10.93, samples=21 write: IOPS=35, BW=565KiB/s (579kB/s)(6032KiB/10677msec) clat (usec): min=823, max=68984, avg=12390.02, stdev=12993.29 lat (usec): min=825, max=68986, avg=12391.65, stdev=12993.25 clat percentiles (usec): | 1.00th=[ 955], 5.00th=[ 1090], 10.00th=[ 1172], 20.00th=[ 1582], | 30.00th=[ 2540], 40.00th=[ 4359], 50.00th=[ 7898], 60.00th=[11994], | 70.00th=[15270], 80.00th=[21627], 90.00th=[31065], 95.00th=[38536], | 99.00th=[54264], 99.50th=[64226], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 255, max= 958, per=24.53%, avg=553.19, stdev=161.35, samples=21 iops : min= 15, max= 59, avg=34.05, stdev=10.09, samples=21 lat (usec) : 100=1.04%, 250=5.60%, 500=0.26%, 750=0.26%, 1000=0.78% lat (msec) : 2=11.33%, 4=7.68%, 10=16.02%, 20=33.20%, 50=22.27% lat (msec) : 100=1.56% cpu : usr=0.15%, sys=11.70%, ctx=1275, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26796: Thu May 1 22:26:47 2025 read: IOPS=33, BW=540KiB/s (553kB/s)(6032KiB/11173msec) clat (usec): min=80, max=67246, avg=16193.86, stdev=12122.08 lat (usec): min=81, max=67247, avg=16195.33, stdev=12121.54 clat percentiles (usec): | 1.00th=[ 95], 5.00th=[ 110], 10.00th=[ 125], 20.00th=[ 7177], | 30.00th=[10028], 40.00th=[11994], 50.00th=[14222], 60.00th=[16712], | 70.00th=[19792], 80.00th=[24511], 90.00th=[30016], 95.00th=[38536], | 99.00th=[60556], 99.50th=[66847], 99.90th=[67634], 99.95th=[67634], | 99.99th=[67634] bw ( KiB/s): min= 127, max= 798, per=24.64%, avg=527.95, stdev=155.15, samples=22 iops : min= 7, max= 49, avg=32.50, stdev= 9.66, samples=22 write: IOPS=34, BW=560KiB/s (573kB/s)(6256KiB/11173msec) clat (usec): min=791, max=67585, avg=12395.78, stdev=13201.02 lat (usec): min=792, max=67586, avg=12397.87, stdev=13200.64 clat percentiles (usec): | 1.00th=[ 930], 5.00th=[ 1106], 10.00th=[ 1156], 20.00th=[ 1483], | 30.00th=[ 2507], 40.00th=[ 4228], 50.00th=[ 7504], 60.00th=[11076], | 70.00th=[15926], 80.00th=[22676], 90.00th=[30802], 95.00th=[39584], | 99.00th=[55837], 99.50th=[63701], 99.90th=[67634], 99.95th=[67634], | 99.99th=[67634] bw ( KiB/s): min= 223, max= 862, per=24.51%, avg=552.59, stdev=155.65, samples=22 iops : min= 13, max= 53, avg=34.05, stdev= 9.58, samples=22 lat (usec) : 100=1.17%, 250=5.21%, 500=0.26%, 1000=1.17% lat (msec) : 2=12.11%, 4=7.55%, 10=16.67%, 20=28.78%, 50=25.00% lat (msec) : 100=2.08% cpu : usr=0.15%, sys=11.27%, ctx=1241, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2144KiB/s (2195kB/s), 527KiB/s-586KiB/s (540kB/s-600kB/s), io=23.4MiB (24.5MB), run=10677-11173msec WRITE: bw=2255KiB/s (2310kB/s), 560KiB/s-600KiB/s (573kB/s-615kB/s), io=24.6MiB (25.8MB), run=10677-11173msec rand-rw: (groupid=0, jobs=1): err= 0: pid=26797: Thu May 1 22:26:53 2025 read: IOPS=21, BW=345KiB/s (353kB/s)(5920KiB/17151msec) clat (msec): min=4, max=123, avg=16.97, stdev=13.79 lat (msec): min=4, max=123, avg=16.97, stdev=13.79 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 10], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 25], 90.00th=[ 35], 95.00th=[ 44], | 99.00th=[ 68], 99.50th=[ 73], 99.90th=[ 125], 99.95th=[ 125], | 99.99th=[ 125] bw ( KiB/s): min= 128, max= 704, per=24.63%, avg=341.68, stdev=149.51, samples=34 iops : min= 8, max= 44, avg=20.88, stdev= 9.43, samples=34 write: IOPS=23, BW=371KiB/s (380kB/s)(6368KiB/17151msec) clat (msec): min=7, max=112, avg=27.14, stdev=16.06 lat (msec): min=7, max=112, avg=27.14, stdev=16.06 clat percentiles (msec): | 1.00th=[ 9], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 16], | 30.00th=[ 18], 40.00th=[ 20], 50.00th=[ 22], 60.00th=[ 25], | 70.00th=[ 29], 80.00th=[ 37], 90.00th=[ 52], 95.00th=[ 63], | 99.00th=[ 81], 99.50th=[ 84], 99.90th=[ 113], 99.95th=[ 113], | 99.99th=[ 113] bw ( KiB/s): min= 95, max= 698, per=25.35%, avg=369.79, stdev=162.74, samples=34 iops : min= 5, max= 43, avg=22.65, stdev=10.32, samples=34 lat (msec) : 10=19.92%, 20=38.15%, 50=34.51%, 100=7.16%, 250=0.26% cpu : usr=0.08%, sys=4.87%, ctx=891, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26798: Thu May 1 22:26:53 2025 read: IOPS=20, BW=333KiB/s (341kB/s)(5744KiB/17263msec) clat (usec): min=3671, max=85350, avg=17048.46, stdev=14429.88 lat (usec): min=3672, max=85351, avg=17049.80, stdev=14429.78 clat percentiles (usec): | 1.00th=[ 3884], 5.00th=[ 5800], 10.00th=[ 6259], 20.00th=[ 7832], | 30.00th=[ 9241], 40.00th=[10683], 50.00th=[12256], 60.00th=[13960], | 70.00th=[16450], 80.00th=[21103], 90.00th=[33817], 95.00th=[53740], | 99.00th=[74974], 99.50th=[79168], 99.90th=[85459], 99.95th=[85459], | 99.99th=[85459] bw ( KiB/s): min= 95, max= 640, per=23.82%, avg=330.38, stdev=156.45, samples=34 iops : min= 5, max= 40, avg=20.24, stdev= 9.95, samples=34 write: IOPS=23, BW=379KiB/s (388kB/s)(6544KiB/17263msec) clat (usec): min=5981, max=88571, avg=27126.89, stdev=15402.34 lat (usec): min=5983, max=88573, avg=27129.69, stdev=15401.82 clat percentiles (usec): | 1.00th=[ 8029], 5.00th=[11731], 10.00th=[13042], 20.00th=[15533], | 30.00th=[17695], 40.00th=[19792], 50.00th=[22152], 60.00th=[24773], | 70.00th=[30016], 80.00th=[36963], 90.00th=[48497], 95.00th=[59507], | 99.00th=[80217], 99.50th=[84411], 99.90th=[88605], 99.95th=[88605], | 99.99th=[88605] bw ( KiB/s): min= 192, max= 736, per=25.86%, avg=377.29, stdev=146.88, samples=34 iops : min= 12, max= 46, avg=23.18, stdev= 9.35, samples=34 lat (msec) : 4=0.52%, 10=17.45%, 20=39.97%, 50=34.77%, 100=7.29% cpu : usr=0.00%, sys=4.70%, ctx=883, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26799: Thu May 1 22:26:53 2025 read: IOPS=22, BW=362KiB/s (371kB/s)(6256KiB/17267msec) clat (msec): min=3, max=120, avg=16.68, stdev=14.62 lat (msec): min=3, max=120, avg=16.68, stdev=14.62 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 16], 80.00th=[ 21], 90.00th=[ 36], 95.00th=[ 48], | 99.00th=[ 68], 99.50th=[ 107], 99.90th=[ 122], 99.95th=[ 122], | 99.99th=[ 122] bw ( KiB/s): min= 96, max= 832, per=26.05%, avg=361.32, stdev=206.30, samples=34 iops : min= 6, max= 52, avg=22.12, stdev=12.96, samples=34 write: IOPS=21, BW=349KiB/s (358kB/s)(6032KiB/17267msec) clat (msec): min=7, max=115, avg=28.21, stdev=16.64 lat (msec): min=7, max=115, avg=28.22, stdev=16.64 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 12], 10.00th=[ 14], 20.00th=[ 16], | 30.00th=[ 18], 40.00th=[ 21], 50.00th=[ 23], 60.00th=[ 27], | 70.00th=[ 32], 80.00th=[ 37], 90.00th=[ 52], 95.00th=[ 65], | 99.00th=[ 84], 99.50th=[ 90], 99.90th=[ 116], 99.95th=[ 116], | 99.99th=[ 116] bw ( KiB/s): min= 127, max= 606, per=23.60%, avg=344.38, stdev=146.37, samples=34 iops : min= 7, max= 37, avg=21.06, stdev= 9.13, samples=34 lat (msec) : 4=0.13%, 10=20.57%, 20=37.63%, 50=34.37%, 100=6.90% lat (msec) : 250=0.39% cpu : usr=0.20%, sys=4.50%, ctx=866, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26800: Thu May 1 22:26:53 2025 read: IOPS=22, BW=354KiB/s (363kB/s)(6032KiB/17019msec) clat (msec): min=3, max=123, avg=18.03, stdev=16.34 lat (msec): min=3, max=123, avg=18.03, stdev=16.34 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 13], 60.00th=[ 15], | 70.00th=[ 18], 80.00th=[ 24], 90.00th=[ 41], 95.00th=[ 54], | 99.00th=[ 81], 99.50th=[ 86], 99.90th=[ 125], 99.95th=[ 125], | 99.99th=[ 125] bw ( KiB/s): min= 127, max= 766, per=25.40%, avg=352.32, stdev=166.70, samples=34 iops : min= 7, max= 47, avg=21.65, stdev=10.50, samples=34 write: IOPS=22, BW=368KiB/s (376kB/s)(6256KiB/17019msec) clat (usec): min=7138, max=90971, avg=25790.43, stdev=14092.05 lat (usec): min=7140, max=90972, avg=25795.97, stdev=14090.62 clat percentiles (usec): | 1.00th=[ 9372], 5.00th=[11338], 10.00th=[12780], 20.00th=[14746], | 30.00th=[16909], 40.00th=[19530], 50.00th=[22152], 60.00th=[24511], | 70.00th=[27919], 80.00th=[33817], 90.00th=[45351], 95.00th=[56886], | 99.00th=[73925], 99.50th=[86508], 99.90th=[90702], 99.95th=[90702], | 99.99th=[90702] bw ( KiB/s): min= 95, max= 733, per=25.17%, avg=367.24, stdev=167.12, samples=34 iops : min= 5, max= 45, avg=22.59, stdev=10.54, samples=34 lat (msec) : 4=0.26%, 10=18.49%, 20=39.06%, 50=34.90%, 100=7.16% lat (msec) : 250=0.13% cpu : usr=0.25%, sys=4.57%, ctx=888, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1387KiB/s (1420kB/s), 333KiB/s-362KiB/s (341kB/s-371kB/s), io=23.4MiB (24.5MB), run=17019-17267msec WRITE: bw=1459KiB/s (1494kB/s), 349KiB/s-379KiB/s (358kB/s-388kB/s), io=24.6MiB (25.8MB), run=17019-17267msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=26819: Thu May 1 22:26:57 2025 read: IOPS=5, BW=5842KiB/s (5982kB/s)(7168KiB/1227msec) clat (msec): min=18, max=183, avg=78.99, stdev=59.42 lat (msec): min=18, max=183, avg=78.99, stdev=59.42 clat percentiles (msec): | 1.00th=[ 19], 5.00th=[ 19], 10.00th=[ 19], 20.00th=[ 40], | 30.00th=[ 42], 40.00th=[ 42], 50.00th=[ 46], 60.00th=[ 102], | 70.00th=[ 102], 80.00th=[ 124], 90.00th=[ 184], 95.00th=[ 184], | 99.00th=[ 184], 99.50th=[ 184], 99.90th=[ 184], 99.95th=[ 184], | 99.99th=[ 184] bw ( KiB/s): min= 4055, max= 6144, per=54.78%, avg=5099.50, stdev=1477.15, samples=2 iops : min= 3, max= 6, avg= 4.50, stdev= 2.12, samples=2 write: IOPS=4, BW=4173KiB/s (4273kB/s)(5120KiB/1227msec) clat (msec): min=41, max=137, avg=89.73, stdev=35.19 lat (msec): min=41, max=137, avg=89.92, stdev=35.26 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 42], 10.00th=[ 42], 20.00th=[ 42], | 30.00th=[ 75], 40.00th=[ 75], 50.00th=[ 94], 60.00th=[ 94], | 70.00th=[ 101], 80.00th=[ 101], 90.00th=[ 138], 95.00th=[ 138], | 99.00th=[ 138], 99.50th=[ 138], 99.90th=[ 138], 99.95th=[ 138], | 99.99th=[ 138] bw ( KiB/s): min= 6083, max= 6083, per=46.68%, avg=6083.00, stdev= 0.00, samples=1 iops : min= 5, max= 5, avg= 5.00, stdev= 0.00, samples=1 lat (msec) : 20=8.33%, 50=33.33%, 100=16.67%, 250=41.67% cpu : usr=0.00%, sys=4.49%, ctx=35, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26820: Thu May 1 22:26:57 2025 read: IOPS=0, BW=1006KiB/s (1030kB/s)(2048KiB/2036msec) clat (msec): min=93, max=184, avg=139.24, stdev=64.15 lat (msec): min=93, max=184, avg=139.25, stdev=64.15 clat percentiles (msec): | 1.00th=[ 94], 5.00th=[ 94], 10.00th=[ 94], 20.00th=[ 94], | 30.00th=[ 94], 40.00th=[ 94], 50.00th=[ 94], 60.00th=[ 186], | 70.00th=[ 186], 80.00th=[ 186], 90.00th=[ 186], 95.00th=[ 186], | 99.00th=[ 186], 99.50th=[ 186], 99.90th=[ 186], 99.95th=[ 186], | 99.99th=[ 186] bw ( KiB/s): min= 2039, max= 2048, per=21.95%, avg=2043.50, stdev= 6.36, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=4, BW=5029KiB/s (5150kB/s)(10.0MiB/2036msec) clat (msec): min=27, max=631, avg=172.85, stdev=179.35 lat (msec): min=27, max=632, avg=172.93, stdev=179.34 clat percentiles (msec): | 1.00th=[ 28], 5.00th=[ 28], 10.00th=[ 28], 20.00th=[ 45], | 30.00th=[ 46], 40.00th=[ 75], 50.00th=[ 78], 60.00th=[ 174], | 70.00th=[ 215], 80.00th=[ 215], 90.00th=[ 228], 95.00th=[ 634], | 99.00th=[ 634], 99.50th=[ 634], 99.90th=[ 634], 99.95th=[ 634], | 99.99th=[ 634] bw ( KiB/s): min= 2048, max= 6144, per=35.31%, avg=4601.75, stdev=1954.31, samples=4 iops : min= 2, max= 6, avg= 4.25, stdev= 1.71, samples=4 lat (msec) : 50=25.00%, 100=25.00%, 250=41.67%, 750=8.33% cpu : usr=0.00%, sys=2.85%, ctx=38, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26821: Thu May 1 22:26:57 2025 read: IOPS=2, BW=2793KiB/s (2860kB/s)(6144KiB/2200msec) clat (msec): min=49, max=270, avg=167.52, stdev=89.16 lat (msec): min=49, max=270, avg=167.53, stdev=89.16 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 50], 20.00th=[ 71], | 30.00th=[ 71], 40.00th=[ 186], 50.00th=[ 186], 60.00th=[ 192], | 70.00th=[ 239], 80.00th=[ 239], 90.00th=[ 271], 95.00th=[ 271], | 99.00th=[ 271], 99.50th=[ 271], 99.90th=[ 271], 99.95th=[ 271], | 99.99th=[ 271] bw ( KiB/s): min= 2039, max= 4096, per=36.37%, avg=3386.00, stdev=1167.11, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=2, BW=2793KiB/s (2860kB/s)(6144KiB/2200msec) clat (msec): min=96, max=391, avg=177.86, stdev=108.62 lat (msec): min=96, max=392, avg=177.92, stdev=108.62 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 96], 10.00th=[ 96], 20.00th=[ 118], | 30.00th=[ 118], 40.00th=[ 131], 50.00th=[ 131], 60.00th=[ 153], | 70.00th=[ 180], 80.00th=[ 180], 90.00th=[ 393], 95.00th=[ 393], | 99.00th=[ 393], 99.50th=[ 393], 99.90th=[ 393], 99.95th=[ 393], | 99.99th=[ 393] bw ( KiB/s): min= 2011, max= 4096, per=23.45%, avg=3056.25, stdev=1190.86, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.50, samples=4 lat (msec) : 50=8.33%, 100=16.67%, 250=58.33%, 500=16.67% cpu : usr=0.00%, sys=4.09%, ctx=35, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26822: Thu May 1 22:26:57 2025 read: IOPS=2, BW=2397KiB/s (2455kB/s)(5120KiB/2136msec) clat (msec): min=41, max=401, avg=159.56, stdev=148.77 lat (msec): min=41, max=401, avg=159.56, stdev=148.77 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 42], 10.00th=[ 42], 20.00th=[ 42], | 30.00th=[ 47], 40.00th=[ 47], 50.00th=[ 114], 60.00th=[ 114], | 70.00th=[ 194], 80.00th=[ 194], 90.00th=[ 401], 95.00th=[ 401], | 99.00th=[ 401], 99.50th=[ 401], 99.90th=[ 401], 99.95th=[ 401], | 99.99th=[ 401] bw ( KiB/s): min= 8159, max= 8159, per=87.65%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=3, BW=3356KiB/s (3436kB/s)(7168KiB/2136msec) clat (msec): min=45, max=374, avg=180.26, stdev=124.31 lat (msec): min=45, max=374, avg=180.33, stdev=124.33 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 46], 10.00th=[ 46], 20.00th=[ 66], | 30.00th=[ 80], 40.00th=[ 80], 50.00th=[ 203], 60.00th=[ 205], | 70.00th=[ 205], 80.00th=[ 292], 90.00th=[ 376], 95.00th=[ 376], | 99.00th=[ 376], 99.50th=[ 376], 99.90th=[ 376], 99.95th=[ 376], | 99.99th=[ 376] bw ( KiB/s): min= 2039, max= 6131, per=27.42%, avg=3574.00, stdev=1959.07, samples=4 iops : min= 1, max= 5, avg= 2.50, stdev= 1.91, samples=4 lat (msec) : 50=25.00%, 100=16.67%, 250=33.33%, 500=25.00% cpu : usr=0.00%, sys=4.26%, ctx=64, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=9309KiB/s (9533kB/s), 1006KiB/s-5842KiB/s (1030kB/s-5982kB/s), io=20.0MiB (20.0MB), run=1227-2200msec WRITE: bw=12.7MiB/s (13.3MB/s), 2793KiB/s-5029KiB/s (2860kB/s-5150kB/s), io=28.0MiB (29.4MB), run=1227-2200msec rand-rw: (groupid=0, jobs=1): err= 0: pid=26824: Thu May 1 22:26:58 2025 read: IOPS=3, BW=3158KiB/s (3233kB/s)(7168KiB/2270msec) clat (usec): min=340, max=1450.3k, avg=245399.53, stdev=533136.71 lat (usec): min=341, max=1450.3k, avg=245401.15, stdev=533136.76 clat percentiles (usec): | 1.00th=[ 343], 5.00th=[ 343], 10.00th=[ 343], | 20.00th=[ 594], 30.00th=[ 807], 40.00th=[ 807], | 50.00th=[ 79168], 60.00th=[ 89654], 70.00th=[ 89654], | 80.00th=[ 96994], 90.00th=[1451230], 95.00th=[1451230], | 99.00th=[1451230], 99.50th=[1451230], 99.90th=[1451230], | 99.95th=[1451230], 99.99th=[1451230] bw ( KiB/s): min= 4079, max= 5605, per=53.67%, avg=4842.00, stdev=1079.04, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=2, BW=2256KiB/s (2310kB/s)(5120KiB/2270msec) clat (usec): min=28256, max=98995, avg=73934.04, stdev=28202.37 lat (usec): min=28274, max=99068, avg=73983.25, stdev=28222.48 clat percentiles (usec): | 1.00th=[28181], 5.00th=[28181], 10.00th=[28181], 20.00th=[28181], | 30.00th=[69731], 40.00th=[69731], 50.00th=[77071], 60.00th=[77071], | 70.00th=[94897], 80.00th=[94897], 90.00th=[99091], 95.00th=[99091], | 99.00th=[99091], 99.50th=[99091], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 5605, max= 5605, per=44.38%, avg=5605.00, stdev= 0.00, samples=1 iops : min= 5, max= 5, avg= 5.00, stdev= 0.00, samples=1 lat (usec) : 500=8.33%, 750=8.33%, 1000=8.33% lat (msec) : 50=8.33%, 100=58.33% cpu : usr=0.04%, sys=10.40%, ctx=344, majf=0, minf=34 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26825: Thu May 1 22:26:58 2025 read: IOPS=0, BW=1019KiB/s (1044kB/s)(2048KiB/2009msec) clat (msec): min=96, max=100, avg=98.40, stdev= 2.99 lat (msec): min=96, max=100, avg=98.40, stdev= 2.99 clat percentiles (msec): | 1.00th=[ 96], 5.00th=[ 96], 10.00th=[ 96], 20.00th=[ 96], | 30.00th=[ 96], 40.00th=[ 96], 50.00th=[ 96], 60.00th=[ 101], | 70.00th=[ 101], 80.00th=[ 101], 90.00th=[ 101], 95.00th=[ 101], | 99.00th=[ 101], 99.50th=[ 101], 99.90th=[ 101], 99.95th=[ 101], | 99.99th=[ 101] bw ( KiB/s): min= 2043, max= 2048, per=22.67%, avg=2045.50, stdev= 3.54, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=4, BW=5097KiB/s (5219kB/s)(10.0MiB/2009msec) clat (msec): min=35, max=322, avg=157.05, stdev=93.63 lat (msec): min=35, max=323, avg=157.14, stdev=93.64 clat percentiles (msec): | 1.00th=[ 36], 5.00th=[ 36], 10.00th=[ 36], 20.00th=[ 73], | 30.00th=[ 94], 40.00th=[ 104], 50.00th=[ 107], 60.00th=[ 176], | 70.00th=[ 190], 80.00th=[ 190], 90.00th=[ 292], 95.00th=[ 326], | 99.00th=[ 326], 99.50th=[ 326], 99.90th=[ 326], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 4096, max=10199, per=56.59%, avg=7147.50, stdev=4315.47, samples=2 iops : min= 4, max= 9, avg= 6.50, stdev= 3.54, samples=2 lat (msec) : 50=8.33%, 100=25.00%, 250=50.00%, 500=16.67% cpu : usr=0.20%, sys=16.04%, ctx=200, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26826: Thu May 1 22:26:58 2025 read: IOPS=3, BW=3361KiB/s (3442kB/s)(6144KiB/1828msec) clat (usec): min=647, max=170156, avg=98882.32, stdev=61142.32 lat (usec): min=649, max=170157, avg=98884.04, stdev=61142.21 clat percentiles (usec): | 1.00th=[ 652], 5.00th=[ 652], 10.00th=[ 652], 20.00th=[ 52691], | 30.00th=[ 52691], 40.00th=[120062], 50.00th=[120062], 60.00th=[122160], | 70.00th=[128451], 80.00th=[128451], 90.00th=[170918], 95.00th=[170918], | 99.00th=[170918], 99.50th=[170918], 99.90th=[170918], 99.95th=[170918], | 99.99th=[170918] bw ( KiB/s): min= 2048, max= 6144, per=45.40%, avg=4096.00, stdev=2896.31, samples=2 iops : min= 2, max= 6, avg= 4.00, stdev= 2.83, samples=2 write: IOPS=3, BW=3361KiB/s (3442kB/s)(6144KiB/1828msec) clat (msec): min=50, max=365, avg=161.30, stdev=131.69 lat (msec): min=50, max=366, avg=161.37, stdev=131.70 clat percentiles (msec): | 1.00th=[ 52], 5.00th=[ 52], 10.00th=[ 52], 20.00th=[ 56], | 30.00th=[ 56], 40.00th=[ 85], 50.00th=[ 85], 60.00th=[ 130], | 70.00th=[ 284], 80.00th=[ 284], 90.00th=[ 368], 95.00th=[ 368], | 99.00th=[ 368], 99.50th=[ 368], 99.90th=[ 368], 99.95th=[ 368], | 99.99th=[ 368] bw ( KiB/s): min= 2048, max= 4096, per=21.62%, avg=2730.67, stdev=1182.41, samples=3 iops : min= 2, max= 4, avg= 2.67, stdev= 1.15, samples=3 lat (usec) : 750=8.33% lat (msec) : 100=33.33%, 250=41.67%, 500=16.67% cpu : usr=0.00%, sys=13.19%, ctx=318, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26827: Thu May 1 22:26:58 2025 read: IOPS=2, BW=2789KiB/s (2856kB/s)(5120KiB/1836msec) clat (usec): min=428, max=146805, avg=56976.60, stdev=62909.11 lat (usec): min=429, max=146807, avg=56978.05, stdev=62909.50 clat percentiles (usec): | 1.00th=[ 429], 5.00th=[ 429], 10.00th=[ 429], 20.00th=[ 429], | 30.00th=[ 465], 40.00th=[ 465], 50.00th=[ 45351], 60.00th=[ 45351], | 70.00th=[ 91751], 80.00th=[ 91751], 90.00th=[147850], 95.00th=[147850], | 99.00th=[147850], 99.50th=[147850], 99.90th=[147850], 99.95th=[147850], | 99.99th=[147850] bw ( KiB/s): min= 8175, max= 8175, per=90.61%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=3, BW=3904KiB/s (3998kB/s)(7168KiB/1836msec) clat (msec): min=85, max=407, avg=210.66, stdev=120.60 lat (msec): min=85, max=407, avg=210.72, stdev=120.60 clat percentiles (msec): | 1.00th=[ 86], 5.00th=[ 86], 10.00th=[ 86], 20.00th=[ 123], | 30.00th=[ 130], 40.00th=[ 130], 50.00th=[ 169], 60.00th=[ 222], | 70.00th=[ 222], 80.00th=[ 338], 90.00th=[ 409], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 2035, max= 6131, per=26.92%, avg=3400.33, stdev=2364.83, samples=3 iops : min= 1, max= 5, avg= 2.33, stdev= 2.31, samples=3 lat (usec) : 500=16.67% lat (msec) : 50=8.33%, 100=16.67%, 250=41.67%, 500=16.67% cpu : usr=0.00%, sys=12.59%, ctx=103, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=9022KiB/s (9239kB/s), 1019KiB/s-3361KiB/s (1044kB/s-3442kB/s), io=20.0MiB (20.0MB), run=1828-2270msec WRITE: bw=12.3MiB/s (12.9MB/s), 2256KiB/s-5097KiB/s (2310kB/s-5219kB/s), io=28.0MiB (29.4MB), run=1828-2270msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.7 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=26836: Thu May 1 22:27:02 2025 read: IOPS=2, BW=11.1MiB/s (11.7MB/s)(12.0MiB/1079msec) clat (usec): min=1941, max=556589, avg=339339.85, stdev=296201.61 lat (usec): min=1943, max=556590, avg=339341.36, stdev=296201.18 clat percentiles (usec): | 1.00th=[ 1942], 5.00th=[ 1942], 10.00th=[ 1942], 20.00th=[ 1942], | 30.00th=[ 1942], 40.00th=[459277], 50.00th=[459277], 60.00th=[459277], | 70.00th=[557843], 80.00th=[557843], 90.00th=[557843], 95.00th=[557843], | 99.00th=[557843], 99.50th=[557843], 99.90th=[557843], 99.95th=[557843], | 99.99th=[557843] bw ( KiB/s): min= 8192, max= 8192, per=88.35%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 2=33.33%, 500=33.33%, 750=33.33% cpu : usr=0.19%, sys=10.48%, ctx=342, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26838: Thu May 1 22:27:02 2025 read: IOPS=0, BW=2318KiB/s (2374kB/s)(4096KiB/1767msec) clat (nsec): min=508702k, max=508702k, avg=508701740.00, stdev= 0.00 lat (nsec): min=508704k, max=508704k, avg=508704369.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 510], 5.00th=[ 510], 10.00th=[ 510], 20.00th=[ 510], | 30.00th=[ 510], 40.00th=[ 510], 50.00th=[ 510], 60.00th=[ 510], | 70.00th=[ 510], 80.00th=[ 510], 90.00th=[ 510], 95.00th=[ 510], | 99.00th=[ 510], 99.50th=[ 510], 99.90th=[ 510], 99.95th=[ 510], | 99.99th=[ 510] bw ( KiB/s): min= 8175, max= 8175, per=88.17%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4636KiB/s (4747kB/s)(8192KiB/1767msec) clat (msec): min=394, max=690, avg=542.60, stdev=208.91 lat (msec): min=395, max=691, avg=543.16, stdev=209.19 clat percentiles (msec): | 1.00th=[ 397], 5.00th=[ 397], 10.00th=[ 397], 20.00th=[ 397], | 30.00th=[ 397], 40.00th=[ 397], 50.00th=[ 397], 60.00th=[ 693], | 70.00th=[ 693], 80.00th=[ 693], 90.00th=[ 693], 95.00th=[ 693], | 99.00th=[ 693], 99.50th=[ 693], 99.90th=[ 693], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min= 5610, max= 5610, per=30.25%, avg=5610.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.06%, sys=13.14%, ctx=111, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26840: Thu May 1 22:27:02 2025 write: IOPS=2, BW=9170KiB/s (9390kB/s)(12.0MiB/1340msec) clat (msec): min=264, max=533, avg=406.85, stdev=135.23 lat (msec): min=264, max=533, avg=407.15, stdev=135.25 clat percentiles (msec): | 1.00th=[ 266], 5.00th=[ 266], 10.00th=[ 266], 20.00th=[ 266], | 30.00th=[ 266], 40.00th=[ 422], 50.00th=[ 422], 60.00th=[ 422], | 70.00th=[ 535], 80.00th=[ 535], 90.00th=[ 535], 95.00th=[ 535], | 99.00th=[ 535], 99.50th=[ 535], 99.90th=[ 535], 99.95th=[ 535], | 99.99th=[ 535] bw ( KiB/s): min= 8143, max= 8143, per=43.91%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.07%, sys=20.61%, ctx=62, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26842: Thu May 1 22:27:02 2025 write: IOPS=1, BW=7018KiB/s (7186kB/s)(12.0MiB/1751msec) clat (msec): min=326, max=654, avg=537.07, stdev=182.90 lat (msec): min=326, max=654, avg=537.42, stdev=182.93 clat percentiles (msec): | 1.00th=[ 326], 5.00th=[ 326], 10.00th=[ 326], 20.00th=[ 326], | 30.00th=[ 326], 40.00th=[ 634], 50.00th=[ 634], 60.00th=[ 634], | 70.00th=[ 651], 80.00th=[ 651], 90.00th=[ 651], 95.00th=[ 651], | 99.00th=[ 651], 99.50th=[ 651], 99.90th=[ 651], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 8175, max= 8192, per=44.13%, avg=8183.50, stdev=12.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.06%, sys=19.71%, ctx=86, majf=0, minf=28 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=9272KiB/s (9495kB/s), 2318KiB/s-11.1MiB/s (2374kB/s-11.7MB/s), io=16.0MiB (16.8MB), run=1079-1767msec WRITE: bw=18.1MiB/s (18.0MB/s), 4636KiB/s-9170KiB/s (4747kB/s-9390kB/s), io=32.0MiB (33.6MB), run=1340-1767msec rand-rw: (groupid=0, jobs=1): err= 0: pid=26837: Thu May 1 22:27:02 2025 read: IOPS=1, BW=5573KiB/s (5707kB/s)(12.0MiB/2205msec) clat (msec): min=121, max=986, avg=647.54, stdev=462.15 lat (msec): min=121, max=986, avg=647.54, stdev=462.15 clat percentiles (msec): | 1.00th=[ 122], 5.00th=[ 122], 10.00th=[ 122], 20.00th=[ 122], | 30.00th=[ 122], 40.00th=[ 835], 50.00th=[ 835], 60.00th=[ 835], | 70.00th=[ 986], 80.00th=[ 986], 90.00th=[ 986], 95.00th=[ 986], | 99.00th=[ 986], 99.50th=[ 986], 99.90th=[ 986], 99.95th=[ 986], | 99.99th=[ 986] bw ( KiB/s): min= 6187, max= 6311, per=84.10%, avg=6249.00, stdev=87.68, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 250=33.33%, 1000=66.67% cpu : usr=0.32%, sys=1.63%, ctx=33, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26839: Thu May 1 22:27:02 2025 read: IOPS=0, BW=1928KiB/s (1974kB/s)(4096KiB/2125msec) clat (nsec): min=905805k, max=905805k, avg=905804990.00, stdev= 0.00 lat (nsec): min=905807k, max=905807k, avg=905806882.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 902], 5.00th=[ 902], 10.00th=[ 902], 20.00th=[ 902], | 30.00th=[ 902], 40.00th=[ 902], 50.00th=[ 902], 60.00th=[ 902], | 70.00th=[ 902], 80.00th=[ 902], 90.00th=[ 902], 95.00th=[ 902], | 99.00th=[ 902], 99.50th=[ 902], 99.90th=[ 902], 99.95th=[ 902], | 99.99th=[ 902] bw ( KiB/s): min= 6725, max= 6725, per=90.51%, avg=6725.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3855KiB/s (3948kB/s)(8192KiB/2125msec) clat (msec): min=465, max=550, avg=507.77, stdev=60.13 lat (msec): min=465, max=550, avg=508.05, stdev=60.11 clat percentiles (msec): | 1.00th=[ 464], 5.00th=[ 464], 10.00th=[ 464], 20.00th=[ 464], | 30.00th=[ 464], 40.00th=[ 464], 50.00th=[ 464], 60.00th=[ 550], | 70.00th=[ 550], 80.00th=[ 550], 90.00th=[ 550], 95.00th=[ 550], | 99.00th=[ 550], 99.50th=[ 550], 99.90th=[ 550], 99.95th=[ 550], | 99.99th=[ 550] bw ( KiB/s): min= 6860, max= 6860, per=46.79%, avg=6860.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.09%, sys=5.27%, ctx=38, majf=0, minf=32 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26841: Thu May 1 22:27:02 2025 write: IOPS=1, BW=5498KiB/s (5630kB/s)(12.0MiB/2235msec) clat (msec): min=62, max=1179, avg=675.83, stdev=566.52 lat (msec): min=62, max=1179, avg=676.14, stdev=566.54 clat percentiles (msec): | 1.00th=[ 63], 5.00th=[ 63], 10.00th=[ 63], 20.00th=[ 63], | 30.00th=[ 63], 40.00th=[ 785], 50.00th=[ 785], 60.00th=[ 785], | 70.00th=[ 1183], 80.00th=[ 1183], 90.00th=[ 1183], 95.00th=[ 1183], | 99.00th=[ 1183], 99.50th=[ 1183], 99.90th=[ 1183], 99.95th=[ 1183], | 99.99th=[ 1183] bw ( KiB/s): min= 8192, max= 8192, per=55.88%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 100=33.33%, 1000=33.33% cpu : usr=0.04%, sys=7.12%, ctx=81, majf=0, minf=31 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=26843: Thu May 1 22:27:02 2025 write: IOPS=2, BW=9177KiB/s (9397kB/s)(12.0MiB/1339msec) clat (msec): min=253, max=701, avg=416.91, stdev=247.55 lat (msec): min=253, max=701, avg=418.55, stdev=246.55 clat percentiles (msec): | 1.00th=[ 253], 5.00th=[ 253], 10.00th=[ 253], 20.00th=[ 253], | 30.00th=[ 253], 40.00th=[ 296], 50.00th=[ 296], 60.00th=[ 296], | 70.00th=[ 701], 80.00th=[ 701], 90.00th=[ 701], 95.00th=[ 701], | 99.00th=[ 701], 99.50th=[ 701], 99.90th=[ 701], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 8062, max= 8143, per=55.27%, avg=8102.50, stdev=57.28, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.00%, sys=4.04%, ctx=28, majf=0, minf=28 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7430KiB/s (7609kB/s), 1928KiB/s-5573KiB/s (1974kB/s-5707kB/s), io=16.0MiB (16.8MB), run=2125-2205msec WRITE: bw=14.3MiB/s (15.0MB/s), 3855KiB/s-9177KiB/s (3948kB/s-9397kB/s), io=32.0MiB (33.6MB), run=1339-2235msec