== sanity test 398b: DIO and buffer IO race ============== 03:23:08 (1753428188) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 3.10127 s, 16.2 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=156612: Fri Jul 25 03:24:07 2025 read: IOPS=30, BW=123KiB/s (126kB/s)(5992KiB/48793msec) clat (usec): min=99, max=824460, avg=16099.77, stdev=23810.57 lat (usec): min=100, max=824461, avg=16100.72, stdev=23810.57 clat percentiles (usec): | 1.00th=[ 163], 5.00th=[ 383], 10.00th=[ 5735], 20.00th=[ 8455], | 30.00th=[ 9765], 40.00th=[ 11207], 50.00th=[ 12911], 60.00th=[ 15401], | 70.00th=[ 18744], 80.00th=[ 22414], 90.00th=[ 27919], 95.00th=[ 33424], | 99.00th=[ 56886], 99.50th=[ 71828], 99.90th=[145753], 99.95th=[826278], | 99.99th=[826278] bw ( KiB/s): min= 8, max= 184, per=25.36%, avg=122.47, stdev=36.15, samples=97 iops : min= 2, max= 46, avg=30.31, stdev= 9.10, samples=97 write: IOPS=32, BW=129KiB/s (132kB/s)(6296KiB/48793msec); 0 zone resets clat (usec): min=761, max=136118, avg=15539.47, stdev=14377.56 lat (usec): min=761, max=136119, avg=15540.72, stdev=14377.64 clat percentiles (usec): | 1.00th=[ 898], 5.00th=[ 1319], 10.00th=[ 2073], 20.00th=[ 2802], | 30.00th=[ 4490], 40.00th=[ 8717], 50.00th=[ 12125], 60.00th=[ 15795], | 70.00th=[ 20055], 80.00th=[ 26346], 90.00th=[ 33817], 95.00th=[ 42730], | 99.00th=[ 61080], 99.50th=[ 70779], 99.90th=[117965], 99.95th=[135267], | 99.99th=[135267] bw ( KiB/s): min= 63, max= 200, per=26.03%, avg=131.43, stdev=33.03, samples=95 iops : min= 15, max= 50, avg=32.56, stdev= 8.23, samples=95 lat (usec) : 100=0.03%, 250=0.98%, 500=2.54%, 750=0.55%, 1000=1.33% lat (msec) : 2=3.55%, 4=9.51%, 10=18.98%, 20=34.21%, 50=26.14% lat (msec) : 100=2.02%, 250=0.13%, 1000=0.03% cpu : usr=0.17%, sys=23.19%, ctx=7408, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156615: Fri Jul 25 03:24:07 2025 read: IOPS=30, BW=121KiB/s (124kB/s)(5940KiB/48931msec) clat (usec): min=87, max=118833, avg=16009.08, stdev=10941.21 lat (usec): min=87, max=118834, avg=16009.91, stdev=10941.24 clat percentiles (usec): | 1.00th=[ 167], 5.00th=[ 416], 10.00th=[ 6587], 20.00th=[ 8717], | 30.00th=[ 10159], 40.00th=[ 11600], 50.00th=[ 13435], 60.00th=[ 15926], | 70.00th=[ 19006], 80.00th=[ 22938], 90.00th=[ 28181], 95.00th=[ 33817], | 99.00th=[ 53216], 99.50th=[ 65274], 99.90th=[ 98042], 99.95th=[119014], | 99.99th=[119014] bw ( KiB/s): min= 7, max= 208, per=25.32%, avg=122.30, stdev=31.32, samples=96 iops : min= 1, max= 52, avg=30.27, stdev= 7.79, samples=96 write: IOPS=32, BW=130KiB/s (133kB/s)(6348KiB/48931msec); 0 zone resets clat (usec): min=624, max=824406, avg=15712.58, stdev=24157.22 lat (usec): min=625, max=824407, avg=15713.81, stdev=24157.33 clat percentiles (usec): | 1.00th=[ 955], 5.00th=[ 1450], 10.00th=[ 2180], 20.00th=[ 3130], | 30.00th=[ 5014], 40.00th=[ 9372], 50.00th=[ 12518], 60.00th=[ 15533], | 70.00th=[ 19792], 80.00th=[ 25035], 90.00th=[ 33424], 95.00th=[ 41157], | 99.00th=[ 58459], 99.50th=[ 62129], 99.90th=[ 93848], 99.95th=[826278], | 99.99th=[826278] bw ( KiB/s): min= 7, max= 223, per=25.54%, avg=128.97, stdev=37.63, samples=97 iops : min= 1, max= 55, avg=31.94, stdev= 9.42, samples=97 lat (usec) : 100=0.03%, 250=0.85%, 500=1.99%, 750=0.42%, 1000=0.81% lat (msec) : 2=3.74%, 4=9.24%, 10=18.75%, 20=35.94%, 50=26.60% lat (msec) : 100=1.56%, 250=0.03%, 1000=0.03% cpu : usr=0.16%, sys=23.64%, ctx=7525, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156617: Fri Jul 25 03:24:07 2025 read: IOPS=30, BW=123KiB/s (126kB/s)(6028KiB/48841msec) clat (usec): min=89, max=837543, avg=16056.55, stdev=23378.19 lat (usec): min=90, max=838186, avg=16057.98, stdev=23393.18 clat percentiles (usec): | 1.00th=[ 151], 5.00th=[ 363], 10.00th=[ 6390], 20.00th=[ 8586], | 30.00th=[ 10028], 40.00th=[ 11731], 50.00th=[ 13566], 60.00th=[ 15664], | 70.00th=[ 18482], 80.00th=[ 21890], 90.00th=[ 28443], 95.00th=[ 33424], | 99.00th=[ 49021], 99.50th=[ 54789], 99.90th=[ 82314], 99.95th=[834667], | 99.99th=[834667] bw ( KiB/s): min= 8, max= 207, per=25.70%, avg=124.15, stdev=30.14, samples=96 iops : min= 2, max= 51, avg=30.74, stdev= 7.50, samples=96 write: IOPS=32, BW=128KiB/s (131kB/s)(6260KiB/48841msec); 0 zone resets clat (usec): min=667, max=224286, avg=15605.69, stdev=14660.87 lat (usec): min=668, max=224288, avg=15607.26, stdev=14661.05 clat percentiles (usec): | 1.00th=[ 955], 5.00th=[ 1647], 10.00th=[ 2245], 20.00th=[ 3064], | 30.00th=[ 4686], 40.00th=[ 9372], 50.00th=[ 12518], 60.00th=[ 16057], | 70.00th=[ 20317], 80.00th=[ 25297], 90.00th=[ 33817], 95.00th=[ 41681], | 99.00th=[ 62653], 99.50th=[ 70779], 99.90th=[112722], 99.95th=[223347], | 99.99th=[223347] bw ( KiB/s): min= 23, max= 192, per=25.49%, avg=128.72, stdev=30.04, samples=96 iops : min= 5, max= 48, avg=31.87, stdev= 7.51, samples=96 lat (usec) : 100=0.07%, 250=0.88%, 500=2.25%, 750=0.72%, 1000=0.62% lat (msec) : 2=3.26%, 4=10.03%, 10=18.23%, 20=35.94%, 50=26.46% lat (msec) : 100=1.46%, 250=0.07%, 1000=0.03% cpu : usr=0.16%, sys=23.50%, ctx=7572, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156619: Fri Jul 25 03:24:07 2025 read: IOPS=30, BW=122KiB/s (125kB/s)(6052KiB/49697msec) clat (usec): min=80, max=157753, avg=15737.78, stdev=11081.84 lat (usec): min=80, max=157754, avg=15738.64, stdev=11081.88 clat percentiles (usec): | 1.00th=[ 184], 5.00th=[ 367], 10.00th=[ 6194], 20.00th=[ 8455], | 30.00th=[ 9896], 40.00th=[ 11338], 50.00th=[ 13173], 60.00th=[ 15795], | 70.00th=[ 19006], 80.00th=[ 22414], 90.00th=[ 28967], 95.00th=[ 34341], | 99.00th=[ 49546], 99.50th=[ 60031], 99.90th=[101188], 99.95th=[158335], | 99.99th=[158335] bw ( KiB/s): min= 48, max= 240, per=25.68%, avg=124.03, stdev=33.43, samples=97 iops : min= 12, max= 60, avg=30.71, stdev= 8.40, samples=97 write: IOPS=31, BW=125KiB/s (128kB/s)(6236KiB/49697msec); 0 zone resets clat (usec): min=737, max=883732, avg=16459.84, stdev=26391.65 lat (usec): min=738, max=883733, avg=16461.05, stdev=26391.67 clat percentiles (usec): | 1.00th=[ 906], 5.00th=[ 1450], 10.00th=[ 2147], 20.00th=[ 2999], | 30.00th=[ 4555], 40.00th=[ 9110], 50.00th=[ 12649], 60.00th=[ 16319], | 70.00th=[ 20579], 80.00th=[ 26346], 90.00th=[ 34341], 95.00th=[ 43779], | 99.00th=[ 67634], 99.50th=[ 71828], 99.90th=[141558], 99.95th=[884999], | 99.99th=[884999] bw ( KiB/s): min= 63, max= 248, per=25.28%, avg=127.66, stdev=36.19, samples=97 iops : min= 15, max= 62, avg=31.62, stdev= 9.12, samples=97 lat (usec) : 100=0.03%, 250=1.01%, 500=2.34%, 750=0.36%, 1000=1.17% lat (msec) : 2=3.55%, 4=9.47%, 10=18.95%, 20=34.05%, 50=27.18% lat (msec) : 100=1.73%, 250=0.13%, 1000=0.03% cpu : usr=0.15%, sys=22.98%, ctx=7634, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=483KiB/s (495kB/s), 121KiB/s-123KiB/s (124kB/s-126kB/s), io=23.4MiB (24.6MB), run=48793-49697msec WRITE: bw=506KiB/s (518kB/s), 125KiB/s-130KiB/s (128kB/s-133kB/s), io=24.6MiB (25.7MB), run=48793-49697msec rand-rw: (groupid=0, jobs=1): err= 0: pid=156613: Fri Jul 25 03:24:51 2025 read: IOPS=15, BW=63.8KiB/s (65.4kB/s)(5992KiB/93854msec) clat (msec): min=3, max=840, avg=15.76, stdev=25.13 lat (msec): min=3, max=840, avg=15.76, stdev=25.13 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 21], 90.00th=[ 30], 95.00th=[ 41], | 99.00th=[ 70], 99.50th=[ 85], 99.90th=[ 150], 99.95th=[ 844], | 99.99th=[ 844] bw ( KiB/s): min= 8, max= 168, per=25.24%, avg=64.37, stdev=34.16, samples=185 iops : min= 2, max= 42, avg=15.92, stdev= 8.56, samples=185 write: IOPS=16, BW=67.1KiB/s (68.7kB/s)(6296KiB/93854msec); 0 zone resets clat (msec): min=15, max=230, avg=44.49, stdev=19.88 lat (msec): min=15, max=230, avg=44.49, stdev=19.88 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 24], 10.00th=[ 27], 20.00th=[ 30], | 30.00th=[ 32], 40.00th=[ 35], 50.00th=[ 39], 60.00th=[ 44], | 70.00th=[ 49], 80.00th=[ 59], 90.00th=[ 73], 95.00th=[ 84], | 99.00th=[ 110], 99.50th=[ 118], 99.90th=[ 161], 99.95th=[ 230], | 99.99th=[ 230] bw ( KiB/s): min= 15, max= 136, per=25.29%, avg=67.53, stdev=25.67, samples=185 iops : min= 3, max= 34, avg=16.71, stdev= 6.50, samples=185 lat (msec) : 4=0.26%, 10=21.68%, 20=17.32%, 50=44.82%, 100=14.97% lat (msec) : 250=0.91%, 1000=0.03% cpu : usr=0.07%, sys=6.46%, ctx=4136, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156614: Fri Jul 25 03:24:51 2025 read: IOPS=15, BW=63.9KiB/s (65.4kB/s)(5940KiB/93011msec) clat (msec): min=3, max=834, avg=15.32, stdev=24.97 lat (msec): min=3, max=834, avg=15.33, stdev=24.97 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 14], | 70.00th=[ 17], 80.00th=[ 21], 90.00th=[ 30], 95.00th=[ 37], | 99.00th=[ 64], 99.50th=[ 83], 99.90th=[ 153], 99.95th=[ 835], | 99.99th=[ 835] bw ( KiB/s): min= 7, max= 176, per=25.28%, avg=64.47, stdev=35.12, samples=183 iops : min= 1, max= 44, avg=15.95, stdev= 8.83, samples=183 write: IOPS=17, BW=68.2KiB/s (69.9kB/s)(6348KiB/93011msec); 0 zone resets clat (msec): min=15, max=163, avg=44.13, stdev=20.09 lat (msec): min=15, max=163, avg=44.13, stdev=20.09 clat percentiles (msec): | 1.00th=[ 21], 5.00th=[ 25], 10.00th=[ 27], 20.00th=[ 29], | 30.00th=[ 32], 40.00th=[ 35], 50.00th=[ 39], 60.00th=[ 43], | 70.00th=[ 48], 80.00th=[ 58], 90.00th=[ 72], 95.00th=[ 84], | 99.00th=[ 117], 99.50th=[ 144], 99.90th=[ 163], 99.95th=[ 163], | 99.99th=[ 163] bw ( KiB/s): min= 23, max= 128, per=25.73%, avg=68.71, stdev=26.66, samples=183 iops : min= 5, max= 32, avg=17.01, stdev= 6.74, samples=183 lat (msec) : 4=0.33%, 10=22.10%, 20=16.31%, 50=45.80%, 100=14.39% lat (msec) : 250=1.04%, 1000=0.03% cpu : usr=0.09%, sys=6.32%, ctx=4066, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156616: Fri Jul 25 03:24:51 2025 read: IOPS=16, BW=64.3KiB/s (65.8kB/s)(6028KiB/93791msec) clat (msec): min=3, max=156, avg=15.21, stdev=12.58 lat (msec): min=3, max=156, avg=15.21, stdev=12.58 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 10], 50.00th=[ 12], 60.00th=[ 15], | 70.00th=[ 18], 80.00th=[ 22], 90.00th=[ 30], 95.00th=[ 37], | 99.00th=[ 62], 99.50th=[ 73], 99.90th=[ 148], 99.95th=[ 157], | 99.99th=[ 157] bw ( KiB/s): min= 8, max= 160, per=25.57%, avg=65.20, stdev=33.27, samples=184 iops : min= 2, max= 40, avg=16.12, stdev= 8.35, samples=184 write: IOPS=16, BW=66.7KiB/s (68.3kB/s)(6260KiB/93791msec); 0 zone resets clat (msec): min=15, max=860, avg=45.15, stdev=29.68 lat (msec): min=15, max=860, avg=45.15, stdev=29.68 clat percentiles (msec): | 1.00th=[ 21], 5.00th=[ 25], 10.00th=[ 27], 20.00th=[ 29], | 30.00th=[ 32], 40.00th=[ 35], 50.00th=[ 39], 60.00th=[ 43], | 70.00th=[ 48], 80.00th=[ 59], 90.00th=[ 74], 95.00th=[ 88], | 99.00th=[ 118], 99.50th=[ 140], 99.90th=[ 236], 99.95th=[ 860], | 99.99th=[ 860] bw ( KiB/s): min= 15, max= 136, per=25.19%, avg=67.27, stdev=27.20, samples=185 iops : min= 3, max= 34, avg=16.64, stdev= 6.88, samples=185 lat (msec) : 4=0.42%, 10=20.87%, 20=17.12%, 50=46.71%, 100=13.48% lat (msec) : 250=1.37%, 1000=0.03% cpu : usr=0.09%, sys=6.40%, ctx=4200, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156618: Fri Jul 25 03:24:51 2025 read: IOPS=16, BW=65.2KiB/s (66.8kB/s)(6052KiB/92765msec) clat (msec): min=3, max=125, avg=15.24, stdev=13.24 lat (msec): min=3, max=125, avg=15.24, stdev=13.24 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 12], 60.00th=[ 14], | 70.00th=[ 18], 80.00th=[ 21], 90.00th=[ 29], 95.00th=[ 40], | 99.00th=[ 75], 99.50th=[ 89], 99.90th=[ 124], 99.95th=[ 126], | 99.99th=[ 126] bw ( KiB/s): min= 7, max= 159, per=25.83%, avg=65.87, stdev=33.33, samples=183 iops : min= 1, max= 39, avg=16.30, stdev= 8.41, samples=183 write: IOPS=16, BW=67.2KiB/s (68.8kB/s)(6236KiB/92765msec); 0 zone resets clat (msec): min=18, max=879, avg=44.57, stdev=28.99 lat (msec): min=18, max=879, avg=44.58, stdev=28.99 clat percentiles (msec): | 1.00th=[ 22], 5.00th=[ 25], 10.00th=[ 27], 20.00th=[ 30], | 30.00th=[ 32], 40.00th=[ 35], 50.00th=[ 39], 60.00th=[ 43], | 70.00th=[ 48], 80.00th=[ 57], 90.00th=[ 71], 95.00th=[ 85], | 99.00th=[ 107], 99.50th=[ 123], 99.90th=[ 234], 99.95th=[ 877], | 99.99th=[ 877] bw ( KiB/s): min= 16, max= 120, per=25.41%, avg=67.84, stdev=25.66, samples=183 iops : min= 4, max= 30, avg=16.79, stdev= 6.47, samples=183 lat (msec) : 4=0.10%, 10=22.46%, 20=16.57%, 50=45.96%, 100=13.77% lat (msec) : 250=1.11%, 1000=0.03% cpu : usr=0.10%, sys=6.42%, ctx=4139, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=256KiB/s (262kB/s), 63.8KiB/s-65.2KiB/s (65.4kB/s-66.8kB/s), io=23.4MiB (24.6MB), run=92765-93854msec WRITE: bw=268KiB/s (274kB/s), 66.7KiB/s-68.2KiB/s (68.3kB/s-69.9kB/s), io=24.6MiB (25.7MB), run=92765-93854msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=156646: Fri Jul 25 03:25:07 2025 read: IOPS=34, BW=547KiB/s (560kB/s)(5920KiB/10825msec) clat (usec): min=132, max=100680, avg=14622.80, stdev=10357.25 lat (usec): min=133, max=100681, avg=14623.87, stdev=10357.21 clat percentiles (usec): | 1.00th=[ 221], 5.00th=[ 326], 10.00th=[ 400], 20.00th=[ 8979], | 30.00th=[ 10552], 40.00th=[ 12518], 50.00th=[ 13960], 60.00th=[ 15795], | 70.00th=[ 17957], 80.00th=[ 20317], 90.00th=[ 24249], 95.00th=[ 32113], | 99.00th=[ 44303], 99.50th=[ 61080], 99.90th=[101188], 99.95th=[101188], | 99.99th=[101188] bw ( KiB/s): min= 352, max= 864, per=25.50%, avg=548.57, stdev=137.61, samples=21 iops : min= 22, max= 54, avg=33.81, stdev= 8.63, samples=21 write: IOPS=36, BW=588KiB/s (602kB/s)(6368KiB/10825msec); 0 zone resets clat (usec): min=1292, max=105060, avg=13413.59, stdev=11862.31 lat (usec): min=1293, max=105062, avg=13415.09, stdev=11862.34 clat percentiles (usec): | 1.00th=[ 1450], 5.00th=[ 2638], 10.00th=[ 3720], 20.00th=[ 4490], | 30.00th=[ 5276], 40.00th=[ 7177], 50.00th=[ 10552], 60.00th=[ 12780], | 70.00th=[ 15401], 80.00th=[ 20579], 90.00th=[ 26608], 95.00th=[ 34866], | 99.00th=[ 53216], 99.50th=[ 72877], 99.90th=[105382], 99.95th=[105382], | 99.99th=[105382] bw ( KiB/s): min= 352, max= 862, per=26.22%, avg=593.38, stdev=128.74, samples=21 iops : min= 22, max= 53, avg=36.62, stdev= 8.01, samples=21 lat (usec) : 250=0.65%, 500=5.86%, 750=1.30%, 1000=0.13% lat (msec) : 2=1.56%, 4=6.64%, 10=22.40%, 20=39.71%, 50=20.83% lat (msec) : 100=0.65%, 250=0.26% cpu : usr=0.17%, sys=30.78%, ctx=2129, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156647: Fri Jul 25 03:25:07 2025 read: IOPS=32, BW=516KiB/s (528kB/s)(5744KiB/11131msec) clat (usec): min=137, max=80984, avg=15637.95, stdev=10423.50 lat (usec): min=139, max=80985, avg=15638.94, stdev=10423.53 clat percentiles (usec): | 1.00th=[ 208], 5.00th=[ 363], 10.00th=[ 474], 20.00th=[ 9634], | 30.00th=[10945], 40.00th=[12649], 50.00th=[14877], 60.00th=[16450], | 70.00th=[19006], 80.00th=[22414], 90.00th=[26608], 95.00th=[30802], | 99.00th=[54789], 99.50th=[62653], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 318, max= 704, per=23.62%, avg=508.14, stdev=120.88, samples=21 iops : min= 19, max= 44, avg=31.29, stdev= 7.73, samples=21 write: IOPS=36, BW=588KiB/s (602kB/s)(6544KiB/11131msec); 0 zone resets clat (usec): min=1230, max=64876, avg=13330.80, stdev=10749.62 lat (usec): min=1232, max=64878, avg=13332.71, stdev=10749.64 clat percentiles (usec): | 1.00th=[ 1336], 5.00th=[ 2638], 10.00th=[ 3359], 20.00th=[ 4228], | 30.00th=[ 5014], 40.00th=[ 7046], 50.00th=[10421], 60.00th=[13173], | 70.00th=[16712], 80.00th=[22152], 90.00th=[28705], 95.00th=[34341], | 99.00th=[45351], 99.50th=[53740], 99.90th=[64750], 99.95th=[64750], | 99.99th=[64750] bw ( KiB/s): min= 381, max= 864, per=25.35%, avg=573.67, stdev=144.71, samples=21 iops : min= 23, max= 54, avg=35.38, stdev= 9.07, samples=21 lat (usec) : 250=0.78%, 500=4.30%, 750=0.91%, 1000=0.26% lat (msec) : 2=2.08%, 4=7.16%, 10=21.61%, 20=37.11%, 50=24.61% lat (msec) : 100=1.17% cpu : usr=0.14%, sys=31.08%, ctx=2210, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156648: Fri Jul 25 03:25:07 2025 read: IOPS=35, BW=573KiB/s (587kB/s)(6256KiB/10919msec) clat (usec): min=90, max=88240, avg=15134.82, stdev=11277.02 lat (usec): min=92, max=88241, avg=15136.84, stdev=11276.78 clat percentiles (usec): | 1.00th=[ 145], 5.00th=[ 338], 10.00th=[ 424], 20.00th=[ 9241], | 30.00th=[10290], 40.00th=[11600], 50.00th=[13435], 60.00th=[15795], | 70.00th=[18220], 80.00th=[21103], 90.00th=[26608], 95.00th=[33424], | 99.00th=[55313], 99.50th=[76022], 99.90th=[88605], 99.95th=[88605], | 99.99th=[88605] bw ( KiB/s): min= 349, max= 862, per=26.85%, avg=577.52, stdev=141.48, samples=21 iops : min= 21, max= 53, avg=35.62, stdev= 8.81, samples=21 write: IOPS=34, BW=552KiB/s (566kB/s)(6032KiB/10919msec); 0 zone resets clat (usec): min=1093, max=71401, avg=13177.86, stdev=11089.95 lat (usec): min=1095, max=71402, avg=13179.63, stdev=11089.95 clat percentiles (usec): | 1.00th=[ 1450], 5.00th=[ 2900], 10.00th=[ 3589], 20.00th=[ 4228], | 30.00th=[ 4883], 40.00th=[ 6718], 50.00th=[10945], 60.00th=[13042], | 70.00th=[15664], 80.00th=[20841], 90.00th=[28181], 95.00th=[36963], | 99.00th=[50070], 99.50th=[56886], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min= 318, max= 798, per=24.25%, avg=548.86, stdev=139.72, samples=21 iops : min= 19, max= 49, avg=33.86, stdev= 8.77, samples=21 lat (usec) : 100=0.13%, 250=0.78%, 500=5.47%, 750=0.78% lat (msec) : 2=1.56%, 4=6.64%, 10=21.09%, 20=40.49%, 50=22.01% lat (msec) : 100=1.04% cpu : usr=0.17%, sys=30.25%, ctx=2224, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156649: Fri Jul 25 03:25:07 2025 read: IOPS=34, BW=549KiB/s (562kB/s)(6032KiB/10985msec) clat (usec): min=138, max=132795, avg=14902.96, stdev=11365.11 lat (usec): min=139, max=132796, avg=14903.93, stdev=11365.13 clat percentiles (usec): | 1.00th=[ 188], 5.00th=[ 310], 10.00th=[ 400], 20.00th=[ 8717], | 30.00th=[ 10159], 40.00th=[ 12125], 50.00th=[ 13829], 60.00th=[ 16057], | 70.00th=[ 18482], 80.00th=[ 21103], 90.00th=[ 25297], 95.00th=[ 30540], | 99.00th=[ 46924], 99.50th=[ 60031], 99.90th=[132645], 99.95th=[132645], | 99.99th=[132645] bw ( KiB/s): min= 288, max= 766, per=25.43%, avg=547.05, stdev=122.15, samples=21 iops : min= 18, max= 47, avg=33.71, stdev= 7.62, samples=21 write: IOPS=35, BW=570KiB/s (583kB/s)(6256KiB/10985msec); 0 zone resets clat (usec): min=1080, max=69747, avg=13588.20, stdev=11150.29 lat (usec): min=1081, max=69748, avg=13589.87, stdev=11150.30 clat percentiles (usec): | 1.00th=[ 1549], 5.00th=[ 2999], 10.00th=[ 3392], 20.00th=[ 4228], | 30.00th=[ 5080], 40.00th=[ 7046], 50.00th=[10814], 60.00th=[13698], | 70.00th=[16909], 80.00th=[21627], 90.00th=[28181], 95.00th=[33817], | 99.00th=[51643], 99.50th=[69731], 99.90th=[69731], 99.95th=[69731], | 99.99th=[69731] bw ( KiB/s): min= 320, max= 800, per=25.28%, avg=572.10, stdev=113.06, samples=21 iops : min= 20, max= 50, avg=35.29, stdev= 7.17, samples=21 lat (usec) : 250=1.30%, 500=5.47%, 750=0.65%, 1000=0.26% lat (msec) : 2=1.43%, 4=7.16%, 10=22.14%, 20=37.37%, 50=23.18% lat (msec) : 100=0.91%, 250=0.13% cpu : usr=0.12%, sys=31.04%, ctx=2167, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2152KiB/s (2203kB/s), 516KiB/s-573KiB/s (528kB/s-587kB/s), io=23.4MiB (24.5MB), run=10825-11131msec WRITE: bw=2264KiB/s (2318kB/s), 552KiB/s-588KiB/s (566kB/s-602kB/s), io=24.6MiB (25.8MB), run=10825-11131msec rand-rw: (groupid=0, jobs=1): err= 0: pid=156651: Fri Jul 25 03:25:18 2025 read: IOPS=16, BW=267KiB/s (273kB/s)(5920KiB/22204msec) clat (usec): min=3793, max=95789, avg=18072.97, stdev=14303.05 lat (usec): min=3794, max=95789, avg=18073.99, stdev=14302.97 clat percentiles (usec): | 1.00th=[ 4047], 5.00th=[ 4948], 10.00th=[ 5604], 20.00th=[ 7242], | 30.00th=[ 9896], 40.00th=[12649], 50.00th=[15008], 60.00th=[17171], | 70.00th=[19268], 80.00th=[23987], 90.00th=[33817], 95.00th=[46400], | 99.00th=[86508], 99.50th=[87557], 99.90th=[95945], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 96, max= 544, per=24.84%, avg=266.27, stdev=104.23, samples=44 iops : min= 6, max= 34, avg=16.43, stdev= 6.57, samples=44 write: IOPS=17, BW=287KiB/s (294kB/s)(6368KiB/22204msec); 0 zone resets clat (msec): min=16, max=150, avg=38.90, stdev=17.35 lat (msec): min=16, max=150, avg=38.90, stdev=17.35 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 23], 10.00th=[ 25], 20.00th=[ 27], | 30.00th=[ 30], 40.00th=[ 32], 50.00th=[ 34], 60.00th=[ 37], | 70.00th=[ 42], 80.00th=[ 48], 90.00th=[ 61], 95.00th=[ 75], | 99.00th=[ 109], 99.50th=[ 113], 99.90th=[ 150], 99.95th=[ 150], | 99.99th=[ 150] bw ( KiB/s): min= 63, max= 512, per=25.44%, avg=286.91, stdev=112.14, samples=44 iops : min= 3, max= 32, avg=17.70, stdev= 7.23, samples=44 lat (msec) : 4=0.26%, 10=14.32%, 20=21.09%, 50=54.30%, 100=9.24% lat (msec) : 250=0.78% cpu : usr=0.07%, sys=8.81%, ctx=1222, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156652: Fri Jul 25 03:25:18 2025 read: IOPS=16, BW=257KiB/s (263kB/s)(5744KiB/22325msec) clat (usec): min=3415, max=92769, avg=17190.90, stdev=14629.65 lat (usec): min=3416, max=92771, avg=17191.98, stdev=14629.59 clat percentiles (usec): | 1.00th=[ 4015], 5.00th=[ 4621], 10.00th=[ 5145], 20.00th=[ 6652], | 30.00th=[ 8455], 40.00th=[10945], 50.00th=[13829], 60.00th=[16319], | 70.00th=[18744], 80.00th=[22152], 90.00th=[30540], 95.00th=[48497], | 99.00th=[76022], 99.50th=[84411], 99.90th=[92799], 99.95th=[92799], | 99.99th=[92799] bw ( KiB/s): min= 52, max= 544, per=23.83%, avg=255.50, stdev=128.95, samples=44 iops : min= 3, max= 34, avg=15.75, stdev= 8.21, samples=44 write: IOPS=18, BW=293KiB/s (300kB/s)(6544KiB/22325msec); 0 zone resets clat (msec): min=16, max=132, avg=39.24, stdev=17.88 lat (msec): min=16, max=132, avg=39.24, stdev=17.88 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 22], 10.00th=[ 24], 20.00th=[ 26], | 30.00th=[ 29], 40.00th=[ 32], 50.00th=[ 35], 60.00th=[ 39], | 70.00th=[ 43], 80.00th=[ 50], 90.00th=[ 65], 95.00th=[ 80], | 99.00th=[ 104], 99.50th=[ 113], 99.90th=[ 133], 99.95th=[ 133], | 99.99th=[ 133] bw ( KiB/s): min= 96, max= 544, per=25.80%, avg=291.00, stdev=113.67, samples=44 iops : min= 6, max= 34, avg=17.98, stdev= 7.27, samples=44 lat (msec) : 4=0.39%, 10=17.45%, 20=17.58%, 50=52.60%, 100=11.20% lat (msec) : 250=0.78% cpu : usr=0.12%, sys=8.72%, ctx=1231, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156653: Fri Jul 25 03:25:18 2025 read: IOPS=18, BW=289KiB/s (296kB/s)(6256KiB/21655msec) clat (usec): min=3779, max=95795, avg=16751.47, stdev=13301.18 lat (usec): min=3780, max=95796, avg=16752.72, stdev=13301.08 clat percentiles (usec): | 1.00th=[ 4293], 5.00th=[ 5145], 10.00th=[ 5538], 20.00th=[ 6718], | 30.00th=[ 8717], 40.00th=[11207], 50.00th=[13960], 60.00th=[16319], | 70.00th=[19268], 80.00th=[21890], 90.00th=[29230], 95.00th=[41681], | 99.00th=[76022], 99.50th=[92799], 99.90th=[95945], 99.95th=[95945], | 99.99th=[95945] bw ( KiB/s): min= 95, max= 640, per=27.05%, avg=289.93, stdev=140.65, samples=43 iops : min= 5, max= 40, avg=17.88, stdev= 8.97, samples=43 write: IOPS=17, BW=279KiB/s (285kB/s)(6032KiB/21655msec); 0 zone resets clat (msec): min=18, max=124, avg=39.82, stdev=16.78 lat (msec): min=18, max=124, avg=39.82, stdev=16.78 clat percentiles (msec): | 1.00th=[ 22], 5.00th=[ 24], 10.00th=[ 26], 20.00th=[ 28], | 30.00th=[ 30], 40.00th=[ 32], 50.00th=[ 35], 60.00th=[ 38], | 70.00th=[ 44], 80.00th=[ 51], 90.00th=[ 63], 95.00th=[ 78], | 99.00th=[ 102], 99.50th=[ 115], 99.90th=[ 125], 99.95th=[ 125], | 99.99th=[ 125] bw ( KiB/s): min= 127, max= 480, per=24.71%, avg=278.72, stdev=101.32, samples=43 iops : min= 7, max= 30, avg=17.19, stdev= 6.53, samples=43 lat (msec) : 4=0.26%, 10=17.97%, 20=19.01%, 50=51.43%, 100=10.81% lat (msec) : 250=0.52% cpu : usr=0.09%, sys=8.92%, ctx=1205, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156654: Fri Jul 25 03:25:18 2025 read: IOPS=16, BW=272KiB/s (278kB/s)(6032KiB/22194msec) clat (usec): min=3899, max=94569, avg=17014.74, stdev=14353.26 lat (usec): min=3900, max=94570, avg=17015.77, stdev=14353.24 clat percentiles (usec): | 1.00th=[ 4146], 5.00th=[ 4948], 10.00th=[ 5538], 20.00th=[ 6587], | 30.00th=[ 7898], 40.00th=[10028], 50.00th=[13698], 60.00th=[16057], | 70.00th=[19268], 80.00th=[22938], 90.00th=[32113], 95.00th=[47973], | 99.00th=[82314], 99.50th=[89654], 99.90th=[94897], 99.95th=[94897], | 99.99th=[94897] bw ( KiB/s): min= 63, max= 576, per=25.23%, avg=270.48, stdev=127.48, samples=44 iops : min= 3, max= 36, avg=16.68, stdev= 8.18, samples=44 write: IOPS=17, BW=282KiB/s (289kB/s)(6256KiB/22194msec); 0 zone resets clat (msec): min=18, max=121, avg=40.12, stdev=18.04 lat (msec): min=18, max=121, avg=40.12, stdev=18.04 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 23], 10.00th=[ 25], 20.00th=[ 27], | 30.00th=[ 30], 40.00th=[ 33], 50.00th=[ 35], 60.00th=[ 39], | 70.00th=[ 43], 80.00th=[ 51], 90.00th=[ 65], 95.00th=[ 75], | 99.00th=[ 113], 99.50th=[ 120], 99.90th=[ 123], 99.95th=[ 123], | 99.99th=[ 123] bw ( KiB/s): min= 95, max= 480, per=25.04%, avg=282.45, stdev=109.21, samples=44 iops : min= 5, max= 30, avg=17.43, stdev= 6.97, samples=44 lat (msec) : 4=0.39%, 10=19.14%, 20=16.15%, 50=51.69%, 100=11.59% lat (msec) : 250=1.04% cpu : usr=0.06%, sys=8.82%, ctx=1250, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1073KiB/s (1099kB/s), 257KiB/s-289KiB/s (263kB/s-296kB/s), io=23.4MiB (24.5MB), run=21655-22325msec WRITE: bw=1129KiB/s (1156kB/s), 279KiB/s-293KiB/s (285kB/s-300kB/s), io=24.6MiB (25.8MB), run=21655-22325msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=156668: Fri Jul 25 03:25:28 2025 read: IOPS=2, BW=2164KiB/s (2216kB/s)(7168KiB/3312msec) clat (usec): min=856, max=1161.9k, avg=330305.45, stdev=384605.30 lat (usec): min=858, max=1161.9k, avg=330306.89, stdev=384605.23 clat percentiles (usec): | 1.00th=[ 857], 5.00th=[ 857], 10.00th=[ 857], | 20.00th=[ 128451], 30.00th=[ 158335], 40.00th=[ 158335], | 50.00th=[ 223347], 60.00th=[ 270533], 70.00th=[ 270533], | 80.00th=[ 371196], 90.00th=[1166017], 95.00th=[1166017], | 99.00th=[1166017], 99.50th=[1166017], 99.90th=[1166017], | 99.95th=[1166017], 99.99th=[1166017] bw ( KiB/s): min= 1914, max= 4096, per=54.35%, avg=2520.25, stdev=1052.11, samples=4 iops : min= 1, max= 4, avg= 2.00, stdev= 1.41, samples=4 write: IOPS=1, BW=1546KiB/s (1583kB/s)(5120KiB/3312msec); 0 zone resets clat (msec): min=129, max=334, avg=186.06, stdev=83.74 lat (msec): min=129, max=334, avg=186.16, stdev=83.76 clat percentiles (msec): | 1.00th=[ 130], 5.00th=[ 130], 10.00th=[ 130], 20.00th=[ 130], | 30.00th=[ 150], 40.00th=[ 150], 50.00th=[ 155], 60.00th=[ 155], | 70.00th=[ 165], 80.00th=[ 165], 90.00th=[ 334], 95.00th=[ 334], | 99.00th=[ 334], 99.50th=[ 334], 99.90th=[ 334], 99.95th=[ 334], | 99.99th=[ 334] bw ( KiB/s): min= 2048, max= 6144, per=63.09%, avg=4096.00, stdev=2896.31, samples=2 iops : min= 2, max= 6, avg= 4.00, stdev= 2.83, samples=2 lat (usec) : 1000=8.33% lat (msec) : 250=58.33%, 500=25.00%, 2000=8.33% cpu : usr=0.00%, sys=36.24%, ctx=962, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156670: Fri Jul 25 03:25:28 2025 read: IOPS=0, BW=464KiB/s (475kB/s)(2048KiB/4416msec) clat (usec): min=1124, max=140406, avg=70765.83, stdev=98487.03 lat (usec): min=1126, max=140409, avg=70767.71, stdev=98487.93 clat percentiles (usec): | 1.00th=[ 1123], 5.00th=[ 1123], 10.00th=[ 1123], 20.00th=[ 1123], | 30.00th=[ 1123], 40.00th=[ 1123], 50.00th=[ 1123], 60.00th=[139461], | 70.00th=[139461], 80.00th=[139461], 90.00th=[139461], 95.00th=[139461], | 99.00th=[139461], 99.50th=[139461], 99.90th=[139461], 99.95th=[139461], | 99.99th=[139461] bw ( KiB/s): min= 1906, max= 2048, per=42.64%, avg=1977.00, stdev=100.41, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2319KiB/s (2374kB/s)(10.0MiB/4416msec); 0 zone resets clat (msec): min=124, max=1468, avg=412.16, stdev=415.17 lat (msec): min=124, max=1468, avg=412.24, stdev=415.16 clat percentiles (msec): | 1.00th=[ 125], 5.00th=[ 125], 10.00th=[ 125], 20.00th=[ 146], | 30.00th=[ 167], 40.00th=[ 218], 50.00th=[ 228], 60.00th=[ 257], | 70.00th=[ 330], 80.00th=[ 426], 90.00th=[ 760], 95.00th=[ 1469], | 99.00th=[ 1469], 99.50th=[ 1469], 99.90th=[ 1469], 99.95th=[ 1469], | 99.99th=[ 1469] bw ( KiB/s): min= 2023, max= 6144, per=63.00%, avg=4089.75, stdev=1682.41, samples=4 iops : min= 1, max= 6, avg= 3.75, stdev= 2.06, samples=4 lat (msec) : 2=8.33%, 250=50.00%, 500=25.00%, 1000=8.33%, 2000=8.33% cpu : usr=0.05%, sys=29.47%, ctx=171, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156672: Fri Jul 25 03:25:28 2025 read: IOPS=1, BW=1423KiB/s (1457kB/s)(6144KiB/4318msec) clat (usec): min=856, max=212342, avg=140873.75, stdev=74964.00 lat (usec): min=858, max=212343, avg=140874.85, stdev=74963.87 clat percentiles (usec): | 1.00th=[ 857], 5.00th=[ 857], 10.00th=[ 857], 20.00th=[133694], | 30.00th=[133694], 40.00th=[145753], 50.00th=[145753], 60.00th=[156238], | 70.00th=[196084], 80.00th=[196084], 90.00th=[212861], 95.00th=[212861], | 99.00th=[212861], 99.50th=[212861], 99.90th=[212861], 99.95th=[212861], | 99.99th=[212861] bw ( KiB/s): min= 4096, max= 6144, per=100.00%, avg=5120.00, stdev=1448.15, samples=2 iops : min= 4, max= 6, avg= 5.00, stdev= 1.41, samples=2 write: IOPS=1, BW=1423KiB/s (1457kB/s)(6144KiB/4318msec); 0 zone resets clat (msec): min=160, max=1750, avg=558.71, stdev=613.31 lat (msec): min=160, max=1750, avg=558.78, stdev=613.31 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 224], | 30.00th=[ 224], 40.00th=[ 262], 50.00th=[ 262], 60.00th=[ 271], | 70.00th=[ 693], 80.00th=[ 693], 90.00th=[ 1754], 95.00th=[ 1754], | 99.00th=[ 1754], 99.50th=[ 1754], 99.90th=[ 1754], 99.95th=[ 1754], | 99.99th=[ 1754] bw ( KiB/s): min= 1921, max= 2048, per=31.06%, avg=2016.60, stdev=54.43, samples=5 iops : min= 1, max= 2, avg= 1.40, stdev= 0.55, samples=5 lat (usec) : 1000=8.33% lat (msec) : 250=58.33%, 500=16.67%, 750=8.33%, 2000=8.33% cpu : usr=0.02%, sys=25.48%, ctx=857, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156674: Fri Jul 25 03:25:28 2025 read: IOPS=1, BW=1435KiB/s (1469kB/s)(5120KiB/3568msec) clat (usec): min=730, max=229368, avg=104258.24, stdev=101659.18 lat (usec): min=730, max=229368, avg=104259.27, stdev=101659.16 clat percentiles (usec): | 1.00th=[ 734], 5.00th=[ 734], 10.00th=[ 734], 20.00th=[ 734], | 30.00th=[ 1012], 40.00th=[ 1012], 50.00th=[123208], 60.00th=[123208], | 70.00th=[166724], 80.00th=[166724], 90.00th=[229639], 95.00th=[229639], | 99.00th=[229639], 99.50th=[229639], 99.90th=[229639], 99.95th=[229639], | 99.99th=[229639] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=1, BW=2009KiB/s (2057kB/s)(7168KiB/3568msec); 0 zone resets clat (msec): min=141, max=1489, avg=421.01, stdev=476.10 lat (msec): min=141, max=1489, avg=421.08, stdev=476.10 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 174], | 30.00th=[ 243], 40.00th=[ 243], 50.00th=[ 251], 60.00th=[ 313], | 70.00th=[ 313], 80.00th=[ 338], 90.00th=[ 1485], 95.00th=[ 1485], | 99.00th=[ 1485], 99.50th=[ 1485], 99.90th=[ 1485], 99.95th=[ 1485], | 99.99th=[ 1485] bw ( KiB/s): min= 1961, max= 6144, per=54.78%, avg=3556.00, stdev=1990.26, samples=4 iops : min= 1, max= 6, avg= 3.00, stdev= 2.45, samples=4 lat (usec) : 750=8.33% lat (msec) : 2=8.33%, 250=58.33%, 500=16.67%, 2000=8.33% cpu : usr=0.00%, sys=34.88%, ctx=504, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4638KiB/s (4749kB/s), 464KiB/s-2164KiB/s (475kB/s-2216kB/s), io=20.0MiB (20.0MB), run=3312-4416msec WRITE: bw=6493KiB/s (6649kB/s), 1423KiB/s-2319KiB/s (1457kB/s-2374kB/s), io=28.0MiB (29.4MB), run=3312-4416msec rand-rw: (groupid=0, jobs=1): err= 0: pid=156667: Fri Jul 25 03:25:28 2025 read: IOPS=2, BW=2221KiB/s (2275kB/s)(7168KiB/3227msec) clat (msec): min=44, max=925, avg=244.97, stdev=309.03 lat (msec): min=44, max=925, avg=244.98, stdev=309.03 clat percentiles (msec): | 1.00th=[ 45], 5.00th=[ 45], 10.00th=[ 45], 20.00th=[ 78], | 30.00th=[ 95], 40.00th=[ 95], 50.00th=[ 123], 60.00th=[ 190], | 70.00th=[ 190], 80.00th=[ 262], 90.00th=[ 927], 95.00th=[ 927], | 99.00th=[ 927], 99.50th=[ 927], 99.90th=[ 927], 99.95th=[ 927], | 99.99th=[ 927] bw ( KiB/s): min= 2043, max= 5495, per=62.27%, avg=2908.50, stdev=1724.33, samples=4 iops : min= 1, max= 5, avg= 2.50, stdev= 1.73, samples=4 write: IOPS=1, BW=1587KiB/s (1625kB/s)(5120KiB/3227msec); 0 zone resets clat (msec): min=178, max=479, avg=300.16, stdev=122.24 lat (msec): min=178, max=479, avg=300.25, stdev=122.24 clat percentiles (msec): | 1.00th=[ 180], 5.00th=[ 180], 10.00th=[ 180], 20.00th=[ 180], | 30.00th=[ 226], 40.00th=[ 226], 50.00th=[ 249], 60.00th=[ 249], | 70.00th=[ 368], 80.00th=[ 368], 90.00th=[ 481], 95.00th=[ 481], | 99.00th=[ 481], 99.50th=[ 481], 99.90th=[ 481], 99.95th=[ 481], | 99.99th=[ 481] bw ( KiB/s): min= 2023, max= 2048, per=31.20%, avg=2040.50, stdev=11.90, samples=4 iops : min= 1, max= 2, avg= 1.50, stdev= 0.58, samples=4 lat (msec) : 50=8.33%, 100=16.67%, 250=41.67%, 500=25.00%, 1000=8.33% cpu : usr=0.00%, sys=13.76%, ctx=197, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156669: Fri Jul 25 03:25:28 2025 read: IOPS=0, BW=467KiB/s (478kB/s)(2048KiB/4384msec) clat (msec): min=72, max=193, avg=133.31, stdev=85.74 lat (msec): min=72, max=193, avg=133.31, stdev=85.74 clat percentiles (msec): | 1.00th=[ 73], 5.00th=[ 73], 10.00th=[ 73], 20.00th=[ 73], | 30.00th=[ 73], 40.00th=[ 73], 50.00th=[ 73], 60.00th=[ 194], | 70.00th=[ 194], 80.00th=[ 194], 90.00th=[ 194], 95.00th=[ 194], | 99.00th=[ 194], 99.50th=[ 194], 99.90th=[ 194], 99.95th=[ 194], | 99.99th=[ 194] bw ( KiB/s): min= 1882, max= 2048, per=42.07%, avg=1965.00, stdev=117.38, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2336KiB/s (2392kB/s)(10.0MiB/4384msec); 0 zone resets clat (msec): min=104, max=1536, avg=410.04, stdev=434.18 lat (msec): min=104, max=1536, avg=410.14, stdev=434.21 clat percentiles (msec): | 1.00th=[ 105], 5.00th=[ 105], 10.00th=[ 105], 20.00th=[ 111], | 30.00th=[ 140], 40.00th=[ 192], 50.00th=[ 222], 60.00th=[ 279], | 70.00th=[ 351], 80.00th=[ 523], 90.00th=[ 642], 95.00th=[ 1536], | 99.00th=[ 1536], 99.50th=[ 1536], 99.90th=[ 1536], 99.95th=[ 1536], | 99.99th=[ 1536] bw ( KiB/s): min= 2023, max= 6144, per=62.48%, avg=4086.50, stdev=2368.32, samples=4 iops : min= 1, max= 6, avg= 3.50, stdev= 2.38, samples=4 lat (msec) : 100=8.33%, 250=50.00%, 500=16.67%, 750=16.67%, 2000=8.33% cpu : usr=0.02%, sys=13.53%, ctx=151, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156671: Fri Jul 25 03:25:28 2025 read: IOPS=1, BW=1486KiB/s (1522kB/s)(6144KiB/4135msec) clat (msec): min=49, max=338, avg=189.82, stdev=126.01 lat (msec): min=49, max=338, avg=189.82, stdev=126.01 clat percentiles (msec): | 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 50], 20.00th=[ 61], | 30.00th=[ 61], 40.00th=[ 129], 50.00th=[ 129], 60.00th=[ 271], | 70.00th=[ 292], 80.00th=[ 292], 90.00th=[ 338], 95.00th=[ 338], | 99.00th=[ 338], 99.50th=[ 338], 99.90th=[ 338], 99.95th=[ 338], | 99.99th=[ 338] bw ( KiB/s): min= 2023, max= 4096, per=54.67%, avg=2553.75, stdev=1028.23, samples=4 iops : min= 1, max= 4, avg= 2.25, stdev= 1.26, samples=4 write: IOPS=1, BW=1486KiB/s (1522kB/s)(6144KiB/4135msec); 0 zone resets clat (msec): min=122, max=1489, avg=489.55, stdev=520.74 lat (msec): min=122, max=1489, avg=489.63, stdev=520.75 clat percentiles (msec): | 1.00th=[ 124], 5.00th=[ 124], 10.00th=[ 124], 20.00th=[ 133], | 30.00th=[ 133], 40.00th=[ 213], 50.00th=[ 213], 60.00th=[ 397], | 70.00th=[ 584], 80.00th=[ 584], 90.00th=[ 1485], 95.00th=[ 1485], | 99.00th=[ 1485], 99.50th=[ 1485], 99.90th=[ 1485], 99.95th=[ 1485], | 99.99th=[ 1485] bw ( KiB/s): min= 1838, max= 2048, per=30.60%, avg=2001.00, stdev=91.76, samples=5 iops : min= 1, max= 2, avg= 1.60, stdev= 0.55, samples=5 lat (msec) : 50=8.33%, 100=8.33%, 250=33.33%, 500=33.33%, 750=8.33% lat (msec) : 2000=8.33% cpu : usr=0.00%, sys=12.75%, ctx=126, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156673: Fri Jul 25 03:25:28 2025 read: IOPS=1, BW=1520KiB/s (1556kB/s)(5120KiB/3369msec) clat (msec): min=66, max=273, avg=135.58, stdev=84.48 lat (msec): min=66, max=273, avg=135.58, stdev=84.49 clat percentiles (msec): | 1.00th=[ 67], 5.00th=[ 67], 10.00th=[ 67], 20.00th=[ 67], | 30.00th=[ 81], 40.00th=[ 81], 50.00th=[ 102], 60.00th=[ 102], | 70.00th=[ 157], 80.00th=[ 157], 90.00th=[ 275], 95.00th=[ 275], | 99.00th=[ 275], 99.50th=[ 275], 99.90th=[ 275], 99.95th=[ 275], | 99.99th=[ 275] bw ( KiB/s): min= 4047, max= 4096, per=87.17%, avg=4071.50, stdev=34.65, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2128KiB/s (2179kB/s)(7168KiB/3369msec); 0 zone resets clat (msec): min=106, max=1508, avg=379.10, stdev=505.37 lat (msec): min=106, max=1508, avg=379.20, stdev=505.37 clat percentiles (msec): | 1.00th=[ 107], 5.00th=[ 107], 10.00th=[ 107], 20.00th=[ 108], | 30.00th=[ 125], 40.00th=[ 125], 50.00th=[ 222], 60.00th=[ 243], | 70.00th=[ 243], 80.00th=[ 342], 90.00th=[ 1502], 95.00th=[ 1502], | 99.00th=[ 1502], 99.50th=[ 1502], 99.90th=[ 1502], 99.95th=[ 1502], | 99.99th=[ 1502] bw ( KiB/s): min= 1841, max= 4087, per=38.22%, avg=2499.75, stdev=1062.18, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 100=16.67%, 250=58.33%, 500=16.67%, 2000=8.33% cpu : usr=0.00%, sys=13.72%, ctx=85, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4672KiB/s (4784kB/s), 467KiB/s-2221KiB/s (478kB/s-2275kB/s), io=20.0MiB (20.0MB), run=3227-4384msec WRITE: bw=6540KiB/s (6697kB/s), 1486KiB/s-2336KiB/s (1522kB/s-2392kB/s), io=28.0MiB (29.4MB), run=3227-4384msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=156680: Fri Jul 25 03:25:39 2025 read: IOPS=1, BW=6946KiB/s (7113kB/s)(12.0MiB/1769msec) clat (msec): min=286, max=1055, avg=558.39, stdev=431.28 lat (msec): min=286, max=1055, avg=558.39, stdev=431.27 clat percentiles (msec): | 1.00th=[ 288], 5.00th=[ 288], 10.00th=[ 288], 20.00th=[ 288], | 30.00th=[ 288], 40.00th=[ 334], 50.00th=[ 334], 60.00th=[ 334], | 70.00th=[ 1053], 80.00th=[ 1053], 90.00th=[ 1053], 95.00th=[ 1053], | 99.00th=[ 1053], 99.50th=[ 1053], 99.90th=[ 1053], 99.95th=[ 1053], | 99.99th=[ 1053] bw ( KiB/s): min= 6543, max= 8192, per=100.00%, avg=7367.50, stdev=1166.02, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=27.77%, ctx=215, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156682: Fri Jul 25 03:25:39 2025 read: IOPS=0, BW=1018KiB/s (1043kB/s)(4096KiB/4022msec) clat (nsec): min=779666k, max=779666k, avg=779666006.00, stdev= 0.00 lat (nsec): min=779668k, max=779668k, avg=779668270.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 776], 5.00th=[ 776], 10.00th=[ 776], 20.00th=[ 776], | 30.00th=[ 776], 40.00th=[ 776], 50.00th=[ 776], 60.00th=[ 776], | 70.00th=[ 776], 80.00th=[ 776], 90.00th=[ 776], 95.00th=[ 776], | 99.00th=[ 776], 99.50th=[ 776], 99.90th=[ 776], 99.95th=[ 776], | 99.99th=[ 776] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2037KiB/s (2086kB/s)(8192KiB/4022msec); 0 zone resets clat (msec): min=1511, max=1658, avg=1585.20, stdev=104.15 lat (msec): min=1511, max=1659, avg=1585.59, stdev=104.21 clat percentiles (msec): | 1.00th=[ 1519], 5.00th=[ 1519], 10.00th=[ 1519], 20.00th=[ 1519], | 30.00th=[ 1519], 40.00th=[ 1519], 50.00th=[ 1519], 60.00th=[ 1653], | 70.00th=[ 1653], 80.00th=[ 1653], 90.00th=[ 1653], 95.00th=[ 1653], | 99.00th=[ 1653], 99.50th=[ 1653], 99.90th=[ 1653], 99.95th=[ 1653], | 99.99th=[ 1653] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.00%, sys=11.69%, ctx=90, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156684: Fri Jul 25 03:25:39 2025 write: IOPS=0, BW=3994KiB/s (4089kB/s)(12.0MiB/3077msec); 0 zone resets clat (msec): min=280, max=1676, avg=1000.70, stdev=699.07 lat (msec): min=280, max=1676, avg=1001.05, stdev=699.20 clat percentiles (msec): | 1.00th=[ 279], 5.00th=[ 279], 10.00th=[ 279], 20.00th=[ 279], | 30.00th=[ 279], 40.00th=[ 1045], 50.00th=[ 1045], 60.00th=[ 1045], | 70.00th=[ 1670], 80.00th=[ 1670], 90.00th=[ 1670], 95.00th=[ 1670], | 99.00th=[ 1670], 99.50th=[ 1670], 99.90th=[ 1670], 99.95th=[ 1670], | 99.99th=[ 1670] bw ( KiB/s): min= 8143, max= 8143, per=99.95%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.00%, sys=17.33%, ctx=234, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156687: Fri Jul 25 03:25:39 2025 write: IOPS=0, BW=3683KiB/s (3772kB/s)(12.0MiB/3336msec); 0 zone resets clat (msec): min=314, max=1506, avg=1093.90, stdev=675.79 lat (msec): min=314, max=1507, avg=1094.39, stdev=675.88 clat percentiles (msec): | 1.00th=[ 313], 5.00th=[ 313], 10.00th=[ 313], 20.00th=[ 313], | 30.00th=[ 313], 40.00th=[ 1469], 50.00th=[ 1469], 60.00th=[ 1469], | 70.00th=[ 1502], 80.00th=[ 1502], 90.00th=[ 1502], 95.00th=[ 1502], | 99.00th=[ 1502], 99.50th=[ 1502], 99.90th=[ 1502], 99.95th=[ 1502], | 99.99th=[ 1502] bw ( KiB/s): min= 8143, max= 8192, per=100.00%, avg=8167.50, stdev=34.65, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.06%, sys=16.61%, ctx=141, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4074KiB/s (4171kB/s), 1018KiB/s-6946KiB/s (1043kB/s-7113kB/s), io=16.0MiB (16.8MB), run=1769-4022msec WRITE: bw=8147KiB/s (8343kB/s), 2037KiB/s-3994KiB/s (2086kB/s-4089kB/s), io=32.0MiB (33.6MB), run=3077-4022msec rand-rw: (groupid=0, jobs=1): err= 0: pid=156681: Fri Jul 25 03:25:39 2025 read: IOPS=1, BW=4375KiB/s (4479kB/s)(12.0MiB/2809msec) clat (msec): min=551, max=1481, avg=920.46, stdev=494.12 lat (msec): min=551, max=1481, avg=920.47, stdev=494.12 clat percentiles (msec): | 1.00th=[ 550], 5.00th=[ 550], 10.00th=[ 550], 20.00th=[ 550], | 30.00th=[ 550], 40.00th=[ 726], 50.00th=[ 726], 60.00th=[ 726], | 70.00th=[ 1485], 80.00th=[ 1485], 90.00th=[ 1485], 95.00th=[ 1485], | 99.00th=[ 1485], 99.50th=[ 1485], 99.90th=[ 1485], 99.95th=[ 1485], | 99.99th=[ 1485] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 750=66.67%, 2000=33.33% cpu : usr=0.00%, sys=29.67%, ctx=1784, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156683: Fri Jul 25 03:25:39 2025 read: IOPS=0, BW=1066KiB/s (1091kB/s)(4096KiB/3844msec) clat (nsec): min=732138k, max=732138k, avg=732138487.00, stdev= 0.00 lat (nsec): min=732140k, max=732140k, avg=732139890.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 735], 5.00th=[ 735], 10.00th=[ 735], 20.00th=[ 735], | 30.00th=[ 735], 40.00th=[ 735], 50.00th=[ 735], 60.00th=[ 735], | 70.00th=[ 735], 80.00th=[ 735], 90.00th=[ 735], 95.00th=[ 735], | 99.00th=[ 735], 99.50th=[ 735], 99.90th=[ 735], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2131KiB/s (2182kB/s)(8192KiB/3844msec); 0 zone resets clat (msec): min=1261, max=1705, avg=1483.74, stdev=313.79 lat (msec): min=1262, max=1705, avg=1484.02, stdev=313.70 clat percentiles (msec): | 1.00th=[ 1267], 5.00th=[ 1267], 10.00th=[ 1267], 20.00th=[ 1267], | 30.00th=[ 1267], 40.00th=[ 1267], 50.00th=[ 1267], 60.00th=[ 1703], | 70.00th=[ 1703], 80.00th=[ 1703], 90.00th=[ 1703], 95.00th=[ 1703], | 99.00th=[ 1703], 99.50th=[ 1703], 99.90th=[ 1703], 99.95th=[ 1703], | 99.99th=[ 1703] bw ( KiB/s): min= 8192, max= 8192, per=99.71%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 2000=66.67% cpu : usr=0.03%, sys=29.01%, ctx=748, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156685: Fri Jul 25 03:25:39 2025 write: IOPS=0, BW=3081KiB/s (3155kB/s)(12.0MiB/3988msec); 0 zone resets clat (msec): min=496, max=1794, avg=1270.73, stdev=684.03 lat (msec): min=497, max=1794, avg=1272.52, stdev=684.84 clat percentiles (msec): | 1.00th=[ 498], 5.00th=[ 498], 10.00th=[ 498], 20.00th=[ 498], | 30.00th=[ 498], 40.00th=[ 1519], 50.00th=[ 1519], 60.00th=[ 1519], | 70.00th=[ 1787], 80.00th=[ 1787], 90.00th=[ 1787], 95.00th=[ 1787], | 99.00th=[ 1787], 99.50th=[ 1787], 99.90th=[ 1787], 99.95th=[ 1787], | 99.99th=[ 1787] bw ( KiB/s): min= 8192, max= 8192, per=99.71%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.05%, sys=33.33%, ctx=86, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=156686: Fri Jul 25 03:25:39 2025 write: IOPS=0, BW=4066KiB/s (4164kB/s)(12.0MiB/3022msec); 0 zone resets clat (msec): min=437, max=1458, avg=948.96, stdev=510.44 lat (msec): min=437, max=1460, avg=950.09, stdev=511.59 clat percentiles (msec): | 1.00th=[ 439], 5.00th=[ 439], 10.00th=[ 439], 20.00th=[ 439], | 30.00th=[ 439], 40.00th=[ 953], 50.00th=[ 953], 60.00th=[ 953], | 70.00th=[ 1452], 80.00th=[ 1452], 90.00th=[ 1452], 95.00th=[ 1452], | 99.00th=[ 1452], 99.50th=[ 1452], 99.90th=[ 1452], 99.95th=[ 1452], | 99.99th=[ 1452] bw ( KiB/s): min= 8192, max= 8192, per=99.71%, avg=8192.00, stdev= 0.00, samples=2 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.03%, sys=49.92%, ctx=227, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4262KiB/s (4365kB/s), 1066KiB/s-4375KiB/s (1091kB/s-4479kB/s), io=16.0MiB (16.8MB), run=2809-3844msec WRITE: bw=8217KiB/s (8414kB/s), 2131KiB/s-4066KiB/s (2182kB/s-4164kB/s), io=32.0MiB (33.6MB), run=3022-3988msec