== sanity test 398b: DIO and buffer IO race ============== 23:50:05 (1763355005) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.93955 s, 17.1 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=183653: Sun Nov 16 23:51:13 2025 read: IOPS=25, BW=103KiB/s (105kB/s)(5992KiB/58229msec) clat (usec): min=288, max=700861, avg=21333.59, stdev=21157.71 lat (usec): min=289, max=700862, avg=21337.12, stdev=21157.46 clat percentiles (usec): | 1.00th=[ 449], 5.00th=[ 7635], 10.00th=[ 9372], 20.00th=[ 11994], | 30.00th=[ 14091], 40.00th=[ 16450], 50.00th=[ 18482], 60.00th=[ 21103], | 70.00th=[ 24511], 80.00th=[ 27919], 90.00th=[ 34866], 95.00th=[ 42730], | 99.00th=[ 65274], 99.50th=[ 74974], 99.90th=[102237], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 7, max= 247, per=24.76%, avg=102.01, stdev=33.40, samples=115 iops : min= 1, max= 61, avg=25.05, stdev= 8.37, samples=115 write: IOPS=27, BW=108KiB/s (111kB/s)(6296KiB/58229msec); 0 zone resets clat (usec): min=804, max=76282, avg=16553.98, stdev=12557.54 lat (usec): min=805, max=76284, avg=16555.97, stdev=12557.20 clat percentiles (usec): | 1.00th=[ 1352], 5.00th=[ 2638], 10.00th=[ 3261], 20.00th=[ 5145], | 30.00th=[ 7963], 40.00th=[10814], 50.00th=[13304], 60.00th=[16450], | 70.00th=[20841], 80.00th=[26084], 90.00th=[35390], 95.00th=[41681], | 99.00th=[54789], 99.50th=[57934], 99.90th=[64750], 99.95th=[76022], | 99.99th=[76022] bw ( KiB/s): min= 47, max= 199, per=24.98%, avg=107.68, stdev=30.46, samples=114 iops : min= 11, max= 49, avg=26.47, stdev= 7.55, samples=114 lat (usec) : 500=0.65%, 750=0.29%, 1000=0.23% lat (msec) : 2=0.98%, 4=6.02%, 10=16.63%, 20=37.40%, 50=35.42% lat (msec) : 100=2.31%, 250=0.03%, 750=0.03% cpu : usr=0.14%, sys=25.17%, ctx=9680, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183654: Sun Nov 16 23:51:13 2025 read: IOPS=25, BW=104KiB/s (106kB/s)(5940KiB/57209msec) clat (usec): min=143, max=112845, avg=20753.11, stdev=10674.68 lat (usec): min=143, max=112847, avg=20754.18, stdev=10674.76 clat percentiles (usec): | 1.00th=[ 494], 5.00th=[ 8291], 10.00th=[ 9765], 20.00th=[ 12125], | 30.00th=[ 14222], 40.00th=[ 16450], 50.00th=[ 19006], 60.00th=[ 21627], | 70.00th=[ 24511], 80.00th=[ 28443], 90.00th=[ 34341], 95.00th=[ 40633], | 99.00th=[ 54789], 99.50th=[ 56886], 99.90th=[ 71828], 99.95th=[112722], | 99.99th=[112722] bw ( KiB/s): min= 7, max= 176, per=25.07%, avg=103.30, stdev=27.43, samples=113 iops : min= 1, max= 44, avg=25.39, stdev= 6.83, samples=113 write: IOPS=27, BW=111KiB/s (114kB/s)(6348KiB/57209msec); 0 zone resets clat (usec): min=778, max=700247, avg=16493.28, stdev=21192.43 lat (usec): min=779, max=700247, avg=16494.82, stdev=21192.57 clat percentiles (usec): | 1.00th=[ 1106], 5.00th=[ 2507], 10.00th=[ 3163], 20.00th=[ 4621], | 30.00th=[ 7177], 40.00th=[ 10683], 50.00th=[ 13698], 60.00th=[ 16581], | 70.00th=[ 20317], 80.00th=[ 25035], 90.00th=[ 32900], 95.00th=[ 40109], | 99.00th=[ 54264], 99.50th=[ 65799], 99.90th=[ 87557], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 56, max= 199, per=25.73%, avg=110.89, stdev=32.86, samples=112 iops : min= 14, max= 49, avg=27.29, stdev= 8.20, samples=112 lat (usec) : 250=0.07%, 500=0.42%, 750=0.29%, 1000=0.42% lat (msec) : 2=1.17%, 4=7.03%, 10=15.40%, 20=37.08%, 50=36.56% lat (msec) : 100=1.50%, 250=0.03%, 750=0.03% cpu : usr=0.13%, sys=25.46%, ctx=9628, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183655: Sun Nov 16 23:51:13 2025 read: IOPS=26, BW=105KiB/s (108kB/s)(6028KiB/57160msec) clat (usec): min=122, max=76683, avg=20332.91, stdev=10812.65 lat (usec): min=123, max=76683, avg=20335.42, stdev=10811.45 clat percentiles (usec): | 1.00th=[ 482], 5.00th=[ 8029], 10.00th=[ 9503], 20.00th=[11731], | 30.00th=[13829], 40.00th=[15926], 50.00th=[17957], 60.00th=[21103], | 70.00th=[23725], 80.00th=[27919], 90.00th=[34341], 95.00th=[40109], | 99.00th=[55837], 99.50th=[63701], 99.90th=[73925], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 48, max= 176, per=25.75%, avg=106.11, stdev=29.76, samples=112 iops : min= 12, max= 44, avg=26.09, stdev= 7.40, samples=112 write: IOPS=27, BW=110KiB/s (112kB/s)(6260KiB/57160msec); 0 zone resets clat (usec): min=714, max=702483, avg=16798.41, stdev=21498.20 lat (usec): min=716, max=702484, avg=16799.86, stdev=21498.19 clat percentiles (usec): | 1.00th=[ 1106], 5.00th=[ 2507], 10.00th=[ 3130], 20.00th=[ 4752], | 30.00th=[ 7177], 40.00th=[ 10945], 50.00th=[ 13566], 60.00th=[ 16581], | 70.00th=[ 20841], 80.00th=[ 26346], 90.00th=[ 33424], 95.00th=[ 41157], | 99.00th=[ 56361], 99.50th=[ 64750], 99.90th=[ 82314], 99.95th=[700449], | 99.99th=[700449] bw ( KiB/s): min= 14, max= 174, per=25.25%, avg=108.83, stdev=29.49, samples=113 iops : min= 3, max= 43, avg=26.77, stdev= 7.38, samples=113 lat (usec) : 250=0.16%, 500=0.39%, 750=0.42%, 1000=0.49% lat (msec) : 2=1.30%, 4=6.22%, 10=16.18%, 20=37.92%, 50=34.80% lat (msec) : 100=2.08%, 750=0.03% cpu : usr=0.13%, sys=25.43%, ctx=9527, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183657: Sun Nov 16 23:51:13 2025 read: IOPS=26, BW=105KiB/s (108kB/s)(6052KiB/57583msec) clat (usec): min=100, max=718820, avg=21342.10, stdev=21003.20 lat (usec): min=102, max=718821, avg=21343.31, stdev=21003.26 clat percentiles (usec): | 1.00th=[ 553], 5.00th=[ 7963], 10.00th=[ 9896], 20.00th=[ 12125], | 30.00th=[ 14222], 40.00th=[ 16581], 50.00th=[ 19006], 60.00th=[ 21890], | 70.00th=[ 24511], 80.00th=[ 28705], 90.00th=[ 33817], 95.00th=[ 40633], | 99.00th=[ 58459], 99.50th=[ 68682], 99.90th=[ 83362], 99.95th=[717226], | 99.99th=[717226] bw ( KiB/s): min= 7, max= 167, per=25.27%, avg=104.11, stdev=26.63, samples=114 iops : min= 1, max= 41, avg=25.58, stdev= 6.64, samples=114 write: IOPS=27, BW=108KiB/s (111kB/s)(6236KiB/57583msec); 0 zone resets clat (usec): min=841, max=100357, avg=16067.78, stdev=12520.70 lat (usec): min=842, max=100359, avg=16069.28, stdev=12520.81 clat percentiles (usec): | 1.00th=[ 1123], 5.00th=[ 2474], 10.00th=[ 3195], 20.00th=[ 4621], | 30.00th=[ 6718], 40.00th=[ 10290], 50.00th=[ 13173], 60.00th=[ 16319], | 70.00th=[ 20579], 80.00th=[ 26346], 90.00th=[ 33817], 95.00th=[ 39060], | 99.00th=[ 52691], 99.50th=[ 58983], 99.90th=[ 85459], 99.95th=[100140], | 99.99th=[100140] bw ( KiB/s): min= 28, max= 223, per=24.94%, avg=107.51, stdev=36.91, samples=114 iops : min= 7, max= 55, avg=26.44, stdev= 9.24, samples=114 lat (usec) : 250=0.03%, 500=0.26%, 750=0.55%, 1000=0.42% lat (msec) : 2=1.27%, 4=6.64%, 10=15.72%, 20=36.56%, 50=36.52% lat (msec) : 100=1.95%, 250=0.03%, 750=0.03% cpu : usr=0.13%, sys=25.32%, ctx=9675, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=412KiB/s (422kB/s), 103KiB/s-105KiB/s (105kB/s-108kB/s), io=23.4MiB (24.6MB), run=57160-58229msec WRITE: bw=432KiB/s (442kB/s), 108KiB/s-111KiB/s (111kB/s-114kB/s), io=24.6MiB (25.7MB), run=57160-58229msec rand-rw: (groupid=0, jobs=1): err= 0: pid=183656: Sun Nov 16 23:51:20 2025 read: IOPS=22, BW=91.3KiB/s (93.4kB/s)(5992KiB/65665msec) clat (msec): min=2, max=710, avg=18.59, stdev=23.14 lat (msec): min=2, max=710, avg=18.59, stdev=23.14 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 21], 80.00th=[ 28], 90.00th=[ 40], 95.00th=[ 50], | 99.00th=[ 69], 99.50th=[ 82], 99.90th=[ 116], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 15, max= 328, per=25.00%, avg=91.25, stdev=62.11, samples=129 iops : min= 3, max= 82, avg=22.47, stdev=15.64, samples=129 write: IOPS=23, BW=95.9KiB/s (98.2kB/s)(6296KiB/65665msec); 0 zone resets clat (msec): min=4, max=113, avg=23.90, stdev=16.35 lat (msec): min=4, max=113, avg=23.90, stdev=16.35 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 12], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 30], 80.00th=[ 37], 90.00th=[ 47], 95.00th=[ 56], | 99.00th=[ 80], 99.50th=[ 90], 99.90th=[ 101], 99.95th=[ 113], | 99.99th=[ 113] bw ( KiB/s): min= 23, max= 328, per=24.97%, avg=95.40, stdev=62.94, samples=129 iops : min= 5, max= 82, avg=23.50, stdev=15.84, samples=129 lat (msec) : 4=0.78%, 10=28.52%, 20=30.89%, 50=33.76%, 100=5.92% lat (msec) : 250=0.10%, 750=0.03% cpu : usr=0.11%, sys=11.80%, ctx=5914, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183658: Sun Nov 16 23:51:20 2025 read: IOPS=22, BW=90.6KiB/s (92.8kB/s)(5940KiB/65574msec) clat (msec): min=3, max=721, avg=18.40, stdev=23.18 lat (msec): min=3, max=721, avg=18.41, stdev=23.18 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 20], 80.00th=[ 28], 90.00th=[ 40], 95.00th=[ 48], | 99.00th=[ 69], 99.50th=[ 79], 99.90th=[ 99], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 15, max= 368, per=24.88%, avg=90.81, stdev=64.55, samples=129 iops : min= 3, max= 92, avg=22.36, stdev=16.26, samples=129 write: IOPS=24, BW=96.8KiB/s (99.1kB/s)(6348KiB/65574msec); 0 zone resets clat (msec): min=5, max=128, avg=23.97, stdev=16.38 lat (msec): min=5, max=128, avg=23.97, stdev=16.38 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 12], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 25], | 70.00th=[ 31], 80.00th=[ 36], 90.00th=[ 47], 95.00th=[ 56], | 99.00th=[ 80], 99.50th=[ 91], 99.90th=[ 107], 99.95th=[ 129], | 99.99th=[ 129] bw ( KiB/s): min= 31, max= 336, per=25.33%, avg=96.77, stdev=63.04, samples=129 iops : min= 7, max= 84, avg=23.84, stdev=15.87, samples=129 lat (msec) : 4=0.78%, 10=28.48%, 20=31.45%, 50=33.40%, 100=5.79% lat (msec) : 250=0.07%, 750=0.03% cpu : usr=0.09%, sys=11.80%, ctx=5982, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183659: Sun Nov 16 23:51:20 2025 read: IOPS=22, BW=91.9KiB/s (94.1kB/s)(6028KiB/65572msec) clat (usec): min=2872, max=99536, avg=18438.25, stdev=14761.67 lat (usec): min=2873, max=99537, avg=18439.37, stdev=14761.80 clat percentiles (usec): | 1.00th=[ 3752], 5.00th=[ 4555], 10.00th=[ 5211], 20.00th=[ 6521], | 30.00th=[ 8356], 40.00th=[11338], 50.00th=[13698], 60.00th=[16712], | 70.00th=[20841], 80.00th=[28443], 90.00th=[40633], 95.00th=[49021], | 99.00th=[65274], 99.50th=[77071], 99.90th=[96994], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 31, max= 352, per=25.25%, avg=92.18, stdev=65.39, samples=129 iops : min= 7, max= 88, avg=22.70, stdev=16.47, samples=129 write: IOPS=23, BW=95.5KiB/s (97.8kB/s)(6260KiB/65572msec); 0 zone resets clat (msec): min=4, max=738, avg=24.02, stdev=24.10 lat (msec): min=4, max=738, avg=24.02, stdev=24.10 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 12], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 30], 80.00th=[ 36], 90.00th=[ 45], 95.00th=[ 53], | 99.00th=[ 79], 99.50th=[ 95], 99.90th=[ 113], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 23, max= 344, per=24.99%, avg=95.46, stdev=62.67, samples=129 iops : min= 5, max= 86, avg=23.52, stdev=15.78, samples=129 lat (msec) : 4=1.04%, 10=27.90%, 20=30.73%, 50=34.80%, 100=5.37% lat (msec) : 250=0.13%, 750=0.03% cpu : usr=0.10%, sys=11.90%, ctx=5872, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183660: Sun Nov 16 23:51:20 2025 read: IOPS=23, BW=92.3KiB/s (94.5kB/s)(6052KiB/65600msec) clat (msec): min=2, max=116, avg=18.40, stdev=14.41 lat (msec): min=2, max=116, avg=18.41, stdev=14.41 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 12], 50.00th=[ 15], 60.00th=[ 17], | 70.00th=[ 22], 80.00th=[ 29], 90.00th=[ 39], 95.00th=[ 48], | 99.00th=[ 67], 99.50th=[ 75], 99.90th=[ 104], 99.95th=[ 116], | 99.99th=[ 116] bw ( KiB/s): min= 24, max= 352, per=25.22%, avg=92.04, stdev=62.39, samples=129 iops : min= 6, max= 88, avg=22.67, stdev=15.71, samples=129 write: IOPS=23, BW=95.1KiB/s (97.3kB/s)(6236KiB/65600msec); 0 zone resets clat (msec): min=4, max=724, avg=24.09, stdev=24.31 lat (msec): min=4, max=724, avg=24.09, stdev=24.31 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 10], | 30.00th=[ 12], 40.00th=[ 16], 50.00th=[ 20], 60.00th=[ 24], | 70.00th=[ 30], 80.00th=[ 36], 90.00th=[ 46], 95.00th=[ 55], | 99.00th=[ 81], 99.50th=[ 100], 99.90th=[ 112], 99.95th=[ 726], | 99.99th=[ 726] bw ( KiB/s): min= 31, max= 336, per=24.91%, avg=95.16, stdev=62.71, samples=129 iops : min= 7, max= 84, avg=23.44, stdev=15.80, samples=129 lat (msec) : 4=0.78%, 10=28.45%, 20=30.53%, 50=34.70%, 100=5.27% lat (msec) : 250=0.23%, 750=0.03% cpu : usr=0.10%, sys=11.78%, ctx=5996, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=366KiB/s (374kB/s), 90.6KiB/s-92.3KiB/s (92.8kB/s-94.5kB/s), io=23.4MiB (24.6MB), run=65572-65665msec WRITE: bw=383KiB/s (392kB/s), 95.1KiB/s-96.8KiB/s (97.3kB/s-99.1kB/s), io=24.6MiB (25.7MB), run=65572-65665msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=183688: Sun Nov 16 23:51:46 2025 read: IOPS=20, BW=328KiB/s (336kB/s)(5920KiB/18054msec) clat (usec): min=185, max=86548, avg=24787.09, stdev=12727.11 lat (usec): min=185, max=86550, avg=24788.23, stdev=12727.17 clat percentiles (usec): | 1.00th=[ 334], 5.00th=[ 1336], 10.00th=[11207], 20.00th=[15008], | 30.00th=[17957], 40.00th=[20841], 50.00th=[23725], 60.00th=[26608], | 70.00th=[29754], 80.00th=[33424], 90.00th=[40109], 95.00th=[45876], | 99.00th=[64226], 99.50th=[80217], 99.90th=[86508], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 192, max= 606, per=25.08%, avg=322.03, stdev=94.67, samples=35 iops : min= 12, max= 37, avg=19.51, stdev= 6.04, samples=35 write: IOPS=22, BW=353KiB/s (361kB/s)(6368KiB/18054msec); 0 zone resets clat (usec): min=1088, max=87805, avg=22244.41, stdev=14737.06 lat (usec): min=1089, max=87808, avg=22246.51, stdev=14737.03 clat percentiles (usec): | 1.00th=[ 1795], 5.00th=[ 4293], 10.00th=[ 5538], 20.00th=[ 8848], | 30.00th=[12780], 40.00th=[16319], 50.00th=[20055], 60.00th=[23725], | 70.00th=[27395], 80.00th=[32637], 90.00th=[43254], 95.00th=[51119], | 99.00th=[68682], 99.50th=[72877], 99.90th=[87557], 99.95th=[87557], | 99.99th=[87557] bw ( KiB/s): min= 190, max= 640, per=26.01%, avg=351.34, stdev=104.55, samples=35 iops : min= 11, max= 40, avg=21.37, stdev= 6.57, samples=35 lat (usec) : 250=0.26%, 500=0.78%, 750=1.17%, 1000=0.13% lat (msec) : 2=0.78%, 4=1.56%, 10=10.94%, 20=28.65%, 50=51.17% lat (msec) : 100=4.56% cpu : usr=0.08%, sys=28.69%, ctx=3118, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183689: Sun Nov 16 23:51:46 2025 read: IOPS=19, BW=308KiB/s (315kB/s)(5744KiB/18649msec) clat (usec): min=158, max=83302, avg=26073.33, stdev=13818.14 lat (usec): min=158, max=83302, avg=26077.12, stdev=13817.31 clat percentiles (usec): | 1.00th=[ 433], 5.00th=[ 7439], 10.00th=[12125], 20.00th=[15533], | 30.00th=[18220], 40.00th=[21627], 50.00th=[23725], 60.00th=[27395], | 70.00th=[31065], 80.00th=[35390], 90.00th=[41157], 95.00th=[51643], | 99.00th=[74974], 99.50th=[82314], 99.90th=[83362], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 158, max= 640, per=24.04%, avg=308.68, stdev=103.82, samples=37 iops : min= 9, max= 40, avg=18.70, stdev= 6.52, samples=37 write: IOPS=21, BW=351KiB/s (359kB/s)(6544KiB/18649msec); 0 zone resets clat (usec): min=1661, max=86452, avg=22661.62, stdev=14861.44 lat (usec): min=1662, max=86455, avg=22663.97, stdev=14861.83 clat percentiles (usec): | 1.00th=[ 2802], 5.00th=[ 4555], 10.00th=[ 6194], 20.00th=[ 9503], | 30.00th=[12649], 40.00th=[16057], 50.00th=[19530], 60.00th=[23987], | 70.00th=[28181], 80.00th=[34341], 90.00th=[43779], 95.00th=[51643], | 99.00th=[66323], 99.50th=[69731], 99.90th=[86508], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 191, max= 832, per=25.92%, avg=350.22, stdev=134.83, samples=37 iops : min= 11, max= 52, avg=21.30, stdev= 8.59, samples=37 lat (usec) : 250=0.26%, 500=0.39%, 750=1.17% lat (msec) : 2=0.13%, 4=1.69%, 10=10.94%, 20=28.52%, 50=51.17% lat (msec) : 100=5.73% cpu : usr=0.06%, sys=27.82%, ctx=3069, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183690: Sun Nov 16 23:51:46 2025 read: IOPS=21, BW=338KiB/s (346kB/s)(6256KiB/18528msec) clat (usec): min=133, max=70410, avg=24760.59, stdev=12702.28 lat (usec): min=134, max=70411, avg=24761.72, stdev=12702.36 clat percentiles (usec): | 1.00th=[ 289], 5.00th=[ 8225], 10.00th=[11207], 20.00th=[14353], | 30.00th=[17171], 40.00th=[20055], 50.00th=[23200], 60.00th=[26870], | 70.00th=[30278], 80.00th=[33817], 90.00th=[41157], 95.00th=[47973], | 99.00th=[66323], 99.50th=[67634], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 159, max= 670, per=25.83%, avg=331.69, stdev=123.11, samples=36 iops : min= 9, max= 41, avg=20.11, stdev= 7.79, samples=36 write: IOPS=20, BW=326KiB/s (333kB/s)(6032KiB/18528msec); 0 zone resets clat (usec): min=1232, max=73045, avg=23390.54, stdev=13962.01 lat (usec): min=1234, max=73047, avg=23392.64, stdev=13962.09 clat percentiles (usec): | 1.00th=[ 1614], 5.00th=[ 4555], 10.00th=[ 6325], 20.00th=[10814], | 30.00th=[14484], 40.00th=[18482], 50.00th=[21627], 60.00th=[25560], | 70.00th=[28967], 80.00th=[33424], 90.00th=[42206], 95.00th=[50070], | 99.00th=[67634], 99.50th=[69731], 99.90th=[72877], 99.95th=[72877], | 99.99th=[72877] bw ( KiB/s): min= 95, max= 574, per=23.05%, avg=311.36, stdev=97.53, samples=36 iops : min= 5, max= 35, avg=18.86, stdev= 6.15, samples=36 lat (usec) : 250=0.26%, 500=1.04%, 750=0.52%, 1000=0.13% lat (msec) : 2=0.65%, 4=1.17%, 10=8.85%, 20=29.82%, 50=52.99% lat (msec) : 100=4.56% cpu : usr=0.12%, sys=28.29%, ctx=3156, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183691: Sun Nov 16 23:51:46 2025 read: IOPS=21, BW=342KiB/s (350kB/s)(6032KiB/17656msec) clat (usec): min=290, max=72389, avg=24229.27, stdev=12444.95 lat (usec): min=291, max=72391, avg=24230.47, stdev=12445.02 clat percentiles (usec): | 1.00th=[ 379], 5.00th=[ 930], 10.00th=[11207], 20.00th=[14353], | 30.00th=[17433], 40.00th=[20055], 50.00th=[22938], 60.00th=[26346], | 70.00th=[29492], 80.00th=[32900], 90.00th=[39060], 95.00th=[45351], | 99.00th=[62129], 99.50th=[71828], 99.90th=[72877], 99.95th=[72877], | 99.99th=[72877] bw ( KiB/s): min= 96, max= 638, per=26.51%, avg=340.40, stdev=105.15, samples=35 iops : min= 6, max= 39, avg=20.69, stdev= 6.59, samples=35 write: IOPS=22, BW=354KiB/s (363kB/s)(6256KiB/17656msec); 0 zone resets clat (usec): min=1714, max=78084, avg=21747.91, stdev=13975.55 lat (usec): min=1716, max=78086, avg=21749.99, stdev=13975.53 clat percentiles (usec): | 1.00th=[ 2057], 5.00th=[ 4490], 10.00th=[ 6390], 20.00th=[ 9110], | 30.00th=[12387], 40.00th=[15926], 50.00th=[19006], 60.00th=[22676], | 70.00th=[27657], 80.00th=[33162], 90.00th=[40633], 95.00th=[47449], | 99.00th=[68682], 99.50th=[76022], 99.90th=[78119], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 191, max= 640, per=26.28%, avg=355.03, stdev=101.14, samples=35 iops : min= 11, max= 40, avg=21.60, stdev= 6.40, samples=35 lat (usec) : 500=0.65%, 750=1.30%, 1000=0.52% lat (msec) : 2=0.52%, 4=1.69%, 10=11.20%, 20=30.34%, 50=50.13% lat (msec) : 100=3.65% cpu : usr=0.13%, sys=29.38%, ctx=3099, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1284KiB/s (1315kB/s), 308KiB/s-342KiB/s (315kB/s-350kB/s), io=23.4MiB (24.5MB), run=17656-18649msec WRITE: bw=1351KiB/s (1384kB/s), 326KiB/s-354KiB/s (333kB/s-363kB/s), io=24.6MiB (25.8MB), run=17656-18649msec rand-rw: (groupid=0, jobs=1): err= 0: pid=183683: Sun Nov 16 23:51:47 2025 read: IOPS=18, BW=291KiB/s (298kB/s)(5920KiB/20370msec) clat (usec): min=3786, max=89796, avg=23980.01, stdev=16742.56 lat (usec): min=3786, max=89798, avg=23981.52, stdev=16742.46 clat percentiles (usec): | 1.00th=[ 5473], 5.00th=[ 6587], 10.00th=[ 8586], 20.00th=[11338], | 30.00th=[13435], 40.00th=[15664], 50.00th=[18744], 60.00th=[21627], | 70.00th=[27395], 80.00th=[34341], 90.00th=[48497], 95.00th=[58983], | 99.00th=[83362], 99.50th=[88605], 99.90th=[89654], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 64, max= 704, per=24.46%, avg=284.75, stdev=132.47, samples=40 iops : min= 4, max= 44, avg=17.20, stdev= 8.45, samples=40 write: IOPS=19, BW=313KiB/s (320kB/s)(6368KiB/20370msec); 0 zone resets clat (msec): min=6, max=122, avg=28.77, stdev=16.45 lat (msec): min=6, max=122, avg=28.77, stdev=16.45 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 16], | 30.00th=[ 19], 40.00th=[ 22], 50.00th=[ 26], 60.00th=[ 31], | 70.00th=[ 34], 80.00th=[ 42], 90.00th=[ 51], 95.00th=[ 61], | 99.00th=[ 86], 99.50th=[ 103], 99.90th=[ 124], 99.95th=[ 124], | 99.99th=[ 124] bw ( KiB/s): min= 127, max= 800, per=25.34%, avg=310.40, stdev=132.59, samples=40 iops : min= 7, max= 50, avg=18.80, stdev= 8.46, samples=40 lat (msec) : 4=0.13%, 10=10.55%, 20=34.24%, 50=45.70%, 100=9.11% lat (msec) : 250=0.26% cpu : usr=0.12%, sys=12.81%, ctx=1734, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183684: Sun Nov 16 23:51:47 2025 read: IOPS=17, BW=281KiB/s (288kB/s)(5744KiB/20449msec) clat (usec): min=4177, max=88582, avg=22955.80, stdev=15884.07 lat (usec): min=4179, max=88584, avg=22957.00, stdev=15883.99 clat percentiles (usec): | 1.00th=[ 4752], 5.00th=[ 6325], 10.00th=[ 7701], 20.00th=[11076], | 30.00th=[13304], 40.00th=[15664], 50.00th=[17957], 60.00th=[21365], | 70.00th=[26084], 80.00th=[32637], 90.00th=[44827], 95.00th=[58459], | 99.00th=[78119], 99.50th=[88605], 99.90th=[88605], 99.95th=[88605], | 99.99th=[88605] bw ( KiB/s): min= 96, max= 704, per=23.80%, avg=277.00, stdev=129.09, samples=40 iops : min= 6, max= 44, avg=16.73, stdev= 8.22, samples=40 write: IOPS=20, BW=320KiB/s (328kB/s)(6544KiB/20449msec); 0 zone resets clat (msec): min=6, max=105, avg=29.73, stdev=17.40 lat (msec): min=6, max=105, avg=29.73, stdev=17.40 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 15], | 30.00th=[ 19], 40.00th=[ 22], 50.00th=[ 27], 60.00th=[ 31], | 70.00th=[ 36], 80.00th=[ 42], 90.00th=[ 52], 95.00th=[ 67], | 99.00th=[ 83], 99.50th=[ 95], 99.90th=[ 106], 99.95th=[ 106], | 99.99th=[ 106] bw ( KiB/s): min= 160, max= 832, per=25.30%, avg=309.98, stdev=144.92, samples=40 iops : min= 10, max= 52, avg=18.77, stdev= 9.22, samples=40 lat (msec) : 10=9.90%, 20=35.42%, 50=45.31%, 100=9.24%, 250=0.13% cpu : usr=0.09%, sys=12.75%, ctx=1709, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183685: Sun Nov 16 23:51:47 2025 read: IOPS=19, BW=306KiB/s (314kB/s)(6256KiB/20423msec) clat (msec): min=4, max=102, avg=24.09, stdev=17.13 lat (msec): min=4, max=102, avg=24.09, stdev=17.13 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 16], 50.00th=[ 19], 60.00th=[ 23], | 70.00th=[ 28], 80.00th=[ 37], 90.00th=[ 51], 95.00th=[ 58], | 99.00th=[ 86], 99.50th=[ 94], 99.90th=[ 103], 99.95th=[ 103], | 99.99th=[ 103] bw ( KiB/s): min= 95, max= 960, per=25.98%, avg=302.42, stdev=159.16, samples=40 iops : min= 5, max= 60, avg=18.30, stdev=10.12, samples=40 write: IOPS=18, BW=295KiB/s (302kB/s)(6032KiB/20423msec); 0 zone resets clat (msec): min=5, max=124, avg=29.06, stdev=17.44 lat (msec): min=5, max=124, avg=29.06, stdev=17.44 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 16], | 30.00th=[ 18], 40.00th=[ 20], 50.00th=[ 25], 60.00th=[ 29], | 70.00th=[ 36], 80.00th=[ 43], 90.00th=[ 53], 95.00th=[ 64], | 99.00th=[ 80], 99.50th=[ 105], 99.90th=[ 125], 99.95th=[ 125], | 99.99th=[ 125] bw ( KiB/s): min= 127, max= 704, per=23.44%, avg=287.15, stdev=123.00, samples=40 iops : min= 7, max= 44, avg=17.35, stdev= 7.85, samples=40 lat (msec) : 10=12.37%, 20=36.07%, 50=40.36%, 100=10.81%, 250=0.39% cpu : usr=0.17%, sys=12.64%, ctx=1660, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183686: Sun Nov 16 23:51:47 2025 read: IOPS=18, BW=293KiB/s (300kB/s)(6032KiB/20560msec) clat (msec): min=3, max=110, avg=24.22, stdev=17.11 lat (msec): min=3, max=110, avg=24.22, stdev=17.11 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 8], 20.00th=[ 11], | 30.00th=[ 14], 40.00th=[ 16], 50.00th=[ 19], 60.00th=[ 23], | 70.00th=[ 29], 80.00th=[ 38], 90.00th=[ 48], 95.00th=[ 59], | 99.00th=[ 81], 99.50th=[ 89], 99.90th=[ 111], 99.95th=[ 111], | 99.99th=[ 111] bw ( KiB/s): min= 95, max= 768, per=24.22%, avg=281.88, stdev=138.09, samples=40 iops : min= 5, max= 48, avg=17.02, stdev= 8.85, samples=40 write: IOPS=19, BW=304KiB/s (312kB/s)(6256KiB/20560msec); 0 zone resets clat (msec): min=5, max=109, avg=29.16, stdev=16.91 lat (msec): min=5, max=109, avg=29.16, stdev=16.91 clat percentiles (msec): | 1.00th=[ 6], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 16], | 30.00th=[ 19], 40.00th=[ 22], 50.00th=[ 26], 60.00th=[ 30], | 70.00th=[ 35], 80.00th=[ 42], 90.00th=[ 52], 95.00th=[ 63], | 99.00th=[ 85], 99.50th=[ 90], 99.90th=[ 110], 99.95th=[ 110], | 99.99th=[ 110] bw ( KiB/s): min= 96, max= 672, per=23.90%, avg=292.82, stdev=114.26, samples=40 iops : min= 6, max= 42, avg=17.70, stdev= 7.29, samples=40 lat (msec) : 4=0.26%, 10=11.59%, 20=33.07%, 50=45.05%, 100=9.77% lat (msec) : 250=0.26% cpu : usr=0.12%, sys=12.63%, ctx=1656, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1165KiB/s (1193kB/s), 281KiB/s-306KiB/s (288kB/s-314kB/s), io=23.4MiB (24.5MB), run=20370-20560msec WRITE: bw=1226KiB/s (1255kB/s), 295KiB/s-320KiB/s (302kB/s-328kB/s), io=24.6MiB (25.8MB), run=20370-20560msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=183704: Sun Nov 16 23:51:55 2025 read: IOPS=3, BW=3904KiB/s (3998kB/s)(7168KiB/1836msec) clat (usec): min=795, max=154848, avg=117314.89, stdev=53835.00 lat (usec): min=796, max=154849, avg=117316.17, stdev=53835.04 clat percentiles (usec): | 1.00th=[ 799], 5.00th=[ 799], 10.00th=[ 799], 20.00th=[111674], | 30.00th=[125305], 40.00th=[125305], 50.00th=[127402], 60.00th=[149947], | 70.00th=[149947], 80.00th=[149947], 90.00th=[154141], 95.00th=[154141], | 99.00th=[154141], 99.50th=[154141], 99.90th=[154141], 99.95th=[154141], | 99.99th=[154141] bw ( KiB/s): min= 2048, max= 5036, per=50.14%, avg=3726.67, stdev=1527.86, samples=3 iops : min= 2, max= 4, avg= 3.33, stdev= 1.15, samples=3 write: IOPS=2, BW=2789KiB/s (2856kB/s)(5120KiB/1836msec); 0 zone resets clat (msec): min=122, max=257, avg=183.18, stdev=63.00 lat (msec): min=122, max=257, avg=183.25, stdev=62.97 clat percentiles (msec): | 1.00th=[ 123], 5.00th=[ 123], 10.00th=[ 123], 20.00th=[ 123], | 30.00th=[ 133], 40.00th=[ 133], 50.00th=[ 161], 60.00th=[ 161], | 70.00th=[ 245], 80.00th=[ 245], 90.00th=[ 257], 95.00th=[ 257], | 99.00th=[ 257], 99.50th=[ 257], 99.90th=[ 257], 99.95th=[ 257], | 99.99th=[ 257] bw ( KiB/s): min= 2048, max= 6144, per=39.36%, avg=4096.00, stdev=2896.31, samples=2 iops : min= 2, max= 6, avg= 4.00, stdev= 2.83, samples=2 lat (usec) : 1000=8.33% lat (msec) : 250=83.33%, 500=8.33% cpu : usr=0.00%, sys=60.49%, ctx=758, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183705: Sun Nov 16 23:51:55 2025 read: IOPS=0, BW=743KiB/s (761kB/s)(2048KiB/2755msec) clat (usec): min=793, max=215642, avg=108217.60, stdev=151921.10 lat (usec): min=794, max=215644, avg=108219.30, stdev=151921.69 clat percentiles (usec): | 1.00th=[ 791], 5.00th=[ 791], 10.00th=[ 791], 20.00th=[ 791], | 30.00th=[ 791], 40.00th=[ 791], 50.00th=[ 791], 60.00th=[214959], | 70.00th=[214959], 80.00th=[214959], 90.00th=[214959], 95.00th=[214959], | 99.00th=[214959], 99.50th=[214959], 99.90th=[214959], 99.95th=[214959], | 99.99th=[214959] bw ( KiB/s): min= 1692, max= 2048, per=25.16%, avg=1870.00, stdev=251.73, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=3, BW=3717KiB/s (3806kB/s)(10.0MiB/2755msec); 0 zone resets clat (msec): min=45, max=437, avg=248.49, stdev=123.39 lat (msec): min=46, max=437, avg=248.56, stdev=123.38 clat percentiles (msec): | 1.00th=[ 46], 5.00th=[ 46], 10.00th=[ 46], 20.00th=[ 125], | 30.00th=[ 178], 40.00th=[ 184], 50.00th=[ 222], 60.00th=[ 243], | 70.00th=[ 300], 80.00th=[ 363], 90.00th=[ 388], 95.00th=[ 439], | 99.00th=[ 439], 99.50th=[ 439], 99.90th=[ 439], 99.95th=[ 439], | 99.99th=[ 439] bw ( KiB/s): min= 2048, max= 6144, per=39.26%, avg=4085.75, stdev=1672.31, samples=4 iops : min= 2, max= 6, avg= 3.75, stdev= 1.71, samples=4 lat (usec) : 1000=8.33% lat (msec) : 50=8.33%, 250=50.00%, 500=33.33% cpu : usr=0.07%, sys=49.53%, ctx=196, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183706: Sun Nov 16 23:51:55 2025 read: IOPS=2, BW=2800KiB/s (2868kB/s)(6144KiB/2194msec) clat (usec): min=717, max=153630, avg=26734.11, stdev=62175.77 lat (usec): min=718, max=153631, avg=26734.90, stdev=62175.65 clat percentiles (usec): | 1.00th=[ 717], 5.00th=[ 717], 10.00th=[ 717], 20.00th=[ 766], | 30.00th=[ 766], 40.00th=[ 865], 50.00th=[ 865], 60.00th=[ 922], | 70.00th=[ 3490], 80.00th=[ 3490], 90.00th=[154141], 95.00th=[154141], | 99.00th=[154141], 99.50th=[154141], 99.90th=[154141], 99.95th=[154141], | 99.99th=[154141] bw ( KiB/s): min= 2048, max= 8192, per=68.88%, avg=5120.00, stdev=4344.46, samples=2 iops : min= 2, max= 8, avg= 5.00, stdev= 4.24, samples=2 write: IOPS=2, BW=2800KiB/s (2868kB/s)(6144KiB/2194msec); 0 zone resets clat (msec): min=183, max=673, avg=326.00, stdev=181.74 lat (msec): min=183, max=673, avg=326.05, stdev=181.75 clat percentiles (msec): | 1.00th=[ 184], 5.00th=[ 184], 10.00th=[ 184], 20.00th=[ 186], | 30.00th=[ 186], 40.00th=[ 259], 50.00th=[ 259], 60.00th=[ 321], | 70.00th=[ 334], 80.00th=[ 334], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min= 3357, max= 4096, per=36.99%, avg=3849.67, stdev=426.66, samples=3 iops : min= 3, max= 4, avg= 3.67, stdev= 0.58, samples=3 lat (usec) : 750=8.33%, 1000=25.00% lat (msec) : 4=8.33%, 250=25.00%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=41.13%, ctx=141, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183707: Sun Nov 16 23:51:55 2025 read: IOPS=2, BW=2397KiB/s (2455kB/s)(5120KiB/2136msec) clat (usec): min=638, max=271286, avg=82453.70, stdev=121306.27 lat (usec): min=639, max=271288, avg=82454.70, stdev=121306.63 clat percentiles (usec): | 1.00th=[ 635], 5.00th=[ 635], 10.00th=[ 635], 20.00th=[ 635], | 30.00th=[ 758], 40.00th=[ 758], 50.00th=[ 824], 60.00th=[ 824], | 70.00th=[139461], 80.00th=[139461], 90.00th=[270533], 95.00th=[270533], | 99.00th=[270533], 99.50th=[270533], 99.90th=[270533], 99.95th=[270533], | 99.99th=[270533] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 8, max= 8, avg= 8.00, stdev= 0.00, samples=1 write: IOPS=3, BW=3356KiB/s (3436kB/s)(7168KiB/2136msec); 0 zone resets clat (msec): min=175, max=421, avg=243.43, stdev=90.00 lat (msec): min=175, max=422, avg=243.52, stdev=90.00 clat percentiles (msec): | 1.00th=[ 176], 5.00th=[ 176], 10.00th=[ 176], 20.00th=[ 178], | 30.00th=[ 197], 40.00th=[ 197], 50.00th=[ 209], 60.00th=[ 222], | 70.00th=[ 222], 80.00th=[ 305], 90.00th=[ 422], 95.00th=[ 422], | 99.00th=[ 422], 99.50th=[ 422], 99.90th=[ 422], 99.95th=[ 422], | 99.99th=[ 422] bw ( KiB/s): min= 1665, max= 6144, per=38.13%, avg=3968.33, stdev=2242.23, samples=3 iops : min= 1, max= 6, avg= 3.67, stdev= 2.52, samples=3 lat (usec) : 750=8.33%, 1000=16.67% lat (msec) : 250=50.00%, 500=25.00% cpu : usr=0.00%, sys=49.04%, ctx=156, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7434KiB/s (7612kB/s), 743KiB/s-3904KiB/s (761kB/s-3998kB/s), io=20.0MiB (20.0MB), run=1836-2755msec WRITE: bw=10.2MiB/s (10.7MB/s), 2789KiB/s-3717KiB/s (2856kB/s-3806kB/s), io=28.0MiB (29.4MB), run=1836-2755msec rand-rw: (groupid=0, jobs=1): err= 0: pid=183700: Sun Nov 16 23:51:55 2025 read: IOPS=2, BW=2852KiB/s (2921kB/s)(7168KiB/2513msec) clat (msec): min=55, max=508, avg=192.37, stdev=178.30 lat (msec): min=55, max=508, avg=192.37, stdev=178.30 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 56], 10.00th=[ 56], 20.00th=[ 77], | 30.00th=[ 82], 40.00th=[ 82], 50.00th=[ 104], 60.00th=[ 142], | 70.00th=[ 142], 80.00th=[ 380], 90.00th=[ 510], 95.00th=[ 510], | 99.00th=[ 510], 99.50th=[ 510], 99.90th=[ 510], 99.95th=[ 510], | 99.99th=[ 510] bw ( KiB/s): min= 4096, max= 5180, per=62.06%, avg=4638.00, stdev=766.50, samples=2 iops : min= 4, max= 5, avg= 4.50, stdev= 0.71, samples=2 write: IOPS=1, BW=2037KiB/s (2086kB/s)(5120KiB/2513msec); 0 zone resets clat (msec): min=111, max=326, avg=212.78, stdev=87.18 lat (msec): min=111, max=326, avg=212.85, stdev=87.18 clat percentiles (msec): | 1.00th=[ 112], 5.00th=[ 112], 10.00th=[ 112], 20.00th=[ 112], | 30.00th=[ 169], 40.00th=[ 169], 50.00th=[ 180], 60.00th=[ 180], | 70.00th=[ 279], 80.00th=[ 279], 90.00th=[ 326], 95.00th=[ 326], | 99.00th=[ 326], 99.50th=[ 326], 99.90th=[ 326], 99.95th=[ 326], | 99.99th=[ 326] bw ( KiB/s): min= 1726, max= 3799, per=26.40%, avg=2762.50, stdev=1465.83, samples=2 iops : min= 1, max= 3, avg= 2.00, stdev= 1.41, samples=2 lat (msec) : 100=25.00%, 250=41.67%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=21.97%, ctx=143, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183701: Sun Nov 16 23:51:55 2025 read: IOPS=0, BW=752KiB/s (770kB/s)(2048KiB/2722msec) clat (usec): min=66989, max=86233, avg=76611.52, stdev=13607.18 lat (usec): min=66991, max=86234, avg=76613.21, stdev=13606.71 clat percentiles (usec): | 1.00th=[66847], 5.00th=[66847], 10.00th=[66847], 20.00th=[66847], | 30.00th=[66847], 40.00th=[66847], 50.00th=[66847], 60.00th=[86508], | 70.00th=[86508], 80.00th=[86508], 90.00th=[86508], 95.00th=[86508], | 99.00th=[86508], 99.50th=[86508], 99.90th=[86508], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 1721, max= 1899, per=24.22%, avg=1810.00, stdev=125.87, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3762KiB/s (3852kB/s)(10.0MiB/2722msec); 0 zone resets clat (msec): min=65, max=467, avg=249.37, stdev=155.48 lat (msec): min=65, max=468, avg=249.44, stdev=155.48 clat percentiles (msec): | 1.00th=[ 66], 5.00th=[ 66], 10.00th=[ 66], 20.00th=[ 81], | 30.00th=[ 94], 40.00th=[ 186], 50.00th=[ 190], 60.00th=[ 205], | 70.00th=[ 368], 80.00th=[ 388], 90.00th=[ 451], 95.00th=[ 468], | 99.00th=[ 468], 99.50th=[ 468], 99.90th=[ 468], 99.95th=[ 468], | 99.99th=[ 468] bw ( KiB/s): min= 1721, max= 5919, per=33.61%, avg=3516.60, stdev=1700.58, samples=5 iops : min= 1, max= 5, avg= 3.00, stdev= 1.58, samples=5 lat (msec) : 100=41.67%, 250=25.00%, 500=33.33% cpu : usr=0.07%, sys=21.68%, ctx=229, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183702: Sun Nov 16 23:51:55 2025 read: IOPS=2, BW=2242KiB/s (2296kB/s)(6144KiB/2740msec) clat (msec): min=47, max=330, avg=179.87, stdev=102.49 lat (msec): min=47, max=330, avg=179.87, stdev=102.49 clat percentiles (msec): | 1.00th=[ 48], 5.00th=[ 48], 10.00th=[ 48], 20.00th=[ 92], | 30.00th=[ 92], 40.00th=[ 159], 50.00th=[ 159], 60.00th=[ 218], | 70.00th=[ 234], 80.00th=[ 234], 90.00th=[ 330], 95.00th=[ 330], | 99.00th=[ 330], 99.50th=[ 330], 99.90th=[ 330], 99.95th=[ 330], | 99.99th=[ 330] bw ( KiB/s): min= 1899, max= 4096, per=44.34%, avg=3313.67, stdev=1227.43, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 write: IOPS=2, BW=2242KiB/s (2296kB/s)(6144KiB/2740msec); 0 zone resets clat (msec): min=54, max=641, avg=269.54, stdev=219.39 lat (msec): min=54, max=641, avg=269.61, stdev=219.39 clat percentiles (msec): | 1.00th=[ 55], 5.00th=[ 55], 10.00th=[ 55], 20.00th=[ 56], | 30.00th=[ 56], 40.00th=[ 226], 50.00th=[ 226], 60.00th=[ 279], | 70.00th=[ 363], 80.00th=[ 363], 90.00th=[ 642], 95.00th=[ 642], | 99.00th=[ 642], 99.50th=[ 642], 99.90th=[ 642], 99.95th=[ 642], | 99.99th=[ 642] bw ( KiB/s): min= 1899, max= 3447, per=22.38%, avg=2341.75, stdev=739.34, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 50=8.33%, 100=25.00%, 250=33.33%, 500=25.00%, 750=8.33% cpu : usr=0.00%, sys=17.63%, ctx=125, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183703: Sun Nov 16 23:51:55 2025 read: IOPS=1, BW=1914KiB/s (1960kB/s)(5120KiB/2675msec) clat (msec): min=52, max=357, avg=207.26, stdev=138.78 lat (msec): min=52, max=357, avg=207.26, stdev=138.78 clat percentiles (msec): | 1.00th=[ 53], 5.00th=[ 53], 10.00th=[ 53], 20.00th=[ 53], | 30.00th=[ 93], 40.00th=[ 93], 50.00th=[ 194], 60.00th=[ 194], | 70.00th=[ 338], 80.00th=[ 338], 90.00th=[ 359], 95.00th=[ 359], | 99.00th=[ 359], 99.50th=[ 359], 99.90th=[ 359], 99.95th=[ 359], | 99.99th=[ 359] bw ( KiB/s): min= 4096, max= 4096, per=54.80%, avg=4096.00, stdev= 0.00, samples=2 iops : min= 4, max= 4, avg= 4.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2680KiB/s (2744kB/s)(7168KiB/2675msec); 0 zone resets clat (msec): min=57, max=648, avg=232.53, stdev=213.22 lat (msec): min=57, max=648, avg=232.63, stdev=213.22 clat percentiles (msec): | 1.00th=[ 58], 5.00th=[ 58], 10.00th=[ 58], 20.00th=[ 67], | 30.00th=[ 84], 40.00th=[ 84], 50.00th=[ 159], 60.00th=[ 271], | 70.00th=[ 271], 80.00th=[ 342], 90.00th=[ 651], 95.00th=[ 651], | 99.00th=[ 651], 99.50th=[ 651], 99.90th=[ 651], 99.95th=[ 651], | 99.99th=[ 651] bw ( KiB/s): min= 5919, max= 6918, per=61.34%, avg=6418.50, stdev=706.40, samples=2 iops : min= 5, max= 6, avg= 5.50, stdev= 0.71, samples=2 lat (msec) : 100=41.67%, 250=16.67%, 500=33.33%, 750=8.33% cpu : usr=0.15%, sys=19.04%, ctx=132, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7474KiB/s (7654kB/s), 752KiB/s-2852KiB/s (770kB/s-2921kB/s), io=20.0MiB (20.0MB), run=2513-2740msec WRITE: bw=10.2MiB/s (10.7MB/s), 2037KiB/s-3762KiB/s (2086kB/s-3852kB/s), io=28.0MiB (29.4MB), run=2513-2740msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=183714: Sun Nov 16 23:52:04 2025 read: IOPS=2, BW=8331KiB/s (8531kB/s)(12.0MiB/1475msec) clat (msec): min=433, max=468, avg=446.31, stdev=19.54 lat (msec): min=433, max=468, avg=446.31, stdev=19.54 clat percentiles (msec): | 1.00th=[ 435], 5.00th=[ 435], 10.00th=[ 435], 20.00th=[ 435], | 30.00th=[ 435], 40.00th=[ 439], 50.00th=[ 439], 60.00th=[ 439], | 70.00th=[ 468], 80.00th=[ 468], 90.00th=[ 468], 95.00th=[ 468], | 99.00th=[ 468], 99.50th=[ 468], 99.90th=[ 468], 99.95th=[ 468], | 99.99th=[ 468] bw ( KiB/s): min= 5919, max= 8192, per=68.86%, avg=7055.50, stdev=1607.25, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=100.00% cpu : usr=0.00%, sys=49.86%, ctx=2331, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183716: Sun Nov 16 23:52:04 2025 read: IOPS=0, BW=2562KiB/s (2623kB/s)(4096KiB/1599msec) clat (nsec): min=466426k, max=466426k, avg=466425874.00, stdev= 0.00 lat (nsec): min=466428k, max=466428k, avg=466427631.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 468], 5.00th=[ 468], 10.00th=[ 468], 20.00th=[ 468], | 30.00th=[ 468], 40.00th=[ 468], 50.00th=[ 468], 60.00th=[ 468], | 70.00th=[ 468], 80.00th=[ 468], 90.00th=[ 468], 95.00th=[ 468], | 99.00th=[ 468], 99.50th=[ 468], 99.90th=[ 468], 99.95th=[ 468], | 99.99th=[ 468] bw ( KiB/s): min= 6606, max= 6606, per=64.47%, avg=6606.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=5123KiB/s (5246kB/s)(8192KiB/1599msec); 0 zone resets clat (msec): min=523, max=542, avg=533.10, stdev=13.08 lat (msec): min=524, max=542, avg=533.44, stdev=13.10 clat percentiles (msec): | 1.00th=[ 523], 5.00th=[ 523], 10.00th=[ 523], 20.00th=[ 523], | 30.00th=[ 523], 40.00th=[ 523], 50.00th=[ 523], 60.00th=[ 542], | 70.00th=[ 542], 80.00th=[ 542], 90.00th=[ 542], 95.00th=[ 542], | 99.00th=[ 542], 99.50th=[ 542], 99.90th=[ 542], 99.95th=[ 542], | 99.99th=[ 542] bw ( KiB/s): min= 8192, max= 8192, per=51.38%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.00%, sys=24.16%, ctx=898, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183718: Sun Nov 16 23:52:04 2025 write: IOPS=1, BW=5980KiB/s (6123kB/s)(12.0MiB/2055msec); 0 zone resets clat (msec): min=201, max=1180, avg=662.14, stdev=491.95 lat (msec): min=201, max=1180, avg=662.35, stdev=492.01 clat percentiles (msec): | 1.00th=[ 201], 5.00th=[ 201], 10.00th=[ 201], 20.00th=[ 201], | 30.00th=[ 201], 40.00th=[ 609], 50.00th=[ 609], 60.00th=[ 609], | 70.00th=[ 1183], 80.00th=[ 1183], 90.00th=[ 1183], 95.00th=[ 1183], | 99.00th=[ 1183], 99.50th=[ 1183], 99.90th=[ 1183], 99.95th=[ 1183], | 99.99th=[ 1183] bw ( KiB/s): min= 8192, max= 8192, per=51.38%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.05%, sys=19.72%, ctx=85, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183720: Sun Nov 16 23:52:04 2025 write: IOPS=1, BW=6447KiB/s (6602kB/s)(12.0MiB/1906msec); 0 zone resets clat (msec): min=269, max=1017, avg=605.68, stdev=379.96 lat (msec): min=269, max=1018, avg=606.04, stdev=379.94 clat percentiles (msec): | 1.00th=[ 271], 5.00th=[ 271], 10.00th=[ 271], 20.00th=[ 271], | 30.00th=[ 271], 40.00th=[ 531], 50.00th=[ 531], 60.00th=[ 531], | 70.00th=[ 1020], 80.00th=[ 1020], 90.00th=[ 1020], 95.00th=[ 1020], | 99.00th=[ 1020], 99.50th=[ 1020], 99.90th=[ 1020], 99.95th=[ 1020], | 99.99th=[ 1020] bw ( KiB/s): min= 6340, max= 8192, per=45.57%, avg=7266.00, stdev=1309.56, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=22.20%, ctx=91, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=10.0MiB/s (10.5MB/s), 2562KiB/s-8331KiB/s (2623kB/s-8531kB/s), io=16.0MiB (16.8MB), run=1475-1599msec WRITE: bw=15.6MiB/s (16.3MB/s), 5123KiB/s-6447KiB/s (5246kB/s-6602kB/s), io=32.0MiB (33.6MB), run=1599-2055msec rand-rw: (groupid=0, jobs=1): err= 0: pid=183713: Sun Nov 16 23:52:04 2025 read: IOPS=1, BW=8052KiB/s (8246kB/s)(12.0MiB/1526msec) clat (msec): min=238, max=871, avg=475.82, stdev=344.55 lat (msec): min=238, max=871, avg=475.82, stdev=344.55 clat percentiles (msec): | 1.00th=[ 239], 5.00th=[ 239], 10.00th=[ 239], 20.00th=[ 239], | 30.00th=[ 239], 40.00th=[ 317], 50.00th=[ 317], 60.00th=[ 317], | 70.00th=[ 869], 80.00th=[ 869], 90.00th=[ 869], 95.00th=[ 869], | 99.00th=[ 869], 99.50th=[ 869], 99.90th=[ 869], 99.95th=[ 869], | 99.99th=[ 869] bw ( KiB/s): min= 6627, max= 8192, per=100.00%, avg=7409.50, stdev=1106.62, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 500=33.33%, 1000=33.33% cpu : usr=0.00%, sys=17.25%, ctx=283, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183715: Sun Nov 16 23:52:04 2025 read: IOPS=0, BW=1841KiB/s (1885kB/s)(4096KiB/2225msec) clat (nsec): min=527767k, max=527767k, avg=527767355.00, stdev= 0.00 lat (nsec): min=527769k, max=527769k, avg=527768577.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 527], 5.00th=[ 527], 10.00th=[ 527], 20.00th=[ 527], | 30.00th=[ 527], 40.00th=[ 527], 50.00th=[ 527], 60.00th=[ 527], | 70.00th=[ 527], 80.00th=[ 527], 90.00th=[ 527], 95.00th=[ 527], | 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527], | 99.99th=[ 527] bw ( KiB/s): min= 8192, max= 8192, per=100.00%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=0, BW=3682KiB/s (3770kB/s)(8192KiB/2225msec); 0 zone resets clat (msec): min=404, max=1192, avg=798.51, stdev=556.64 lat (msec): min=405, max=1192, avg=798.80, stdev=556.49 clat percentiles (msec): | 1.00th=[ 405], 5.00th=[ 405], 10.00th=[ 405], 20.00th=[ 405], | 30.00th=[ 405], 40.00th=[ 405], 50.00th=[ 405], 60.00th=[ 1200], | 70.00th=[ 1200], 80.00th=[ 1200], 90.00th=[ 1200], 95.00th=[ 1200], | 99.00th=[ 1200], 99.50th=[ 1200], 99.90th=[ 1200], 99.95th=[ 1200], | 99.99th=[ 1200] bw ( KiB/s): min= 8192, max= 8192, per=57.30%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=16.28%, ctx=92, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183717: Sun Nov 16 23:52:04 2025 write: IOPS=1, BW=5361KiB/s (5490kB/s)(12.0MiB/2292msec); 0 zone resets clat (msec): min=165, max=1315, avg=707.55, stdev=578.11 lat (msec): min=165, max=1316, avg=707.99, stdev=577.93 clat percentiles (msec): | 1.00th=[ 165], 5.00th=[ 165], 10.00th=[ 165], 20.00th=[ 165], | 30.00th=[ 165], 40.00th=[ 642], 50.00th=[ 642], 60.00th=[ 642], | 70.00th=[ 1318], 80.00th=[ 1318], 90.00th=[ 1318], 95.00th=[ 1318], | 99.00th=[ 1318], 99.50th=[ 1318], 99.90th=[ 1318], 99.95th=[ 1318], | 99.99th=[ 1318] bw ( KiB/s): min= 8159, max= 8192, per=57.19%, avg=8175.50, stdev=23.33, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 750=33.33%, 2000=33.33% cpu : usr=0.00%, sys=16.94%, ctx=90, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=183719: Sun Nov 16 23:52:04 2025 write: IOPS=1, BW=7367KiB/s (7544kB/s)(12.0MiB/1668msec); 0 zone resets clat (msec): min=450, max=553, avg=515.38, stdev=56.70 lat (msec): min=450, max=553, avg=515.70, stdev=56.69 clat percentiles (msec): | 1.00th=[ 451], 5.00th=[ 451], 10.00th=[ 451], 20.00th=[ 451], | 30.00th=[ 451], 40.00th=[ 542], 50.00th=[ 542], 60.00th=[ 542], | 70.00th=[ 550], 80.00th=[ 550], 90.00th=[ 550], 95.00th=[ 550], | 99.00th=[ 550], 99.50th=[ 550], 99.90th=[ 550], 99.95th=[ 550], | 99.99th=[ 550] bw ( KiB/s): min= 8159, max= 8192, per=57.19%, avg=8175.50, stdev=23.33, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.00%, sys=20.64%, ctx=69, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7364KiB/s (7540kB/s), 1841KiB/s-8052KiB/s (1885kB/s-8246kB/s), io=16.0MiB (16.8MB), run=1526-2225msec WRITE: bw=13.0MiB/s (14.6MB/s), 3682KiB/s-7367KiB/s (3770kB/s-7544kB/s), io=32.0MiB (33.6MB), run=1668-2292msec