== sanity test 398b: DIO and buffer IO race ============== 10:40:14 (1774881614) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 6.06003 s, 8.3 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=181809: Mon Mar 30 10:42:55 2026 read: IOPS=10, BW=43.3KiB/s (44.3kB/s)(5992KiB/138487msec) clat (usec): min=46, max=989676, avg=51983.38, stdev=41278.73 lat (usec): min=47, max=989677, avg=51985.24, stdev=41278.56 clat percentiles (usec): | 1.00th=[ 52], 5.00th=[ 12387], 10.00th=[ 18744], 20.00th=[ 25822], | 30.00th=[ 31851], 40.00th=[ 37487], 50.00th=[ 44303], 60.00th=[ 52167], | 70.00th=[ 61080], 80.00th=[ 72877], 90.00th=[ 93848], 95.00th=[115868], | 99.00th=[164627], 99.50th=[200279], 99.90th=[238027], 99.95th=[985662], | 99.99th=[985662] bw ( KiB/s): min= 7, max= 119, per=25.15%, avg=43.00, stdev=17.64, samples=271 iops : min= 1, max= 29, avg=10.09, stdev= 4.44, samples=271 write: IOPS=11, BW=45.5KiB/s (46.6kB/s)(6296KiB/138487msec); 0 zone resets clat (usec): min=797, max=302251, avg=38195.92, stdev=41002.94 lat (usec): min=799, max=302253, avg=38200.71, stdev=41002.39 clat percentiles (usec): | 1.00th=[ 1188], 5.00th=[ 1352], 10.00th=[ 1582], 20.00th=[ 3884], | 30.00th=[ 8586], 40.00th=[ 17695], 50.00th=[ 27132], 60.00th=[ 35390], | 70.00th=[ 48497], 80.00th=[ 65274], 90.00th=[ 90702], 95.00th=[116917], | 99.00th=[189793], 99.50th=[227541], 99.90th=[256902], 99.95th=[304088], | 99.99th=[304088] bw ( KiB/s): min= 7, max= 117, per=25.24%, avg=45.18, stdev=20.08, samples=271 iops : min= 1, max= 29, avg=10.65, stdev= 5.06, samples=271 lat (usec) : 50=0.26%, 100=1.66%, 250=0.13%, 1000=0.07% lat (msec) : 2=6.48%, 4=3.91%, 10=6.18%, 20=8.53%, 50=37.60% lat (msec) : 100=27.15%, 250=7.91%, 500=0.10%, 1000=0.03% cpu : usr=0.06%, sys=4.52%, ctx=6438, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181810: Mon Mar 30 10:42:55 2026 read: IOPS=10, BW=42.5KiB/s (43.6kB/s)(5940KiB/139624msec) clat (usec): min=39, max=994331, avg=52004.60, stdev=41807.96 lat (usec): min=41, max=994332, avg=52007.50, stdev=41807.74 clat percentiles (usec): | 1.00th=[ 55], 5.00th=[ 12387], 10.00th=[ 18744], 20.00th=[ 25822], | 30.00th=[ 31327], 40.00th=[ 36439], 50.00th=[ 44303], 60.00th=[ 51119], | 70.00th=[ 61080], 80.00th=[ 73925], 90.00th=[ 92799], 95.00th=[119014], | 99.00th=[175113], 99.50th=[202376], 99.90th=[240124], 99.95th=[994051], | 99.99th=[994051] bw ( KiB/s): min= 7, max= 104, per=24.73%, avg=42.30, stdev=16.72, samples=273 iops : min= 1, max= 26, avg= 9.91, stdev= 4.22, samples=273 write: IOPS=11, BW=45.5KiB/s (46.6kB/s)(6348KiB/139624msec); 0 zone resets clat (usec): min=722, max=268212, avg=38975.32, stdev=40336.02 lat (usec): min=722, max=268213, avg=38990.62, stdev=40330.84 clat percentiles (usec): | 1.00th=[ 1205], 5.00th=[ 1369], 10.00th=[ 1729], 20.00th=[ 4080], | 30.00th=[ 8979], 40.00th=[ 18482], 50.00th=[ 27395], 60.00th=[ 38011], | 70.00th=[ 49546], 80.00th=[ 66847], 90.00th=[ 92799], 95.00th=[117965], | 99.00th=[175113], 99.50th=[204473], 99.90th=[248513], 99.95th=[267387], | 99.99th=[267387] bw ( KiB/s): min= 7, max= 135, per=25.05%, avg=44.84, stdev=22.40, samples=274 iops : min= 1, max= 33, avg=10.55, stdev= 5.62, samples=274 lat (usec) : 50=0.23%, 100=1.40%, 250=0.13%, 750=0.07%, 1000=0.03% lat (msec) : 2=5.83%, 4=4.20%, 10=6.61%, 20=8.17%, 50=37.89% lat (msec) : 100=27.15%, 250=8.24%, 500=0.03%, 1000=0.03% cpu : usr=0.07%, sys=4.49%, ctx=6547, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181811: Mon Mar 30 10:42:55 2026 read: IOPS=10, BW=43.6KiB/s (44.7kB/s)(6028KiB/138135msec) clat (usec): min=48, max=286655, avg=51340.02, stdev=34705.58 lat (usec): min=49, max=286655, avg=51341.65, stdev=34705.54 clat percentiles (usec): | 1.00th=[ 55], 5.00th=[ 11469], 10.00th=[ 18482], 20.00th=[ 24773], | 30.00th=[ 31065], 40.00th=[ 36963], 50.00th=[ 43779], 60.00th=[ 51643], | 70.00th=[ 61080], 80.00th=[ 72877], 90.00th=[ 94897], 95.00th=[114820], | 99.00th=[170918], 99.50th=[196084], 99.90th=[246416], 99.95th=[287310], | 99.99th=[287310] bw ( KiB/s): min= 7, max= 102, per=25.50%, avg=43.60, stdev=17.33, samples=269 iops : min= 1, max= 25, avg=10.24, stdev= 4.36, samples=269 write: IOPS=11, BW=45.3KiB/s (46.4kB/s)(6260KiB/138135msec); 0 zone resets clat (usec): min=726, max=936928, avg=38525.62, stdev=45929.59 lat (usec): min=726, max=936930, avg=38537.50, stdev=45928.46 clat percentiles (usec): | 1.00th=[ 1205], 5.00th=[ 1336], 10.00th=[ 1532], 20.00th=[ 3523], | 30.00th=[ 7767], 40.00th=[ 16057], 50.00th=[ 26084], 60.00th=[ 35914], | 70.00th=[ 47973], 80.00th=[ 65799], 90.00th=[ 95945], 95.00th=[121111], | 99.00th=[173016], 99.50th=[187696], 99.90th=[223347], 99.95th=[935330], | 99.99th=[935330] bw ( KiB/s): min= 7, max= 111, per=25.17%, avg=45.06, stdev=20.17, samples=270 iops : min= 1, max= 27, avg=10.61, stdev= 5.05, samples=270 lat (usec) : 50=0.13%, 100=1.82%, 250=0.03%, 500=0.07%, 750=0.03% lat (usec) : 1000=0.10% lat (msec) : 2=6.61%, 4=4.04%, 10=6.64%, 20=8.53%, 50=36.78% lat (msec) : 100=26.20%, 250=8.95%, 500=0.03%, 1000=0.03% cpu : usr=0.07%, sys=4.50%, ctx=6572, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181812: Mon Mar 30 10:42:55 2026 read: IOPS=10, BW=43.2KiB/s (44.3kB/s)(6052KiB/140008msec) clat (usec): min=32, max=979774, avg=51208.90, stdev=43080.37 lat (usec): min=33, max=979775, avg=51210.75, stdev=43080.27 clat percentiles (usec): | 1.00th=[ 53], 5.00th=[ 8717], 10.00th=[ 16909], 20.00th=[ 24511], | 30.00th=[ 30540], 40.00th=[ 36439], 50.00th=[ 43254], 60.00th=[ 51119], | 70.00th=[ 60031], 80.00th=[ 70779], 90.00th=[ 91751], 95.00th=[113771], | 99.00th=[187696], 99.50th=[229639], 99.90th=[320865], 99.95th=[977273], | 99.99th=[977273] bw ( KiB/s): min= 7, max= 111, per=25.07%, avg=42.87, stdev=18.47, samples=275 iops : min= 1, max= 27, avg=10.05, stdev= 4.65, samples=275 write: IOPS=11, BW=44.5KiB/s (45.6kB/s)(6236KiB/140008msec); 0 zone resets clat (usec): min=961, max=326558, avg=39812.02, stdev=40917.98 lat (usec): min=963, max=326560, avg=39817.46, stdev=40918.07 clat percentiles (usec): | 1.00th=[ 1205], 5.00th=[ 1336], 10.00th=[ 1549], 20.00th=[ 4178], | 30.00th=[ 9372], 40.00th=[ 18220], 50.00th=[ 27919], 60.00th=[ 40109], | 70.00th=[ 53740], 80.00th=[ 69731], 90.00th=[ 94897], 95.00th=[114820], | 99.00th=[181404], 99.50th=[208667], 99.90th=[295699], 99.95th=[325059], | 99.99th=[325059] bw ( KiB/s): min= 7, max= 111, per=24.65%, avg=44.13, stdev=20.43, samples=275 iops : min= 1, max= 27, avg=10.37, stdev= 5.14, samples=275 lat (usec) : 50=0.29%, 100=1.73%, 250=0.23%, 750=0.03%, 1000=0.16% lat (msec) : 2=6.25%, 4=3.55%, 10=5.92%, 20=9.60%, 50=35.22% lat (msec) : 100=28.94%, 250=7.88%, 500=0.16%, 1000=0.03% cpu : usr=0.07%, sys=4.46%, ctx=6453, majf=0, minf=18 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=172KiB/s (176kB/s), 42.5KiB/s-43.6KiB/s (43.6kB/s-44.7kB/s), io=23.4MiB (24.6MB), run=138135-140008msec WRITE: bw=180KiB/s (184kB/s), 44.5KiB/s-45.5KiB/s (45.6kB/s-46.6kB/s), io=24.6MiB (25.7MB), run=138135-140008msec rand-rw: (groupid=0, jobs=1): err= 0: pid=181814: Mon Mar 30 10:43:38 2026 read: IOPS=8, BW=32.9KiB/s (33.7kB/s)(5992KiB/181922msec) clat (msec): min=5, max=1034, avg=47.58, stdev=52.70 lat (msec): min=5, max=1034, avg=47.58, stdev=52.70 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 13], 20.00th=[ 17], | 30.00th=[ 20], 40.00th=[ 25], 50.00th=[ 30], 60.00th=[ 38], | 70.00th=[ 51], 80.00th=[ 68], 90.00th=[ 104], 95.00th=[ 146], | 99.00th=[ 234], 99.50th=[ 271], 99.90th=[ 368], 99.95th=[ 1036], | 99.99th=[ 1036] bw ( KiB/s): min= 7, max= 136, per=25.66%, avg=33.62, stdev=25.14, samples=344 iops : min= 1, max= 34, avg= 7.76, stdev= 6.34, samples=344 write: IOPS=8, BW=34.6KiB/s (35.4kB/s)(6296KiB/181922msec); 0 zone resets clat (msec): min=8, max=443, avg=70.21, stdev=51.11 lat (msec): min=8, max=443, avg=70.21, stdev=51.11 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 21], 10.00th=[ 25], 20.00th=[ 32], | 30.00th=[ 38], 40.00th=[ 45], 50.00th=[ 53], 60.00th=[ 64], | 70.00th=[ 83], 80.00th=[ 107], 90.00th=[ 140], 95.00th=[ 174], | 99.00th=[ 253], 99.50th=[ 268], 99.90th=[ 355], 99.95th=[ 443], | 99.99th=[ 443] bw ( KiB/s): min= 7, max= 134, per=25.55%, avg=35.00, stdev=23.16, samples=348 iops : min= 1, max= 33, avg= 8.10, stdev= 5.85, samples=348 lat (msec) : 10=2.51%, 20=14.84%, 50=41.11%, 100=24.54%, 250=16.11% lat (msec) : 500=0.85%, 2000=0.03% cpu : usr=0.06%, sys=1.76%, ctx=4797, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181815: Mon Mar 30 10:43:38 2026 read: IOPS=8, BW=32.6KiB/s (33.4kB/s)(5940KiB/182270msec) clat (msec): min=4, max=1030, avg=46.41, stdev=51.11 lat (msec): min=4, max=1030, avg=46.41, stdev=51.11 clat percentiles (msec): | 1.00th=[ 7], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 16], | 30.00th=[ 20], 40.00th=[ 24], 50.00th=[ 30], 60.00th=[ 37], | 70.00th=[ 47], 80.00th=[ 67], 90.00th=[ 106], 95.00th=[ 138], | 99.00th=[ 220], 99.50th=[ 255], 99.90th=[ 330], 99.95th=[ 1028], | 99.99th=[ 1028] bw ( KiB/s): min= 7, max= 159, per=25.84%, avg=33.85, stdev=25.85, samples=339 iops : min= 1, max= 39, avg= 7.81, stdev= 6.52, samples=339 write: IOPS=8, BW=34.8KiB/s (35.7kB/s)(6348KiB/182270msec); 0 zone resets clat (msec): min=8, max=447, avg=71.30, stdev=54.65 lat (msec): min=8, max=447, avg=71.30, stdev=54.65 clat percentiles (msec): | 1.00th=[ 14], 5.00th=[ 20], 10.00th=[ 24], 20.00th=[ 31], | 30.00th=[ 36], 40.00th=[ 42], 50.00th=[ 52], 60.00th=[ 66], | 70.00th=[ 84], 80.00th=[ 109], 90.00th=[ 146], 95.00th=[ 184], | 99.00th=[ 259], 99.50th=[ 300], 99.90th=[ 393], 99.95th=[ 447], | 99.99th=[ 447] bw ( KiB/s): min= 7, max= 119, per=25.14%, avg=34.44, stdev=23.30, samples=355 iops : min= 1, max= 29, avg= 7.96, stdev= 5.88, samples=355 lat (msec) : 10=3.19%, 20=14.13%, 50=42.38%, 100=22.85%, 250=16.57% lat (msec) : 500=0.85%, 2000=0.03% cpu : usr=0.05%, sys=1.81%, ctx=4842, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181816: Mon Mar 30 10:43:38 2026 read: IOPS=8, BW=33.2KiB/s (33.0kB/s)(6028KiB/181600msec) clat (msec): min=4, max=365, avg=46.68, stdev=46.08 lat (msec): min=4, max=365, avg=46.68, stdev=46.08 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 16], | 30.00th=[ 21], 40.00th=[ 25], 50.00th=[ 30], 60.00th=[ 37], | 70.00th=[ 50], 80.00th=[ 67], 90.00th=[ 107], 95.00th=[ 144], | 99.00th=[ 228], 99.50th=[ 268], 99.90th=[ 334], 99.95th=[ 368], | 99.99th=[ 368] bw ( KiB/s): min= 7, max= 119, per=26.28%, avg=34.43, stdev=24.54, samples=340 iops : min= 1, max= 29, avg= 7.96, stdev= 6.20, samples=340 write: IOPS=8, BW=34.5KiB/s (35.3kB/s)(6260KiB/181600msec); 0 zone resets clat (msec): min=9, max=1114, avg=71.01, stdev=57.74 lat (msec): min=9, max=1114, avg=71.02, stdev=57.74 clat percentiles (msec): | 1.00th=[ 15], 5.00th=[ 21], 10.00th=[ 24], 20.00th=[ 31], | 30.00th=[ 37], 40.00th=[ 44], 50.00th=[ 54], 60.00th=[ 66], | 70.00th=[ 85], 80.00th=[ 105], 90.00th=[ 140], 95.00th=[ 178], | 99.00th=[ 253], 99.50th=[ 279], 99.90th=[ 342], 99.95th=[ 1116], | 99.99th=[ 1116] bw ( KiB/s): min= 7, max= 119, per=25.12%, avg=34.42, stdev=24.37, samples=353 iops : min= 1, max= 29, avg= 7.95, stdev= 6.14, samples=353 lat (msec) : 10=3.06%, 20=13.77%, 50=41.80%, 100=24.45%, 250=16.11% lat (msec) : 500=0.78%, 2000=0.03% cpu : usr=0.05%, sys=1.73%, ctx=4830, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181817: Mon Mar 30 10:43:38 2026 read: IOPS=8, BW=33.3KiB/s (34.1kB/s)(6052KiB/181809msec) clat (msec): min=5, max=322, avg=46.57, stdev=44.05 lat (msec): min=5, max=322, avg=46.57, stdev=44.05 clat percentiles (msec): | 1.00th=[ 8], 5.00th=[ 11], 10.00th=[ 13], 20.00th=[ 17], | 30.00th=[ 21], 40.00th=[ 26], 50.00th=[ 32], 60.00th=[ 38], | 70.00th=[ 47], 80.00th=[ 69], 90.00th=[ 107], 95.00th=[ 138], | 99.00th=[ 205], 99.50th=[ 253], 99.90th=[ 317], 99.95th=[ 321], | 99.99th=[ 321] bw ( KiB/s): min= 7, max= 159, per=26.11%, avg=34.20, stdev=25.38, samples=343 iops : min= 1, max= 39, avg= 7.90, stdev= 6.40, samples=343 write: IOPS=8, BW=34.3KiB/s (35.1kB/s)(6236KiB/181809msec); 0 zone resets clat (msec): min=10, max=1061, avg=71.30, stdev=57.69 lat (msec): min=10, max=1061, avg=71.31, stdev=57.69 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 21], 10.00th=[ 24], 20.00th=[ 31], | 30.00th=[ 37], 40.00th=[ 44], 50.00th=[ 53], 60.00th=[ 66], | 70.00th=[ 83], 80.00th=[ 107], 90.00th=[ 142], 95.00th=[ 178], | 99.00th=[ 251], 99.50th=[ 292], 99.90th=[ 384], 99.95th=[ 1062], | 99.99th=[ 1062] bw ( KiB/s): min= 7, max= 136, per=25.23%, avg=34.56, stdev=23.39, samples=349 iops : min= 1, max= 34, avg= 7.99, stdev= 5.91, samples=349 lat (msec) : 10=1.99%, 20=14.29%, 50=43.16%, 100=23.70%, 250=16.08% lat (msec) : 500=0.75%, 2000=0.03% cpu : usr=0.05%, sys=1.82%, ctx=4940, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=132KiB/s (135kB/s), 32.6KiB/s-33.3KiB/s (33.4kB/s-34.1kB/s), io=23.4MiB (24.6MB), run=181600-182270msec WRITE: bw=138KiB/s (141kB/s), 34.3KiB/s-34.8KiB/s (35.1kB/s-35.7kB/s), io=24.6MiB (25.7MB), run=181600-182270msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=181862: Mon Mar 30 10:44:22 2026 read: IOPS=10, BW=165KiB/s (169kB/s)(5920KiB/35913msec) clat (usec): min=49, max=244835, avg=52805.89, stdev=35536.59 lat (usec): min=49, max=244836, avg=52807.39, stdev=35536.59 clat percentiles (usec): | 1.00th=[ 55], 5.00th=[ 65], 10.00th=[ 14353], 20.00th=[ 25822], | 30.00th=[ 31589], 40.00th=[ 39584], 50.00th=[ 46924], 60.00th=[ 56361], | 70.00th=[ 64750], 80.00th=[ 76022], 90.00th=[ 95945], 95.00th=[111674], | 99.00th=[168821], 99.50th=[202376], 99.90th=[244319], 99.95th=[244319], | 99.99th=[244319] bw ( KiB/s): min= 27, max= 349, per=24.26%, avg=161.30, stdev=67.57, samples=70 iops : min= 1, max= 21, avg= 9.27, stdev= 4.32, samples=70 write: IOPS=11, BW=177KiB/s (182kB/s)(6368KiB/35913msec); 0 zone resets clat (usec): min=1424, max=217397, avg=39834.57, stdev=40746.49 lat (usec): min=1426, max=217399, avg=39864.63, stdev=40726.81 clat percentiles (usec): | 1.00th=[ 1647], 5.00th=[ 1926], 10.00th=[ 2573], 20.00th=[ 6456], | 30.00th=[ 11600], 40.00th=[ 17957], 50.00th=[ 27919], 60.00th=[ 38011], | 70.00th=[ 47449], 80.00th=[ 66323], 90.00th=[ 95945], 95.00th=[125305], | 99.00th=[193987], 99.50th=[212861], 99.90th=[217056], 99.95th=[217056], | 99.99th=[217056] bw ( KiB/s): min= 31, max= 412, per=25.43%, avg=177.75, stdev=75.30, samples=69 iops : min= 1, max= 25, avg=10.32, stdev= 4.80, samples=69 lat (usec) : 50=0.13%, 100=3.39%, 250=0.26%, 500=0.13% lat (msec) : 2=3.12%, 4=4.56%, 10=6.77%, 20=9.77%, 50=34.64% lat (msec) : 100=28.65%, 250=8.59% cpu : usr=0.09%, sys=4.44%, ctx=1645, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181863: Mon Mar 30 10:44:22 2026 read: IOPS=10, BW=162KiB/s (166kB/s)(5744KiB/35432msec) clat (usec): min=40, max=196109, avg=52433.76, stdev=32375.08 lat (usec): min=41, max=196111, avg=52436.47, stdev=32374.67 clat percentiles (usec): | 1.00th=[ 60], 5.00th=[ 86], 10.00th=[ 19530], 20.00th=[ 27919], | 30.00th=[ 34341], 40.00th=[ 40109], 50.00th=[ 46400], 60.00th=[ 54789], | 70.00th=[ 64226], 80.00th=[ 73925], 90.00th=[ 93848], 95.00th=[108528], | 99.00th=[173016], 99.50th=[183501], 99.90th=[196084], 99.95th=[196084], | 99.99th=[196084] bw ( KiB/s): min= 30, max= 320, per=24.26%, avg=161.32, stdev=65.59, samples=69 iops : min= 1, max= 20, avg= 9.28, stdev= 4.17, samples=69 write: IOPS=11, BW=185KiB/s (189kB/s)(6544KiB/35432msec); 0 zone resets clat (usec): min=1440, max=324996, avg=40255.44, stdev=40548.98 lat (usec): min=1443, max=324998, avg=40261.18, stdev=40548.08 clat percentiles (usec): | 1.00th=[ 1680], 5.00th=[ 1942], 10.00th=[ 2769], 20.00th=[ 6325], | 30.00th=[ 12125], 40.00th=[ 19530], 50.00th=[ 27395], 60.00th=[ 40109], | 70.00th=[ 52691], 80.00th=[ 66847], 90.00th=[ 92799], 95.00th=[114820], | 99.00th=[173016], 99.50th=[196084], 99.90th=[325059], 99.95th=[325059], | 99.99th=[325059] bw ( KiB/s): min= 31, max= 473, per=26.30%, avg=183.81, stdev=91.02, samples=69 iops : min= 1, max= 29, avg=10.70, stdev= 5.79, samples=69 lat (usec) : 50=0.13%, 100=2.34%, 250=0.52%, 1000=0.13% lat (msec) : 2=3.26%, 4=4.43%, 10=6.25%, 20=9.77%, 50=34.51% lat (msec) : 100=30.86%, 250=7.68%, 500=0.13% cpu : usr=0.07%, sys=4.65%, ctx=1639, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181864: Mon Mar 30 10:44:22 2026 read: IOPS=10, BW=174KiB/s (178kB/s)(6256KiB/36006msec) clat (usec): min=31, max=229117, avg=50835.91, stdev=34149.86 lat (usec): min=32, max=229118, avg=50837.54, stdev=34149.84 clat percentiles (usec): | 1.00th=[ 47], 5.00th=[ 69], 10.00th=[ 16581], 20.00th=[ 25560], | 30.00th=[ 31589], 40.00th=[ 37487], 50.00th=[ 44303], 60.00th=[ 53216], | 70.00th=[ 62129], 80.00th=[ 74974], 90.00th=[ 96994], 95.00th=[115868], | 99.00th=[160433], 99.50th=[166724], 99.90th=[229639], 99.95th=[229639], | 99.99th=[229639] bw ( KiB/s): min= 30, max= 479, per=26.49%, avg=176.17, stdev=77.84, samples=69 iops : min= 1, max= 29, avg=10.20, stdev= 4.96, samples=69 write: IOPS=10, BW=168KiB/s (172kB/s)(6032KiB/36006msec); 0 zone resets clat (usec): min=1183, max=237870, avg=41471.32, stdev=39620.50 lat (usec): min=1187, max=237872, avg=41473.78, stdev=39620.56 clat percentiles (usec): | 1.00th=[ 1729], 5.00th=[ 1942], 10.00th=[ 3261], 20.00th=[ 7308], | 30.00th=[ 12518], 40.00th=[ 21103], 50.00th=[ 30540], 60.00th=[ 40109], | 70.00th=[ 52167], 80.00th=[ 69731], 90.00th=[ 93848], 95.00th=[117965], | 99.00th=[185598], 99.50th=[208667], 99.90th=[238027], 99.95th=[238027], | 99.99th=[238027] bw ( KiB/s): min= 61, max= 351, per=24.23%, avg=169.40, stdev=63.45, samples=68 iops : min= 3, max= 21, avg= 9.78, stdev= 3.98, samples=68 lat (usec) : 50=0.52%, 100=3.26%, 250=0.26%, 500=0.13%, 750=0.13% lat (msec) : 2=2.73%, 4=2.99%, 10=6.64%, 20=9.11%, 50=36.59% lat (msec) : 100=28.78%, 250=8.85% cpu : usr=0.06%, sys=4.77%, ctx=1664, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181865: Mon Mar 30 10:44:22 2026 read: IOPS=10, BW=170KiB/s (174kB/s)(6032KiB/35474msec) clat (usec): min=32, max=232311, avg=52280.72, stdev=35245.64 lat (usec): min=32, max=232313, avg=52282.21, stdev=35245.67 clat percentiles (usec): | 1.00th=[ 55], 5.00th=[ 71], 10.00th=[ 10421], 20.00th=[ 27657], | 30.00th=[ 33817], 40.00th=[ 40109], 50.00th=[ 47973], 60.00th=[ 55313], | 70.00th=[ 63701], 80.00th=[ 72877], 90.00th=[ 95945], 95.00th=[115868], | 99.00th=[185598], 99.50th=[212861], 99.90th=[231736], 99.95th=[231736], | 99.99th=[231736] bw ( KiB/s): min= 31, max= 348, per=25.46%, avg=169.29, stdev=76.21, samples=68 iops : min= 1, max= 21, avg= 9.78, stdev= 4.84, samples=68 write: IOPS=11, BW=176KiB/s (181kB/s)(6256KiB/35474msec); 0 zone resets clat (usec): min=1557, max=188108, avg=38910.17, stdev=41035.60 lat (usec): min=1560, max=188110, avg=38912.57, stdev=41035.62 clat percentiles (usec): | 1.00th=[ 1811], 5.00th=[ 2008], 10.00th=[ 2704], 20.00th=[ 5800], | 30.00th=[ 8717], 40.00th=[ 16450], 50.00th=[ 23987], 60.00th=[ 34341], | 70.00th=[ 49021], 80.00th=[ 66847], 90.00th=[ 99091], 95.00th=[126354], | 99.00th=[177210], 99.50th=[183501], 99.90th=[187696], 99.95th=[187696], | 99.99th=[187696] bw ( KiB/s): min= 28, max= 416, per=24.91%, avg=174.09, stdev=78.00, samples=69 iops : min= 1, max= 26, avg=10.09, stdev= 4.98, samples=69 lat (usec) : 50=0.26%, 100=3.78%, 250=0.26%, 500=0.13% lat (msec) : 2=2.47%, 4=5.34%, 10=9.51%, 20=7.94%, 50=32.81% lat (msec) : 100=29.17%, 250=8.33% cpu : usr=0.07%, sys=5.03%, ctx=1642, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=665KiB/s (681kB/s), 162KiB/s-174KiB/s (166kB/s-178kB/s), io=23.4MiB (24.5MB), run=35432-36006msec WRITE: bw=700KiB/s (717kB/s), 168KiB/s-185KiB/s (172kB/s-189kB/s), io=24.6MiB (25.8MB), run=35432-36006msec rand-rw: (groupid=0, jobs=1): err= 0: pid=181858: Mon Mar 30 10:44:37 2026 read: IOPS=7, BW=118KiB/s (120kB/s)(5920KiB/50334msec) clat (msec): min=6, max=297, avg=52.42, stdev=46.46 lat (msec): min=6, max=297, avg=52.42, stdev=46.46 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 15], 10.00th=[ 18], 20.00th=[ 22], | 30.00th=[ 27], 40.00th=[ 31], 50.00th=[ 36], 60.00th=[ 43], | 70.00th=[ 55], 80.00th=[ 73], 90.00th=[ 106], 95.00th=[ 161], | 99.00th=[ 230], 99.50th=[ 259], 99.90th=[ 296], 99.95th=[ 296], | 99.99th=[ 296] bw ( KiB/s): min= 29, max= 318, per=26.11%, avg=123.49, stdev=70.94, samples=94 iops : min= 1, max= 19, avg= 6.87, stdev= 4.44, samples=94 write: IOPS=7, BW=127KiB/s (130kB/s)(6368KiB/50334msec); 0 zone resets clat (msec): min=17, max=330, avg=76.57, stdev=51.90 lat (msec): min=17, max=330, avg=76.59, stdev=51.89 clat percentiles (msec): | 1.00th=[ 21], 5.00th=[ 28], 10.00th=[ 32], 20.00th=[ 39], | 30.00th=[ 45], 40.00th=[ 52], 50.00th=[ 58], 60.00th=[ 70], | 70.00th=[ 87], 80.00th=[ 111], 90.00th=[ 144], 95.00th=[ 188], | 99.00th=[ 279], 99.50th=[ 326], 99.90th=[ 330], 99.95th=[ 330], | 99.99th=[ 330] bw ( KiB/s): min= 31, max= 318, per=26.12%, avg=129.81, stdev=72.46, samples=96 iops : min= 1, max= 19, avg= 7.27, stdev= 4.58, samples=96 lat (msec) : 10=0.52%, 20=7.42%, 50=43.88%, 100=30.86%, 250=16.41% lat (msec) : 500=0.91% cpu : usr=0.05%, sys=1.45%, ctx=1113, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181859: Mon Mar 30 10:44:37 2026 read: IOPS=7, BW=113KiB/s (116kB/s)(5744KiB/50621msec) clat (msec): min=6, max=293, avg=51.46, stdev=43.96 lat (msec): min=6, max=293, avg=51.46, stdev=43.96 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 14], 10.00th=[ 17], 20.00th=[ 23], | 30.00th=[ 27], 40.00th=[ 31], 50.00th=[ 36], 60.00th=[ 42], | 70.00th=[ 53], 80.00th=[ 72], 90.00th=[ 110], 95.00th=[ 153], | 99.00th=[ 207], 99.50th=[ 218], 99.90th=[ 292], 99.95th=[ 292], | 99.99th=[ 292] bw ( KiB/s): min= 28, max= 382, per=25.79%, avg=121.98, stdev=77.54, samples=91 iops : min= 1, max= 23, avg= 6.78, stdev= 4.89, samples=91 write: IOPS=8, BW=129KiB/s (132kB/s)(6544KiB/50621msec); 0 zone resets clat (msec): min=17, max=290, avg=77.82, stdev=47.38 lat (msec): min=17, max=290, avg=77.83, stdev=47.39 clat percentiles (msec): | 1.00th=[ 21], 5.00th=[ 28], 10.00th=[ 33], 20.00th=[ 41], | 30.00th=[ 47], 40.00th=[ 55], 50.00th=[ 64], 60.00th=[ 73], | 70.00th=[ 90], 80.00th=[ 115], 90.00th=[ 148], 95.00th=[ 171], | 99.00th=[ 226], 99.50th=[ 279], 99.90th=[ 292], 99.95th=[ 292], | 99.99th=[ 292] bw ( KiB/s): min= 28, max= 351, per=25.83%, avg=128.38, stdev=70.34, samples=98 iops : min= 1, max= 21, avg= 7.17, stdev= 4.44, samples=98 lat (msec) : 10=0.65%, 20=6.64%, 50=43.36%, 100=29.82%, 250=19.01% lat (msec) : 500=0.52% cpu : usr=0.04%, sys=1.76%, ctx=1193, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181860: Mon Mar 30 10:44:37 2026 read: IOPS=7, BW=124KiB/s (127kB/s)(6256KiB/50540msec) clat (msec): min=7, max=438, avg=50.59, stdev=46.81 lat (msec): min=7, max=438, avg=50.59, stdev=46.81 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 14], 10.00th=[ 18], 20.00th=[ 22], | 30.00th=[ 27], 40.00th=[ 31], 50.00th=[ 36], 60.00th=[ 43], | 70.00th=[ 52], 80.00th=[ 68], 90.00th=[ 103], 95.00th=[ 133], | 99.00th=[ 228], 99.50th=[ 305], 99.90th=[ 439], 99.95th=[ 439], | 99.99th=[ 439] bw ( KiB/s): min= 29, max= 351, per=26.94%, avg=127.41, stdev=81.79, samples=96 iops : min= 1, max= 21, avg= 7.13, stdev= 5.19, samples=96 write: IOPS=7, BW=119KiB/s (122kB/s)(6032KiB/50540msec); 0 zone resets clat (msec): min=19, max=337, avg=80.38, stdev=53.76 lat (msec): min=19, max=337, avg=80.40, stdev=53.75 clat percentiles (msec): | 1.00th=[ 24], 5.00th=[ 29], 10.00th=[ 34], 20.00th=[ 41], | 30.00th=[ 46], 40.00th=[ 53], 50.00th=[ 61], 60.00th=[ 75], | 70.00th=[ 89], 80.00th=[ 118], 90.00th=[ 157], 95.00th=[ 190], | 99.00th=[ 313], 99.50th=[ 334], 99.90th=[ 338], 99.95th=[ 338], | 99.99th=[ 338] bw ( KiB/s): min= 29, max= 286, per=23.77%, avg=118.14, stdev=66.85, samples=98 iops : min= 1, max= 17, avg= 6.53, stdev= 4.18, samples=98 lat (msec) : 10=0.52%, 20=7.55%, 50=44.79%, 100=29.04%, 250=17.19% lat (msec) : 500=0.91% cpu : usr=0.05%, sys=1.70%, ctx=1232, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181861: Mon Mar 30 10:44:37 2026 read: IOPS=7, BW=120KiB/s (123kB/s)(6032KiB/50302msec) clat (msec): min=7, max=347, avg=50.58, stdev=44.40 lat (msec): min=7, max=347, avg=50.59, stdev=44.40 clat percentiles (msec): | 1.00th=[ 10], 5.00th=[ 14], 10.00th=[ 18], 20.00th=[ 23], | 30.00th=[ 27], 40.00th=[ 32], 50.00th=[ 35], 60.00th=[ 42], | 70.00th=[ 53], 80.00th=[ 68], 90.00th=[ 100], 95.00th=[ 136], | 99.00th=[ 241], 99.50th=[ 266], 99.90th=[ 347], 99.95th=[ 347], | 99.99th=[ 347] bw ( KiB/s): min= 31, max= 351, per=26.77%, avg=126.62, stdev=71.90, samples=93 iops : min= 1, max= 21, avg= 7.06, stdev= 4.55, samples=93 write: IOPS=7, BW=124KiB/s (127kB/s)(6256KiB/50302msec); 0 zone resets clat (msec): min=15, max=338, avg=79.66, stdev=52.23 lat (msec): min=15, max=338, avg=79.68, stdev=52.23 clat percentiles (msec): | 1.00th=[ 23], 5.00th=[ 30], 10.00th=[ 34], 20.00th=[ 40], | 30.00th=[ 46], 40.00th=[ 53], 50.00th=[ 62], 60.00th=[ 73], | 70.00th=[ 89], 80.00th=[ 118], 90.00th=[ 150], 95.00th=[ 190], | 99.00th=[ 249], 99.50th=[ 326], 99.90th=[ 338], 99.95th=[ 338], | 99.99th=[ 338] bw ( KiB/s): min= 30, max= 319, per=25.24%, avg=125.44, stdev=69.54, samples=98 iops : min= 1, max= 19, avg= 7.00, stdev= 4.37, samples=98 lat (msec) : 10=0.52%, 20=7.16%, 50=44.66%, 100=29.30%, 250=17.58% lat (msec) : 500=0.78% cpu : usr=0.03%, sys=1.58%, ctx=1138, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=473KiB/s (485kB/s), 113KiB/s-124KiB/s (116kB/s-127kB/s), io=23.4MiB (24.5MB), run=50302-50621msec WRITE: bw=498KiB/s (510kB/s), 119KiB/s-129KiB/s (122kB/s-132kB/s), io=24.6MiB (25.8MB), run=50302-50621msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=181884: Mon Mar 30 10:44:53 2026 read: IOPS=2, BW=2137KiB/s (2188kB/s)(7168KiB/3355msec) clat (usec): min=1147, max=430967, avg=176843.15, stdev=142393.33 lat (usec): min=1148, max=430968, avg=176844.72, stdev=142393.40 clat percentiles (usec): | 1.00th=[ 1156], 5.00th=[ 1156], 10.00th=[ 1156], 20.00th=[ 55837], | 30.00th=[124257], 40.00th=[124257], 50.00th=[170918], 60.00th=[183501], | 70.00th=[183501], 80.00th=[270533], 90.00th=[429917], 95.00th=[429917], | 99.00th=[429917], 99.50th=[429917], 99.90th=[429917], 99.95th=[429917], | 99.99th=[429917] bw ( KiB/s): min= 1914, max= 3771, per=61.43%, avg=2811.75, stdev=975.19, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 write: IOPS=1, BW=1526KiB/s (1563kB/s)(5120KiB/3355msec); 0 zone resets clat (msec): min=178, max=607, avg=393.43, stdev=170.04 lat (msec): min=179, max=607, avg=393.90, stdev=169.76 clat percentiles (msec): | 1.00th=[ 180], 5.00th=[ 180], 10.00th=[ 180], 20.00th=[ 180], | 30.00th=[ 268], 40.00th=[ 268], 50.00th=[ 447], 60.00th=[ 447], | 70.00th=[ 464], 80.00th=[ 464], 90.00th=[ 609], 95.00th=[ 609], | 99.00th=[ 609], 99.50th=[ 609], 99.90th=[ 609], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 1885, max= 2043, per=30.49%, avg=1953.75, stdev=69.86, samples=4 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=4 lat (msec) : 2=8.33%, 100=8.33%, 250=33.33%, 500=41.67%, 750=8.33% cpu : usr=0.00%, sys=12.02%, ctx=139, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181885: Mon Mar 30 10:44:54 2026 read: IOPS=0, BW=458KiB/s (469kB/s)(2048KiB/4474msec) clat (msec): min=99, max=411, avg=255.47, stdev=220.95 lat (msec): min=99, max=411, avg=255.47, stdev=220.95 clat percentiles (msec): | 1.00th=[ 100], 5.00th=[ 100], 10.00th=[ 100], 20.00th=[ 100], | 30.00th=[ 100], 40.00th=[ 100], 50.00th=[ 100], 60.00th=[ 414], | 70.00th=[ 414], 80.00th=[ 414], 90.00th=[ 414], 95.00th=[ 414], | 99.00th=[ 414], 99.50th=[ 414], 99.90th=[ 414], 99.95th=[ 414], | 99.99th=[ 414] bw ( KiB/s): min= 1519, max= 1973, per=38.15%, avg=1746.00, stdev=321.03, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2289KiB/s (2344kB/s)(10.0MiB/4474msec); 0 zone resets clat (msec): min=132, max=768, avg=386.32, stdev=178.18 lat (msec): min=132, max=768, avg=386.42, stdev=178.18 clat percentiles (msec): | 1.00th=[ 133], 5.00th=[ 133], 10.00th=[ 133], 20.00th=[ 220], | 30.00th=[ 279], 40.00th=[ 330], 50.00th=[ 351], 60.00th=[ 368], | 70.00th=[ 414], 80.00th=[ 477], 90.00th=[ 535], 95.00th=[ 768], | 99.00th=[ 768], 99.50th=[ 768], 99.90th=[ 768], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 1519, max= 4087, per=39.01%, avg=2499.86, stdev=1091.53, samples=7 iops : min= 1, max= 3, avg= 1.57, stdev= 0.98, samples=7 lat (msec) : 100=8.33%, 250=16.67%, 500=58.33%, 750=8.33%, 1000=8.33% cpu : usr=0.07%, sys=11.83%, ctx=232, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181887: Mon Mar 30 10:44:54 2026 read: IOPS=2, BW=2053KiB/s (2103kB/s)(6144KiB/2992msec) clat (usec): min=458, max=348722, avg=194463.21, stdev=131941.85 lat (usec): min=460, max=348724, avg=194464.58, stdev=131941.92 clat percentiles (usec): | 1.00th=[ 461], 5.00th=[ 461], 10.00th=[ 461], 20.00th=[116917], | 30.00th=[116917], 40.00th=[133694], 50.00th=[133694], 60.00th=[274727], | 70.00th=[291505], 80.00th=[291505], 90.00th=[350225], 95.00th=[350225], | 99.00th=[350225], 99.50th=[350225], 99.90th=[350225], 99.95th=[350225], | 99.99th=[350225] bw ( KiB/s): min= 3828, max= 4087, per=86.46%, avg=3957.50, stdev=183.14, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2053KiB/s (2103kB/s)(6144KiB/2992msec); 0 zone resets clat (msec): min=66, max=476, avg=261.88, stdev=167.67 lat (msec): min=66, max=477, avg=262.09, stdev=167.84 clat percentiles (msec): | 1.00th=[ 67], 5.00th=[ 67], 10.00th=[ 67], 20.00th=[ 74], | 30.00th=[ 74], 40.00th=[ 257], 50.00th=[ 257], 60.00th=[ 292], | 70.00th=[ 405], 80.00th=[ 405], 90.00th=[ 477], 95.00th=[ 477], | 99.00th=[ 477], 99.50th=[ 477], 99.90th=[ 477], 99.95th=[ 477], | 99.99th=[ 477] bw ( KiB/s): min= 1885, max= 3442, per=35.95%, avg=2303.50, stdev=759.88, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (usec) : 500=8.33% lat (msec) : 100=16.67%, 250=16.67%, 500=58.33% cpu : usr=0.00%, sys=12.60%, ctx=191, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181888: Mon Mar 30 10:44:54 2026 read: IOPS=1, BW=1733KiB/s (1775kB/s)(5120KiB/2954msec) clat (msec): min=4, max=393, avg=164.90, stdev=158.96 lat (msec): min=4, max=393, avg=164.90, stdev=158.96 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 5], | 30.00th=[ 26], 40.00th=[ 26], 50.00th=[ 180], 60.00th=[ 180], | 70.00th=[ 224], 80.00th=[ 224], 90.00th=[ 393], 95.00th=[ 393], | 99.00th=[ 393], 99.50th=[ 393], 99.90th=[ 393], 99.95th=[ 393], | 99.99th=[ 393] bw ( KiB/s): min= 1885, max= 5742, per=83.32%, avg=3813.50, stdev=2727.31, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=2, BW=2427KiB/s (2485kB/s)(7168KiB/2954msec); 0 zone resets clat (msec): min=82, max=720, avg=278.21, stdev=214.43 lat (msec): min=83, max=720, avg=278.49, stdev=214.31 clat percentiles (msec): | 1.00th=[ 84], 5.00th=[ 84], 10.00th=[ 84], 20.00th=[ 131], | 30.00th=[ 161], 40.00th=[ 161], 50.00th=[ 224], 60.00th=[ 300], | 70.00th=[ 300], 80.00th=[ 330], 90.00th=[ 718], 95.00th=[ 718], | 99.00th=[ 718], 99.50th=[ 718], 99.90th=[ 718], 99.95th=[ 718], | 99.99th=[ 718] bw ( KiB/s): min= 1885, max= 4891, per=55.77%, avg=3574.00, stdev=1537.14, samples=3 iops : min= 1, max= 4, avg= 2.67, stdev= 1.53, samples=3 lat (msec) : 10=8.33%, 50=8.33%, 100=8.33%, 250=41.67%, 500=25.00% lat (msec) : 750=8.33% cpu : usr=0.00%, sys=13.85%, ctx=162, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4578KiB/s (4687kB/s), 458KiB/s-2137KiB/s (469kB/s-2188kB/s), io=20.0MiB (20.0MB), run=2954-4474msec WRITE: bw=6409KiB/s (6562kB/s), 1526KiB/s-2427KiB/s (1563kB/s-2485kB/s), io=28.0MiB (29.4MB), run=2954-4474msec rand-rw: (groupid=0, jobs=1): err= 0: pid=181886: Mon Mar 30 10:44:54 2026 read: IOPS=1, BW=1882KiB/s (1928kB/s)(7168KiB/3808msec) clat (msec): min=101, max=524, avg=222.31, stdev=149.27 lat (msec): min=101, max=524, avg=222.31, stdev=149.27 clat percentiles (msec): | 1.00th=[ 102], 5.00th=[ 102], 10.00th=[ 102], 20.00th=[ 106], | 30.00th=[ 120], 40.00th=[ 120], 50.00th=[ 188], 60.00th=[ 255], | 70.00th=[ 255], 80.00th=[ 264], 90.00th=[ 523], 95.00th=[ 523], | 99.00th=[ 523], 99.50th=[ 523], 99.90th=[ 523], 99.95th=[ 523], | 99.99th=[ 523] bw ( KiB/s): min= 1572, max= 4023, per=56.37%, avg=2341.60, stdev=961.32, samples=5 iops : min= 1, max= 3, avg= 1.40, stdev= 0.89, samples=5 write: IOPS=1, BW=1345KiB/s (1377kB/s)(5120KiB/3808msec); 0 zone resets clat (msec): min=147, max=835, avg=404.23, stdev=278.48 lat (msec): min=147, max=835, avg=404.33, stdev=278.49 clat percentiles (msec): | 1.00th=[ 148], 5.00th=[ 148], 10.00th=[ 148], 20.00th=[ 148], | 30.00th=[ 167], 40.00th=[ 167], 50.00th=[ 422], 60.00th=[ 422], | 70.00th=[ 451], 80.00th=[ 451], 90.00th=[ 835], 95.00th=[ 835], | 99.00th=[ 835], 99.50th=[ 835], 99.90th=[ 835], 99.95th=[ 835], | 99.99th=[ 835] bw ( KiB/s): min= 2027, max= 4071, per=46.67%, avg=2713.67, stdev=1175.51, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 250=50.00%, 500=33.33%, 750=8.33%, 1000=8.33% cpu : usr=0.00%, sys=5.67%, ctx=138, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181889: Mon Mar 30 10:44:54 2026 read: IOPS=0, BW=415KiB/s (425kB/s)(2048KiB/4930msec) clat (msec): min=68, max=103, avg=86.08, stdev=24.70 lat (msec): min=68, max=103, avg=86.08, stdev=24.70 clat percentiles (msec): | 1.00th=[ 69], 5.00th=[ 69], 10.00th=[ 69], 20.00th=[ 69], | 30.00th=[ 69], 40.00th=[ 69], 50.00th=[ 69], 60.00th=[ 104], | 70.00th=[ 104], 80.00th=[ 104], 90.00th=[ 104], 95.00th=[ 104], | 99.00th=[ 104], 99.50th=[ 104], 99.90th=[ 104], 99.95th=[ 104], | 99.99th=[ 104] bw ( KiB/s): min= 1558, max= 2043, per=43.34%, avg=1800.50, stdev=342.95, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=2, BW=2077KiB/s (2127kB/s)(10.0MiB/4930msec); 0 zone resets clat (msec): min=165, max=872, avg=468.76, stdev=198.56 lat (msec): min=165, max=872, avg=469.34, stdev=198.64 clat percentiles (msec): | 1.00th=[ 167], 5.00th=[ 167], 10.00th=[ 167], 20.00th=[ 334], | 30.00th=[ 334], 40.00th=[ 368], 50.00th=[ 388], 60.00th=[ 493], | 70.00th=[ 535], 80.00th=[ 542], 90.00th=[ 659], 95.00th=[ 877], | 99.00th=[ 877], 99.50th=[ 877], 99.90th=[ 877], 99.95th=[ 877], | 99.99th=[ 877] bw ( KiB/s): min= 1984, max= 4079, per=44.76%, avg=2602.57, stdev=995.18, samples=7 iops : min= 1, max= 3, avg= 1.57, stdev= 0.98, samples=7 lat (msec) : 100=8.33%, 250=16.67%, 500=41.67%, 750=25.00%, 1000=8.33% cpu : usr=0.00%, sys=4.48%, ctx=100, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181890: Mon Mar 30 10:44:54 2026 read: IOPS=1, BW=1260KiB/s (1291kB/s)(6144KiB/4875msec) clat (msec): min=133, max=619, avg=279.88, stdev=174.16 lat (msec): min=133, max=619, avg=279.88, stdev=174.16 clat percentiles (msec): | 1.00th=[ 134], 5.00th=[ 134], 10.00th=[ 134], 20.00th=[ 192], | 30.00th=[ 192], 40.00th=[ 213], 50.00th=[ 213], 60.00th=[ 228], | 70.00th=[ 296], 80.00th=[ 296], 90.00th=[ 617], 95.00th=[ 617], | 99.00th=[ 617], 99.50th=[ 617], 99.90th=[ 617], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 1984, max= 2039, per=48.70%, avg=2023.20, stdev=23.41, samples=5 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=5 write: IOPS=1, BW=1260KiB/s (1291kB/s)(6144KiB/4875msec); 0 zone resets clat (msec): min=135, max=946, avg=492.81, stdev=319.96 lat (msec): min=135, max=947, avg=492.92, stdev=319.94 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 157], | 30.00th=[ 157], 40.00th=[ 468], 50.00th=[ 468], 60.00th=[ 506], | 70.00th=[ 743], 80.00th=[ 743], 90.00th=[ 944], 95.00th=[ 944], | 99.00th=[ 944], 99.50th=[ 944], 99.90th=[ 944], 99.95th=[ 944], | 99.99th=[ 944] bw ( KiB/s): min= 2035, max= 4055, per=52.31%, avg=3042.00, stdev=1160.49, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 1.15, samples=4 lat (msec) : 250=50.00%, 500=16.67%, 750=25.00%, 1000=8.33% cpu : usr=0.00%, sys=4.82%, ctx=121, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181891: Mon Mar 30 10:44:54 2026 read: IOPS=1, BW=1102KiB/s (1128kB/s)(5120KiB/4647msec) clat (msec): min=126, max=408, avg=207.46, stdev=114.12 lat (msec): min=126, max=408, avg=207.46, stdev=114.12 clat percentiles (msec): | 1.00th=[ 127], 5.00th=[ 127], 10.00th=[ 127], 20.00th=[ 127], | 30.00th=[ 150], 40.00th=[ 150], 50.00th=[ 171], 60.00th=[ 171], | 70.00th=[ 182], 80.00th=[ 182], 90.00th=[ 409], 95.00th=[ 409], | 99.00th=[ 409], 99.50th=[ 409], 99.90th=[ 409], 99.95th=[ 409], | 99.99th=[ 409] bw ( KiB/s): min= 4071, max= 4087, per=98.19%, avg=4079.00, stdev=11.31, samples=2 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=2 write: IOPS=1, BW=1543KiB/s (1580kB/s)(7168KiB/4647msec); 0 zone resets clat (msec): min=169, max=801, avg=495.66, stdev=229.89 lat (msec): min=169, max=801, avg=496.02, stdev=229.74 clat percentiles (msec): | 1.00th=[ 169], 5.00th=[ 169], 10.00th=[ 169], 20.00th=[ 317], | 30.00th=[ 372], 40.00th=[ 372], 50.00th=[ 460], 60.00th=[ 642], | 70.00th=[ 642], 80.00th=[ 709], 90.00th=[ 802], 95.00th=[ 802], | 99.00th=[ 802], 99.50th=[ 802], 99.90th=[ 802], 99.95th=[ 802], | 99.99th=[ 802] bw ( KiB/s): min= 1984, max= 4079, per=43.43%, avg=2525.25, stdev=1035.99, samples=4 iops : min= 1, max= 3, avg= 1.50, stdev= 1.00, samples=4 lat (msec) : 250=41.67%, 500=33.33%, 750=16.67%, 1000=8.33% cpu : usr=0.06%, sys=5.83%, ctx=124, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4154KiB/s (4254kB/s), 415KiB/s-1882KiB/s (425kB/s-1928kB/s), io=20.0MiB (20.0MB), run=3808-4930msec WRITE: bw=5816KiB/s (5955kB/s), 1260KiB/s-2077KiB/s (1291kB/s-2127kB/s), io=28.0MiB (29.4MB), run=3808-4930msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=181902: Mon Mar 30 10:45:09 2026 read: IOPS=1, BW=4361KiB/s (4465kB/s)(12.0MiB/2818msec) clat (msec): min=542, max=1303, avg=894.10, stdev=383.75 lat (msec): min=542, max=1303, avg=894.10, stdev=383.75 clat percentiles (msec): | 1.00th=[ 542], 5.00th=[ 542], 10.00th=[ 542], 20.00th=[ 542], | 30.00th=[ 542], 40.00th=[ 835], 50.00th=[ 835], 60.00th=[ 835], | 70.00th=[ 1301], 80.00th=[ 1301], 90.00th=[ 1301], 95.00th=[ 1301], | 99.00th=[ 1301], 99.50th=[ 1301], 99.90th=[ 1301], 99.95th=[ 1301], | 99.99th=[ 1301] bw ( KiB/s): min= 8126, max= 8159, per=100.00%, avg=8142.50, stdev=23.33, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.04%, sys=9.34%, ctx=366, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181904: Mon Mar 30 10:45:09 2026 read: IOPS=0, BW=1282KiB/s (1312kB/s)(4096KiB/3196msec) clat (nsec): min=736369k, max=736369k, avg=736369140.00, stdev= 0.00 lat (nsec): min=736372k, max=736372k, avg=736371928.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 735], 5.00th=[ 735], 10.00th=[ 735], 20.00th=[ 735], | 30.00th=[ 735], 40.00th=[ 735], 50.00th=[ 735], 60.00th=[ 735], | 70.00th=[ 735], 80.00th=[ 735], 90.00th=[ 735], 95.00th=[ 735], | 99.00th=[ 735], 99.50th=[ 735], 99.90th=[ 735], 99.95th=[ 735], | 99.99th=[ 735] bw ( KiB/s): min= 8126, max= 8126, per=100.00%, avg=8126.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2563KiB/s (2625kB/s)(8192KiB/3196msec); 0 zone resets clat (msec): min=815, max=1560, avg=1188.43, stdev=526.82 lat (msec): min=817, max=1561, avg=1189.12, stdev=526.20 clat percentiles (msec): | 1.00th=[ 818], 5.00th=[ 818], 10.00th=[ 818], 20.00th=[ 818], | 30.00th=[ 818], 40.00th=[ 818], 50.00th=[ 818], 60.00th=[ 1569], | 70.00th=[ 1569], 80.00th=[ 1569], 90.00th=[ 1569], 95.00th=[ 1569], | 99.00th=[ 1569], 99.50th=[ 1569], 99.90th=[ 1569], 99.95th=[ 1569], | 99.99th=[ 1569] bw ( KiB/s): min= 7953, max= 7953, per=87.82%, avg=7953.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=3.94%, ctx=150, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181905: Mon Mar 30 10:45:09 2026 write: IOPS=0, BW=3396KiB/s (3478kB/s)(12.0MiB/3618msec); 0 zone resets clat (msec): min=349, max=1900, avg=1173.29, stdev=779.92 lat (msec): min=349, max=1900, avg=1173.92, stdev=779.99 clat percentiles (msec): | 1.00th=[ 351], 5.00th=[ 351], 10.00th=[ 351], 20.00th=[ 351], | 30.00th=[ 351], 40.00th=[ 1267], 50.00th=[ 1267], 60.00th=[ 1267], | 70.00th=[ 1905], 80.00th=[ 1905], 90.00th=[ 1905], 95.00th=[ 1905], | 99.00th=[ 1905], 99.50th=[ 1905], 99.90th=[ 1905], 99.95th=[ 1905], | 99.99th=[ 1905] bw ( KiB/s): min= 8094, max= 8159, per=89.74%, avg=8126.50, stdev=45.96, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 2000=66.67% cpu : usr=0.14%, sys=3.21%, ctx=51, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181906: Mon Mar 30 10:45:09 2026 write: IOPS=0, BW=3987KiB/s (4083kB/s)(12.0MiB/3082msec); 0 zone resets clat (msec): min=660, max=1434, avg=996.72, stdev=396.59 lat (msec): min=664, max=1435, avg=999.81, stdev=395.08 clat percentiles (msec): | 1.00th=[ 659], 5.00th=[ 659], 10.00th=[ 659], 20.00th=[ 659], | 30.00th=[ 659], 40.00th=[ 894], 50.00th=[ 894], 60.00th=[ 894], | 70.00th=[ 1435], 80.00th=[ 1435], 90.00th=[ 1435], 95.00th=[ 1435], | 99.00th=[ 1435], 99.50th=[ 1435], 99.90th=[ 1435], 99.95th=[ 1435], | 99.99th=[ 1435] bw ( KiB/s): min= 8126, max= 8159, per=89.91%, avg=8142.50, stdev=23.33, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 750=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.00%, sys=3.93%, ctx=52, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5126KiB/s (5249kB/s), 1282KiB/s-4361KiB/s (1312kB/s-4465kB/s), io=16.0MiB (16.8MB), run=2818-3196msec WRITE: bw=9057KiB/s (9274kB/s), 2563KiB/s-3987KiB/s (2625kB/s-4083kB/s), io=32.0MiB (33.6MB), run=3082-3618msec rand-rw: (groupid=0, jobs=1): err= 0: pid=181899: Mon Mar 30 10:45:09 2026 read: IOPS=0, BW=3828KiB/s (3920kB/s)(12.0MiB/3210msec) clat (msec): min=364, max=1803, avg=1025.64, stdev=726.47 lat (msec): min=364, max=1803, avg=1025.64, stdev=726.47 clat percentiles (msec): | 1.00th=[ 363], 5.00th=[ 363], 10.00th=[ 363], 20.00th=[ 363], | 30.00th=[ 363], 40.00th=[ 911], 50.00th=[ 911], 60.00th=[ 911], | 70.00th=[ 1804], 80.00th=[ 1804], 90.00th=[ 1804], 95.00th=[ 1804], | 99.00th=[ 1804], 99.50th=[ 1804], 99.90th=[ 1804], 99.95th=[ 1804], | 99.99th=[ 1804] bw ( KiB/s): min= 7937, max= 8175, per=100.00%, avg=8056.00, stdev=168.29, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=33.33%, 1000=33.33%, 2000=33.33% cpu : usr=0.03%, sys=2.71%, ctx=40, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181900: Mon Mar 30 10:45:09 2026 read: IOPS=0, BW=1069KiB/s (1094kB/s)(4096KiB/3833msec) clat (nsec): min=1014.9M, max=1014.9M, avg=1014908352.00, stdev= 0.00 lat (nsec): min=1014.9M, max=1014.9M, avg=1014911507.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 1011], 5.00th=[ 1011], 10.00th=[ 1011], 20.00th=[ 1011], | 30.00th=[ 1011], 40.00th=[ 1011], 50.00th=[ 1011], 60.00th=[ 1011], | 70.00th=[ 1011], 80.00th=[ 1011], 90.00th=[ 1011], 95.00th=[ 1011], | 99.00th=[ 1011], 99.50th=[ 1011], 99.90th=[ 1011], 99.95th=[ 1011], | 99.99th=[ 1011] bw ( KiB/s): min= 8159, max= 8159, per=100.00%, avg=8159.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=0, BW=2137KiB/s (2189kB/s)(8192KiB/3833msec); 0 zone resets clat (msec): min=994, max=1629, avg=1312.19, stdev=448.95 lat (msec): min=997, max=1630, avg=1313.93, stdev=447.35 clat percentiles (msec): | 1.00th=[ 995], 5.00th=[ 995], 10.00th=[ 995], 20.00th=[ 995], | 30.00th=[ 995], 40.00th=[ 995], 50.00th=[ 995], 60.00th=[ 1636], | 70.00th=[ 1636], 80.00th=[ 1636], 90.00th=[ 1636], 95.00th=[ 1636], | 99.00th=[ 1636], 99.50th=[ 1636], 99.90th=[ 1636], 99.95th=[ 1636], | 99.99th=[ 1636] bw ( KiB/s): min= 8175, max= 8175, per=95.64%, avg=8175.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 1000=33.33%, 2000=66.67% cpu : usr=0.03%, sys=2.30%, ctx=36, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181901: Mon Mar 30 10:45:09 2026 write: IOPS=2, BW=9918KiB/s (10.2MB/s)(12.0MiB/1239msec); 0 zone resets clat (msec): min=204, max=671, avg=395.68, stdev=244.74 lat (msec): min=205, max=672, avg=396.37, stdev=244.74 clat percentiles (msec): | 1.00th=[ 205], 5.00th=[ 205], 10.00th=[ 205], 20.00th=[ 205], | 30.00th=[ 205], 40.00th=[ 313], 50.00th=[ 313], 60.00th=[ 313], | 70.00th=[ 676], 80.00th=[ 676], 90.00th=[ 676], 95.00th=[ 676], | 99.00th=[ 676], 99.50th=[ 676], 99.90th=[ 676], 99.95th=[ 676], | 99.99th=[ 676] bw ( KiB/s): min=14296, max=14296, per=100.00%, avg=14296.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 750=33.33% cpu : usr=0.00%, sys=6.54%, ctx=38, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=181903: Mon Mar 30 10:45:09 2026 write: IOPS=1, BW=4560KiB/s (4669kB/s)(12.0MiB/2695msec); 0 zone resets clat (msec): min=264, max=1948, avg=868.46, stdev=937.45 lat (msec): min=265, max=1949, avg=868.93, stdev=937.56 clat percentiles (msec): | 1.00th=[ 266], 5.00th=[ 266], 10.00th=[ 266], 20.00th=[ 266], | 30.00th=[ 266], 40.00th=[ 393], 50.00th=[ 393], 60.00th=[ 393], | 70.00th=[ 1955], 80.00th=[ 1955], 90.00th=[ 1955], 95.00th=[ 1955], | 99.00th=[ 1955], 99.50th=[ 1955], 99.90th=[ 1955], 99.95th=[ 1955], | 99.99th=[ 1955] bw ( KiB/s): min= 6826, max= 8159, per=87.65%, avg=7492.50, stdev=942.57, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 lat (msec) : 500=66.67%, 2000=33.33% cpu : usr=0.00%, sys=4.49%, ctx=54, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=4274KiB/s (4377kB/s), 1069KiB/s-3828KiB/s (1094kB/s-3920kB/s), io=16.0MiB (16.8MB), run=3210-3833msec WRITE: bw=8549KiB/s (8754kB/s), 2137KiB/s-9918KiB/s (2189kB/s-10.2MB/s), io=32.0MiB (33.6MB), run=1239-3833msec