== sanity test 398b: DIO and buffer IO race ============== 13:16:49 (1763489809) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.9189 s, 17.2 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146664: Tue Nov 18 13:17:47 2025 read: IOPS=30, BW=124KiB/s (127kB/s)(5992KiB/48468msec) clat (usec): min=80, max=139491, avg=16540.37, stdev=10963.79 lat (usec): min=80, max=139493, avg=16542.21, stdev=10963.63 clat percentiles (usec): | 1.00th=[ 245], 5.00th=[ 506], 10.00th=[ 7308], 20.00th=[ 9110], | 30.00th=[ 10683], 40.00th=[ 12518], 50.00th=[ 14484], 60.00th=[ 16909], | 70.00th=[ 19268], 80.00th=[ 22938], 90.00th=[ 28705], 95.00th=[ 34341], | 99.00th=[ 52691], 99.50th=[ 65799], 99.90th=[113771], 99.95th=[139461], | 99.99th=[139461] bw ( KiB/s): min= 32, max= 224, per=25.11%, avg=123.28, stdev=34.46, samples=96 iops : min= 8, max= 56, avg=30.46, stdev= 8.65, samples=96 write: IOPS=32, BW=130KiB/s (133kB/s)(6296KiB/48468msec); 0 zone resets clat (usec): min=711, max=691418, avg=14921.69, stdev=21058.42 lat (usec): min=712, max=691419, avg=14925.46, stdev=21057.28 clat percentiles (usec): | 1.00th=[ 1029], 5.00th=[ 2147], 10.00th=[ 2606], 20.00th=[ 3556], | 30.00th=[ 5276], 40.00th=[ 8586], 50.00th=[ 11863], 60.00th=[ 14877], | 70.00th=[ 18482], 80.00th=[ 23200], 90.00th=[ 31065], 95.00th=[ 36963], | 99.00th=[ 56886], 99.50th=[ 71828], 99.90th=[ 92799], 99.95th=[692061], | 99.99th=[692061] bw ( KiB/s): min= 47, max= 248, per=25.26%, avg=129.81, stdev=41.10, samples=96 iops : min= 11, max= 62, avg=32.08, stdev=10.31, samples=96 lat (usec) : 100=0.03%, 250=0.46%, 500=1.92%, 750=0.46%, 1000=0.36% lat (msec) : 2=1.95%, 4=9.47%, 10=20.41%, 20=38.28%, 50=25.29% lat (msec) : 100=1.27%, 250=0.07%, 750=0.03% cpu : usr=0.15%, sys=26.18%, ctx=8541, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146665: Tue Nov 18 13:17:47 2025 read: IOPS=30, BW=123KiB/s (126kB/s)(5940KiB/48401msec) clat (usec): min=101, max=125252, avg=16532.36, stdev=10835.07 lat (usec): min=102, max=125263, avg=16533.29, stdev=10835.21 clat percentiles (usec): | 1.00th=[ 223], 5.00th=[ 4948], 10.00th=[ 7308], 20.00th=[ 9241], | 30.00th=[ 10552], 40.00th=[ 12125], 50.00th=[ 14091], 60.00th=[ 16319], | 70.00th=[ 19268], 80.00th=[ 23200], 90.00th=[ 28967], 95.00th=[ 34866], | 99.00th=[ 56886], 99.50th=[ 64750], 99.90th=[107480], 99.95th=[125305], | 99.99th=[125305] bw ( KiB/s): min= 32, max= 207, per=24.93%, avg=122.39, stdev=36.95, samples=96 iops : min= 8, max= 51, avg=30.23, stdev= 9.28, samples=96 write: IOPS=32, BW=131KiB/s (134kB/s)(6348KiB/48401msec); 0 zone resets clat (usec): min=692, max=692445, avg=14899.77, stdev=21209.05 lat (usec): min=692, max=692446, avg=14901.04, stdev=21209.09 clat percentiles (usec): | 1.00th=[ 1045], 5.00th=[ 2057], 10.00th=[ 2638], 20.00th=[ 3589], | 30.00th=[ 5276], 40.00th=[ 8586], 50.00th=[ 11600], 60.00th=[ 14615], | 70.00th=[ 18744], 80.00th=[ 23725], 90.00th=[ 30278], 95.00th=[ 38011], | 99.00th=[ 54789], 99.50th=[ 81265], 99.90th=[122160], 99.95th=[692061], | 99.99th=[692061] bw ( KiB/s): min= 40, max= 224, per=25.40%, avg=130.55, stdev=38.55, samples=96 iops : min= 10, max= 56, avg=32.27, stdev= 9.67, samples=96 lat (usec) : 250=0.59%, 500=0.88%, 750=0.68%, 1000=0.33% lat (msec) : 2=2.25%, 4=8.95%, 10=21.78%, 20=37.57%, 50=25.49% lat (msec) : 100=1.37%, 250=0.10%, 750=0.03% cpu : usr=0.15%, sys=26.54%, ctx=8616, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146666: Tue Nov 18 13:17:47 2025 read: IOPS=30, BW=124KiB/s (127kB/s)(6028KiB/48618msec) clat (usec): min=122, max=686422, avg=16460.64, stdev=19898.82 lat (usec): min=122, max=686422, avg=16461.70, stdev=19898.92 clat percentiles (usec): | 1.00th=[ 277], 5.00th=[ 594], 10.00th=[ 6980], 20.00th=[ 9110], | 30.00th=[ 10683], 40.00th=[ 12125], 50.00th=[ 14484], 60.00th=[ 16712], | 70.00th=[ 19268], 80.00th=[ 22414], 90.00th=[ 27395], 95.00th=[ 31851], | 99.00th=[ 50594], 99.50th=[ 72877], 99.90th=[ 83362], 99.95th=[683672], | 99.99th=[683672] bw ( KiB/s): min= 32, max= 191, per=25.06%, avg=123.02, stdev=32.32, samples=96 iops : min= 8, max= 47, avg=30.40, stdev= 8.14, samples=96 write: IOPS=32, BW=129KiB/s (132kB/s)(6260KiB/48618msec); 0 zone resets clat (usec): min=784, max=113345, avg=15081.77, stdev=12348.25 lat (usec): min=784, max=113346, avg=15083.32, stdev=12348.57 clat percentiles (usec): | 1.00th=[ 1090], 5.00th=[ 1975], 10.00th=[ 2540], 20.00th=[ 3556], | 30.00th=[ 5800], 40.00th=[ 9765], 50.00th=[ 12518], 60.00th=[ 15664], | 70.00th=[ 19530], 80.00th=[ 24511], 90.00th=[ 31851], 95.00th=[ 37487], | 99.00th=[ 51643], 99.50th=[ 63177], 99.90th=[ 96994], 99.95th=[113771], | 99.99th=[113771] bw ( KiB/s): min= 47, max= 208, per=24.80%, avg=127.45, stdev=35.15, samples=96 iops : min= 11, max= 52, avg=31.49, stdev= 8.86, samples=96 lat (usec) : 250=0.36%, 500=1.79%, 750=0.62%, 1000=0.42% lat (msec) : 2=2.57%, 4=9.15%, 10=18.62%, 20=38.09%, 50=27.21% lat (msec) : 100=1.11%, 250=0.03%, 750=0.03% cpu : usr=0.13%, sys=26.34%, ctx=8746, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146667: Tue Nov 18 13:17:47 2025 read: IOPS=30, BW=124KiB/s (127kB/s)(6052KiB/48836msec) clat (usec): min=74, max=99106, avg=16055.79, stdev=10101.86 lat (usec): min=74, max=99107, avg=16057.57, stdev=10102.90 clat percentiles (usec): | 1.00th=[ 243], 5.00th=[ 627], 10.00th=[ 7111], 20.00th=[ 8848], | 30.00th=[10159], 40.00th=[11994], 50.00th=[13960], 60.00th=[16712], | 70.00th=[19268], 80.00th=[22152], 90.00th=[28443], 95.00th=[34866], | 99.00th=[50594], 99.50th=[53740], 99.90th=[79168], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 32, max= 232, per=25.05%, avg=123.00, stdev=35.38, samples=97 iops : min= 8, max= 58, avg=30.39, stdev= 8.87, samples=97 write: IOPS=31, BW=128KiB/s (131kB/s)(6236KiB/48836msec); 0 zone resets clat (usec): min=742, max=680442, avg=15613.28, stdev=21507.01 lat (usec): min=742, max=680443, avg=15614.53, stdev=21507.04 clat percentiles (usec): | 1.00th=[ 996], 5.00th=[ 1876], 10.00th=[ 2507], 20.00th=[ 3752], | 30.00th=[ 5997], 40.00th=[ 9634], 50.00th=[ 12387], 60.00th=[ 15139], | 70.00th=[ 18744], 80.00th=[ 23725], 90.00th=[ 31327], 95.00th=[ 40109], | 99.00th=[ 64226], 99.50th=[ 74974], 99.90th=[120062], 99.95th=[683672], | 99.99th=[683672] bw ( KiB/s): min= 63, max= 240, per=24.68%, avg=126.87, stdev=36.33, samples=97 iops : min= 15, max= 60, avg=31.35, stdev= 9.16, samples=97 lat (usec) : 100=0.03%, 250=0.49%, 500=1.63%, 750=0.65%, 1000=0.59% lat (msec) : 2=2.54%, 4=7.94%, 10=21.45%, 20=37.73%, 50=25.23% lat (msec) : 100=1.60%, 250=0.10%, 750=0.03% cpu : usr=0.14%, sys=26.16%, ctx=8556, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=492KiB/s (503kB/s), 123KiB/s-124KiB/s (126kB/s-127kB/s), io=23.4MiB (24.6MB), run=48401-48836msec WRITE: bw=515KiB/s (527kB/s), 128KiB/s-131KiB/s (131kB/s-134kB/s), io=24.6MiB (25.7MB), run=48401-48836msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146669: Tue Nov 18 13:18:17 2025 read: IOPS=19, BW=76.2KiB/s (77.0kB/s)(5992KiB/78686msec) clat (msec): min=2, max=698, avg=13.74, stdev=22.60 lat (msec): min=2, max=698, avg=13.74, stdev=22.60 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 5], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 11], | 70.00th=[ 14], 80.00th=[ 20], 90.00th=[ 29], 95.00th=[ 41], | 99.00th=[ 75], 99.50th=[ 88], 99.90th=[ 153], 99.95th=[ 701], | 99.99th=[ 701] bw ( KiB/s): min= 23, max= 240, per=24.93%, avg=76.05, stdev=44.29, samples=155 iops : min= 5, max= 60, avg=18.72, stdev=11.19, samples=155 write: IOPS=20, BW=80.0KiB/s (81.9kB/s)(6296KiB/78686msec); 0 zone resets clat (msec): min=12, max=154, avg=36.88, stdev=18.25 lat (msec): min=12, max=154, avg=36.88, stdev=18.25 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 19], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 25], 40.00th=[ 28], 50.00th=[ 32], 60.00th=[ 35], | 70.00th=[ 42], 80.00th=[ 51], 90.00th=[ 63], 95.00th=[ 71], | 99.00th=[ 99], 99.50th=[ 113], 99.90th=[ 136], 99.95th=[ 155], | 99.99th=[ 155] bw ( KiB/s): min= 23, max= 160, per=24.97%, avg=79.66, stdev=37.18, samples=155 iops : min= 5, max= 40, avg=19.62, stdev= 9.40, samples=155 lat (msec) : 4=3.12%, 10=25.42%, 20=16.76%, 50=42.55%, 100=11.49% lat (msec) : 250=0.62%, 750=0.03% cpu : usr=0.09%, sys=7.20%, ctx=5601, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146670: Tue Nov 18 13:18:17 2025 read: IOPS=18, BW=75.9KiB/s (77.7kB/s)(5940KiB/78311msec) clat (msec): min=2, max=710, avg=13.60, stdev=22.23 lat (msec): min=2, max=710, avg=13.60, stdev=22.23 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 11], | 70.00th=[ 14], 80.00th=[ 20], 90.00th=[ 28], 95.00th=[ 41], | 99.00th=[ 68], 99.50th=[ 79], 99.90th=[ 115], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 7, max= 192, per=24.95%, avg=76.10, stdev=43.65, samples=155 iops : min= 1, max= 48, avg=18.73, stdev=10.99, samples=155 write: IOPS=20, BW=81.1KiB/s (83.0kB/s)(6348KiB/78311msec); 0 zone resets clat (msec): min=14, max=166, avg=36.58, stdev=18.54 lat (msec): min=14, max=166, avg=36.59, stdev=18.54 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 19], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 25], 40.00th=[ 28], 50.00th=[ 31], 60.00th=[ 35], | 70.00th=[ 42], 80.00th=[ 51], 90.00th=[ 62], 95.00th=[ 73], | 99.00th=[ 103], 99.50th=[ 115], 99.90th=[ 140], 99.95th=[ 167], | 99.99th=[ 167] bw ( KiB/s): min= 23, max= 160, per=25.47%, avg=81.26, stdev=38.29, samples=155 iops : min= 5, max= 40, avg=20.02, stdev= 9.70, samples=155 lat (msec) : 4=2.31%, 10=26.53%, 20=16.41%, 50=42.90%, 100=11.17% lat (msec) : 250=0.65%, 750=0.03% cpu : usr=0.11%, sys=7.31%, ctx=5669, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146671: Tue Nov 18 13:18:17 2025 read: IOPS=19, BW=76.0KiB/s (78.8kB/s)(6028KiB/78292msec) clat (msec): min=2, max=144, avg=13.47, stdev=13.60 lat (msec): min=2, max=144, avg=13.47, stdev=13.60 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 11], | 70.00th=[ 15], 80.00th=[ 20], 90.00th=[ 30], 95.00th=[ 39], | 99.00th=[ 66], 99.50th=[ 86], 99.90th=[ 116], 99.95th=[ 146], | 99.99th=[ 146] bw ( KiB/s): min= 15, max= 192, per=25.50%, avg=77.79, stdev=42.54, samples=154 iops : min= 3, max= 48, avg=19.15, stdev=10.75, samples=154 write: IOPS=19, BW=79.0KiB/s (81.9kB/s)(6260KiB/78292msec); 0 zone resets clat (msec): min=14, max=767, avg=37.02, stdev=25.87 lat (msec): min=14, max=767, avg=37.02, stdev=25.87 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 19], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 25], 40.00th=[ 28], 50.00th=[ 31], 60.00th=[ 35], | 70.00th=[ 42], 80.00th=[ 51], 90.00th=[ 62], 95.00th=[ 71], | 99.00th=[ 99], 99.50th=[ 112], 99.90th=[ 150], 99.95th=[ 768], | 99.99th=[ 768] bw ( KiB/s): min= 23, max= 160, per=25.15%, avg=80.23, stdev=38.67, samples=155 iops : min= 5, max= 40, avg=19.76, stdev= 9.77, samples=155 lat (msec) : 4=2.67%, 10=25.85%, 20=16.57%, 50=43.29%, 100=10.94% lat (msec) : 250=0.65%, 1000=0.03% cpu : usr=0.11%, sys=7.38%, ctx=5773, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146672: Tue Nov 18 13:18:17 2025 read: IOPS=19, BW=77.1KiB/s (78.9kB/s)(6052KiB/78538msec) clat (msec): min=2, max=147, avg=13.50, stdev=14.01 lat (msec): min=2, max=147, avg=13.50, stdev=14.01 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 11], | 70.00th=[ 15], 80.00th=[ 20], 90.00th=[ 30], 95.00th=[ 41], | 99.00th=[ 74], 99.50th=[ 86], 99.90th=[ 113], 99.95th=[ 148], | 99.99th=[ 148] bw ( KiB/s): min= 7, max= 216, per=25.20%, avg=76.87, stdev=44.56, samples=155 iops : min= 1, max= 54, avg=18.92, stdev=11.22, samples=155 write: IOPS=19, BW=79.4KiB/s (81.3kB/s)(6236KiB/78538msec); 0 zone resets clat (msec): min=13, max=738, avg=37.24, stdev=26.08 lat (msec): min=13, max=738, avg=37.24, stdev=26.08 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 18], 10.00th=[ 20], 20.00th=[ 23], | 30.00th=[ 25], 40.00th=[ 28], 50.00th=[ 31], 60.00th=[ 35], | 70.00th=[ 41], 80.00th=[ 51], 90.00th=[ 63], 95.00th=[ 77], | 99.00th=[ 101], 99.50th=[ 110], 99.90th=[ 169], 99.95th=[ 743], | 99.99th=[ 743] bw ( KiB/s): min= 24, max= 152, per=24.87%, avg=79.35, stdev=38.13, samples=155 iops : min= 6, max= 38, avg=19.54, stdev= 9.65, samples=155 lat (msec) : 4=2.67%, 10=26.33%, 20=16.80%, 50=42.68%, 100=10.90% lat (msec) : 250=0.59%, 750=0.03% cpu : usr=0.11%, sys=7.28%, ctx=5786, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=305KiB/s (312kB/s), 75.9KiB/s-77.1KiB/s (77.7kB/s-78.9kB/s), io=23.4MiB (24.6MB), run=78292-78686msec WRITE: bw=319KiB/s (327kB/s), 79.4KiB/s-81.1KiB/s (81.3kB/s-83.0kB/s), io=24.6MiB (25.7MB), run=78292-78686msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146704: Tue Nov 18 13:18:34 2025 read: IOPS=34, BW=548KiB/s (561kB/s)(5920KiB/10801msec) clat (usec): min=89, max=72617, avg=14562.35, stdev=9850.70 lat (usec): min=89, max=72617, avg=14563.09, stdev=9850.77 clat percentiles (usec): | 1.00th=[ 202], 5.00th=[ 314], 10.00th=[ 396], 20.00th=[ 8455], | 30.00th=[10552], 40.00th=[12125], 50.00th=[13435], 60.00th=[15270], | 70.00th=[17957], 80.00th=[21365], 90.00th=[25822], 95.00th=[29230], | 99.00th=[49021], 99.50th=[52167], 99.90th=[72877], 99.95th=[72877], | 99.99th=[72877] bw ( KiB/s): min= 319, max= 862, per=25.92%, avg=545.81, stdev=150.09, samples=21 iops : min= 19, max= 53, avg=33.67, stdev= 9.43, samples=21 write: IOPS=36, BW=590KiB/s (604kB/s)(6368KiB/10801msec); 0 zone resets clat (usec): min=981, max=99837, avg=13442.53, stdev=12147.39 lat (usec): min=982, max=99838, avg=13443.95, stdev=12147.41 clat percentiles (usec): | 1.00th=[ 1385], 5.00th=[ 2835], 10.00th=[ 3654], 20.00th=[ 4555], | 30.00th=[ 5080], 40.00th=[ 6915], 50.00th=[ 10159], 60.00th=[ 12387], | 70.00th=[ 15926], 80.00th=[ 21627], 90.00th=[ 28181], 95.00th=[ 35390], | 99.00th=[ 70779], 99.50th=[ 73925], 99.90th=[100140], 99.95th=[100140], | 99.99th=[100140] bw ( KiB/s): min= 286, max= 830, per=26.71%, avg=591.62, stdev=156.13, samples=21 iops : min= 17, max= 51, avg=36.57, stdev= 9.73, samples=21 lat (usec) : 100=0.13%, 250=0.91%, 500=5.60%, 750=0.39%, 1000=0.13% lat (msec) : 2=1.43%, 4=6.25%, 10=23.96%, 20=37.89%, 50=22.27% lat (msec) : 100=1.04% cpu : usr=0.08%, sys=31.10%, ctx=2314, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146705: Tue Nov 18 13:18:34 2025 read: IOPS=31, BW=505KiB/s (517kB/s)(5744KiB/11372msec) clat (usec): min=147, max=101491, avg=16313.63, stdev=11286.50 lat (usec): min=148, max=101492, avg=16314.45, stdev=11286.47 clat percentiles (usec): | 1.00th=[ 258], 5.00th=[ 371], 10.00th=[ 611], 20.00th=[ 9765], | 30.00th=[ 11076], 40.00th=[ 12649], 50.00th=[ 14484], 60.00th=[ 16712], | 70.00th=[ 19006], 80.00th=[ 22414], 90.00th=[ 28181], 95.00th=[ 36963], | 99.00th=[ 57410], 99.50th=[ 65799], 99.90th=[101188], 99.95th=[101188], | 99.99th=[101188] bw ( KiB/s): min= 320, max= 670, per=23.90%, avg=503.41, stdev=106.35, samples=22 iops : min= 20, max= 41, avg=31.05, stdev= 6.66, samples=22 write: IOPS=35, BW=575KiB/s (589kB/s)(6544KiB/11372msec); 0 zone resets clat (usec): min=1005, max=65148, avg=13247.73, stdev=10372.82 lat (usec): min=1006, max=65148, avg=13249.19, stdev=10372.81 clat percentiles (usec): | 1.00th=[ 1254], 5.00th=[ 2442], 10.00th=[ 3523], 20.00th=[ 4424], | 30.00th=[ 5604], 40.00th=[ 7898], 50.00th=[10421], 60.00th=[13042], | 70.00th=[15926], 80.00th=[21627], 90.00th=[28443], 95.00th=[35390], | 99.00th=[41157], 99.50th=[46924], 99.90th=[65274], 99.95th=[65274], | 99.99th=[65274] bw ( KiB/s): min= 320, max= 800, per=25.37%, avg=562.05, stdev=142.55, samples=22 iops : min= 20, max= 50, avg=34.73, stdev= 9.02, samples=22 lat (usec) : 250=0.39%, 500=3.91%, 750=0.65% lat (msec) : 2=1.95%, 4=5.86%, 10=22.40%, 20=39.97%, 50=23.83% lat (msec) : 100=0.91%, 250=0.13% cpu : usr=0.12%, sys=31.08%, ctx=2402, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146706: Tue Nov 18 13:18:34 2025 read: IOPS=35, BW=560KiB/s (574kB/s)(6256KiB/11166msec) clat (usec): min=92, max=91958, avg=15667.69, stdev=10943.58 lat (usec): min=93, max=91959, avg=15668.61, stdev=10943.48 clat percentiles (usec): | 1.00th=[ 217], 5.00th=[ 383], 10.00th=[ 498], 20.00th=[ 9241], | 30.00th=[11076], 40.00th=[12518], 50.00th=[13960], 60.00th=[16319], | 70.00th=[18482], 80.00th=[21365], 90.00th=[26346], 95.00th=[32113], | 99.00th=[58983], 99.50th=[76022], 99.90th=[91751], 99.95th=[91751], | 99.99th=[91751] bw ( KiB/s): min= 254, max= 864, per=26.74%, avg=563.14, stdev=150.16, samples=22 iops : min= 15, max= 54, avg=34.77, stdev= 9.51, samples=22 write: IOPS=33, BW=540KiB/s (553kB/s)(6032KiB/11166msec); 0 zone resets clat (usec): min=763, max=129483, avg=13239.55, stdev=12022.07 lat (usec): min=764, max=129484, avg=13241.19, stdev=12022.17 clat percentiles (usec): | 1.00th=[ 1516], 5.00th=[ 3359], 10.00th=[ 3916], 20.00th=[ 4490], | 30.00th=[ 5342], 40.00th=[ 6718], 50.00th=[ 9765], 60.00th=[ 11994], | 70.00th=[ 15664], 80.00th=[ 21365], 90.00th=[ 28181], 95.00th=[ 32637], | 99.00th=[ 49546], 99.50th=[ 68682], 99.90th=[129500], 99.95th=[129500], | 99.99th=[129500] bw ( KiB/s): min= 286, max= 896, per=24.27%, avg=537.64, stdev=148.11, samples=22 iops : min= 17, max= 56, avg=33.18, stdev= 9.19, samples=22 lat (usec) : 100=0.13%, 250=0.39%, 500=4.69%, 750=0.52%, 1000=0.26% lat (msec) : 2=0.65%, 4=4.30%, 10=26.43%, 20=38.80%, 50=22.66% lat (msec) : 100=1.04%, 250=0.13% cpu : usr=0.11%, sys=31.15%, ctx=2394, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146707: Tue Nov 18 13:18:34 2025 read: IOPS=34, BW=552KiB/s (565kB/s)(6032KiB/10931msec) clat (usec): min=110, max=59506, avg=15086.94, stdev=10217.24 lat (usec): min=110, max=59506, avg=15087.68, stdev=10217.27 clat percentiles (usec): | 1.00th=[ 235], 5.00th=[ 338], 10.00th=[ 408], 20.00th=[ 8717], | 30.00th=[10552], 40.00th=[12256], 50.00th=[14091], 60.00th=[15795], | 70.00th=[18482], 80.00th=[21627], 90.00th=[26346], 95.00th=[33424], | 99.00th=[52691], 99.50th=[59507], 99.90th=[59507], 99.95th=[59507], | 99.99th=[59507] bw ( KiB/s): min= 286, max= 830, per=25.77%, avg=542.67, stdev=134.03, samples=21 iops : min= 17, max= 51, avg=33.48, stdev= 8.41, samples=21 write: IOPS=35, BW=572KiB/s (586kB/s)(6256KiB/10931msec); 0 zone resets clat (usec): min=998, max=70957, avg=13350.74, stdev=10825.42 lat (usec): min=999, max=70957, avg=13352.16, stdev=10825.49 clat percentiles (usec): | 1.00th=[ 1418], 5.00th=[ 2671], 10.00th=[ 3687], 20.00th=[ 4359], | 30.00th=[ 5276], 40.00th=[ 7308], 50.00th=[11076], 60.00th=[13304], | 70.00th=[16057], 80.00th=[21627], 90.00th=[27657], 95.00th=[35914], | 99.00th=[50070], 99.50th=[55313], 99.90th=[70779], 99.95th=[70779], | 99.99th=[70779] bw ( KiB/s): min= 352, max= 862, per=25.62%, avg=567.57, stdev=115.68, samples=21 iops : min= 22, max= 53, avg=35.05, stdev= 7.08, samples=21 lat (usec) : 250=0.52%, 500=5.86%, 750=0.78%, 1000=0.13% lat (msec) : 2=1.17%, 4=5.73%, 10=23.44%, 20=39.19%, 50=22.14% lat (msec) : 100=1.04% cpu : usr=0.12%, sys=30.64%, ctx=2341, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=2106KiB/s (2157kB/s), 505KiB/s-560KiB/s (517kB/s-574kB/s), io=23.4MiB (24.5MB), run=10801-11372msec WRITE: bw=2216KiB/s (2269kB/s), 540KiB/s-590KiB/s (553kB/s-604kB/s), io=24.6MiB (25.8MB), run=10801-11372msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146708: Tue Nov 18 13:18:43 2025 read: IOPS=18, BW=293KiB/s (300kB/s)(5920KiB/20227msec) clat (usec): min=3079, max=80959, avg=15727.00, stdev=13405.40 lat (usec): min=3080, max=80960, avg=15728.05, stdev=13405.35 clat percentiles (usec): | 1.00th=[ 3392], 5.00th=[ 4080], 10.00th=[ 4752], 20.00th=[ 6259], | 30.00th=[ 7701], 40.00th=[ 9765], 50.00th=[11994], 60.00th=[13829], | 70.00th=[16712], 80.00th=[20317], 90.00th=[30802], 95.00th=[47973], | 99.00th=[67634], 99.50th=[74974], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 96, max= 544, per=24.43%, avg=289.27, stdev=123.61, samples=40 iops : min= 6, max= 34, avg=17.80, stdev= 7.87, samples=40 write: IOPS=19, BW=315KiB/s (322kB/s)(6368KiB/20227msec); 0 zone resets clat (msec): min=14, max=135, avg=36.14, stdev=17.30 lat (msec): min=14, max=135, avg=36.14, stdev=17.30 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 24], | 30.00th=[ 27], 40.00th=[ 29], 50.00th=[ 32], 60.00th=[ 35], | 70.00th=[ 40], 80.00th=[ 46], 90.00th=[ 56], 95.00th=[ 72], | 99.00th=[ 112], 99.50th=[ 127], 99.90th=[ 136], 99.95th=[ 136], | 99.99th=[ 136] bw ( KiB/s): min= 127, max= 576, per=25.14%, avg=312.97, stdev=126.43, samples=40 iops : min= 7, max= 36, avg=19.27, stdev= 8.13, samples=40 lat (msec) : 4=1.95%, 10=17.58%, 20=22.14%, 50=48.70%, 100=8.98% lat (msec) : 250=0.65% cpu : usr=0.10%, sys=8.62%, ctx=1561, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146709: Tue Nov 18 13:18:43 2025 read: IOPS=17, BW=288KiB/s (295kB/s)(5744KiB/19955msec) clat (usec): min=3309, max=78762, avg=14311.14, stdev=12135.51 lat (usec): min=3310, max=78763, avg=14311.97, stdev=12135.46 clat percentiles (usec): | 1.00th=[ 3490], 5.00th=[ 4293], 10.00th=[ 4948], 20.00th=[ 5932], | 30.00th=[ 6915], 40.00th=[ 8225], 50.00th=[10290], 60.00th=[12780], | 70.00th=[15795], 80.00th=[19530], 90.00th=[27919], 95.00th=[41681], | 99.00th=[61080], 99.50th=[76022], 99.90th=[79168], 99.95th=[79168], | 99.99th=[79168] bw ( KiB/s): min= 64, max= 704, per=24.33%, avg=288.03, stdev=156.14, samples=39 iops : min= 4, max= 44, avg=17.72, stdev= 9.91, samples=39 write: IOPS=20, BW=328KiB/s (336kB/s)(6544KiB/19955msec); 0 zone resets clat (msec): min=15, max=125, avg=35.96, stdev=16.52 lat (msec): min=15, max=125, avg=35.96, stdev=16.52 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 24], | 30.00th=[ 27], 40.00th=[ 29], 50.00th=[ 31], 60.00th=[ 35], | 70.00th=[ 39], 80.00th=[ 44], 90.00th=[ 58], 95.00th=[ 73], | 99.00th=[ 90], 99.50th=[ 102], 99.90th=[ 126], 99.95th=[ 126], | 99.99th=[ 126] bw ( KiB/s): min= 159, max= 576, per=26.28%, avg=327.23, stdev=117.83, samples=39 iops : min= 9, max= 36, avg=20.15, stdev= 7.60, samples=39 lat (msec) : 4=1.30%, 10=21.74%, 20=18.49%, 50=49.74%, 100=8.20% lat (msec) : 250=0.52% cpu : usr=0.09%, sys=8.72%, ctx=1558, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146710: Tue Nov 18 13:18:43 2025 read: IOPS=19, BW=310KiB/s (318kB/s)(6256KiB/20153msec) clat (usec): min=2790, max=81164, avg=16080.40, stdev=13689.58 lat (usec): min=2791, max=81166, avg=16081.43, stdev=13689.64 clat percentiles (usec): | 1.00th=[ 3490], 5.00th=[ 4490], 10.00th=[ 5014], 20.00th=[ 6128], | 30.00th=[ 7439], 40.00th=[ 9765], 50.00th=[11600], 60.00th=[14222], | 70.00th=[17433], 80.00th=[22414], 90.00th=[33162], 95.00th=[49546], | 99.00th=[68682], 99.50th=[81265], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 95, max= 704, per=26.31%, avg=311.52, stdev=156.03, samples=40 iops : min= 5, max= 44, avg=19.18, stdev= 9.96, samples=40 write: IOPS=18, BW=299KiB/s (306kB/s)(6032KiB/20153msec); 0 zone resets clat (msec): min=15, max=135, avg=36.56, stdev=18.12 lat (msec): min=15, max=135, avg=36.56, stdev=18.12 clat percentiles (msec): | 1.00th=[ 18], 5.00th=[ 20], 10.00th=[ 22], 20.00th=[ 25], | 30.00th=[ 27], 40.00th=[ 29], 50.00th=[ 31], 60.00th=[ 34], | 70.00th=[ 39], 80.00th=[ 46], 90.00th=[ 61], 95.00th=[ 73], | 99.00th=[ 108], 99.50th=[ 115], 99.90th=[ 136], 99.95th=[ 136], | 99.99th=[ 136] bw ( KiB/s): min= 96, max= 544, per=24.03%, avg=299.17, stdev=124.32, samples=40 iops : min= 6, max= 34, avg=18.40, stdev= 8.01, samples=40 lat (msec) : 4=1.30%, 10=20.05%, 20=19.79%, 50=48.57%, 100=9.64% lat (msec) : 250=0.65% cpu : usr=0.10%, sys=9.04%, ctx=1606, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146711: Tue Nov 18 13:18:43 2025 read: IOPS=18, BW=303KiB/s (310kB/s)(6032KiB/19915msec) clat (usec): min=3494, max=77093, avg=15722.74, stdev=11993.97 lat (usec): min=3502, max=77094, avg=15723.94, stdev=11994.03 clat percentiles (usec): | 1.00th=[ 3687], 5.00th=[ 4359], 10.00th=[ 5080], 20.00th=[ 6456], | 30.00th=[ 8160], 40.00th=[ 9765], 50.00th=[12649], 60.00th=[15139], | 70.00th=[17695], 80.00th=[22152], 90.00th=[31327], 95.00th=[40109], | 99.00th=[63177], 99.50th=[72877], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 127, max= 672, per=25.46%, avg=301.46, stdev=126.93, samples=39 iops : min= 7, max= 42, avg=18.54, stdev= 8.13, samples=39 write: IOPS=19, BW=314KiB/s (322kB/s)(6256KiB/19915msec); 0 zone resets clat (msec): min=14, max=127, avg=35.49, stdev=15.29 lat (msec): min=14, max=127, avg=35.49, stdev=15.29 clat percentiles (msec): | 1.00th=[ 17], 5.00th=[ 20], 10.00th=[ 21], 20.00th=[ 24], | 30.00th=[ 27], 40.00th=[ 29], 50.00th=[ 32], 60.00th=[ 35], | 70.00th=[ 40], 80.00th=[ 46], 90.00th=[ 56], 95.00th=[ 65], | 99.00th=[ 95], 99.50th=[ 114], 99.90th=[ 128], 99.95th=[ 128], | 99.99th=[ 128] bw ( KiB/s): min= 96, max= 544, per=25.13%, avg=312.85, stdev=119.11, samples=39 iops : min= 6, max= 34, avg=19.26, stdev= 7.70, samples=39 lat (msec) : 4=1.17%, 10=18.62%, 20=19.66%, 50=52.34%, 100=7.94% lat (msec) : 250=0.26% cpu : usr=0.10%, sys=8.83%, ctx=1599, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1184KiB/s (1213kB/s), 288KiB/s-310KiB/s (295kB/s-318kB/s), io=23.4MiB (24.5MB), run=19915-20227msec WRITE: bw=1246KiB/s (1276kB/s), 299KiB/s-328KiB/s (306kB/s-336kB/s), io=24.6MiB (25.8MB), run=19915-20227msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146719: Tue Nov 18 13:18:51 2025 read: IOPS=2, BW=2393KiB/s (2450kB/s)(7168KiB/2996msec) clat (usec): min=861, max=817362, avg=235104.31, stdev=279310.73 lat (usec): min=862, max=817363, avg=235105.73, stdev=279310.66 clat percentiles (usec): | 1.00th=[ 865], 5.00th=[ 865], 10.00th=[ 865], 20.00th=[ 83362], | 30.00th=[ 84411], 40.00th=[ 84411], 50.00th=[ 84411], 60.00th=[283116], | 70.00th=[283116], 80.00th=[291505], 90.00th=[817890], 95.00th=[817890], | 99.00th=[817890], 99.50th=[817890], 99.90th=[817890], 99.95th=[817890], | 99.99th=[817890] bw ( KiB/s): min= 2035, max= 3430, per=41.15%, avg=2390.25, stdev=693.19, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 write: IOPS=1, BW=1709KiB/s (1750kB/s)(5120KiB/2996msec); 0 zone resets clat (msec): min=146, max=338, avg=267.89, stdev=77.83 lat (msec): min=146, max=338, avg=267.96, stdev=77.81 clat percentiles (msec): | 1.00th=[ 146], 5.00th=[ 146], 10.00th=[ 146], 20.00th=[ 146], | 30.00th=[ 247], 40.00th=[ 247], 50.00th=[ 275], 60.00th=[ 275], | 70.00th=[ 334], 80.00th=[ 334], 90.00th=[ 338], 95.00th=[ 338], | 99.00th=[ 338], 99.50th=[ 338], 99.90th=[ 338], 99.95th=[ 338], | 99.99th=[ 338] bw ( KiB/s): min= 2035, max= 2048, per=25.13%, avg=2043.67, stdev= 7.51, samples=3 iops : min= 1, max= 2, avg= 1.67, stdev= 0.58, samples=3 lat (usec) : 1000=8.33% lat (msec) : 100=25.00%, 250=16.67%, 500=41.67%, 1000=8.33% cpu : usr=0.00%, sys=38.20%, ctx=1560, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146720: Tue Nov 18 13:18:51 2025 read: IOPS=0, BW=581KiB/s (595kB/s)(2048KiB/3526msec) clat (msec): min=86, max=170, avg=128.20, stdev=59.46 lat (msec): min=86, max=170, avg=128.20, stdev=59.46 clat percentiles (msec): | 1.00th=[ 87], 5.00th=[ 87], 10.00th=[ 87], 20.00th=[ 87], | 30.00th=[ 87], 40.00th=[ 87], 50.00th=[ 87], 60.00th=[ 171], | 70.00th=[ 171], 80.00th=[ 171], 90.00th=[ 171], 95.00th=[ 171], | 99.00th=[ 171], 99.50th=[ 171], 99.90th=[ 171], 99.95th=[ 171], | 99.99th=[ 171] bw ( KiB/s): min= 1712, max= 2048, per=32.37%, avg=1880.00, stdev=237.59, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2904KiB/s (2974kB/s)(10.0MiB/3526msec); 0 zone resets clat (msec): min=88, max=1102, avg=324.65, stdev=309.97 lat (msec): min=88, max=1102, avg=324.76, stdev=309.96 clat percentiles (msec): | 1.00th=[ 89], 5.00th=[ 89], 10.00th=[ 89], 20.00th=[ 100], | 30.00th=[ 103], 40.00th=[ 186], 50.00th=[ 197], 60.00th=[ 266], | 70.00th=[ 296], 80.00th=[ 334], 90.00th=[ 575], 95.00th=[ 1099], | 99.00th=[ 1099], 99.50th=[ 1099], 99.90th=[ 1099], 99.95th=[ 1099], | 99.99th=[ 1099] bw ( KiB/s): min= 2048, max= 6144, per=50.30%, avg=4089.75, stdev=1672.23, samples=4 iops : min= 2, max= 6, avg= 3.75, stdev= 1.71, samples=4 lat (msec) : 100=25.00%, 250=33.33%, 500=25.00%, 750=8.33%, 2000=8.33% cpu : usr=0.06%, sys=40.11%, ctx=533, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146721: Tue Nov 18 13:18:51 2025 read: IOPS=2, BW=2156KiB/s (2208kB/s)(6144KiB/2850msec) clat (usec): min=717, max=243712, avg=103286.39, stdev=115953.09 lat (usec): min=719, max=243713, avg=103287.63, stdev=115953.16 clat percentiles (usec): | 1.00th=[ 717], 5.00th=[ 717], 10.00th=[ 717], 20.00th=[ 742], | 30.00th=[ 742], 40.00th=[ 3032], 50.00th=[ 3032], 60.00th=[147850], | 70.00th=[223347], 80.00th=[223347], 90.00th=[244319], 95.00th=[244319], | 99.00th=[244319], 99.50th=[244319], 99.90th=[244319], 99.95th=[244319], | 99.99th=[244319] bw ( KiB/s): min= 4071, max= 4096, per=70.31%, avg=4083.50, stdev=17.68, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2156KiB/s (2208kB/s)(6144KiB/2850msec); 0 zone resets clat (msec): min=128, max=1082, avg=367.73, stdev=359.04 lat (msec): min=128, max=1082, avg=367.80, stdev=359.05 clat percentiles (msec): | 1.00th=[ 129], 5.00th=[ 129], 10.00th=[ 129], 20.00th=[ 140], | 30.00th=[ 140], 40.00th=[ 251], 50.00th=[ 251], 60.00th=[ 275], | 70.00th=[ 334], 80.00th=[ 334], 90.00th=[ 1083], 95.00th=[ 1083], | 99.00th=[ 1083], 99.50th=[ 1083], 99.90th=[ 1083], 99.95th=[ 1083], | 99.99th=[ 1083] bw ( KiB/s): min= 2035, max= 3424, per=29.38%, avg=2388.75, stdev=690.19, samples=4 iops : min= 1, max= 3, avg= 2.00, stdev= 0.82, samples=4 lat (usec) : 750=16.67% lat (msec) : 4=8.33%, 250=41.67%, 500=25.00%, 2000=8.33% cpu : usr=0.00%, sys=32.26%, ctx=613, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146722: Tue Nov 18 13:18:51 2025 read: IOPS=1, BW=1884KiB/s (1930kB/s)(5120KiB/2717msec) clat (usec): min=650, max=117669, avg=68268.53, stdev=56620.02 lat (usec): min=651, max=117670, avg=68269.44, stdev=56619.82 clat percentiles (usec): | 1.00th=[ 652], 5.00th=[ 652], 10.00th=[ 652], 20.00th=[ 652], | 30.00th=[ 12780], 40.00th=[ 12780], 50.00th=[103285], 60.00th=[103285], | 70.00th=[106431], 80.00th=[106431], 90.00th=[117965], 95.00th=[117965], | 99.00th=[117965], 99.50th=[117965], 99.90th=[117965], 99.95th=[117965], | 99.99th=[117965] bw ( KiB/s): min= 8143, max= 8143, per=100.00%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=2, BW=2638KiB/s (2702kB/s)(7168KiB/2717msec); 0 zone resets clat (msec): min=160, max=856, avg=336.88, stdev=234.87 lat (msec): min=160, max=856, avg=336.95, stdev=234.88 clat percentiles (msec): | 1.00th=[ 161], 5.00th=[ 161], 10.00th=[ 161], 20.00th=[ 228], | 30.00th=[ 232], 40.00th=[ 232], 50.00th=[ 268], 60.00th=[ 288], | 70.00th=[ 288], 80.00th=[ 330], 90.00th=[ 860], 95.00th=[ 860], | 99.00th=[ 860], 99.50th=[ 860], 99.90th=[ 860], 99.95th=[ 860], | 99.99th=[ 860] bw ( KiB/s): min= 1732, max= 4096, per=36.77%, avg=2989.75, stdev=1283.36, samples=4 iops : min= 1, max= 4, avg= 2.50, stdev= 1.73, samples=4 lat (usec) : 750=8.33% lat (msec) : 20=8.33%, 250=50.00%, 500=25.00%, 1000=8.33% cpu : usr=0.00%, sys=40.72%, ctx=574, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=5808KiB/s (5948kB/s), 581KiB/s-2393KiB/s (595kB/s-2450kB/s), io=20.0MiB (20.0MB), run=2717-3526msec WRITE: bw=8132KiB/s (8327kB/s), 1709KiB/s-2904KiB/s (1750kB/s-2974kB/s), io=28.0MiB (29.4MB), run=2717-3526msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146724: Tue Nov 18 13:18:51 2025 read: IOPS=2, BW=2137KiB/s (2188kB/s)(7168KiB/3354msec) clat (msec): min=21, max=1142, avg=396.95, stdev=426.63 lat (msec): min=21, max=1142, avg=396.95, stdev=426.63 clat percentiles (msec): | 1.00th=[ 22], 5.00th=[ 22], 10.00th=[ 22], 20.00th=[ 44], | 30.00th=[ 53], 40.00th=[ 53], 50.00th=[ 296], 60.00th=[ 456], | 70.00th=[ 456], 80.00th=[ 768], 90.00th=[ 1150], 95.00th=[ 1150], | 99.00th=[ 1150], 99.50th=[ 1150], 99.90th=[ 1150], 99.95th=[ 1150], | 99.99th=[ 1150] bw ( KiB/s): min= 1969, max= 4096, per=41.81%, avg=2529.75, stdev=1044.61, samples=4 iops : min= 1, max= 4, avg= 1.75, stdev= 1.50, samples=4 write: IOPS=1, BW=1527KiB/s (1563kB/s)(5120KiB/3354msec); 0 zone resets clat (msec): min=70, max=169, avg=105.09, stdev=40.65 lat (msec): min=70, max=169, avg=105.18, stdev=40.68 clat percentiles (msec): | 1.00th=[ 71], 5.00th=[ 71], 10.00th=[ 71], 20.00th=[ 71], | 30.00th=[ 77], 40.00th=[ 77], 50.00th=[ 90], 60.00th=[ 90], | 70.00th=[ 121], 80.00th=[ 121], 90.00th=[ 169], 95.00th=[ 169], | 99.00th=[ 169], 99.50th=[ 169], 99.90th=[ 169], 99.95th=[ 169], | 99.99th=[ 169] bw ( KiB/s): min= 2011, max= 2048, per=23.96%, avg=2029.50, stdev=26.16, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 50=16.67%, 100=33.33%, 250=16.67%, 500=16.67%, 1000=8.33% lat (msec) : 2000=8.33% cpu : usr=0.03%, sys=10.14%, ctx=62, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146725: Tue Nov 18 13:18:51 2025 read: IOPS=0, BW=605KiB/s (620kB/s)(2048KiB/3385msec) clat (msec): min=123, max=208, avg=165.72, stdev=60.35 lat (msec): min=123, max=208, avg=165.72, stdev=60.35 clat percentiles (msec): | 1.00th=[ 124], 5.00th=[ 124], 10.00th=[ 124], 20.00th=[ 124], | 30.00th=[ 124], 40.00th=[ 124], 50.00th=[ 124], 60.00th=[ 209], | 70.00th=[ 209], 80.00th=[ 209], 90.00th=[ 209], 95.00th=[ 209], | 99.00th=[ 209], 99.50th=[ 209], 99.90th=[ 209], 99.95th=[ 209], | 99.99th=[ 209] bw ( KiB/s): min= 1943, max= 2048, per=32.98%, avg=1995.50, stdev=74.25, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 write: IOPS=2, BW=3025KiB/s (3098kB/s)(10.0MiB/3385msec); 0 zone resets clat (msec): min=84, max=1465, avg=297.84, stdev=415.78 lat (msec): min=84, max=1465, avg=297.94, stdev=415.78 clat percentiles (msec): | 1.00th=[ 85], 5.00th=[ 85], 10.00th=[ 85], 20.00th=[ 112], | 30.00th=[ 130], 40.00th=[ 146], 50.00th=[ 163], 60.00th=[ 163], | 70.00th=[ 174], 80.00th=[ 211], 90.00th=[ 334], 95.00th=[ 1469], | 99.00th=[ 1469], 99.50th=[ 1469], 99.90th=[ 1469], 99.95th=[ 1469], | 99.99th=[ 1469] bw ( KiB/s): min= 3938, max= 6035, per=55.37%, avg=4689.67, stdev=1167.77, samples=3 iops : min= 3, max= 5, avg= 4.00, stdev= 1.00, samples=3 lat (msec) : 100=8.33%, 250=75.00%, 500=8.33%, 2000=8.33% cpu : usr=0.00%, sys=15.19%, ctx=346, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146726: Tue Nov 18 13:18:51 2025 read: IOPS=1, BW=1918KiB/s (1964kB/s)(6144KiB/3203msec) clat (msec): min=44, max=263, avg=102.87, stdev=81.44 lat (msec): min=44, max=263, avg=102.87, stdev=81.44 clat percentiles (msec): | 1.00th=[ 45], 5.00th=[ 45], 10.00th=[ 45], 20.00th=[ 47], | 30.00th=[ 47], 40.00th=[ 81], 50.00th=[ 81], 60.00th=[ 84], | 70.00th=[ 100], 80.00th=[ 100], 90.00th=[ 264], 95.00th=[ 264], | 99.00th=[ 264], 99.50th=[ 264], 99.90th=[ 264], 99.95th=[ 264], | 99.99th=[ 264] bw ( KiB/s): min= 2048, max= 8047, per=83.43%, avg=5047.50, stdev=4241.93, samples=2 iops : min= 2, max= 7, avg= 4.50, stdev= 3.54, samples=2 write: IOPS=1, BW=1918KiB/s (1964kB/s)(6144KiB/3203msec); 0 zone resets clat (msec): min=93, max=1520, avg=417.79, stdev=546.98 lat (msec): min=93, max=1520, avg=417.88, stdev=546.97 clat percentiles (msec): | 1.00th=[ 94], 5.00th=[ 94], 10.00th=[ 94], 20.00th=[ 148], | 30.00th=[ 148], 40.00th=[ 176], 50.00th=[ 176], 60.00th=[ 220], | 70.00th=[ 351], 80.00th=[ 351], 90.00th=[ 1519], 95.00th=[ 1519], | 99.00th=[ 1519], 99.50th=[ 1519], 99.90th=[ 1519], 99.95th=[ 1519], | 99.99th=[ 1519] bw ( KiB/s): min= 1969, max= 3842, per=29.13%, avg=2467.50, stdev=916.90, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 50=16.67%, 100=33.33%, 250=25.00%, 500=16.67%, 2000=8.33% cpu : usr=0.06%, sys=11.71%, ctx=462, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146727: Tue Nov 18 13:18:51 2025 read: IOPS=1, BW=1523KiB/s (1559kB/s)(5120KiB/3362msec) clat (msec): min=52, max=190, avg=110.37, stdev=59.84 lat (msec): min=52, max=190, avg=110.37, stdev=59.84 clat percentiles (msec): | 1.00th=[ 54], 5.00th=[ 54], 10.00th=[ 54], 20.00th=[ 54], | 30.00th=[ 72], 40.00th=[ 72], 50.00th=[ 80], 60.00th=[ 80], | 70.00th=[ 157], 80.00th=[ 157], 90.00th=[ 190], 95.00th=[ 190], | 99.00th=[ 190], 99.50th=[ 190], 99.90th=[ 190], 99.95th=[ 190], | 99.99th=[ 190] bw ( KiB/s): min= 2048, max= 6035, per=66.80%, avg=4041.50, stdev=2819.23, samples=2 iops : min= 2, max= 5, avg= 3.50, stdev= 2.12, samples=2 write: IOPS=2, BW=2132KiB/s (2183kB/s)(7168KiB/3362msec); 0 zone resets clat (msec): min=131, max=1445, avg=385.99, stdev=473.51 lat (msec): min=131, max=1445, avg=386.08, stdev=473.51 clat percentiles (msec): | 1.00th=[ 132], 5.00th=[ 132], 10.00th=[ 132], 20.00th=[ 155], | 30.00th=[ 155], 40.00th=[ 155], 50.00th=[ 220], 60.00th=[ 234], | 70.00th=[ 234], 80.00th=[ 363], 90.00th=[ 1452], 95.00th=[ 1452], | 99.00th=[ 1452], 99.50th=[ 1452], 99.90th=[ 1452], 99.95th=[ 1452], | 99.99th=[ 1452] bw ( KiB/s): min= 1828, max= 3938, per=29.00%, avg=2456.25, stdev=992.50, samples=4 iops : min= 1, max= 3, avg= 1.75, stdev= 0.96, samples=4 lat (msec) : 100=25.00%, 250=58.33%, 500=8.33%, 2000=8.33% cpu : usr=0.06%, sys=17.58%, ctx=133, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=6050KiB/s (6195kB/s), 605KiB/s-2137KiB/s (620kB/s-2188kB/s), io=20.0MiB (20.0MB), run=3203-3385msec WRITE: bw=8470KiB/s (8674kB/s), 1527KiB/s-3025KiB/s (1563kB/s-3098kB/s), io=28.0MiB (29.4MB), run=3203-3385msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=146734: Tue Nov 18 13:18:59 2025 read: IOPS=2, BW=8840KiB/s (9052kB/s)(12.0MiB/1390msec) clat (msec): min=362, max=526, avg=453.29, stdev=83.18 lat (msec): min=362, max=526, avg=453.29, stdev=83.18 clat percentiles (msec): | 1.00th=[ 363], 5.00th=[ 363], 10.00th=[ 363], 20.00th=[ 363], | 30.00th=[ 363], 40.00th=[ 472], 50.00th=[ 472], 60.00th=[ 472], | 70.00th=[ 527], 80.00th=[ 527], 90.00th=[ 527], 95.00th=[ 527], | 99.00th=[ 527], 99.50th=[ 527], 99.90th=[ 527], 99.95th=[ 527], | 99.99th=[ 527] bw ( KiB/s): min= 7670, max= 8192, per=90.04%, avg=7931.00, stdev=369.11, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.07%, sys=55.80%, ctx=2819, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146735: Tue Nov 18 13:18:59 2025 read: IOPS=0, BW=2202KiB/s (2255kB/s)(4096KiB/1860msec) clat (nsec): min=347296k, max=347296k, avg=347295549.00, stdev= 0.00 lat (nsec): min=347302k, max=347302k, avg=347301729.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 347], 5.00th=[ 347], 10.00th=[ 347], 20.00th=[ 347], | 30.00th=[ 347], 40.00th=[ 347], 50.00th=[ 347], 60.00th=[ 347], | 70.00th=[ 347], 80.00th=[ 347], 90.00th=[ 347], 95.00th=[ 347], | 99.00th=[ 347], 99.50th=[ 347], 99.90th=[ 347], 99.95th=[ 347], | 99.99th=[ 347] bw ( KiB/s): min= 7953, max= 7953, per=90.29%, avg=7953.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4404KiB/s (4510kB/s)(8192KiB/1860msec); 0 zone resets clat (msec): min=601, max=884, avg=743.28, stdev=200.39 lat (msec): min=601, max=885, avg=743.60, stdev=200.43 clat percentiles (msec): | 1.00th=[ 600], 5.00th=[ 600], 10.00th=[ 600], 20.00th=[ 600], | 30.00th=[ 600], 40.00th=[ 600], 50.00th=[ 600], 60.00th=[ 885], | 70.00th=[ 885], 80.00th=[ 885], 90.00th=[ 885], 95.00th=[ 885], | 99.00th=[ 885], 99.50th=[ 885], 99.90th=[ 885], 99.95th=[ 885], | 99.99th=[ 885] bw ( KiB/s): min= 8192, max= 8192, per=48.43%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=12.69%, ctx=962, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146736: Tue Nov 18 13:18:59 2025 write: IOPS=1, BW=6344KiB/s (6496kB/s)(12.0MiB/1937msec); 0 zone resets clat (msec): min=196, max=1210, avg=635.00, stdev=520.51 lat (msec): min=196, max=1210, avg=635.45, stdev=520.77 clat percentiles (msec): | 1.00th=[ 197], 5.00th=[ 197], 10.00th=[ 197], 20.00th=[ 197], | 30.00th=[ 197], 40.00th=[ 498], 50.00th=[ 498], 60.00th=[ 498], | 70.00th=[ 1217], 80.00th=[ 1217], 90.00th=[ 1217], 95.00th=[ 1217], | 99.00th=[ 1217], 99.50th=[ 1217], 99.90th=[ 1217], 99.95th=[ 1217], | 99.99th=[ 1217] bw ( KiB/s): min= 8192, max= 8192, per=48.43%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.05%, sys=18.29%, ctx=120, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146737: Tue Nov 18 13:18:59 2025 write: IOPS=1, BW=7812KiB/s (7999kB/s)(12.0MiB/1573msec); 0 zone resets clat (msec): min=343, max=627, avg=502.03, stdev=145.09 lat (msec): min=343, max=627, avg=502.35, stdev=145.07 clat percentiles (msec): | 1.00th=[ 342], 5.00th=[ 342], 10.00th=[ 342], 20.00th=[ 342], | 30.00th=[ 342], 40.00th=[ 535], 50.00th=[ 535], 60.00th=[ 535], | 70.00th=[ 625], 80.00th=[ 625], 90.00th=[ 625], 95.00th=[ 625], | 99.00th=[ 625], 99.50th=[ 625], 99.90th=[ 625], 99.95th=[ 625], | 99.99th=[ 625] bw ( KiB/s): min= 5728, max= 8192, per=41.14%, avg=6960.00, stdev=1742.31, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=33.33%, 750=66.67% cpu : usr=0.06%, sys=21.37%, ctx=75, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8809KiB/s (9020kB/s), 2202KiB/s-8840KiB/s (2255kB/s-9052kB/s), io=16.0MiB (16.8MB), run=1390-1860msec WRITE: bw=16.5MiB/s (17.3MB/s), 4404KiB/s-7812KiB/s (4510kB/s-7999kB/s), io=32.0MiB (33.6MB), run=1573-1937msec rand-rw: (groupid=0, jobs=1): err= 0: pid=146738: Tue Nov 18 13:18:59 2025 read: IOPS=1, BW=8047KiB/s (8240kB/s)(12.0MiB/1527msec) clat (msec): min=328, max=784, avg=493.11, stdev=253.23 lat (msec): min=328, max=784, avg=493.11, stdev=253.23 clat percentiles (msec): | 1.00th=[ 330], 5.00th=[ 330], 10.00th=[ 330], 20.00th=[ 330], | 30.00th=[ 330], 40.00th=[ 368], 50.00th=[ 368], 60.00th=[ 368], | 70.00th=[ 785], 80.00th=[ 785], 90.00th=[ 785], 95.00th=[ 785], | 99.00th=[ 785], 99.50th=[ 785], 99.90th=[ 785], 99.95th=[ 785], | 99.99th=[ 785] bw ( KiB/s): min=16286, max=16286, per=100.00%, avg=16286.00, stdev= 0.00, samples=1 iops : min= 3, max= 3, avg= 3.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 1000=33.33% cpu : usr=0.00%, sys=17.37%, ctx=99, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146739: Tue Nov 18 13:18:59 2025 read: IOPS=0, BW=2147KiB/s (2198kB/s)(4096KiB/1908msec) clat (nsec): min=645628k, max=645628k, avg=645627579.00, stdev= 0.00 lat (nsec): min=645630k, max=645630k, avg=645630235.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 642], 5.00th=[ 642], 10.00th=[ 642], 20.00th=[ 642], | 30.00th=[ 642], 40.00th=[ 642], 50.00th=[ 642], 60.00th=[ 642], | 70.00th=[ 642], 80.00th=[ 642], 90.00th=[ 642], 95.00th=[ 642], | 99.00th=[ 642], 99.50th=[ 642], 99.90th=[ 642], 99.95th=[ 642], | 99.99th=[ 642] bw ( KiB/s): min= 5557, max= 5557, per=64.71%, avg=5557.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4294KiB/s (4397kB/s)(8192KiB/1908msec); 0 zone resets clat (msec): min=300, max=944, avg=622.67, stdev=455.16 lat (msec): min=301, max=944, avg=623.06, stdev=455.24 clat percentiles (msec): | 1.00th=[ 300], 5.00th=[ 300], 10.00th=[ 300], 20.00th=[ 300], | 30.00th=[ 300], 40.00th=[ 300], 50.00th=[ 300], 60.00th=[ 944], | 70.00th=[ 944], 80.00th=[ 944], 90.00th=[ 944], 95.00th=[ 944], | 99.00th=[ 944], 99.50th=[ 944], 99.90th=[ 944], 99.95th=[ 944], | 99.99th=[ 944] bw ( KiB/s): min= 8192, max= 8192, per=47.70%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=33.33%, 750=33.33%, 1000=33.33% cpu : usr=0.00%, sys=14.58%, ctx=67, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146740: Tue Nov 18 13:18:59 2025 write: IOPS=1, BW=7165KiB/s (7337kB/s)(12.0MiB/1715msec); 0 zone resets clat (msec): min=239, max=994, avg=551.44, stdev=394.29 lat (msec): min=239, max=994, avg=551.77, stdev=394.31 clat percentiles (msec): | 1.00th=[ 241], 5.00th=[ 241], 10.00th=[ 241], 20.00th=[ 241], | 30.00th=[ 241], 40.00th=[ 422], 50.00th=[ 422], 60.00th=[ 422], | 70.00th=[ 995], 80.00th=[ 995], 90.00th=[ 995], 95.00th=[ 995], | 99.00th=[ 995], 99.50th=[ 995], 99.90th=[ 995], 99.95th=[ 995], | 99.99th=[ 995] bw ( KiB/s): min= 8143, max= 8143, per=47.41%, avg=8143.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 1000=33.33% cpu : usr=0.00%, sys=15.93%, ctx=83, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=146741: Tue Nov 18 13:18:59 2025 write: IOPS=2, BW=9022KiB/s (9239kB/s)(12.0MiB/1362msec); 0 zone resets clat (msec): min=180, max=694, avg=443.45, stdev=257.61 lat (msec): min=180, max=695, avg=443.78, stdev=257.64 clat percentiles (msec): | 1.00th=[ 180], 5.00th=[ 180], 10.00th=[ 180], 20.00th=[ 180], | 30.00th=[ 180], 40.00th=[ 456], 50.00th=[ 456], 60.00th=[ 456], | 70.00th=[ 693], 80.00th=[ 693], 90.00th=[ 693], 95.00th=[ 693], | 99.00th=[ 693], 99.50th=[ 693], 99.90th=[ 693], 99.95th=[ 693], | 99.99th=[ 693] bw ( KiB/s): min=11538, max=11538, per=67.18%, avg=11538.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 750=33.33% cpu : usr=0.15%, sys=22.26%, ctx=75, majf=0, minf=10 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=8587KiB/s (8793kB/s), 2147KiB/s-8047KiB/s (2198kB/s-8240kB/s), io=16.0MiB (16.8MB), run=1527-1908msec WRITE: bw=16.8MiB/s (17.6MB/s), 4294KiB/s-9022KiB/s (4397kB/s-9239kB/s), io=32.0MiB (33.6MB), run=1362-1908msec