== sanity test 398b: DIO and buffer IO race ============== 14:48:41 (1758826121) /usr/bin/fio 48+0 records in 48+0 records out 50331648 bytes (50 MB, 48 MiB) copied, 2.98829 s, 16.8 MB/s mix direct rw 4096 by fio with 4 jobs... mix buffer rw 4096 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=191564: Thu Sep 25 14:49:31 2025 read: IOPS=36, BW=145KiB/s (148kB/s)(5992KiB/41326msec) clat (usec): min=221, max=615019, avg=15207.86, stdev=17144.38 lat (usec): min=222, max=615020, avg=15208.86, stdev=17144.31 clat percentiles (usec): | 1.00th=[ 367], 5.00th=[ 6521], 10.00th=[ 7504], 20.00th=[ 8979], | 30.00th=[ 10290], 40.00th=[ 11994], 50.00th=[ 13829], 60.00th=[ 15533], | 70.00th=[ 17171], 80.00th=[ 19268], 90.00th=[ 23200], 95.00th=[ 27657], | 99.00th=[ 41681], 99.50th=[ 45876], 99.90th=[ 56361], 99.95th=[616563], | 99.99th=[616563] bw ( KiB/s): min= 79, max= 232, per=25.30%, avg=146.75, stdev=29.79, samples=81 iops : min= 19, max= 58, avg=36.35, stdev= 7.48, samples=81 write: IOPS=38, BW=152KiB/s (156kB/s)(6296KiB/41326msec); 0 zone resets clat (usec): min=545, max=89833, avg=11672.58, stdev=8799.88 lat (usec): min=545, max=89834, avg=11673.18, stdev=8799.88 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 2311], 10.00th=[ 2606], 20.00th=[ 3261], | 30.00th=[ 5407], 40.00th=[ 8160], 50.00th=[ 9896], 60.00th=[11994], | 70.00th=[14746], 80.00th=[17957], 90.00th=[23725], 95.00th=[28705], | 99.00th=[38536], 99.50th=[42730], 99.90th=[47973], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 40, max= 248, per=25.40%, avg=154.19, stdev=39.00, samples=81 iops : min= 10, max= 62, avg=38.20, stdev= 9.76, samples=81 lat (usec) : 250=0.03%, 500=0.68%, 750=0.29%, 1000=0.20% lat (msec) : 2=1.50%, 4=11.07%, 10=25.68%, 20=43.95%, 50=16.47% lat (msec) : 100=0.10%, 750=0.03% cpu : usr=0.07%, sys=26.03%, ctx=9006, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191565: Thu Sep 25 14:49:31 2025 read: IOPS=36, BW=145KiB/s (149kB/s)(5940KiB/40890msec) clat (usec): min=121, max=58878, avg=14477.99, stdev=7098.62 lat (usec): min=121, max=58878, avg=14478.99, stdev=7098.86 clat percentiles (usec): | 1.00th=[ 363], 5.00th=[ 6652], 10.00th=[ 7504], 20.00th=[ 8979], | 30.00th=[10290], 40.00th=[11600], 50.00th=[13173], 60.00th=[15008], | 70.00th=[16909], 80.00th=[19006], 90.00th=[22414], 95.00th=[27657], | 99.00th=[40109], 99.50th=[44827], 99.90th=[56886], 99.95th=[58983], | 99.99th=[58983] bw ( KiB/s): min= 6, max= 231, per=25.04%, avg=145.26, stdev=34.91, samples=81 iops : min= 1, max= 57, avg=35.96, stdev= 8.67, samples=81 write: IOPS=38, BW=155KiB/s (159kB/s)(6348KiB/40890msec); 0 zone resets clat (usec): min=638, max=607228, avg=12121.41, stdev=17420.33 lat (usec): min=639, max=607229, avg=12122.04, stdev=17420.36 clat percentiles (usec): | 1.00th=[ 1057], 5.00th=[ 2212], 10.00th=[ 2606], 20.00th=[ 3228], | 30.00th=[ 5473], 40.00th=[ 8094], 50.00th=[ 9896], 60.00th=[ 11863], | 70.00th=[ 14877], 80.00th=[ 18744], 90.00th=[ 24249], 95.00th=[ 28705], | 99.00th=[ 40633], 99.50th=[ 46400], 99.90th=[ 64226], 99.95th=[608175], | 99.99th=[608175] bw ( KiB/s): min= 55, max= 240, per=25.87%, avg=157.01, stdev=38.33, samples=80 iops : min= 13, max= 60, avg=38.90, stdev= 9.71, samples=80 lat (usec) : 250=0.03%, 500=0.78%, 750=0.29%, 1000=0.26% lat (msec) : 2=1.37%, 4=11.36%, 10=25.65%, 20=43.13%, 50=16.76% lat (msec) : 100=0.33%, 750=0.03% cpu : usr=0.11%, sys=26.34%, ctx=9041, majf=0, minf=17 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191566: Thu Sep 25 14:49:31 2025 read: IOPS=36, BW=147KiB/s (150kB/s)(6028KiB/41113msec) clat (usec): min=74, max=632211, avg=15272.45, stdev=17624.29 lat (usec): min=75, max=632211, avg=15273.05, stdev=17624.30 clat percentiles (usec): | 1.00th=[ 330], 5.00th=[ 6259], 10.00th=[ 7373], 20.00th=[ 8979], | 30.00th=[ 10290], 40.00th=[ 11863], 50.00th=[ 13435], 60.00th=[ 15401], | 70.00th=[ 17433], 80.00th=[ 19530], 90.00th=[ 23987], 95.00th=[ 28705], | 99.00th=[ 41157], 99.50th=[ 46924], 99.90th=[ 60556], 99.95th=[633340], | 99.99th=[633340] bw ( KiB/s): min= 56, max= 240, per=25.54%, avg=148.11, stdev=33.17, samples=80 iops : min= 14, max= 60, avg=36.67, stdev= 8.35, samples=80 write: IOPS=38, BW=152KiB/s (156kB/s)(6260KiB/41113msec); 0 zone resets clat (usec): min=602, max=57880, avg=11463.67, stdev=8715.70 lat (usec): min=602, max=57881, avg=11464.32, stdev=8715.72 clat percentiles (usec): | 1.00th=[ 947], 5.00th=[ 2245], 10.00th=[ 2573], 20.00th=[ 3195], | 30.00th=[ 4883], 40.00th=[ 7898], 50.00th=[ 9503], 60.00th=[11994], | 70.00th=[14484], 80.00th=[17957], 90.00th=[23200], 95.00th=[28181], | 99.00th=[40633], 99.50th=[45351], 99.90th=[51119], 99.95th=[57934], | 99.99th=[57934] bw ( KiB/s): min= 20, max= 256, per=24.94%, avg=151.37, stdev=42.98, samples=81 iops : min= 5, max= 64, avg=37.49, stdev=10.76, samples=81 lat (usec) : 100=0.03%, 250=0.20%, 500=0.72%, 750=0.33%, 1000=0.36% lat (msec) : 2=1.20%, 4=11.85%, 10=25.36%, 20=43.00%, 50=16.73% lat (msec) : 100=0.20%, 750=0.03% cpu : usr=0.11%, sys=26.04%, ctx=8824, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191567: Thu Sep 25 14:49:31 2025 read: IOPS=36, BW=146KiB/s (150kB/s)(6052KiB/41364msec) clat (usec): min=84, max=49862, avg=14559.39, stdev=7126.53 lat (usec): min=85, max=49862, avg=14559.93, stdev=7126.54 clat percentiles (usec): | 1.00th=[ 396], 5.00th=[ 6063], 10.00th=[ 7439], 20.00th=[ 8848], | 30.00th=[10028], 40.00th=[11600], 50.00th=[13304], 60.00th=[15270], | 70.00th=[17171], 80.00th=[19792], 90.00th=[23462], 95.00th=[27919], | 99.00th=[38011], 99.50th=[41157], 99.90th=[47449], 99.95th=[50070], | 99.99th=[50070] bw ( KiB/s): min= 64, max= 224, per=25.44%, avg=147.58, stdev=33.34, samples=81 iops : min= 16, max= 56, avg=36.54, stdev= 8.45, samples=81 write: IOPS=37, BW=151KiB/s (154kB/s)(6236KiB/41364msec); 0 zone resets clat (usec): min=569, max=626724, avg=12296.38, stdev=18022.28 lat (usec): min=569, max=626726, avg=12296.93, stdev=18022.32 clat percentiles (usec): | 1.00th=[ 873], 5.00th=[ 2114], 10.00th=[ 2540], 20.00th=[ 3097], | 30.00th=[ 5211], 40.00th=[ 8094], 50.00th=[ 10159], 60.00th=[ 12256], | 70.00th=[ 15139], 80.00th=[ 19006], 90.00th=[ 24249], 95.00th=[ 29754], | 99.00th=[ 41681], 99.50th=[ 46400], 99.90th=[ 54789], 99.95th=[624952], | 99.99th=[624952] bw ( KiB/s): min= 64, max= 232, per=25.09%, avg=152.30, stdev=35.85, samples=81 iops : min= 16, max= 58, avg=37.73, stdev= 8.97, samples=81 lat (usec) : 100=0.03%, 250=0.07%, 500=0.78%, 750=0.33%, 1000=0.52% lat (msec) : 2=1.50%, 4=11.39%, 10=24.64%, 20=42.32%, 50=18.29% lat (msec) : 100=0.10%, 750=0.03% cpu : usr=0.07%, sys=25.84%, ctx=8981, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=581KiB/s (594kB/s), 145KiB/s-147KiB/s (148kB/s-150kB/s), io=23.4MiB (24.6MB), run=40890-41364msec WRITE: bw=608KiB/s (622kB/s), 151KiB/s-155KiB/s (154kB/s-159kB/s), io=24.6MiB (25.7MB), run=40890-41364msec rand-rw: (groupid=0, jobs=1): err= 0: pid=191560: Thu Sep 25 14:49:37 2025 read: IOPS=31, BW=126KiB/s (129kB/s)(5992KiB/47402msec) clat (msec): min=2, max=617, avg=14.06, stdev=18.87 lat (msec): min=2, max=617, avg=14.06, stdev=18.87 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 9], 50.00th=[ 11], 60.00th=[ 12], | 70.00th=[ 16], 80.00th=[ 22], 90.00th=[ 30], 95.00th=[ 36], | 99.00th=[ 52], 99.50th=[ 57], 99.90th=[ 70], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 40, max= 392, per=25.10%, avg=126.99, stdev=81.53, samples=93 iops : min= 10, max= 98, avg=31.38, stdev=20.50, samples=93 write: IOPS=33, BW=133KiB/s (136kB/s)(6296KiB/47402msec); 0 zone resets clat (usec): min=3226, max=83393, avg=16631.98, stdev=11359.76 lat (usec): min=3226, max=83394, avg=16632.73, stdev=11359.76 clat percentiles (usec): | 1.00th=[ 4555], 5.00th=[ 5211], 10.00th=[ 5735], 20.00th=[ 6849], | 30.00th=[ 8225], 40.00th=[10421], 50.00th=[13435], 60.00th=[16909], | 70.00th=[20841], 80.00th=[25297], 90.00th=[31851], 95.00th=[38011], | 99.00th=[56886], 99.50th=[61080], 99.90th=[79168], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 32, max= 432, per=25.15%, avg=133.27, stdev=82.20, samples=93 iops : min= 8, max= 108, avg=32.95, stdev=20.67, samples=93 lat (msec) : 4=2.70%, 10=40.92%, 20=29.39%, 50=25.68%, 100=1.27% lat (msec) : 750=0.03% cpu : usr=0.09%, sys=12.72%, ctx=5410, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1498,1574,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191561: Thu Sep 25 14:49:37 2025 read: IOPS=31, BW=126KiB/s (129kB/s)(5940KiB/47250msec) clat (msec): min=2, max=633, avg=14.06, stdev=19.31 lat (msec): min=2, max=633, avg=14.06, stdev=19.31 clat percentiles (msec): | 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6], | 30.00th=[ 7], 40.00th=[ 9], 50.00th=[ 10], 60.00th=[ 12], | 70.00th=[ 16], 80.00th=[ 21], 90.00th=[ 30], 95.00th=[ 37], | 99.00th=[ 53], 99.50th=[ 57], 99.90th=[ 78], 99.95th=[ 634], | 99.99th=[ 634] bw ( KiB/s): min= 32, max= 416, per=25.10%, avg=126.99, stdev=79.18, samples=93 iops : min= 8, max= 104, avg=31.38, stdev=19.91, samples=93 write: IOPS=33, BW=134KiB/s (138kB/s)(6348KiB/47250msec); 0 zone resets clat (usec): min=4145, max=78105, avg=16517.28, stdev=11123.59 lat (usec): min=4145, max=78106, avg=16518.05, stdev=11123.58 clat percentiles (usec): | 1.00th=[ 4752], 5.00th=[ 5276], 10.00th=[ 5866], 20.00th=[ 6980], | 30.00th=[ 8586], 40.00th=[10814], 50.00th=[13304], 60.00th=[16909], | 70.00th=[20317], 80.00th=[24773], 90.00th=[30802], 95.00th=[37487], | 99.00th=[57410], 99.50th=[65799], 99.90th=[76022], 99.95th=[78119], | 99.99th=[78119] bw ( KiB/s): min= 55, max= 408, per=25.62%, avg=135.76, stdev=81.57, samples=93 iops : min= 13, max= 102, avg=33.57, stdev=20.53, samples=93 lat (msec) : 4=2.18%, 10=41.44%, 20=30.08%, 50=24.84%, 100=1.43% lat (msec) : 750=0.03% cpu : usr=0.11%, sys=12.68%, ctx=5433, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1485,1587,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191562: Thu Sep 25 14:49:37 2025 read: IOPS=32, BW=128KiB/s (131kB/s)(6028KiB/47060msec) clat (usec): min=2342, max=69602, avg=13261.40, stdev=10296.05 lat (usec): min=2342, max=69603, avg=13262.04, stdev=10296.05 clat percentiles (usec): | 1.00th=[ 3294], 5.00th=[ 4015], 10.00th=[ 4359], 20.00th=[ 5145], | 30.00th=[ 6587], 40.00th=[ 8291], 50.00th=[ 9896], 60.00th=[11469], | 70.00th=[14615], 80.00th=[19792], 90.00th=[28181], 95.00th=[34341], | 99.00th=[49546], 99.50th=[56886], 99.90th=[63177], 99.95th=[69731], | 99.99th=[69731] bw ( KiB/s): min= 39, max= 432, per=25.51%, avg=129.07, stdev=84.38, samples=92 iops : min= 9, max= 108, avg=31.89, stdev=21.23, samples=92 write: IOPS=33, BW=133KiB/s (136kB/s)(6260KiB/47060msec); 0 zone resets clat (msec): min=4, max=633, avg=17.20, stdev=19.27 lat (msec): min=4, max=633, avg=17.20, stdev=19.27 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 14], 60.00th=[ 18], | 70.00th=[ 22], 80.00th=[ 26], 90.00th=[ 32], 95.00th=[ 38], | 99.00th=[ 59], 99.50th=[ 65], 99.90th=[ 86], 99.95th=[ 634], | 99.99th=[ 634] bw ( KiB/s): min= 32, max= 368, per=25.22%, avg=133.67, stdev=78.74, samples=92 iops : min= 8, max= 92, avg=33.04, stdev=19.80, samples=92 lat (msec) : 4=2.44%, 10=40.95%, 20=30.44%, 50=24.93%, 100=1.20% lat (msec) : 750=0.03% cpu : usr=0.08%, sys=12.77%, ctx=5412, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1507,1565,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191563: Thu Sep 25 14:49:37 2025 read: IOPS=31, BW=128KiB/s (131kB/s)(6052KiB/47348msec) clat (usec): min=2270, max=77560, avg=13510.46, stdev=10631.92 lat (usec): min=2271, max=77561, avg=13511.16, stdev=10631.81 clat percentiles (usec): | 1.00th=[ 3228], 5.00th=[ 3982], 10.00th=[ 4490], 20.00th=[ 5145], | 30.00th=[ 6521], 40.00th=[ 8225], 50.00th=[ 9503], 60.00th=[11600], | 70.00th=[15533], 80.00th=[20841], 90.00th=[28443], 95.00th=[34866], | 99.00th=[52167], 99.50th=[57934], 99.90th=[65274], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 31, max= 424, per=25.39%, avg=128.45, stdev=80.09, samples=93 iops : min= 7, max= 106, avg=31.74, stdev=20.17, samples=93 write: IOPS=32, BW=132KiB/s (135kB/s)(6236KiB/47348msec); 0 zone resets clat (msec): min=3, max=636, avg=17.16, stdev=19.43 lat (msec): min=3, max=636, avg=17.16, stdev=19.43 clat percentiles (msec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 14], 60.00th=[ 17], | 70.00th=[ 22], 80.00th=[ 27], 90.00th=[ 33], 95.00th=[ 39], | 99.00th=[ 57], 99.50th=[ 63], 99.90th=[ 85], 99.95th=[ 634], | 99.99th=[ 634] bw ( KiB/s): min= 48, max= 424, per=25.02%, avg=132.58, stdev=81.77, samples=93 iops : min= 12, max= 106, avg=32.77, stdev=20.56, samples=93 lat (msec) : 4=2.60%, 10=42.12%, 20=28.39%, 50=25.52%, 100=1.33% lat (msec) : 750=0.03% cpu : usr=0.10%, sys=12.67%, ctx=5337, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1513,1559,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=507KiB/s (519kB/s), 126KiB/s-128KiB/s (129kB/s-131kB/s), io=23.4MiB (24.6MB), run=47060-47402msec WRITE: bw=530KiB/s (543kB/s), 132KiB/s-134KiB/s (135kB/s-138kB/s), io=24.6MiB (25.7MB), run=47060-47402msec mix direct rw 16384 by fio with 4 jobs... mix buffer rw 16384 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=191589: Thu Sep 25 14:49:55 2025 read: IOPS=28, BW=455KiB/s (466kB/s)(5920KiB/13012msec) clat (usec): min=154, max=71610, avg=18770.93, stdev=10883.15 lat (usec): min=155, max=71610, avg=18771.45, stdev=10883.22 clat percentiles (usec): | 1.00th=[ 306], 5.00th=[ 6783], 10.00th=[ 8586], 20.00th=[11207], | 30.00th=[12518], 40.00th=[14091], 50.00th=[16188], 60.00th=[19006], | 70.00th=[21103], 80.00th=[25297], 90.00th=[32637], 95.00th=[40633], | 99.00th=[61604], 99.50th=[66323], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min= 224, max= 672, per=24.73%, avg=447.92, stdev=135.49, samples=25 iops : min= 14, max= 42, avg=27.40, stdev= 8.54, samples=25 write: IOPS=30, BW=489KiB/s (501kB/s)(6368KiB/13012msec); 0 zone resets clat (usec): min=982, max=77086, avg=15186.68, stdev=10016.20 lat (usec): min=984, max=77088, avg=15188.58, stdev=10015.42 clat percentiles (usec): | 1.00th=[ 1549], 5.00th=[ 3621], 10.00th=[ 4228], 20.00th=[ 5473], | 30.00th=[ 8717], 40.00th=[10945], 50.00th=[13042], 60.00th=[15664], | 70.00th=[19006], 80.00th=[23200], 90.00th=[29754], 95.00th=[33817], | 99.00th=[40633], 99.50th=[45876], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 160, max= 764, per=25.55%, avg=486.96, stdev=163.76, samples=25 iops : min= 10, max= 47, avg=29.88, stdev=10.24, samples=25 lat (usec) : 250=0.26%, 500=1.04%, 750=0.26%, 1000=0.13% lat (msec) : 2=0.39%, 4=3.65%, 10=20.18%, 20=43.36%, 50=29.95% lat (msec) : 100=0.78% cpu : usr=0.08%, sys=28.61%, ctx=2927, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191590: Thu Sep 25 14:49:55 2025 read: IOPS=27, BW=435KiB/s (445kB/s)(5744KiB/13219msec) clat (usec): min=207, max=83133, avg=19013.50, stdev=9895.07 lat (usec): min=207, max=83134, avg=19014.05, stdev=9895.12 clat percentiles (usec): | 1.00th=[ 306], 5.00th=[ 8356], 10.00th=[10028], 20.00th=[11731], | 30.00th=[13304], 40.00th=[15401], 50.00th=[17433], 60.00th=[19268], | 70.00th=[21627], 80.00th=[25035], 90.00th=[31327], 95.00th=[38536], | 99.00th=[54264], 99.50th=[62129], 99.90th=[83362], 99.95th=[83362], | 99.99th=[83362] bw ( KiB/s): min= 223, max= 608, per=23.94%, avg=433.58, stdev=114.19, samples=26 iops : min= 13, max= 38, avg=26.50, stdev= 7.21, samples=26 write: IOPS=30, BW=495KiB/s (507kB/s)(6544KiB/13219msec); 0 zone resets clat (usec): min=915, max=68310, avg=15450.27, stdev=10874.63 lat (usec): min=916, max=68313, avg=15451.27, stdev=10874.77 clat percentiles (usec): | 1.00th=[ 3130], 5.00th=[ 3720], 10.00th=[ 4490], 20.00th=[ 5735], | 30.00th=[ 8225], 40.00th=[10290], 50.00th=[12387], 60.00th=[15401], | 70.00th=[19530], 80.00th=[23725], 90.00th=[28705], 95.00th=[35390], | 99.00th=[54789], 99.50th=[56886], 99.90th=[68682], 99.95th=[68682], | 99.99th=[68682] bw ( KiB/s): min= 224, max= 766, per=25.93%, avg=494.19, stdev=144.37, samples=26 iops : min= 14, max= 47, avg=30.31, stdev= 8.92, samples=26 lat (usec) : 250=0.26%, 500=0.91%, 1000=0.13% lat (msec) : 2=0.13%, 4=3.26%, 10=20.70%, 20=41.93%, 50=31.51% lat (msec) : 100=1.17% cpu : usr=0.11%, sys=29.32%, ctx=2894, majf=0, minf=16 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191591: Thu Sep 25 14:49:55 2025 read: IOPS=30, BW=484KiB/s (496kB/s)(6256KiB/12927msec) clat (usec): min=110, max=89730, avg=17459.63, stdev=10339.60 lat (usec): min=111, max=89733, avg=17460.30, stdev=10339.68 clat percentiles (usec): | 1.00th=[ 163], 5.00th=[ 5407], 10.00th=[ 8225], 20.00th=[10552], | 30.00th=[11863], 40.00th=[13698], 50.00th=[16057], 60.00th=[17433], | 70.00th=[19792], 80.00th=[22676], 90.00th=[29492], 95.00th=[35390], | 99.00th=[57934], 99.50th=[62129], 99.90th=[89654], 99.95th=[89654], | 99.99th=[89654] bw ( KiB/s): min= 223, max= 734, per=26.86%, avg=486.44, stdev=146.98, samples=25 iops : min= 13, max= 45, avg=29.84, stdev= 9.27, samples=25 write: IOPS=29, BW=467KiB/s (478kB/s)(6032KiB/12927msec); 0 zone resets clat (usec): min=1018, max=76723, avg=16076.77, stdev=10675.24 lat (usec): min=1018, max=76724, avg=16077.92, stdev=10675.35 clat percentiles (usec): | 1.00th=[ 3294], 5.00th=[ 4047], 10.00th=[ 4948], 20.00th=[ 6980], | 30.00th=[ 9241], 40.00th=[11338], 50.00th=[13566], 60.00th=[15926], | 70.00th=[20055], 80.00th=[23725], 90.00th=[29754], 95.00th=[37487], | 99.00th=[53216], 99.50th=[64226], 99.90th=[77071], 99.95th=[77071], | 99.99th=[77071] bw ( KiB/s): min= 223, max= 701, per=24.14%, avg=460.16, stdev=137.21, samples=25 iops : min= 13, max= 43, avg=28.20, stdev= 8.51, samples=25 lat (usec) : 250=0.65%, 500=1.43%, 750=0.13% lat (msec) : 2=0.39%, 4=1.82%, 10=20.83%, 20=45.57%, 50=27.60% lat (msec) : 100=1.56% cpu : usr=0.08%, sys=29.08%, ctx=2877, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191592: Thu Sep 25 14:49:55 2025 read: IOPS=28, BW=458KiB/s (469kB/s)(6032KiB/13161msec) clat (usec): min=105, max=86061, avg=17209.23, stdev=10421.77 lat (usec): min=106, max=86062, avg=17209.76, stdev=10421.81 clat percentiles (usec): | 1.00th=[ 297], 5.00th=[ 445], 10.00th=[ 8029], 20.00th=[10290], | 30.00th=[11863], 40.00th=[13435], 50.00th=[15008], 60.00th=[17433], | 70.00th=[20317], 80.00th=[23462], 90.00th=[29754], 95.00th=[34866], | 99.00th=[51643], 99.50th=[70779], 99.90th=[86508], 99.95th=[86508], | 99.99th=[86508] bw ( KiB/s): min= 192, max= 672, per=25.30%, avg=458.15, stdev=132.17, samples=26 iops : min= 12, max= 42, avg=28.04, stdev= 8.33, samples=26 write: IOPS=29, BW=475KiB/s (487kB/s)(6256KiB/13161msec); 0 zone resets clat (usec): min=1159, max=66989, avg=16967.86, stdev=12249.90 lat (usec): min=1160, max=66991, avg=16968.86, stdev=12249.97 clat percentiles (usec): | 1.00th=[ 1876], 5.00th=[ 3916], 10.00th=[ 4490], 20.00th=[ 6456], | 30.00th=[ 9241], 40.00th=[10945], 50.00th=[13304], 60.00th=[16319], | 70.00th=[21103], 80.00th=[26084], 90.00th=[33424], 95.00th=[41681], | 99.00th=[56886], 99.50th=[65799], 99.90th=[66847], 99.95th=[66847], | 99.99th=[66847] bw ( KiB/s): min= 223, max= 606, per=24.94%, avg=475.27, stdev=129.17, samples=26 iops : min= 13, max= 37, avg=29.12, stdev= 8.05, samples=26 lat (usec) : 250=0.13%, 500=2.60%, 750=0.26%, 1000=0.26% lat (msec) : 2=0.52%, 4=2.73%, 10=19.66%, 20=42.58%, 50=29.56% lat (msec) : 100=1.69% cpu : usr=0.08%, sys=28.11%, ctx=2890, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1812KiB/s (1855kB/s), 435KiB/s-484KiB/s (445kB/s-496kB/s), io=23.4MiB (24.5MB), run=12927-13219msec WRITE: bw=1906KiB/s (1952kB/s), 467KiB/s-495KiB/s (478kB/s-507kB/s), io=24.6MiB (25.8MB), run=12927-13219msec rand-rw: (groupid=0, jobs=1): err= 0: pid=191594: Thu Sep 25 14:49:57 2025 read: IOPS=25, BW=404KiB/s (414kB/s)(5920KiB/14653msec) clat (usec): min=2596, max=98908, avg=17630.22, stdev=13718.75 lat (usec): min=2596, max=98909, avg=17630.78, stdev=13718.78 clat percentiles (usec): | 1.00th=[ 4555], 5.00th=[ 5276], 10.00th=[ 5932], 20.00th=[ 7767], | 30.00th=[ 9372], 40.00th=[10814], 50.00th=[12911], 60.00th=[15401], | 70.00th=[19792], 80.00th=[25560], 90.00th=[35390], 95.00th=[43254], | 99.00th=[72877], 99.50th=[86508], 99.90th=[99091], 99.95th=[99091], | 99.99th=[99091] bw ( KiB/s): min= 128, max= 1184, per=23.76%, avg=384.86, stdev=199.20, samples=28 iops : min= 8, max= 74, avg=23.61, stdev=12.54, samples=28 write: IOPS=27, BW=435KiB/s (445kB/s)(6368KiB/14653msec); 0 zone resets clat (usec): min=4595, max=92045, avg=20202.55, stdev=13363.84 lat (usec): min=4596, max=92047, avg=20204.69, stdev=13363.88 clat percentiles (usec): | 1.00th=[ 5276], 5.00th=[ 5866], 10.00th=[ 6783], 20.00th=[ 9241], | 30.00th=[11469], 40.00th=[13173], 50.00th=[16319], 60.00th=[20579], | 70.00th=[24511], 80.00th=[29754], 90.00th=[37487], 95.00th=[44827], | 99.00th=[67634], 99.50th=[85459], 99.90th=[91751], 99.95th=[91751], | 99.99th=[91751] bw ( KiB/s): min= 128, max= 1056, per=24.46%, avg=417.07, stdev=200.69, samples=28 iops : min= 8, max= 66, avg=25.64, stdev=12.66, samples=28 lat (msec) : 4=0.26%, 10=28.91%, 20=34.77%, 50=32.94%, 100=3.12% cpu : usr=0.10%, sys=13.39%, ctx=1472, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=370,398,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191595: Thu Sep 25 14:49:57 2025 read: IOPS=24, BW=392KiB/s (401kB/s)(5744KiB/14659msec) clat (usec): min=3856, max=81581, avg=17252.25, stdev=12287.44 lat (usec): min=3856, max=81582, avg=17252.81, stdev=12287.46 clat percentiles (usec): | 1.00th=[ 4146], 5.00th=[ 4883], 10.00th=[ 5735], 20.00th=[ 8029], | 30.00th=[ 9372], 40.00th=[10683], 50.00th=[12649], 60.00th=[15795], | 70.00th=[19792], 80.00th=[26608], 90.00th=[35914], 95.00th=[42730], | 99.00th=[52691], 99.50th=[74974], 99.90th=[81265], 99.95th=[81265], | 99.99th=[81265] bw ( KiB/s): min= 127, max= 1088, per=23.52%, avg=381.00, stdev=187.04, samples=28 iops : min= 7, max= 68, avg=23.36, stdev=11.80, samples=28 write: IOPS=27, BW=446KiB/s (457kB/s)(6544KiB/14659msec); 0 zone resets clat (usec): min=4678, max=89030, avg=20446.87, stdev=13520.40 lat (usec): min=4679, max=89031, avg=20448.58, stdev=13520.14 clat percentiles (usec): | 1.00th=[ 5342], 5.00th=[ 5997], 10.00th=[ 6718], 20.00th=[ 8717], | 30.00th=[10421], 40.00th=[13173], 50.00th=[16319], 60.00th=[20579], | 70.00th=[25297], 80.00th=[31327], 90.00th=[39060], 95.00th=[46400], | 99.00th=[58983], 99.50th=[65799], 99.90th=[88605], 99.95th=[88605], | 99.99th=[88605] bw ( KiB/s): min= 222, max= 1216, per=25.07%, avg=427.39, stdev=229.27, samples=28 iops : min= 13, max= 76, avg=26.29, stdev=14.46, samples=28 lat (msec) : 4=0.39%, 10=29.69%, 20=33.98%, 50=33.85%, 100=2.08% cpu : usr=0.08%, sys=13.34%, ctx=1496, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=359,409,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191596: Thu Sep 25 14:49:57 2025 read: IOPS=26, BW=425KiB/s (435kB/s)(6256KiB/14730msec) clat (usec): min=3227, max=73756, avg=17147.38, stdev=12915.08 lat (usec): min=3228, max=73757, avg=17148.04, stdev=12915.14 clat percentiles (usec): | 1.00th=[ 3949], 5.00th=[ 5014], 10.00th=[ 5800], 20.00th=[ 7308], | 30.00th=[ 9110], 40.00th=[10421], 50.00th=[12911], 60.00th=[15270], | 70.00th=[19530], 80.00th=[25560], 90.00th=[35390], 95.00th=[43779], | 99.00th=[65274], 99.50th=[73925], 99.90th=[73925], 99.95th=[73925], | 99.99th=[73925] bw ( KiB/s): min= 159, max= 1376, per=26.15%, avg=423.62, stdev=251.99, samples=29 iops : min= 9, max= 86, avg=26.07, stdev=15.86, samples=29 write: IOPS=25, BW=410KiB/s (419kB/s)(6032KiB/14730msec); 0 zone resets clat (usec): min=3062, max=84128, avg=21211.28, stdev=13377.92 lat (usec): min=3062, max=84129, avg=21213.07, stdev=13378.07 clat percentiles (usec): | 1.00th=[ 4293], 5.00th=[ 6194], 10.00th=[ 7177], 20.00th=[ 9372], | 30.00th=[11863], 40.00th=[14484], 50.00th=[17433], 60.00th=[21365], | 70.00th=[26608], 80.00th=[32375], 90.00th=[40109], 95.00th=[49021], | 99.00th=[58983], 99.50th=[65274], 99.90th=[84411], 99.95th=[84411], | 99.99th=[84411] bw ( KiB/s): min= 159, max= 864, per=23.54%, avg=401.41, stdev=174.88, samples=29 iops : min= 9, max= 54, avg=24.66, stdev=11.06, samples=29 lat (msec) : 4=0.78%, 10=29.30%, 20=33.72%, 50=32.55%, 100=3.65% cpu : usr=0.12%, sys=13.24%, ctx=1465, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=391,377,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191597: Thu Sep 25 14:49:57 2025 read: IOPS=25, BW=408KiB/s (418kB/s)(6032KiB/14778msec) clat (msec): min=2, max=102, avg=16.67, stdev=13.10 lat (msec): min=2, max=102, avg=16.67, stdev=13.10 clat percentiles (msec): | 1.00th=[ 3], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7], | 30.00th=[ 9], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 15], | 70.00th=[ 19], 80.00th=[ 26], 90.00th=[ 34], 95.00th=[ 41], | 99.00th=[ 73], 99.50th=[ 83], 99.90th=[ 103], 99.95th=[ 103], | 99.99th=[ 103] bw ( KiB/s): min= 127, max= 1120, per=24.38%, avg=394.97, stdev=225.64, samples=29 iops : min= 7, max= 70, avg=24.28, stdev=14.24, samples=29 write: IOPS=26, BW=423KiB/s (433kB/s)(6256KiB/14778msec); 0 zone resets clat (usec): min=2810, max=71304, avg=21510.27, stdev=14227.05 lat (usec): min=2811, max=71305, avg=21511.69, stdev=14227.15 clat percentiles (usec): | 1.00th=[ 3720], 5.00th=[ 5866], 10.00th=[ 7046], 20.00th=[ 9503], | 30.00th=[11469], 40.00th=[13829], 50.00th=[17171], 60.00th=[21627], | 70.00th=[27132], 80.00th=[33162], 90.00th=[40109], 95.00th=[51119], | 99.00th=[63701], 99.50th=[70779], 99.90th=[71828], 99.95th=[71828], | 99.99th=[71828] bw ( KiB/s): min= 190, max= 1024, per=24.31%, avg=414.45, stdev=189.01, samples=29 iops : min= 11, max= 64, avg=25.48, stdev=11.98, samples=29 lat (msec) : 4=2.47%, 10=27.34%, 20=33.72%, 50=32.81%, 100=3.52% lat (msec) : 250=0.13% cpu : usr=0.12%, sys=13.01%, ctx=1470, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=377,391,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1621KiB/s (1660kB/s), 392KiB/s-425KiB/s (401kB/s-435kB/s), io=23.4MiB (24.5MB), run=14653-14778msec WRITE: bw=1705KiB/s (1746kB/s), 410KiB/s-446KiB/s (419kB/s-457kB/s), io=24.6MiB (25.8MB), run=14653-14778msec mix direct rw 1048576 by fio with 4 jobs... mix buffer rw 1048576 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=191604: Thu Sep 25 14:50:05 2025 read: IOPS=3, BW=3831KiB/s (3923kB/s)(7168KiB/1871msec) clat (msec): min=19, max=284, avg=140.47, stdev=88.13 lat (msec): min=19, max=284, avg=140.47, stdev=88.13 clat percentiles (msec): | 1.00th=[ 20], 5.00th=[ 20], 10.00th=[ 20], 20.00th=[ 47], | 30.00th=[ 136], 40.00th=[ 136], 50.00th=[ 155], 60.00th=[ 169], | 70.00th=[ 169], 80.00th=[ 176], 90.00th=[ 284], 95.00th=[ 284], | 99.00th=[ 284], 99.50th=[ 284], 99.90th=[ 284], 99.95th=[ 284], | 99.99th=[ 284] bw ( KiB/s): min= 2000, max= 8325, per=69.53%, avg=5162.50, stdev=4472.45, samples=2 iops : min= 1, max= 8, avg= 4.50, stdev= 4.95, samples=2 write: IOPS=2, BW=2737KiB/s (2802kB/s)(5120KiB/1871msec); 0 zone resets clat (msec): min=73, max=328, avg=175.90, stdev=117.00 lat (msec): min=73, max=328, avg=175.99, stdev=117.00 clat percentiles (msec): | 1.00th=[ 74], 5.00th=[ 74], 10.00th=[ 74], 20.00th=[ 74], | 30.00th=[ 79], 40.00th=[ 79], 50.00th=[ 128], 60.00th=[ 128], | 70.00th=[ 271], 80.00th=[ 271], 90.00th=[ 330], 95.00th=[ 330], | 99.00th=[ 330], 99.50th=[ 330], 99.90th=[ 330], 99.95th=[ 330], | 99.99th=[ 330] bw ( KiB/s): min= 1665, max= 4000, per=24.72%, avg=2569.33, stdev=1253.33, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (msec) : 20=8.33%, 50=8.33%, 100=16.67%, 250=41.67%, 500=25.00% cpu : usr=0.11%, sys=18.45%, ctx=94, majf=0, minf=14 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191605: Thu Sep 25 14:50:05 2025 read: IOPS=0, BW=810KiB/s (829kB/s)(2048KiB/2529msec) clat (msec): min=261, max=428, avg=345.08, stdev=117.65 lat (msec): min=261, max=428, avg=345.08, stdev=117.65 clat percentiles (msec): | 1.00th=[ 262], 5.00th=[ 262], 10.00th=[ 262], 20.00th=[ 262], | 30.00th=[ 262], 40.00th=[ 262], 50.00th=[ 262], 60.00th=[ 430], | 70.00th=[ 430], 80.00th=[ 430], 90.00th=[ 430], 95.00th=[ 430], | 99.00th=[ 430], 99.50th=[ 430], 99.90th=[ 430], 99.95th=[ 430], | 99.99th=[ 430] bw ( KiB/s): min= 1665, max= 1965, per=24.44%, avg=1815.00, stdev=212.13, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=4049KiB/s (4146kB/s)(10.0MiB/2529msec); 0 zone resets clat (msec): min=55, max=353, avg=179.34, stdev=97.90 lat (msec): min=55, max=354, avg=179.44, stdev=97.89 clat percentiles (msec): | 1.00th=[ 56], 5.00th=[ 56], 10.00th=[ 56], 20.00th=[ 64], | 30.00th=[ 115], 40.00th=[ 140], 50.00th=[ 148], 60.00th=[ 155], | 70.00th=[ 222], 80.00th=[ 259], 90.00th=[ 288], 95.00th=[ 355], | 99.00th=[ 355], 99.50th=[ 355], 99.90th=[ 355], 99.95th=[ 355], | 99.99th=[ 355] bw ( KiB/s): min= 1665, max= 6131, per=33.01%, avg=3431.50, stdev=2057.95, samples=4 iops : min= 1, max= 5, avg= 2.50, stdev= 1.91, samples=4 lat (msec) : 100=16.67%, 250=41.67%, 500=41.67% cpu : usr=0.04%, sys=18.67%, ctx=125, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191606: Thu Sep 25 14:50:05 2025 read: IOPS=2, BW=2228KiB/s (2281kB/s)(6144KiB/2758msec) clat (msec): min=14, max=392, avg=180.86, stdev=135.86 lat (msec): min=14, max=392, avg=180.86, stdev=135.86 clat percentiles (msec): | 1.00th=[ 16], 5.00th=[ 16], 10.00th=[ 16], 20.00th=[ 71], | 30.00th=[ 71], 40.00th=[ 153], 50.00th=[ 153], 60.00th=[ 192], | 70.00th=[ 264], 80.00th=[ 264], 90.00th=[ 393], 95.00th=[ 393], | 99.00th=[ 393], 99.50th=[ 393], 99.90th=[ 393], 99.95th=[ 393], | 99.99th=[ 393] bw ( KiB/s): min= 2000, max= 3930, per=35.82%, avg=2659.33, stdev=1100.69, samples=3 iops : min= 1, max= 3, avg= 2.00, stdev= 1.00, samples=3 write: IOPS=2, BW=2228KiB/s (2281kB/s)(6144KiB/2758msec); 0 zone resets clat (msec): min=41, max=468, avg=264.39, stdev=146.31 lat (msec): min=41, max=468, avg=264.47, stdev=146.32 clat percentiles (msec): | 1.00th=[ 42], 5.00th=[ 42], 10.00th=[ 42], 20.00th=[ 199], | 30.00th=[ 199], 40.00th=[ 245], 50.00th=[ 245], 60.00th=[ 264], | 70.00th=[ 372], 80.00th=[ 372], 90.00th=[ 468], 95.00th=[ 468], | 99.00th=[ 468], 99.50th=[ 468], 99.90th=[ 468], 99.95th=[ 468], | 99.99th=[ 468] bw ( KiB/s): min= 1659, max= 2048, per=18.69%, avg=1943.00, stdev=162.34, samples=5 iops : min= 1, max= 2, avg= 1.20, stdev= 0.45, samples=5 lat (msec) : 20=8.33%, 50=8.33%, 100=8.33%, 250=33.33%, 500=41.67% cpu : usr=0.00%, sys=17.41%, ctx=159, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191608: Thu Sep 25 14:50:05 2025 read: IOPS=2, BW=2329KiB/s (2385kB/s)(5120KiB/2198msec) clat (msec): min=58, max=371, avg=147.65, stdev=131.01 lat (msec): min=58, max=371, avg=147.65, stdev=131.01 clat percentiles (msec): | 1.00th=[ 59], 5.00th=[ 59], 10.00th=[ 59], 20.00th=[ 59], | 30.00th=[ 67], 40.00th=[ 67], 50.00th=[ 85], 60.00th=[ 85], | 70.00th=[ 159], 80.00th=[ 159], 90.00th=[ 372], 95.00th=[ 372], | 99.00th=[ 372], 99.50th=[ 372], 99.90th=[ 372], 99.95th=[ 372], | 99.99th=[ 372] bw ( KiB/s): min= 2043, max= 6000, per=54.16%, avg=4021.50, stdev=2798.02, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 write: IOPS=3, BW=3261KiB/s (3339kB/s)(7168KiB/2198msec); 0 zone resets clat (msec): min=64, max=280, avg=203.13, stdev=81.97 lat (msec): min=64, max=281, avg=203.22, stdev=81.96 clat percentiles (msec): | 1.00th=[ 65], 5.00th=[ 65], 10.00th=[ 65], 20.00th=[ 136], | 30.00th=[ 180], 40.00th=[ 180], 50.00th=[ 220], 60.00th=[ 266], | 70.00th=[ 266], 80.00th=[ 279], 90.00th=[ 279], 95.00th=[ 279], | 99.00th=[ 279], 99.50th=[ 279], 99.90th=[ 279], 99.95th=[ 279], | 99.99th=[ 279] bw ( KiB/s): min= 3319, max= 5896, per=42.66%, avg=4434.00, stdev=1323.08, samples=3 iops : min= 3, max= 5, avg= 3.67, stdev= 1.15, samples=3 lat (msec) : 100=33.33%, 250=33.33%, 500=33.33% cpu : usr=0.00%, sys=24.81%, ctx=101, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7426KiB/s (7604kB/s), 810KiB/s-3831KiB/s (829kB/s-3923kB/s), io=20.0MiB (20.0MB), run=1871-2758msec WRITE: bw=10.2MiB/s (10.6MB/s), 2228KiB/s-4049KiB/s (2281kB/s-4146kB/s), io=28.0MiB (29.4MB), run=1871-2758msec rand-rw: (groupid=0, jobs=1): err= 0: pid=191609: Thu Sep 25 14:50:05 2025 read: IOPS=3, BW=3771KiB/s (3861kB/s)(7168KiB/1901msec) clat (usec): min=826, max=199359, avg=118707.87, stdev=67459.66 lat (usec): min=827, max=199360, avg=118709.00, stdev=67459.81 clat percentiles (usec): | 1.00th=[ 824], 5.00th=[ 824], 10.00th=[ 824], 20.00th=[ 86508], | 30.00th=[ 87557], 40.00th=[ 87557], 50.00th=[124257], 60.00th=[152044], | 70.00th=[152044], 80.00th=[179307], 90.00th=[200279], 95.00th=[200279], | 99.00th=[200279], 99.50th=[200279], 99.90th=[200279], 99.95th=[200279], | 99.99th=[200279] bw ( KiB/s): min= 3968, max= 4938, per=57.86%, avg=4453.00, stdev=685.89, samples=2 iops : min= 3, max= 4, avg= 3.50, stdev= 0.71, samples=2 write: IOPS=2, BW=2693KiB/s (2758kB/s)(5120KiB/1901msec); 0 zone resets clat (msec): min=135, max=348, avg=205.61, stdev=82.88 lat (msec): min=135, max=349, avg=205.66, stdev=82.86 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 136], | 30.00th=[ 176], 40.00th=[ 176], 50.00th=[ 178], 60.00th=[ 178], | 70.00th=[ 192], 80.00th=[ 192], 90.00th=[ 351], 95.00th=[ 351], | 99.00th=[ 351], 99.50th=[ 351], 99.90th=[ 351], 99.95th=[ 351], | 99.99th=[ 351] bw ( KiB/s): min= 1646, max= 5919, per=35.11%, avg=3782.50, stdev=3021.47, samples=2 iops : min= 1, max= 5, avg= 3.00, stdev= 2.83, samples=2 lat (usec) : 1000=8.33% lat (msec) : 100=16.67%, 250=66.67%, 500=8.33% cpu : usr=0.00%, sys=58.53%, ctx=855, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=7,5,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191610: Thu Sep 25 14:50:05 2025 read: IOPS=0, BW=770KiB/s (788kB/s)(2048KiB/2661msec) clat (msec): min=26, max=281, avg=154.32, stdev=180.34 lat (msec): min=26, max=281, avg=154.32, stdev=180.34 clat percentiles (msec): | 1.00th=[ 27], 5.00th=[ 27], 10.00th=[ 27], 20.00th=[ 27], | 30.00th=[ 27], 40.00th=[ 27], 50.00th=[ 27], 60.00th=[ 284], | 70.00th=[ 284], 80.00th=[ 284], 90.00th=[ 284], 95.00th=[ 284], | 99.00th=[ 284], 99.50th=[ 284], 99.90th=[ 284], 99.95th=[ 284], | 99.99th=[ 284] bw ( KiB/s): min= 1643, max= 2019, per=23.79%, avg=1831.00, stdev=265.87, samples=2 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=2 write: IOPS=3, BW=3848KiB/s (3941kB/s)(10.0MiB/2661msec); 0 zone resets clat (msec): min=103, max=445, avg=230.32, stdev=144.94 lat (msec): min=103, max=445, avg=230.40, stdev=144.94 clat percentiles (msec): | 1.00th=[ 104], 5.00th=[ 104], 10.00th=[ 104], 20.00th=[ 105], | 30.00th=[ 105], 40.00th=[ 109], 50.00th=[ 180], 60.00th=[ 184], | 70.00th=[ 228], 80.00th=[ 418], 90.00th=[ 430], 95.00th=[ 447], | 99.00th=[ 447], 99.50th=[ 447], 99.90th=[ 447], 99.95th=[ 447], | 99.99th=[ 447] bw ( KiB/s): min= 2019, max= 3968, per=30.73%, avg=3311.00, stdev=1118.96, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 lat (msec) : 50=8.33%, 250=58.33%, 500=33.33% cpu : usr=0.04%, sys=48.20%, ctx=257, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2,10,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191611: Thu Sep 25 14:50:05 2025 read: IOPS=3, BW=3227KiB/s (3304kB/s)(6144KiB/1904msec) clat (usec): min=547, max=150327, avg=58536.64, stdev=71812.55 lat (usec): min=547, max=150327, avg=58537.27, stdev=71812.69 clat percentiles (usec): | 1.00th=[ 545], 5.00th=[ 545], 10.00th=[ 545], 20.00th=[ 701], | 30.00th=[ 701], 40.00th=[ 898], 50.00th=[ 898], 60.00th=[ 54789], | 70.00th=[143655], 80.00th=[143655], 90.00th=[149947], 95.00th=[149947], | 99.00th=[149947], 99.50th=[149947], 99.90th=[149947], 99.95th=[149947], | 99.99th=[149947] bw ( KiB/s): min= 3968, max= 5919, per=64.23%, avg=4943.50, stdev=1379.57, samples=2 iops : min= 3, max= 5, avg= 4.00, stdev= 1.41, samples=2 write: IOPS=3, BW=3227KiB/s (3304kB/s)(6144KiB/1904msec); 0 zone resets clat (msec): min=116, max=398, avg=256.84, stdev=118.99 lat (msec): min=116, max=398, avg=256.89, stdev=118.98 clat percentiles (msec): | 1.00th=[ 116], 5.00th=[ 116], 10.00th=[ 116], 20.00th=[ 155], | 30.00th=[ 155], 40.00th=[ 222], 50.00th=[ 222], 60.00th=[ 257], | 70.00th=[ 397], 80.00th=[ 397], 90.00th=[ 397], 95.00th=[ 397], | 99.00th=[ 397], 99.50th=[ 397], 99.90th=[ 397], 99.95th=[ 397], | 99.99th=[ 397] bw ( KiB/s): min= 1984, max= 3946, per=28.48%, avg=3068.67, stdev=997.30, samples=3 iops : min= 1, max= 3, avg= 2.33, stdev= 1.15, samples=3 lat (usec) : 750=16.67%, 1000=8.33% lat (msec) : 100=8.33%, 250=41.67%, 500=25.00% cpu : usr=0.00%, sys=47.92%, ctx=222, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=6,6,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191612: Thu Sep 25 14:50:05 2025 read: IOPS=1, BW=1994KiB/s (2042kB/s)(5120KiB/2568msec) clat (usec): min=846, max=321818, avg=131573.31, stdev=118679.53 lat (usec): min=846, max=321819, avg=131574.34, stdev=118679.62 clat percentiles (usec): | 1.00th=[ 848], 5.00th=[ 848], 10.00th=[ 848], 20.00th=[ 848], | 30.00th=[ 84411], 40.00th=[ 84411], 50.00th=[106431], 60.00th=[106431], | 70.00th=[143655], 80.00th=[143655], 90.00th=[320865], 95.00th=[320865], | 99.00th=[320865], 99.50th=[320865], 99.90th=[320865], 99.95th=[320865], | 99.99th=[320865] bw ( KiB/s): min= 8078, max= 8078, per=100.00%, avg=8078.00, stdev= 0.00, samples=1 iops : min= 7, max= 7, avg= 7.00, stdev= 0.00, samples=1 write: IOPS=2, BW=2791KiB/s (2858kB/s)(7168KiB/2568msec); 0 zone resets clat (msec): min=134, max=401, avg=263.74, stdev=118.19 lat (msec): min=134, max=401, avg=263.82, stdev=118.20 clat percentiles (msec): | 1.00th=[ 136], 5.00th=[ 136], 10.00th=[ 136], 20.00th=[ 148], | 30.00th=[ 161], 40.00th=[ 161], 50.00th=[ 257], 60.00th=[ 363], | 70.00th=[ 363], 80.00th=[ 384], 90.00th=[ 401], 95.00th=[ 401], | 99.00th=[ 401], 99.50th=[ 401], 99.90th=[ 401], 99.95th=[ 401], | 99.99th=[ 401] bw ( KiB/s): min= 1656, max= 3968, per=23.50%, avg=2532.33, stdev=1253.39, samples=3 iops : min= 1, max= 3, avg= 1.67, stdev= 1.15, samples=3 lat (usec) : 1000=8.33% lat (msec) : 100=8.33%, 250=41.67%, 500=41.67% cpu : usr=0.00%, sys=42.89%, ctx=665, majf=0, minf=12 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=5,7,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=7696KiB/s (7881kB/s), 770KiB/s-3771KiB/s (788kB/s-3861kB/s), io=20.0MiB (20.0MB), run=1901-2661msec WRITE: bw=10.5MiB/s (11.0MB/s), 2693KiB/s-3848KiB/s (2758kB/s-3941kB/s), io=28.0MiB (29.4MB), run=1901-2661msec mix direct rw 4194304 by fio with 4 jobs... mix buffer rw 4194304 by fio with 4 jobs... rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (g=0): rw=randrw, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=16 ... fio-3.19 Starting 4 processes rand-rw: (groupid=0, jobs=1): err= 0: pid=191617: Thu Sep 25 14:50:12 2025 read: IOPS=2, BW=9373KiB/s (9598kB/s)(12.0MiB/1311msec) clat (msec): min=364, max=431, avg=399.14, stdev=33.46 lat (msec): min=364, max=431, avg=399.14, stdev=33.46 clat percentiles (msec): | 1.00th=[ 363], 5.00th=[ 363], 10.00th=[ 363], 20.00th=[ 363], | 30.00th=[ 363], 40.00th=[ 401], 50.00th=[ 401], 60.00th=[ 401], | 70.00th=[ 430], 80.00th=[ 430], 90.00th=[ 430], 95.00th=[ 430], | 99.00th=[ 430], 99.50th=[ 430], 99.90th=[ 430], 99.95th=[ 430], | 99.99th=[ 430] bw ( KiB/s): min= 7474, max= 8192, per=63.40%, avg=7833.00, stdev=507.70, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=100.00% cpu : usr=0.00%, sys=63.28%, ctx=2174, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191618: Thu Sep 25 14:50:12 2025 read: IOPS=0, BW=3089KiB/s (3163kB/s)(4096KiB/1326msec) clat (nsec): min=422334k, max=422334k, avg=422334311.00, stdev= 0.00 lat (nsec): min=422338k, max=422338k, avg=422337776.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 422], 5.00th=[ 422], 10.00th=[ 422], 20.00th=[ 422], | 30.00th=[ 422], 40.00th=[ 422], 50.00th=[ 422], 60.00th=[ 422], | 70.00th=[ 422], 80.00th=[ 422], 90.00th=[ 422], 95.00th=[ 422], | 99.00th=[ 422], 99.50th=[ 422], 99.90th=[ 422], 99.95th=[ 422], | 99.99th=[ 422] bw ( KiB/s): min= 7447, max= 7447, per=60.28%, avg=7447.00, stdev= 0.00, samples=1 iops : min= 1, max= 1, avg= 1.00, stdev= 0.00, samples=1 write: IOPS=1, BW=6178KiB/s (6326kB/s)(8192KiB/1326msec); 0 zone resets clat (msec): min=228, max=616, avg=422.39, stdev=274.21 lat (msec): min=229, max=616, avg=422.76, stdev=273.89 clat percentiles (msec): | 1.00th=[ 228], 5.00th=[ 228], 10.00th=[ 228], 20.00th=[ 228], | 30.00th=[ 228], 40.00th=[ 228], 50.00th=[ 228], 60.00th=[ 617], | 70.00th=[ 617], 80.00th=[ 617], 90.00th=[ 617], 95.00th=[ 617], | 99.00th=[ 617], 99.50th=[ 617], 99.90th=[ 617], 99.95th=[ 617], | 99.99th=[ 617] bw ( KiB/s): min= 8192, max= 8192, per=40.75%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 750=33.33% cpu : usr=0.00%, sys=24.36%, ctx=466, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191619: Thu Sep 25 14:50:12 2025 write: IOPS=1, BW=7539KiB/s (7720kB/s)(12.0MiB/1630msec); 0 zone resets clat (msec): min=168, max=705, avg=523.08, stdev=307.51 lat (msec): min=168, max=705, avg=523.42, stdev=307.56 clat percentiles (msec): | 1.00th=[ 169], 5.00th=[ 169], 10.00th=[ 169], 20.00th=[ 169], | 30.00th=[ 169], 40.00th=[ 693], 50.00th=[ 693], 60.00th=[ 693], | 70.00th=[ 709], 80.00th=[ 709], 90.00th=[ 709], 95.00th=[ 709], | 99.00th=[ 709], 99.50th=[ 709], 99.90th=[ 709], 99.95th=[ 709], | 99.99th=[ 709] bw ( KiB/s): min= 8000, max= 8192, per=40.27%, avg=8096.00, stdev=135.76, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 750=66.67% cpu : usr=0.00%, sys=19.77%, ctx=92, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191620: Thu Sep 25 14:50:12 2025 write: IOPS=2, BW=8521KiB/s (8726kB/s)(12.0MiB/1442msec); 0 zone resets clat (msec): min=232, max=887, avg=463.01, stdev=367.74 lat (msec): min=233, max=887, avg=463.38, stdev=367.73 clat percentiles (msec): | 1.00th=[ 234], 5.00th=[ 234], 10.00th=[ 234], 20.00th=[ 234], | 30.00th=[ 234], 40.00th=[ 271], 50.00th=[ 271], 60.00th=[ 271], | 70.00th=[ 885], 80.00th=[ 885], 90.00th=[ 885], 95.00th=[ 885], | 99.00th=[ 885], 99.50th=[ 885], 99.90th=[ 885], 99.95th=[ 885], | 99.99th=[ 885] bw ( KiB/s): min= 7474, max= 8192, per=38.96%, avg=7833.00, stdev=507.70, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 250=33.33%, 500=33.33%, 1000=33.33% cpu : usr=0.07%, sys=19.08%, ctx=68, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=12.1MiB/s (12.7MB/s), 3089KiB/s-9373KiB/s (3163kB/s-9598kB/s), io=16.0MiB (16.8MB), run=1311-1326msec WRITE: bw=19.6MiB/s (20.6MB/s), 6178KiB/s-8521KiB/s (6326kB/s-8726kB/s), io=32.0MiB (33.6MB), run=1326-1630msec rand-rw: (groupid=0, jobs=1): err= 0: pid=191622: Thu Sep 25 14:50:13 2025 read: IOPS=1, BW=6884KiB/s (7049kB/s)(12.0MiB/1785msec) clat (msec): min=141, max=1063, avg=557.29, stdev=467.36 lat (msec): min=141, max=1063, avg=557.29, stdev=467.36 clat percentiles (msec): | 1.00th=[ 142], 5.00th=[ 142], 10.00th=[ 142], 20.00th=[ 142], | 30.00th=[ 142], 40.00th=[ 468], 50.00th=[ 468], 60.00th=[ 468], | 70.00th=[ 1062], 80.00th=[ 1062], 90.00th=[ 1062], 95.00th=[ 1062], | 99.00th=[ 1062], 99.50th=[ 1062], 99.90th=[ 1062], 99.95th=[ 1062], | 99.99th=[ 1062] bw ( KiB/s): min= 8192, max= 8192, per=89.26%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 500=33.33%, 2000=33.33% cpu : usr=0.00%, sys=15.25%, ctx=102, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=3,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191623: Thu Sep 25 14:50:13 2025 read: IOPS=0, BW=2339KiB/s (2395kB/s)(4096KiB/1751msec) clat (nsec): min=791925k, max=791925k, avg=791924529.00, stdev= 0.00 lat (nsec): min=791928k, max=791928k, avg=791927508.00, stdev= 0.00 clat percentiles (msec): | 1.00th=[ 793], 5.00th=[ 793], 10.00th=[ 793], 20.00th=[ 793], | 30.00th=[ 793], 40.00th=[ 793], 50.00th=[ 793], 60.00th=[ 793], | 70.00th=[ 793], 80.00th=[ 793], 90.00th=[ 793], 95.00th=[ 793], | 99.00th=[ 793], 99.50th=[ 793], 99.90th=[ 793], 99.95th=[ 793], | 99.99th=[ 793] bw ( KiB/s): min= 8192, max= 8192, per=89.26%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 write: IOPS=1, BW=4678KiB/s (4791kB/s)(8192KiB/1751msec); 0 zone resets clat (msec): min=427, max=439, avg=433.15, stdev= 8.65 lat (msec): min=427, max=439, avg=433.36, stdev= 8.58 clat percentiles (msec): | 1.00th=[ 426], 5.00th=[ 426], 10.00th=[ 426], 20.00th=[ 426], | 30.00th=[ 426], 40.00th=[ 426], 50.00th=[ 426], 60.00th=[ 439], | 70.00th=[ 439], 80.00th=[ 439], 90.00th=[ 439], 95.00th=[ 439], | 99.00th=[ 439], 99.50th=[ 439], 99.90th=[ 439], 99.95th=[ 439], | 99.99th=[ 439] bw ( KiB/s): min= 8192, max= 8192, per=43.78%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 500=66.67%, 1000=33.33% cpu : usr=0.00%, sys=21.71%, ctx=96, majf=0, minf=9 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=1,2,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191624: Thu Sep 25 14:50:13 2025 write: IOPS=1, BW=7497KiB/s (7677kB/s)(12.0MiB/1639msec); 0 zone resets clat (msec): min=132, max=747, avg=518.87, stdev=336.31 lat (msec): min=133, max=747, avg=519.18, stdev=336.29 clat percentiles (msec): | 1.00th=[ 133], 5.00th=[ 133], 10.00th=[ 133], 20.00th=[ 133], | 30.00th=[ 133], 40.00th=[ 676], 50.00th=[ 676], 60.00th=[ 676], | 70.00th=[ 751], 80.00th=[ 751], 90.00th=[ 751], 95.00th=[ 751], | 99.00th=[ 751], 99.50th=[ 751], 99.90th=[ 751], 99.95th=[ 751], | 99.99th=[ 751] bw ( KiB/s): min= 8192, max= 8192, per=43.78%, avg=8192.00, stdev= 0.00, samples=1 iops : min= 2, max= 2, avg= 2.00, stdev= 0.00, samples=1 lat (msec) : 250=33.33%, 750=66.67% cpu : usr=0.00%, sys=17.09%, ctx=68, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 rand-rw: (groupid=0, jobs=1): err= 0: pid=191625: Thu Sep 25 14:50:13 2025 write: IOPS=1, BW=8106KiB/s (8300kB/s)(12.0MiB/1516msec); 0 zone resets clat (msec): min=447, max=605, avg=501.23, stdev=90.17 lat (msec): min=448, max=605, avg=501.54, stdev=90.12 clat percentiles (msec): | 1.00th=[ 447], 5.00th=[ 447], 10.00th=[ 447], 20.00th=[ 447], | 30.00th=[ 447], 40.00th=[ 451], 50.00th=[ 451], 60.00th=[ 451], | 70.00th=[ 609], 80.00th=[ 609], 90.00th=[ 609], 95.00th=[ 609], | 99.00th=[ 609], 99.50th=[ 609], 99.90th=[ 609], 99.95th=[ 609], | 99.99th=[ 609] bw ( KiB/s): min= 7757, max= 8192, per=42.61%, avg=7974.50, stdev=307.59, samples=2 iops : min= 1, max= 2, avg= 1.50, stdev= 0.71, samples=2 lat (msec) : 500=66.67%, 750=33.33% cpu : usr=0.00%, sys=25.08%, ctx=93, majf=0, minf=7 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,3,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=9179KiB/s (9399kB/s), 2339KiB/s-6884KiB/s (2395kB/s-7049kB/s), io=16.0MiB (16.8MB), run=1751-1785msec WRITE: bw=18.3MiB/s (19.2MB/s), 4678KiB/s-8106KiB/s (4791kB/s-8300kB/s), io=32.0MiB (33.6MB), run=1516-1751msec