== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 23:48:06 (1735102086)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.243287 s, 69.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00817632 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01086 s, 96.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0227166 s, 46.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184069 s, 57.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0289903 s, 36.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286178 s, 36.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0219873 s, 47.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0322699 s, 32.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226958 s, 46.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0272307 s, 38.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0448324 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0454696 s, 23.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0545949 s, 19.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0491955 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0629149 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0614989 s, 17.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00847406 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113242 s, 92.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00851027 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00917011 s, 114 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0121724 s, 86.1 MB/s
, 0.0133102 s, 78.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114987 s, 91.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117544 s, 89.2 MB/s
1048576 bytes (1.0 MB) copied, 0.00311287 s, 337 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109744 s, 95.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133948 s, 78.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123624 s, 84.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119017 s, 88.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118709 s, 88.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117373 s, 89.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134037 s, 78.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg129-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.94273 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73242 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.29@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.29@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.13585 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121404 s, 86.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0210021 s, 49.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203452 s, 51.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0355154 s, 29.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365267 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264838 s, 39.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0392122 s, 26.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0370368 s, 28.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0304886 s, 34.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0362751 s, 28.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0437206 s, 24.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0450038 s, 23.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.184118 s, 5.7 MB/s
, 0.183772 s, 5.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.225321 s, 4.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.226655 s, 4.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0080223 s, 131 MB/s
, 0.00192814 s, 544 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00946536 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106631 s, 98.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112048 s, 93.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100072 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00971959 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010338 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101009 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103027 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103983 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115329 s, 90.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108744 s, 96.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114284 s, 91.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118072 s, 88.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130718 s, 80.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg129-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93879 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73951 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.29@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.102248 s, 164 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00798882 s, 131 MB/s
1048576 bytes (1.0 MB) copied, 0.00688691 s, 152 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0216192 s, 48.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0237623 s, 44.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0258011 s, 40.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0243632 s, 43.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0220409 s, 47.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0245213 s, 42.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.026852 s, 39.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203269 s, 51.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239285 s, 43.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0266455 s, 39.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0241183 s, 43.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0271217 s, 38.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016657 s, 63.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106432 s, 98.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00759258 s, 138 MB/s
, 0.0086417 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00769064 s, 136 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00773678 s, 136 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00836053 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00808297 s, 130 MB/s
, 0.00812105 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00863545 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00937364 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010996 s, 95.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00922071 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00952388 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00974726 s, 108 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00996768 s, 105 MB/s
, 0.0100771 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116307 s, 90.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg129-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93589 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88023 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.120282 s, 139 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00873883 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159256 s, 65.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0315899 s, 33.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0344984 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0304234 s, 34.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0377392 s, 27.8 MB/s
, 0.036188 s, 29.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0420374 s, 24.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0454752 s, 23.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.045777 s, 22.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0513562 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0455107 s, 23.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.105671 s, 9.9 MB/s
, 0.11018 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.111059 s, 9.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106824 s, 9.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00976116 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00983733 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100254 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103155 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113983 s, 92.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0110141 s, 95.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0141861 s, 73.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122857 s, 85.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125184 s, 83.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146033 s, 71.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136532 s, 76.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157754 s, 66.5 MB/s
1048576 bytes (1.0 MB) copied, 0.015102 s, 69.4 MB/s
1+0 records in
1+0 records out
, 0.0150724 s, 69.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0145104 s, 72.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186461 s, 56.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg129-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80959 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72902 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92374 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73109 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.115704 s, 145 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151331 s, 69.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0213004 s, 49.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0335055 s, 31.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328062 s, 32.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0351711 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0296774 s, 35.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0330676 s, 31.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0312963 s, 33.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105703 s, 9.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.112051 s, 9.4 MB/s
1048576 bytes (1.0 MB) copied, 0.12454 s, 8.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.115057 s, 9.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.122686 s, 8.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.132958 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.142306 s, 7.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.137127 s, 7.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105214 s, 99.7 MB/s
, 0.00881131 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00957275 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00923728 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00983585 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104788 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113119 s, 92.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116497 s, 90.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113806 s, 92.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123638 s, 84.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111862 s, 93.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117117 s, 89.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108931 s, 96.3 MB/s
1048576 bytes (1.0 MB) copied, 0.014624 s, 71.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121228 s, 86.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136243 s, 77.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg129-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.6672 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4319 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.103122 s, 163 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00821839 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00909414 s, 115 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.020939 s, 50.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0193333 s, 54.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0220512 s, 47.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0240204 s, 43.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0218263 s, 48.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0293913 s, 35.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0299178 s, 35.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.027623 s, 38.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0237147 s, 44.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0293272 s, 35.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0273954 s, 38.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0333027 s, 31.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0359269 s, 29.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0356163 s, 29.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00846233 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00787926 s, 133 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00961166 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00693144 s, 151 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00773363 s, 136 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00955639 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010669 s, 98.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00173785 s, 603 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00182574 s, 574 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00831544 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010028 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103236 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00979886 s, 107 MB/s
, 0.0097757 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109056 s, 96.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122838 s, 85.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg129-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.94077 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72715 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.0982944 s, 171 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148941 s, 70.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0216325 s, 48.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0230145 s, 45.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0202742 s, 51.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.023293 s, 45.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0231167 s, 45.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0263227 s, 39.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0225415 s, 46.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165456 s, 63.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0212041 s, 49.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0698929 s, 15.0 MB/s
, 0.0629185 s, 16.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.119996 s, 8.7 MB/s
, 0.123422 s, 8.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.128604 s, 8.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.120378 s, 8.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00960845 s, 109 MB/s
, 0.00870542 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00842934 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116985 s, 89.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101229 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010438 s, 100 MB/s
, 0.0105398 s, 99.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112846 s, 92.9 MB/s
, 0.0105898 s, 99.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0097003 s, 108 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0111134 s, 94.4 MB/s
, 0.0112226 s, 93.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123225 s, 85.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122063 s, 85.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0127762 s, 82.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137027 s, 76.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg129-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg129-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76925 s, 10.7 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72782 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77267 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72989 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo