== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 00:46:25 (1746074785)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.315214 s, 53.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181897 s, 57.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161732 s, 64.8 MB/s
, 0.0245699 s, 42.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0711682 s, 14.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0880854 s, 11.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0860454 s, 12.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107754 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105488 s, 9.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.114962 s, 9.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0963769 s, 10.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.114421 s, 9.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.102666 s, 10.2 MB/s
, 0.103828 s, 10.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105349 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.124499 s, 8.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.134758 s, 7.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0218342 s, 48.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0185053 s, 56.7 MB/s
1048576 bytes (1.0 MB) copied, 0.01825 s, 57.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0206041 s, 50.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0209305 s, 50.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0210104 s, 49.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0231407 s, 45.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0250556 s, 41.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.025823 s, 40.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0265333 s, 39.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0282625 s, 37.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0269064 s, 39.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0272885 s, 38.4 MB/s
, 0.0272595 s, 38.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0297617 s, 35.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0306992 s, 34.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg452-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.3669 s, 19.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81691 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.52@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.52@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.384282 s, 43.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0290107 s, 36.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0337506 s, 31.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.123432 s, 8.5 MB/s
1048576 bytes (1.0 MB) copied, 0.125181 s, 8.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.120848 s, 8.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.125815 s, 8.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.120693 s, 8.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.129936 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.140771 s, 7.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.140203 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.140147 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.146189 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.163196 s, 6.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.156102 s, 6.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.198673 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.198878 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0088093 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137431 s, 76.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00191258 s, 548 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00223704 s, 469 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00265183 s, 395 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00353224 s, 297 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.047796 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0262646 s, 39.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0703508 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.031598 s, 33.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0201645 s, 52.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0206966 s, 50.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0156444 s, 67.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129225 s, 81.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00692198 s, 151 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00556981 s, 188 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg452-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.77553 s, 15.5 MB/s
Write runtime is 8 s, speed is 12.500000 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81899 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.52@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.260566 s, 64.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156818 s, 66.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0185169 s, 56.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0642051 s, 16.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0626688 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0699211 s, 15.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0688051 s, 15.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0678464 s, 15.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0705994 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0764579 s, 13.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0767015 s, 13.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0800955 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0799844 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0803638 s, 13.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0793866 s, 13.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102651 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.112974 s, 9.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0132359 s, 79.2 MB/s
, 0.0131887 s, 79.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0187908 s, 55.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136946 s, 76.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153527 s, 68.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0207932 s, 50.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162636 s, 64.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165262 s, 63.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176382 s, 59.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016735 s, 62.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191802 s, 54.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02005 s, 52.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189112 s, 55.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0197872 s, 53.0 MB/s
, 0.0212818 s, 49.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0261624 s, 40.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg452-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.2189 s, 20.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.94474 s, 21.2 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.4474 s, 37.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260949 s, 40.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105548 s, 99.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0454065 s, 23.1 MB/s
, 0.048347 s, 21.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0513425 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0558042 s, 18.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0575487 s, 18.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0611626 s, 17.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.05681 s, 18.5 MB/s
, 0.0523058 s, 20.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.050752 s, 20.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.157611 s, 6.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.176545 s, 5.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.172105 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.162978 s, 6.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.183696 s, 5.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00294559 s, 356 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00314157 s, 334 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00263486 s, 398 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010443 s, 100 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177123 s, 59.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00298933 s, 351 MB/s
, 0.00196191 s, 534 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00188109 s, 557 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00209337 s, 501 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0353093 s, 29.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01412 s, 74.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0252585 s, 41.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0321745 s, 32.6 MB/s
, 0.00585465 s, 179 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0307919 s, 34.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0070955 s, 148 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg452-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.93298 s, 10.6 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.79809 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.05905 s, 20.7 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77709 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.862691 s, 19.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149044 s, 70.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0350829 s, 29.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0401734 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0431838 s, 24.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.042208 s, 24.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0456137 s, 23.0 MB/s
, 0.0448688 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0434127 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.143985 s, 7.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.158476 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.166165 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.16956 s, 6.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.180615 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.179947 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.180867 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.185952 s, 5.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0415997 s, 25.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0432042 s, 24.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151407 s, 69.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0272582 s, 38.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0282881 s, 37.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0290302 s, 36.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0201891 s, 51.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226123 s, 46.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0227293 s, 46.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0267288 s, 39.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0249174 s, 42.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.026158 s, 40.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254836 s, 41.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0313259 s, 33.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0342459 s, 30.6 MB/s
, 0.034127 s, 30.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg452-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7294 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.489 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.273065 s, 61.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146965 s, 71.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191142 s, 54.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0349612 s, 30.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0332659 s, 31.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0404322 s, 25.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0393987 s, 26.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0392742 s, 26.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0504374 s, 20.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0470742 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0501394 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0472547 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0601506 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0727722 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0757371 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0862941 s, 12.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0871365 s, 12.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00378929 s, 277 MB/s
, 0.0211901 s, 49.5 MB/s
1048576 bytes (1.0 MB) copied, 0.00246709 s, 425 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0178066 s, 58.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158996 s, 65.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00224148 s, 468 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00219323 s, 478 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151478 s, 69.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198499 s, 52.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137611 s, 76.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0219775 s, 47.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150016 s, 69.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168174 s, 62.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017147 s, 61.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0251371 s, 41.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0220813 s, 47.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg452-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.34087 s, 19.6 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.83183 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.475714 s, 35.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0104401 s, 100 MB/s
, 0.0110664 s, 94.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173737 s, 60.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0398415 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0566755 s, 18.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0691564 s, 15.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0634754 s, 16.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0593899 s, 17.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0667796 s, 15.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0839698 s, 12.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.10471 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.108749 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.310796 s, 3.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.307255 s, 3.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.352961 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.356612 s, 2.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00213237 s, 492 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00996439 s, 105 MB/s
, 0.0157571 s, 66.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00167334 s, 627 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144772 s, 72.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126534 s, 82.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130449 s, 80.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134109 s, 78.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142749 s, 73.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137698 s, 76.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0164167 s, 63.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169336 s, 61.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172566 s, 60.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126975 s, 82.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159313 s, 65.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128115 s, 81.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg452-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg452-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.94598 s, 10.5 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76851 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.96405 s, 10.5 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.78081 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo