== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 03:29:46 (1747812586)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.246387 s, 68.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108622 s, 96.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0270789 s, 38.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253327 s, 41.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.025089 s, 41.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0223996 s, 46.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0282789 s, 37.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0277973 s, 37.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0317466 s, 33.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0278792 s, 37.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.048404 s, 21.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0410824 s, 25.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0511157 s, 20.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0663913 s, 15.8 MB/s
, 0.0548152 s, 19.1 MB/s
, 0.0646586 s, 16.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0670441 s, 15.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107091 s, 97.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00891559 s, 118 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010237 s, 102 MB/s
1048576 bytes (1.0 MB) copied, 0.00951186 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00973222 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110081 s, 95.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00898824 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106391 s, 98.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100308 s, 105 MB/s
1048576 bytes (1.0 MB) copied, 0.00998177 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103012 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00996889 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111173 s, 94.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112911 s, 92.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116129 s, 90.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110167 s, 95.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93417 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73666 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.40@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.40@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.234439 s, 71.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00975904 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0233335 s, 44.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0204657 s, 51.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0232929 s, 45.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0238312 s, 44.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286085 s, 36.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0209554 s, 50.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.023107 s, 45.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0273857 s, 38.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.037805 s, 27.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0448806 s, 23.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0466215 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.276466 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.275273 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.319308 s, 3.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.331142 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00567176 s, 185 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00753889 s, 139 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108578 s, 96.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00667606 s, 157 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107021 s, 98.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00721072 s, 145 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00760041 s, 138 MB/s
, 0.00912861 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00904479 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00953807 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00908146 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00932644 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010499 s, 99.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0108432 s, 96.7 MB/s
, 0.0108873 s, 96.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102329 s, 102 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.94423 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73079 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.40@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.235196 s, 71.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00802769 s, 131 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179773 s, 58.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0185507 s, 56.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0273258 s, 38.4 MB/s
, 0.0254187 s, 41.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157463 s, 66.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0284788 s, 36.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130379 s, 80.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260867 s, 40.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198206 s, 52.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0424748 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0416058 s, 25.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0528684 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0510606 s, 20.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0466326 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.064476 s, 16.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
, 0.00698311 s, 150 MB/s
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0106853 s, 98.1 MB/s
, 0.00749859 s, 140 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00874222 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00875016 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118254 s, 88.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00955027 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109203 s, 96.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104392 s, 100 MB/s
, 0.0105756 s, 99.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119662 s, 87.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110651 s, 94.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00992605 s, 106 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0133416 s, 78.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0104447 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132407 s, 79.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.94482 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.87967 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.429674 s, 39.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121258 s, 86.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0268736 s, 39.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0292962 s, 35.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0224334 s, 46.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0247734 s, 42.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0221519 s, 47.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0269732 s, 38.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.028768 s, 36.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0751617 s, 14.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0900232 s, 11.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0933187 s, 11.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.090398 s, 11.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.337098 s, 3.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.3297 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.382168 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.380868 s, 2.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100364 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00763242 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105038 s, 99.8 MB/s
, 0.00962495 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00861053 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00967612 s, 108 MB/s
1048576 bytes (1.0 MB) copied, 0.00887768 s, 118 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103749 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0087749 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00888746 s, 118 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00908117 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104064 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105294 s, 99.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011007 s, 95.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0112601 s, 93.1 MB/s
, 0.0103761 s, 101 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81414 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73089 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91173 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73087 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.83129 s, 20.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115041 s, 91.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0204938 s, 51.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264997 s, 39.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0208528 s, 50.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0233188 s, 45.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.021612 s, 48.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0287385 s, 36.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0288097 s, 36.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.173348 s, 6.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.189242 s, 5.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.182172 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.194789 s, 5.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.18824 s, 5.6 MB/s
, 0.20074 s, 5.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.200377 s, 5.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.205394 s, 5.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00664424 s, 158 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00570874 s, 184 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00594056 s, 177 MB/s
, 0.0102322 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00774411 s, 135 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00147264 s, 712 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0113109 s, 92.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00884447 s, 119 MB/s
, 0.00815061 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104784 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00984848 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114689 s, 91.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110258 s, 95.1 MB/s
, 0.0120177 s, 87.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110112 s, 95.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106426 s, 98.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.8297 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.434 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.233468 s, 71.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118113 s, 88.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170803 s, 61.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0346577 s, 30.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0364704 s, 28.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.034917 s, 30.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0398491 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0437766 s, 24.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0437185 s, 24.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0395385 s, 26.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0382811 s, 27.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0527929 s, 19.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.03979 s, 26.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0530816 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0507988 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0473635 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0514977 s, 20.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0096365 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00810608 s, 129 MB/s
, 0.00590716 s, 178 MB/s
1048576 bytes (1.0 MB) copied, 0.0112488 s, 93.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109503 s, 95.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00803866 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00872896 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00814158 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00893338 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00971813 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00998235 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00820286 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107928 s, 97.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113255 s, 92.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115836 s, 90.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136086 s, 77.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93785 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73561 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.434345 s, 38.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139809 s, 75.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02578 s, 40.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0293334 s, 35.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0310098 s, 33.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275851 s, 38.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0264404 s, 39.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0304944 s, 34.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329468 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0723224 s, 14.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.092031 s, 11.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0916173 s, 11.4 MB/s
, 0.0931283 s, 11.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.33194 s, 3.2 MB/s
, 0.331807 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.382257 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.380117 s, 2.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011062 s, 94.8 MB/s
, 0.00770734 s, 136 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00880985 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00892714 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110218 s, 95.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112516 s, 93.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00966244 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116694 s, 89.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118395 s, 88.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120621 s, 86.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.012896 s, 81.3 MB/s
, 0.0136217 s, 77.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138673 s, 75.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0142995 s, 73.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145132 s, 72.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148533 s, 70.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.84651 s, 10.6 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7355 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77981 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73569 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo