== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 01:08:52 (1734674932)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.332704 s, 50.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114911 s, 91.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239253 s, 43.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242446 s, 43.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0793507 s, 13.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0869539 s, 12.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0893315 s, 11.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0910754 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.087195 s, 12.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0926232 s, 11.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0987065 s, 10.6 MB/s
1048576 bytes (1.0 MB) copied, 0.104125 s, 10.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109781 s, 9.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.150294 s, 7.0 MB/s
, 0.154283 s, 6.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.167818 s, 6.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.173073 s, 6.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013995 s, 74.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00158808 s, 660 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134319 s, 78.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0133248 s, 78.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0365141 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144763 s, 72.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239496 s, 43.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0321457 s, 32.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00152998 s, 685 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0300459 s, 34.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0284213 s, 36.9 MB/s
, 0.045404 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0466289 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0301085 s, 34.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00626326 s, 167 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0051628 s, 203 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg207-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.54574 s, 18.9 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81673 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.7@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.7@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.270986 s, 61.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0430477 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0616397 s, 17.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0419383 s, 25.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.082054 s, 12.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0736745 s, 14.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0503253 s, 20.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.100239 s, 10.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.120184 s, 8.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0882951 s, 11.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0799811 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.132907 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.134046 s, 7.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0924735 s, 11.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0940261 s, 11.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.10093 s, 10.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.13969 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0234353 s, 44.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239106 s, 43.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0036639 s, 286 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00551068 s, 190 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0197943 s, 53.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0382959 s, 27.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175031 s, 59.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0333639 s, 31.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0343476 s, 30.5 MB/s
, 0.0263092 s, 39.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0313943 s, 33.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357997 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0279063 s, 37.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242212 s, 43.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00663513 s, 158 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253925 s, 41.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg207-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.24139 s, 20.0 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.78283 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.7@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.208604 s, 80.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0298507 s, 35.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0280242 s, 37.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0603675 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0587331 s, 17.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0595605 s, 17.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0703525 s, 14.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0743112 s, 14.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0773903 s, 13.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.085298 s, 12.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0821108 s, 12.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0923632 s, 11.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0910162 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102704 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.100329 s, 10.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105092 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.114326 s, 9.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.025649 s, 40.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179302 s, 58.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191571 s, 54.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286976 s, 36.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161936 s, 64.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0241892 s, 43.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0225199 s, 46.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0231568 s, 45.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226954 s, 46.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239649 s, 43.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0238412 s, 44.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253341 s, 41.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.024765 s, 42.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0265425 s, 39.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0325689 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0303397 s, 34.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg207-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.06217 s, 20.7 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95779 s, 21.2 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.204808 s, 81.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0235736 s, 44.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167138 s, 62.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226104 s, 46.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.054184 s, 19.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0534907 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0541528 s, 19.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0788505 s, 13.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0723964 s, 14.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0801137 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0790316 s, 13.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.104886 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.111299 s, 9.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.16742 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.148793 s, 7.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.199262 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.198112 s, 5.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327255 s, 32.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281375 s, 37.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365075 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0246294 s, 42.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0245067 s, 42.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0398306 s, 26.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0248364 s, 42.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.019192 s, 54.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0452294 s, 23.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0455224 s, 23.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0285334 s, 36.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0245796 s, 42.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0313323 s, 33.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0269085 s, 39.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0272342 s, 38.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0279883 s, 37.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg207-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.94594 s, 10.5 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82127 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.24792 s, 20.0 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.86081 s, 10.6 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.183693 s, 91.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013109 s, 80.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0473762 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0497742 s, 21.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0562703 s, 18.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0559071 s, 18.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0519123 s, 20.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0600515 s, 17.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0553357 s, 18.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0729878 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0671249 s, 15.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.108654 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107771 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.118791 s, 8.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.119274 s, 8.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.141695 s, 7.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.157367 s, 6.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182008 s, 57.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00958207 s, 109 MB/s
, 0.0160098 s, 65.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132561 s, 79.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0240453 s, 43.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152594 s, 68.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254476 s, 41.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122001 s, 85.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114596 s, 91.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119095 s, 88.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121802 s, 86.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131292 s, 79.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139583 s, 75.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142696 s, 73.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147581 s, 71.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154159 s, 68.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg207-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.649 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4842 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.165152 s, 102 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02526 s, 41.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0241884 s, 43.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0751797 s, 13.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0979259 s, 10.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.100717 s, 10.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0956424 s, 11.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0952068 s, 11.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.103187 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102033 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.112079 s, 9.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107032 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.123373 s, 8.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.166744 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.159298 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.159024 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.164439 s, 6.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0255898 s, 41.0 MB/s
, 0.0239094 s, 43.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239551 s, 43.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0194153 s, 54.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0193729 s, 54.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.022156 s, 47.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.020913 s, 50.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0273598 s, 38.3 MB/s
, 0.0273751 s, 38.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.027671 s, 37.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.028651 s, 36.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.029008 s, 36.1 MB/s
, 0.0289716 s, 36.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242805 s, 43.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0258583 s, 40.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0400194 s, 26.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg207-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.36999 s, 19.5 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81318 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.207506 s, 80.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127727 s, 82.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0352067 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329296 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.04537 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.057944 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0747912 s, 14.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.060439 s, 17.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.130305 s, 8.0 MB/s
, 0.100729 s, 10.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105606 s, 9.9 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.135961 s, 7.7 MB/s
, 0.101447 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.149939 s, 7.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.130087 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107651 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.111805 s, 9.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.029918 s, 35.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0213438 s, 49.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0245484 s, 42.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0259917 s, 40.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0270014 s, 38.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0274174 s, 38.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260925 s, 40.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0265499 s, 39.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286721 s, 36.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0297135 s, 35.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328852 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345383 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0339571 s, 30.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0353438 s, 29.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0376329 s, 27.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0441763 s, 23.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg207-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg207-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.99398 s, 10.5 MB/s
Write runtime is 12 s, speed is 8.333333 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80683 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.91938 s, 10.6 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80577 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo