== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 16:33:54 (1746131634)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.517304 s, 32.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.031156 s, 33.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0440262 s, 23.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0846011 s, 12.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106097 s, 9.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.110803 s, 9.5 MB/s
, 0.0942639 s, 11.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106817 s, 9.8 MB/s
, 0.107963 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.125065 s, 8.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.166697 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.17836 s, 5.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.179903 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.204819 s, 5.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.196518 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.205777 s, 5.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.244632 s, 4.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00478209 s, 219 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00221607 s, 473 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0508784 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00211448 s, 496 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0936626 s, 11.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0911059 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0759425 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0685736 s, 15.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00820305 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0432966 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0664307 s, 15.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0469598 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0395651 s, 26.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0352713 s, 29.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0273441 s, 38.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365023 s, 28.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg112-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 8.40846 s, 12.5 MB/s
Write runtime is 9 s, speed is 11.111111 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.87776 s, 10.6 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.12@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.12@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.341715 s, 49.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413025 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0417525 s, 25.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0978956 s, 10.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0955198 s, 11.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.100652 s, 10.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.108732 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.113318 s, 9.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.103194 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106075 s, 9.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107296 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110142 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.11121 s, 9.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.20825 s, 5.0 MB/s
, 0.21294 s, 4.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.245714 s, 4.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.263581 s, 4.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0299662 s, 35.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253058 s, 41.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0309555 s, 33.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0279165 s, 37.6 MB/s
1048576 bytes (1.0 MB) copied, 0.00513696 s, 204 MB/s
1048576 bytes (1.0 MB) copied, 0.0117464 s, 89.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0422753 s, 24.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281313 s, 37.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0302044 s, 34.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0308482 s, 34.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113961 s, 92.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345431 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365182 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0289004 s, 36.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.036674 s, 28.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0321307 s, 32.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg112-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 8.93079 s, 11.7 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.84303 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.12@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.510961 s, 32.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0439107 s, 23.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0459665 s, 22.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.123764 s, 8.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.159132 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.128669 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.139466 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.153884 s, 6.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.156574 s, 6.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.182134 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.211731 s, 5.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.207477 s, 5.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.234397 s, 4.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.220486 s, 4.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.230957 s, 4.5 MB/s
, 0.203246 s, 5.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.270856 s, 3.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0256115 s, 40.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0383474 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0387488 s, 27.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0312207 s, 33.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329478 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357557 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0386181 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0390962 s, 26.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380091 s, 27.6 MB/s
, 0.0438377 s, 23.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0399263 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0401334 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.040393 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0483387 s, 21.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.043693 s, 24.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0440763 s, 23.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg112-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 8.533 s, 12.3 MB/s
Write runtime is 9 s, speed is 11.111111 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.98922 s, 21.0 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.505536 s, 33.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328428 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0430238 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0915306 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.10085 s, 10.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107559 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.119733 s, 8.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.124223 s, 8.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.137889 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138289 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.145173 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.163101 s, 6.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.157898 s, 6.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.198337 s, 5.3 MB/s
, 0.188665 s, 5.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.306256 s, 3.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.3198 s, 3.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0388552 s, 27.0 MB/s
, 0.0562574 s, 18.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0324222 s, 32.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0368704 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0428903 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0491264 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0797361 s, 13.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0645186 s, 16.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0699733 s, 15.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0620563 s, 16.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0623316 s, 16.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0691856 s, 15.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0768371 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0786509 s, 13.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0791395 s, 13.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0848073 s, 12.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg112-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.9905 s, 9.5 MB/s
Write runtime is 12 s, speed is 8.333333 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8934 s, 10.6 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 7.97402 s, 13.1 MB/s
Write runtime is 9 s, speed is 11.111111 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.86171 s, 10.6 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.906647 s, 18.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327729 s, 32.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0920523 s, 11.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.123015 s, 8.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.130567 s, 8.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.118738 s, 8.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.127906 s, 8.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.148087 s, 7.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.136597 s, 7.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.156272 s, 6.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.183646 s, 5.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.181765 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.196163 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.272655 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.269006 s, 3.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.28722 s, 3.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.29081 s, 3.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00238079 s, 440 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00295492 s, 355 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00260738 s, 402 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0495632 s, 21.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0742473 s, 14.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0016099 s, 651 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0643905 s, 16.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0344437 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0572078 s, 18.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0628017 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0644615 s, 16.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0518196 s, 20.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0645539 s, 16.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0554573 s, 18.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0581557 s, 18.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0593626 s, 17.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg112-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7922 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.5477 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.416367 s, 40.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.035677 s, 29.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0496619 s, 21.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0310259 s, 33.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.10934 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102389 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0972562 s, 10.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.112106 s, 9.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.122366 s, 8.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.123305 s, 8.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.136632 s, 7.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.140981 s, 7.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.137716 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.150544 s, 7.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.152729 s, 6.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.160344 s, 6.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.181345 s, 5.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275983 s, 38.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.044241 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0451122 s, 23.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0572544 s, 18.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345604 s, 30.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0279597 s, 37.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0353307 s, 29.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329393 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0438376 s, 23.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0408182 s, 25.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0475544 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0491537 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0491912 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0488359 s, 21.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0508576 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0519093 s, 20.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg112-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 7.73867 s, 13.5 MB/s
Write runtime is 9 s, speed is 11.111111 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82011 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.553152 s, 30.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0351796 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0405668 s, 25.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.067198 s, 15.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0611287 s, 17.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.116263 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.148063 s, 7.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.171504 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.166305 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.220901 s, 4.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.232181 s, 4.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.24737 s, 4.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.251936 s, 4.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.220778 s, 4.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.256327 s, 4.1 MB/s
, 0.219615 s, 4.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.265049 s, 4.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336132 s, 31.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0423119 s, 24.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00620325 s, 169 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0383187 s, 27.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0388523 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148959 s, 70.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0426853 s, 24.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0457065 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0500713 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0592057 s, 17.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0723691 s, 14.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0466767 s, 22.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0400871 s, 26.2 MB/s
, 0.0769383 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0423914 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.045506 s, 23.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg112-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg112-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.4459 s, 10.0 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80482 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.99296 s, 10.5 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.84076 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo