== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 08:14:57 (1741695297)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.470572 s, 35.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174998 s, 59.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0216813 s, 48.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0979528 s, 10.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0943566 s, 11.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0959503 s, 10.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.100473 s, 10.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102856 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.13099 s, 8.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138964 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.129182 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.149435 s, 7.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.152521 s, 6.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.168584 s, 6.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.167571 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.170545 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.195183 s, 5.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00164285 s, 638 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00258418 s, 406 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00158443 s, 662 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0016059 s, 653 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00150937 s, 695 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00152599 s, 687 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00153482 s, 683 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00166905 s, 628 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00743216 s, 141 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00373588 s, 281 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0017564 s, 597 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0928211 s, 11.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0603742 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0595864 s, 17.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0835876 s, 12.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0708566 s, 14.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg209-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.28361 s, 16.7 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.84489 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.9@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.9@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.391427 s, 42.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156119 s, 67.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329793 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0236282 s, 44.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0372591 s, 28.1 MB/s
, 0.026217 s, 40.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0752078 s, 13.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0973989 s, 10.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0846227 s, 12.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.118047 s, 8.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.118188 s, 8.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.128794 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.156117 s, 6.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.185772 s, 5.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.156795 s, 6.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.18147 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.171934 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00409463 s, 256 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.114911 s, 9.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106896 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0951799 s, 11.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0959226 s, 10.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00161519 s, 649 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0939791 s, 11.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0896014 s, 11.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0784209 s, 13.4 MB/s
, 0.0557792 s, 18.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0727085 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0447822 s, 23.4 MB/s
, 0.0491016 s, 21.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0332768 s, 31.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0305696 s, 34.3 MB/s
, 0.0193903 s, 54.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg209-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 8.06708 s, 13.0 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75422 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.9@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.334256 s, 50.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107837 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.121919 s, 8.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.143159 s, 7.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.236068 s, 4.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.213842 s, 4.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.205172 s, 5.1 MB/s
, 0.227627 s, 4.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.18933 s, 5.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.185433 s, 5.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.350764 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.323638 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.340204 s, 3.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.376208 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.377306 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.41441 s, 2.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.422926 s, 2.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0602916 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412599 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0438885 s, 23.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0461317 s, 22.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0483028 s, 21.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0430928 s, 24.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.045803 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0483356 s, 21.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0506536 s, 20.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0535889 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0448491 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0578281 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0486457 s, 21.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0486839 s, 21.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0521043 s, 20.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0736239 s, 14.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg209-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.6347 s, 10.9 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.98834 s, 21.0 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.363406 s, 46.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0314394 s, 33.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0505227 s, 20.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0485351 s, 21.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0626039 s, 16.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.038433 s, 27.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0594671 s, 17.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.196064 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.162878 s, 6.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138442 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.247544 s, 4.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.253327 s, 4.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.258024 s, 4.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.239826 s, 4.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.308163 s, 3.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.26599 s, 3.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.265518 s, 3.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.029598 s, 35.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0339972 s, 30.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00695163 s, 151 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0118523 s, 88.5 MB/s
1048576 bytes (1.0 MB) copied, 0.00830047 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00158347 s, 662 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102278 s, 10.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0735877 s, 14.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0496831 s, 21.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0501774 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0432084 s, 24.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0901973 s, 11.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0907962 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0387262 s, 27.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0334429 s, 31.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134519 s, 77.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg209-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.99031 s, 10.5 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.87576 s, 10.6 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.88821 s, 15.2 MB/s
Write runtime is 8 s, speed is 12.500000 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.87464 s, 10.6 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.380038 s, 44.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145502 s, 72.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0210765 s, 49.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0250388 s, 41.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0661829 s, 15.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0739044 s, 14.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0942219 s, 11.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0853699 s, 12.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0987845 s, 10.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.103755 s, 10.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.119034 s, 8.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.120461 s, 8.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.132598 s, 7.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.12338 s, 8.5 MB/s
1048576 bytes (1.0 MB) copied, 0.123624 s, 8.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.132337 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.135549 s, 7.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00203268 s, 516 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0748171 s, 14.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0668328 s, 15.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00218508 s, 480 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0628751 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0599873 s, 17.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0635305 s, 16.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0404904 s, 25.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0642737 s, 16.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0519059 s, 20.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145641 s, 72.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.066383 s, 15.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169009 s, 62.0 MB/s
, 0.0208332 s, 50.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0569011 s, 18.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196766 s, 53.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg209-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7189 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.5223 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.251102 s, 66.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.020494 s, 51.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0160262 s, 65.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0250326 s, 41.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0468185 s, 22.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0743256 s, 14.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.071225 s, 14.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.080609 s, 13.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0733825 s, 14.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0688689 s, 15.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0862839 s, 12.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0895512 s, 11.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0972144 s, 10.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.145077 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.145198 s, 7.2 MB/s
1048576 bytes (1.0 MB) copied, 0.149954 s, 7.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.163336 s, 6.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0271639 s, 38.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0292524 s, 35.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0293218 s, 35.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0202271 s, 51.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.030309 s, 34.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0209567 s, 50.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336417 s, 31.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0331155 s, 31.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0341284 s, 30.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0369718 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.027804 s, 37.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0408267 s, 25.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281248 s, 37.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0295985 s, 35.4 MB/s
, 0.032733 s, 32.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0523109 s, 20.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg209-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.94443 s, 17.6 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81465 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.25218 s, 66.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0293904 s, 35.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0316604 s, 33.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0764588 s, 13.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0829456 s, 12.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0788093 s, 13.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0808103 s, 13.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0993733 s, 10.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0958783 s, 10.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.103737 s, 10.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0865907 s, 12.1 MB/s
, 0.107382 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105239 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.125849 s, 8.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.115914 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.133449 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.152011 s, 6.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0224446 s, 46.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00481104 s, 218 MB/s
1048576 bytes (1.0 MB) copied, 0.00498101 s, 211 MB/s
, 0.0190558 s, 55.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0279689 s, 37.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0532979 s, 19.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0480819 s, 21.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0423201 s, 24.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.063182 s, 16.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00711568 s, 147 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0369041 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00937008 s, 112 MB/s
1048576 bytes (1.0 MB) copied, 0.015699 s, 66.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105491 s, 99.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170846 s, 61.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0218614 s, 48.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg209-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg209-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.0042 s, 10.5 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81916 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.90196 s, 10.6 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.78769 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo