== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 18:11:37 (1738969897)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.157627 s, 106 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198398 s, 52.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0233189 s, 45.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385604 s, 27.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0497458 s, 21.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0440867 s, 23.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0510693 s, 20.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0420669 s, 24.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0524324 s, 20.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0337246 s, 31.1 MB/s
, 0.0485043 s, 21.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0521804 s, 20.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0476371 s, 22.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0491178 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.054881 s, 19.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.041854 s, 25.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0524392 s, 20.0 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b62e4000.lock_unused_count=2
ldlm.namespaces.lustre-OST0001-osc-ffff8800b62e4000.lock_unused_count=2
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111527 s, 94.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142571 s, 73.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114398 s, 91.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169979 s, 61.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00495772 s, 212 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135815 s, 77.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165447 s, 63.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.016335 s, 64.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0146949 s, 71.4 MB/s
, 0.0167411 s, 62.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151219 s, 69.3 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0144157 s, 72.7 MB/s
1048576 bytes (1.0 MB) copied, 0.019724 s, 53.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0182136 s, 57.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159902 s, 65.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0225643 s, 46.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90681 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73911 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg138-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg138-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.38@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.38@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.15447 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0269565 s, 38.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0253968 s, 41.3 MB/s
, 0.0261195 s, 40.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0211311 s, 49.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015638 s, 67.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0371096 s, 28.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0333323 s, 31.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.039423 s, 26.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.030643 s, 34.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260533 s, 40.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0289308 s, 36.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0384338 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0325843 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0300243 s, 34.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0193985 s, 54.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328613 s, 31.9 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b62e4000.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800b62e4000.lock_unused_count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173158 s, 60.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170385 s, 61.5 MB/s
, 0.014207 s, 73.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00297343 s, 353 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.012353 s, 84.9 MB/s
, 0.0156014 s, 67.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169244 s, 62.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150414 s, 69.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132824 s, 78.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131953 s, 79.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154603 s, 67.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161214 s, 65.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157028 s, 66.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.020568 s, 51.0 MB/s
1048576 bytes (1.0 MB) copied, 0.00832014 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0188075 s, 55.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90882 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73589 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg138-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg138-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.38@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.154543 s, 109 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177471 s, 59.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0230798 s, 45.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0318742 s, 32.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.040335 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0399221 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0397968 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380923 s, 27.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.035286 s, 29.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.047739 s, 22.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.033821 s, 31.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0356309 s, 29.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.040059 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0494377 s, 21.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0538757 s, 19.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0565592 s, 18.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0471155 s, 22.3 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b62e4000.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0144874 s, 72.4 MB/s
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0143996 s, 72.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0124239 s, 84.4 MB/s
, 0.0168018 s, 62.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135258 s, 77.5 MB/s
, 0.0132607 s, 79.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0188889 s, 55.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0144791 s, 72.4 MB/s
, 0.0166244 s, 63.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186845 s, 56.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145856 s, 71.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101592 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156317 s, 67.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162247 s, 64.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155347 s, 67.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00373409 s, 281 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89635 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88942 s, 21.4 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg138-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg138-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.163827 s, 102 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017195 s, 61.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151876 s, 69.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0220014 s, 47.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.039739 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0469437 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0475971 s, 22.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0552419 s, 19.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0510178 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0481647 s, 21.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0530972 s, 19.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0599241 s, 17.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0579853 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0479994 s, 21.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.051744 s, 20.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0571704 s, 18.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0502121 s, 20.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010795 s, 97.1 MB/s
, 0.0106483 s, 98.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132607 s, 79.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014122 s, 74.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156186 s, 67.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0183297 s, 57.2 MB/s
, 0.0203319 s, 51.6 MB/s
1048576 bytes (1.0 MB) copied, 0.00747663 s, 140 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156278 s, 67.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0194671 s, 53.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158485 s, 66.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135021 s, 77.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147957 s, 70.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01518 s, 69.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.020067 s, 52.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198783 s, 52.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75967 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73738 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg138-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg138-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90078 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73773 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg138-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
oleg138-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.0891271 s, 188 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00735873 s, 142 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116537 s, 90.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116196 s, 90.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198849 s, 52.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0178192 s, 58.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0208218 s, 50.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015214 s, 68.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0259995 s, 40.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0255999 s, 41.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184487 s, 56.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187215 s, 56.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00928685 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177349 s, 59.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264219 s, 39.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0259085 s, 40.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0244903 s, 42.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0141616 s, 74.0 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161318 s, 65.0 MB/s
, 0.00975829 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112879 s, 92.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150066 s, 69.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013608 s, 77.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116914 s, 89.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161457 s, 64.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0185318 s, 56.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0151915 s, 69.0 MB/s
, 0.0151188 s, 69.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153991 s, 68.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150424 s, 69.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015724 s, 66.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158864 s, 66.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0178555 s, 58.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4644 s, 5.4 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4398 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg138-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg138-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.153722 s, 109 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0234053 s, 44.8 MB/s
, 0.0195299 s, 53.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183456 s, 57.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200055 s, 52.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0276589 s, 37.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0263196 s, 39.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0453628 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0296287 s, 35.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0367792 s, 28.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0418095 s, 25.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.059657 s, 17.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0501984 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0586057 s, 17.9 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0481566 s, 21.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0462174 s, 22.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0351441 s, 29.8 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b62e4000.lock_unused_count=3
ldlm.namespaces.lustre-OST0001-osc-ffff8800b62e4000.lock_unused_count=7
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0064551 s, 162 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115111 s, 91.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00834647 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00137892 s, 760 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0169438 s, 61.9 MB/s
, 0.00135163 s, 776 MB/s
, 0.0113612 s, 92.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0186062 s, 56.4 MB/s
, 0.0204593 s, 51.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00138639 s, 756 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175933 s, 59.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151333 s, 69.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0055315 s, 190 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146054 s, 71.8 MB/s
1048576 bytes (1.0 MB) copied, 0.00820286 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00752275 s, 139 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b62e4000.lock_unused_count=2
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.9119 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73961 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg138-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg138-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.151803 s, 111 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168603 s, 62.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196218 s, 53.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.035872 s, 29.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264159 s, 39.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228878 s, 45.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0351731 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0513455 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0399015 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0516369 s, 20.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0504682 s, 20.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0359662 s, 29.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0471385 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0446445 s, 23.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0581488 s, 18.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0543785 s, 19.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0530158 s, 19.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122967 s, 85.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163337 s, 64.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124601 s, 84.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123957 s, 84.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124746 s, 84.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168159 s, 62.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127594 s, 82.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158979 s, 66.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.002621 s, 400 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01443 s, 72.7 MB/s
, 0.0188439 s, 55.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016383 s, 64.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133953 s, 78.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0194682 s, 53.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173775 s, 60.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0197474 s, 53.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76634 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73801 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg138-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg138-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7532 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73953 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg138-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg138-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
pdsh@oleg138-client: oleg138-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo