== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 23:59:42 (1735102782)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.145804 s, 115 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143877 s, 72.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0199785 s, 52.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0352278 s, 29.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0478153 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0467407 s, 22.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0517897 s, 20.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0545681 s, 19.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0540677 s, 19.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0506373 s, 20.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0510589 s, 20.5 MB/s
, 0.054513 s, 19.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0524862 s, 20.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0494997 s, 21.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0604711 s, 17.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.052645 s, 19.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0606097 s, 17.3 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b3cba000.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800b3cba000.lock_unused_count=2
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014901 s, 70.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00855441 s, 123 MB/s
, 0.00716337 s, 146 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00211226 s, 496 MB/s
1048576 bytes (1.0 MB) copied, 0.00616108 s, 170 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00974419 s, 108 MB/s
1048576 bytes (1.0 MB) copied, 0.00232059 s, 452 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00161062 s, 651 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0126649 s, 82.8 MB/s
1+0 records in
1+0 records out
, 0.016985 s, 61.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0155949 s, 67.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163437 s, 64.2 MB/s
1+0 records in
1+0 records out
, 0.00172591 s, 608 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171954 s, 61.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0167194 s, 62.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00240043 s, 437 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg133-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92796 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76416 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg133-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg133-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.33@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.33@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.116216 s, 144 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00847161 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00945208 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0201237 s, 52.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0245046 s, 42.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155532 s, 67.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254936 s, 41.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327047 s, 32.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0343251 s, 30.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0353476 s, 29.7 MB/s
, 0.0304913 s, 34.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0351895 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0302291 s, 34.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0396558 s, 26.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0344183 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345125 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0457362 s, 22.9 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b3cba000.lock_unused_count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00904128 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00779098 s, 135 MB/s
1048576 bytes (1.0 MB) copied, 0.00783 s, 134 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00699606 s, 150 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00869644 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00909244 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103138 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00874976 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100853 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105621 s, 99.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119026 s, 88.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112383 s, 93.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110686 s, 94.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012199 s, 86.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121332 s, 86.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126911 s, 82.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg133-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88785 s, 21.5 MB/s
Write runtime is 5 s, speed is 20.000000 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73609 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg133-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg133-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.33@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.131959 s, 127 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155562 s, 67.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0185434 s, 56.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0472438 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0562145 s, 18.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0611706 s, 17.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0598004 s, 17.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0643361 s, 16.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0624863 s, 16.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0620289 s, 16.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0603372 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0664806 s, 15.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0678031 s, 15.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0761354 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0701716 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.069481 s, 15.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0732096 s, 14.3 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b3cba000.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800b3cba000.lock_unused_count=2
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0110587 s, 94.8 MB/s
, 0.00625645 s, 168 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00664023 s, 158 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00497 s, 211 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00505889 s, 207 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00186935 s, 561 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00156476 s, 670 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00753914 s, 139 MB/s
, 0.00559006 s, 188 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00759288 s, 138 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00825939 s, 127 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00279274 s, 375 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179337 s, 58.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0107642 s, 97.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0230664 s, 45.5 MB/s
1048576 bytes (1.0 MB) copied, 0.00474543 s, 221 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg133-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90512 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89061 s, 21.4 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg133-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg133-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.111474 s, 151 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117471 s, 89.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145314 s, 72.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0259937 s, 40.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0259674 s, 40.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0347095 s, 30.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0250836 s, 41.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0256711 s, 40.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0361403 s, 29.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0362473 s, 28.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0338084 s, 31.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0402696 s, 26.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0339377 s, 30.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402905 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0383656 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0465342 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0440276 s, 23.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00914108 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00947497 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106948 s, 98.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00935984 s, 112 MB/s
, 0.00984028 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129449 s, 81.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106357 s, 98.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110872 s, 94.6 MB/s
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0119116 s, 88.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012188 s, 86.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122125 s, 85.9 MB/s
, 0.013778 s, 76.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121651 s, 86.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012908 s, 81.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013891 s, 75.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146436 s, 71.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg133-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74529 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73181 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg133-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg133-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.8875 s, 21.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74251 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg133-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg133-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.119548 s, 140 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00792207 s, 132 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0180551 s, 58.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0116951 s, 89.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00756407 s, 139 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175379 s, 59.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226107 s, 46.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189971 s, 55.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00925591 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108184 s, 96.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0464965 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0453413 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0488713 s, 21.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0473635 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0453481 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0502808 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0565317 s, 18.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00747145 s, 140 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00869839 s, 121 MB/s
, 0.00941083 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00795521 s, 132 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00885904 s, 118 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102141 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00689264 s, 152 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0086976 s, 121 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00912559 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00213742 s, 491 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00891869 s, 118 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102021 s, 103 MB/s
1048576 bytes (1.0 MB) copied, 0.0108217 s, 96.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0102408 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105222 s, 99.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126978 s, 82.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg133-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.643 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4375 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg133-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg133-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.102933 s, 163 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0090907 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121748 s, 86.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149997 s, 69.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0310597 s, 33.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0409256 s, 25.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0390487 s, 26.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0346459 s, 30.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0353827 s, 29.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0484766 s, 21.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0440438 s, 23.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0499505 s, 21.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0532176 s, 19.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0463425 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0479731 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0462663 s, 22.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0506127 s, 20.7 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b3cba000.lock_unused_count=6
ldlm.namespaces.lustre-OST0001-osc-ffff8800b3cba000.lock_unused_count=6
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00216178 s, 485 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0015432 s, 679 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00225222 s, 466 MB/s
, 0.00316754 s, 331 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b3cba000.lock_unused_count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0249861 s, 42.0 MB/s
, 0.0172887 s, 60.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187734 s, 55.9 MB/s
, 0.00128314 s, 817 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00123611 s, 848 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0282592 s, 37.1 MB/s
, 0.0222418 s, 47.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0235985 s, 44.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0166768 s, 62.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0223314 s, 47.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00536606 s, 195 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00481081 s, 218 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg133-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89304 s, 21.4 MB/s
Write runtime is 5 s, speed is 20.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7416 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg133-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
oleg133-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.11825 s, 142 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00857976 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134053 s, 78.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154825 s, 67.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182728 s, 57.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155979 s, 67.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163023 s, 64.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.025466 s, 41.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182662 s, 57.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0247919 s, 42.3 MB/s
, 0.0294878 s, 35.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0389573 s, 26.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0418952 s, 25.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0424134 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275164 s, 38.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0262176 s, 40.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0528849 s, 19.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116657 s, 89.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00999046 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117607 s, 89.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00978275 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114198 s, 91.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00915291 s, 115 MB/s
, 0.0120363 s, 87.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110875 s, 94.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125855 s, 83.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109751 s, 95.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113493 s, 92.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122116 s, 85.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126138 s, 83.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0127184 s, 82.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0127426 s, 82.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013208 s, 79.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg133-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg133-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74351 s, 10.8 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73614 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg133-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg133-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73665 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73262 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg133-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg133-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo