== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 08:13:52 (1741695232)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.291062 s, 57.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017638 s, 59.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0230335 s, 45.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0395764 s, 26.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0483241 s, 21.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109217 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0984845 s, 10.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.15212 s, 6.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.164175 s, 6.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.14002 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.146805 s, 7.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.137881 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.169463 s, 6.2 MB/s
1048576 bytes (1.0 MB) copied, 0.166296 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.13968 s, 7.5 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6381800.lock_unused_count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.145808 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.183708 s, 5.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00194771 s, 538 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6381800.lock_unused_count=3
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00211752 s, 495 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0964073 s, 10.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0709143 s, 14.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0563254 s, 18.6 MB/s
, 0.0881587 s, 11.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0472087 s, 22.2 MB/s
1048576 bytes (1.0 MB) copied, 0.00300036 s, 349 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0411027 s, 25.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0363984 s, 28.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.101316 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0773482 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0295255 s, 35.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0262908 s, 39.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129434 s, 81.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114744 s, 91.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg336-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.05162 s, 20.8 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81201 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg336-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
oleg336-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.36@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.36@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.227982 s, 73.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0216983 s, 48.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0378395 s, 27.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.082872 s, 12.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.114293 s, 9.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106093 s, 9.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.116622 s, 9.0 MB/s
, 0.0951569 s, 11.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.111916 s, 9.4 MB/s
1048576 bytes (1.0 MB) copied, 0.101227 s, 10.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0989352 s, 10.6 MB/s
, 0.100444 s, 10.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0931515 s, 11.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0992507 s, 10.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.129246 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.121659 s, 8.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102346 s, 10.2 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6381800.lock_unused_count=7
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6381800.lock_unused_count=7
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.03524 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0355003 s, 29.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0353957 s, 29.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0307971 s, 34.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0348821 s, 30.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0250301 s, 41.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286431 s, 36.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0291854 s, 35.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0296314 s, 35.4 MB/s
1048576 bytes (1.0 MB) copied, 0.047113 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0320478 s, 32.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0324366 s, 32.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0330076 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0349683 s, 30.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0344638 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0407351 s, 25.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg336-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.97404 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76757 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg336-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg336-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.36@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.288964 s, 58.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133353 s, 78.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.023985 s, 43.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0383172 s, 27.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0318677 s, 32.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0358911 s, 29.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0614038 s, 17.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0628454 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0493358 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0664711 s, 15.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0779653 s, 13.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0969252 s, 10.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.116664 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0779223 s, 13.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.031104 s, 33.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copiedldlm.namespaces.lustre-OST0000-osc-ffff8800b6381800.lock_unused_count=4
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6381800.lock_unused_count=1
, 0.120136 s, 8.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.13804 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0383803 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380483 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0338026 s, 31.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0401498 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016582 s, 63.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0406551 s, 25.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0230559 s, 45.5 MB/s
1+0 records in
1+0 records out
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6381800.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6381800.lock_unused_count=5
1048576 bytes (1.0 MB) copied, 0.0319553 s, 32.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0241791 s, 43.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253175 s, 41.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0276935 s, 37.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286286 s, 36.6 MB/s
, 0.0276854 s, 37.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327148 s, 32.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0331108 s, 31.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0390908 s, 26.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg336-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.99253 s, 21.0 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.02826 s, 20.9 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg336-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
oleg336-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.54997 s, 30.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329975 s, 31.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0330514 s, 31.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0835784 s, 12.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0857571 s, 12.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.104947 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.121144 s, 8.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.126028 s, 8.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.108819 s, 9.6 MB/s
, 0.110834 s, 9.5 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6381800.lock_unused_count=7
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6381800.lock_unused_count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.159921 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.135423 s, 7.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.147884 s, 7.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.16766 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.176024 s, 6.0 MB/s
1048576 bytes (1.0 MB) copied, 0.170767 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.188169 s, 5.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00516255 s, 203 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0593805 s, 17.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0442571 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0464865 s, 22.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0477877 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0637843 s, 16.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0991027 s, 10.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0831355 s, 12.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00721059 s, 145 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.120291 s, 8.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105588 s, 9.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106177 s, 9.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.051217 s, 20.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0908373 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0373071 s, 28.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0392025 s, 26.7 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6381800.lock_unused_count=2
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg336-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.84257 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8847 s, 10.6 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg336-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
oleg336-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.52546 s, 19.0 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.0089 s, 10.5 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg336-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
oleg336-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.282157 s, 59.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0584123 s, 18.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0771479 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105877 s, 9.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.127787 s, 8.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.165037 s, 6.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.153305 s, 6.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.190105 s, 5.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.179624 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.202167 s, 5.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.198869 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.134325 s, 7.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.176222 s, 6.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.213667 s, 4.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.21593 s, 4.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.211154 s, 5.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.218462 s, 4.8 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6381800.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189504 s, 55.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0271072 s, 38.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0235629 s, 44.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00467405 s, 224 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00750028 s, 140 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00555512 s, 189 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00779982 s, 134 MB/s
, 0.0111646 s, 93.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0576534 s, 18.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0447118 s, 23.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0527168 s, 19.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0540258 s, 19.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0654387 s, 16.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0648114 s, 16.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.05039 s, 20.8 MB/s
, 0.0191245 s, 54.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg336-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.6391 s, 5.3 MB/s
Write runtime is 20 s, speed is 5.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.5556 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg336-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
oleg336-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.339667 s, 49.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0632785 s, 16.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0609137 s, 17.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.166148 s, 6.3 MB/s
1048576 bytes (1.0 MB) copied, 0.164046 s, 6.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.178024 s, 5.9 MB/s
1048576 bytes (1.0 MB) copied, 0.169762 s, 6.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.207118 s, 5.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.233536 s, 4.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.198465 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.253034 s, 4.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.268006 s, 3.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.259432 s, 4.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.254786 s, 4.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.278407 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.299868 s, 3.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.317193 s, 3.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.117146 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107048 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110011 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0309612 s, 33.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.122841 s, 8.5 MB/s
, 0.0715799 s, 14.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0796373 s, 13.2 MB/s
, 0.0363945 s, 28.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0311908 s, 33.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0341117 s, 30.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380803 s, 27.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0356324 s, 29.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0429908 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0457535 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0449641 s, 23.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0452819 s, 23.2 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6381800.lock_unused_count=4
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg336-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.13541 s, 20.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.87846 s, 10.6 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg336-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
oleg336-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.326682 s, 51.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0616396 s, 17.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0417331 s, 25.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0780542 s, 13.4 MB/s
, 0.0529505 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.139177 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.119209 s, 8.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.12241 s, 8.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.166782 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.145329 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.177709 s, 5.9 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6381800.lock_unused_count=5
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.147824 s, 7.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.216045 s, 4.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.177489 s, 5.9 MB/s
, 0.155412 s, 6.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.154739 s, 6.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.14254 s, 7.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00648289 s, 162 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6381800.lock_unused_count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00206988 s, 507 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0020364 s, 515 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110913 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.140779 s, 7.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0994485 s, 10.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.121342 s, 8.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.093444 s, 11.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0791668 s, 13.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0612104 s, 17.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0561628 s, 18.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0508048 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0502561 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0440489 s, 23.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0392215 s, 26.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0362362 s, 28.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg336-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg336-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.87864 s, 10.6 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.83732 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg336-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
oleg336-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8765 s, 10.6 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.84763 s, 10.6 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg336-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
oleg336-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg336-client: oleg336-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo