== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 15:06:31 (1722798391)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.166194 s, 101 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0217069 s, 48.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0251929 s, 41.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140323 s, 74.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0427924 s, 24.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0457444 s, 22.9 MB/s
, 0.0374975 s, 28.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0271915 s, 38.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0463929 s, 22.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0341625 s, 30.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286535 s, 36.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0410831 s, 25.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402668 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0358629 s, 29.2 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b58b5000.lock_unused_count=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0602978 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0805543 s, 13.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.073992 s, 14.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112271 s, 93.4 MB/s
1048576 bytes (1.0 MB) copied, 0.01139 s, 92.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0093175 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00906727 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00764917 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00788563 s, 133 MB/s
1048576 bytes (1.0 MB) copied, 0.00946025 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00945623 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00955764 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00978367 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100831 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102852 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105718 s, 99.2 MB/s
, 0.0101415 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0101746 s, 103 MB/s
1048576 bytes (1.0 MB) copied, 0.0100663 s, 104 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b58b5000.lock_unused_count=5
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg335-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91209 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7432 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg335-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg335-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.35@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.35@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.138944 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142781 s, 73.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0262315 s, 40.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0303606 s, 34.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226868 s, 46.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0284249 s, 36.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.029892 s, 35.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0197269 s, 53.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0415609 s, 25.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00824664 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02374 s, 44.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0567267 s, 18.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.06272 s, 16.7 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b58b5000.lock_unused_count=3
ldlm.namespaces.lustre-OST0001-osc-ffff8800b58b5000.lock_unused_count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0627867 s, 16.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0750821 s, 14.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0568549 s, 18.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0626503 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105216 s, 10.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.107191 s, 9.8 MB/s
, 0.10573 s, 9.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106548 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105807 s, 9.9 MB/s
1048576 bytes (1.0 MB) copied, 0.107466 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106006 s, 9.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106998 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.109748 s, 9.6 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107926 s, 9.7 MB/s
, 0.108627 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.111835 s, 9.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.11115 s, 9.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.11118 s, 9.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110919 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.116536 s, 9.0 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b58b5000.lock_unused_count=2
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg335-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91382 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74219 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg335-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg335-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.35@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.138834 s, 121 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010781 s, 97.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177823 s, 59.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0426635 s, 24.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365865 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0430446 s, 24.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0677721 s, 15.5 MB/s
1048576 bytes (1.0 MB) copied, 0.067976 s, 15.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0620314 s, 16.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0621182 s, 16.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0671264 s, 15.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0573904 s, 18.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.062496 s, 16.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0587131 s, 17.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.065989 s, 15.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0711286 s, 14.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0652174 s, 16.1 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b58b5000.lock_unused_count=3
ldlm.namespaces.lustre-OST0001-osc-ffff8800b58b5000.lock_unused_count=7
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016433 s, 63.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125588 s, 83.5 MB/s
1+0 records in
1+0 records out
, 0.0101116 s, 104 MB/s
1048576 bytes (1.0 MB) copied, 0.0163892 s, 64.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010325 s, 102 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0108378 s, 96.8 MB/s
, 0.0199292 s, 52.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128976 s, 81.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130148 s, 80.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133383 s, 78.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142487 s, 73.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150248 s, 69.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154056 s, 68.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173245 s, 60.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174144 s, 60.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173261 s, 60.5 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b58b5000.lock_unused_count=3
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg335-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91236 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89246 s, 21.4 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg335-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg335-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.149853 s, 112 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101311 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112714 s, 93.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146091 s, 71.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0303701 s, 34.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0291411 s, 36.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0243188 s, 43.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365659 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0337835 s, 31.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0337781 s, 31.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0409572 s, 25.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0378717 s, 27.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0349729 s, 30.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0478126 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0425906 s, 24.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0361821 s, 29.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0404293 s, 25.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177931 s, 58.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014459 s, 72.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147259 s, 71.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147864 s, 70.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0182725 s, 57.4 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0140298 s, 74.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0163482 s, 64.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159081 s, 65.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168621 s, 62.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017784 s, 59.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0178175 s, 58.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174099 s, 60.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191902 s, 54.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0184183 s, 56.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177434 s, 59.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186222 s, 56.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg335-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7621 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74219 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg335-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg335-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90298 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74295 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg335-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg335-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.1605 s, 105 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00950682 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125883 s, 83.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152546 s, 68.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0298212 s, 35.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0287705 s, 36.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0319673 s, 32.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0309075 s, 33.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0351706 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357277 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0377402 s, 27.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0361035 s, 29.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0324616 s, 32.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0335916 s, 31.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0372905 s, 28.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0371175 s, 28.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0386941 s, 27.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0140886 s, 74.4 MB/s
, 0.0166215 s, 63.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123875 s, 84.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124014 s, 84.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0193556 s, 54.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167717 s, 62.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150287 s, 69.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141872 s, 73.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0219891 s, 47.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159171 s, 65.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156245 s, 67.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156087 s, 67.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016213 s, 64.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161381 s, 65.0 MB/s
, 0.0160578 s, 65.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175986 s, 59.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg335-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4644 s, 5.4 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4432 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg335-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg335-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.0897806 s, 187 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00941914 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115471 s, 90.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228309 s, 45.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0257426 s, 40.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0240276 s, 43.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183388 s, 57.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0205148 s, 51.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0301286 s, 34.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0259959 s, 40.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0353271 s, 29.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0334599 s, 31.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0356651 s, 29.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0399268 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0405234 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0346095 s, 30.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357543 s, 29.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.005721 s, 183 MB/s
, 0.00482077 s, 218 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00586488 s, 179 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00908982 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00845632 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00634369 s, 165 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00664587 s, 158 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00711781 s, 147 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0079532 s, 132 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00862021 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00960976 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00969469 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0103605 s, 101 MB/s
, 0.00922249 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00851958 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109293 s, 95.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg335-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89107 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72654 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg335-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg335-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.144102 s, 116 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00744389 s, 141 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00883763 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0244546 s, 42.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151351 s, 69.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286647 s, 36.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0267041 s, 39.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0219812 s, 47.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191572 s, 54.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0312755 s, 33.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336626 s, 31.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0284989 s, 36.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0248867 s, 42.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0379333 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0299154 s, 35.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0343475 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0326072 s, 32.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107048 s, 98.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00206048 s, 509 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00994934 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00837907 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00888481 s, 118 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00906767 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00478171 s, 219 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0112495 s, 93.2 MB/s
, 0.0111587 s, 94.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103077 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010728 s, 97.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124176 s, 84.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00424548 s, 247 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124509 s, 84.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127417 s, 82.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142307 s, 73.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg335-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg335-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75094 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73876 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg335-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg335-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74298 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74191 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg335-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg335-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo