== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 11:05:13 (1734019513)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.117009 s, 143 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103681 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00956514 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0323697 s, 32.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0287222 s, 36.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0370728 s, 28.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0393667 s, 26.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0346907 s, 30.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385919 s, 27.2 MB/s
1048576 bytes (1.0 MB) copied, 0.041164 s, 25.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0338589 s, 31.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0417154 s, 25.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0388383 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0471177 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0450877 s, 23.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380034 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0492658 s, 21.3 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b5c58800.lock_unused_count=3
ldlm.namespaces.lustre-OST0001-osc-ffff8800b5c58800.lock_unused_count=2
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0080962 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133099 s, 78.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121559 s, 86.3 MB/s
, 0.00909568 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146342 s, 71.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148263 s, 70.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104809 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125548 s, 83.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00922106 s, 114 MB/s
1048576 bytes (1.0 MB) copied, 0.00900958 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174278 s, 60.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0199325 s, 52.6 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b5c58800.lock_unused_count=4
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114696 s, 91.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115618 s, 90.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122098 s, 85.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135221 s, 77.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg442-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89115 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73667 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg442-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg442-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.42@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.42@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.113475 s, 148 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105994 s, 98.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148428 s, 70.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0352684 s, 29.7 MB/s
1048576 bytes (1.0 MB) copied, 0.038457 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0347152 s, 30.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0263405 s, 39.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0321489 s, 32.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0434074 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0363056 s, 28.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0398016 s, 26.3 MB/s
, 0.0393541 s, 26.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0469301 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0387888 s, 27.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0352806 s, 29.7 MB/s
, 0.0472886 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0480095 s, 21.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102565 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00801746 s, 131 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00802853 s, 131 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.008159 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100045 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0095972 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117524 s, 89.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100949 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106763 s, 98.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104245 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104077 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0100772 s, 104 MB/s
1048576 bytes (1.0 MB) copied, 0.0101227 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114314 s, 91.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0111229 s, 94.3 MB/s
, 0.00993162 s, 106 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg442-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88947 s, 21.4 MB/s
Write runtime is 5 s, speed is 20.000000 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73854 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg442-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg442-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.42@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.146391 s, 115 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132091 s, 79.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168078 s, 62.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0452962 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0446958 s, 23.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0461175 s, 22.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0466444 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0480444 s, 21.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0393334 s, 26.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0453524 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0445984 s, 23.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0541539 s, 19.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0459518 s, 22.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0592182 s, 17.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0525035 s, 20.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0510205 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0563455 s, 18.6 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b5c58800.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800b5c58800.lock_unused_count=3
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00999758 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127937 s, 82.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00719838 s, 146 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0061976 s, 169 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00691958 s, 152 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00690824 s, 152 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126288 s, 83.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00844069 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115069 s, 91.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00883852 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00881929 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00998295 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184489 s, 56.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.011399 s, 92.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0116071 s, 90.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120294 s, 87.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg442-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91161 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89542 s, 21.4 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg442-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg442-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.096738 s, 173 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00849366 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00733737 s, 143 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0222771 s, 47.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0233636 s, 44.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0291387 s, 36.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0313698 s, 33.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0303605 s, 34.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0309676 s, 33.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0283235 s, 37.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0261744 s, 40.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0236558 s, 44.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0313632 s, 33.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.03442 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0315165 s, 33.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0300135 s, 34.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.032566 s, 32.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00673274 s, 156 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00828561 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010143 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00767546 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0095146 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00952076 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104306 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102561 s, 102 MB/s
, 0.00933079 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100072 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105202 s, 99.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010928 s, 96.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00988376 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123396 s, 85.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010624 s, 98.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117854 s, 89.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg442-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73638 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73466 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg442-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
oleg442-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88805 s, 21.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7585 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg442-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg442-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.144546 s, 116 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103875 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010805 s, 97.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154379 s, 67.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0235199 s, 44.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.033 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0335962 s, 31.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0231697 s, 45.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0270934 s, 38.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275391 s, 38.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380058 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0470424 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0451589 s, 23.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0394094 s, 26.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0467102 s, 22.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0455035 s, 23.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0438877 s, 23.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175655 s, 59.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013064 s, 80.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141532 s, 74.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144761 s, 72.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0198268 s, 52.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0224 s, 46.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161601 s, 64.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192645 s, 54.4 MB/s
, 0.0192711 s, 54.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0197989 s, 53.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0225084 s, 46.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0213538 s, 49.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0178378 s, 58.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.022025 s, 47.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203156 s, 51.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239217 s, 43.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg442-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4673 s, 5.4 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4438 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg442-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg442-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.145416 s, 115 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103716 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00929376 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0256403 s, 40.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0199589 s, 52.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0271517 s, 38.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0325296 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0216465 s, 48.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0202703 s, 51.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0290661 s, 36.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380471 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.041419 s, 25.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0351401 s, 29.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0430446 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0337427 s, 31.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0343589 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412478 s, 25.4 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b5c58800.lock_unused_count=7
ldlm.namespaces.lustre-OST0001-osc-ffff8800b5c58800.lock_unused_count=7
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00180301 s, 582 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0214853 s, 48.8 MB/s
, 0.0247896 s, 42.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00138312 s, 758 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0240411 s, 43.6 MB/s
, 0.0193606 s, 54.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174939 s, 59.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0166221 s, 63.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161329 s, 65.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011395 s, 92.0 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b5c58800.lock_unused_count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106359 s, 98.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132033 s, 79.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00956422 s, 110 MB/s
, 0.0105174 s, 99.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00966309 s, 109 MB/s
, 0.00825546 s, 127 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg442-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90697 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7535 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg442-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg442-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.17498 s, 95.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133795 s, 78.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181271 s, 57.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0213892 s, 49.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281041 s, 37.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0405706 s, 25.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0469255 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.046263 s, 22.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0457714 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402188 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0447315 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.053916 s, 19.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0500035 s, 21.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0578094 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0529946 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0631327 s, 16.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0491919 s, 21.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102941 s, 102 MB/s
1048576 bytes (1.0 MB) copied, 0.0114607 s, 91.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137287 s, 76.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017103 s, 61.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149745 s, 70.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148429 s, 70.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0144926 s, 72.4 MB/s
, 0.0145647 s, 72.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147349 s, 71.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173653 s, 60.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0169881 s, 61.7 MB/s
1+0 records in
1+0 records out
, 0.0168751 s, 62.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0154602 s, 67.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171839 s, 61.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017494 s, 59.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184385 s, 56.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg442-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg442-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77537 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74722 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg442-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg442-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74859 s, 10.8 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73138 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg442-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
oleg442-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg442-client: oleg442-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo