== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 00:48:34 (1721882914)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.107841 s, 156 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00845505 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103144 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146041 s, 71.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0344278 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0441712 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0429508 s, 24.4 MB/s
, 0.0348658 s, 30.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0373673 s, 28.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0441567 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0401839 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0418053 s, 25.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.051279 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0467374 s, 22.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.04903 s, 21.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0460782 s, 22.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0488658 s, 21.5 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b5882800.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116702 s, 89.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0120158 s, 87.3 MB/s
1048576 bytes (1.0 MB) copied, 0.00322617 s, 325 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144551 s, 72.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142451 s, 73.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137741 s, 76.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0196203 s, 53.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0145768 s, 71.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153752 s, 68.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176912 s, 59.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153676 s, 68.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0180323 s, 58.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162062 s, 64.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165212 s, 63.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186437 s, 56.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.019527 s, 53.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg154-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90692 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72699 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg154-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg154-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.54@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.54@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.0991885 s, 169 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00747315 s, 140 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109246 s, 96.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0287275 s, 36.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0296938 s, 35.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264063 s, 39.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329736 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0319301 s, 32.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0304668 s, 34.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0315426 s, 33.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02768 s, 37.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286684 s, 36.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0280446 s, 37.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.031514 s, 33.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.03475 s, 30.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.033984 s, 30.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0361055 s, 29.0 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b5882800.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800b5882800.lock_unused_count=2
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00778023 s, 135 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00777878 s, 135 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00592785 s, 177 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00650362 s, 161 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00829571 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108459 s, 96.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00956086 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00779134 s, 135 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00962142 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0097969 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00879657 s, 119 MB/s
1048576 bytes (1.0 MB) copied, 0.00910921 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00926418 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100129 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00998923 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114485 s, 91.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg154-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89372 s, 21.4 MB/s
Write runtime is 5 s, speed is 20.000000 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7447 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg154-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg154-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.54@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.181796 s, 92.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106101 s, 98.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168862 s, 62.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286162 s, 36.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0237254 s, 44.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328304 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0415132 s, 25.3 MB/s
, 0.0290508 s, 36.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0322082 s, 32.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.030762 s, 34.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0443115 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0390488 s, 26.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0330091 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0344335 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327783 s, 32.0 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b5882800.lock_unused_count=7
ldlm.namespaces.lustre-OST0001-osc-ffff8800b5882800.lock_unused_count=5
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.037953 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0401422 s, 26.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122299 s, 85.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01019 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103606 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014062 s, 74.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118809 s, 88.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130042 s, 80.6 MB/s
, 0.0120463 s, 87.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115302 s, 90.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012963 s, 80.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0135132 s, 77.6 MB/s
, 0.0131006 s, 80.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129587 s, 80.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137328 s, 76.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140976 s, 74.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0140797 s, 74.5 MB/s
, 0.0142432 s, 73.6 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b5882800.lock_unused_count=2
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg154-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91151 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.87476 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg154-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg154-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.13582 s, 124 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01132 s, 92.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014665 s, 71.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0218179 s, 48.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0393789 s, 26.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0486743 s, 21.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0362906 s, 28.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0559437 s, 18.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0364405 s, 28.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0371197 s, 28.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0464239 s, 22.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0470416 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.049039 s, 21.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0592071 s, 17.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0567522 s, 18.5 MB/s
, 0.0589502 s, 17.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.061573 s, 17.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113506 s, 92.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162458 s, 64.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0155888 s, 67.3 MB/s
, 0.0127723 s, 82.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161563 s, 64.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183033 s, 57.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140338 s, 74.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150253 s, 69.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013924 s, 75.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148787 s, 70.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0156995 s, 66.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0162773 s, 64.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162746 s, 64.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146462 s, 71.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158312 s, 66.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172427 s, 60.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg154-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75467 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73089 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg154-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
oleg154-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90571 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73871 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg154-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg154-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.146659 s, 114 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151919 s, 69.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196489 s, 53.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203594 s, 51.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0217003 s, 48.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228845 s, 45.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152501 s, 68.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0300117 s, 34.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0389931 s, 26.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.040889 s, 25.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365246 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196083 s, 53.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0188892 s, 55.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0316221 s, 33.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0429689 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00807717 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0638435 s, 16.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133998 s, 78.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0182829 s, 57.4 MB/s
, 0.0133891 s, 78.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140732 s, 74.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0178701 s, 58.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162872 s, 64.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187873 s, 55.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170514 s, 61.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184526 s, 56.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0184697 s, 56.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183465 s, 57.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198261 s, 52.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0201856 s, 51.9 MB/s
, 0.0214117 s, 49.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203601 s, 51.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0214933 s, 48.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg154-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4651 s, 5.4 MB/s
Write runtime is 20 s, speed is 5.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4465 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg154-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg154-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.156924 s, 107 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127511 s, 82.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0129295 s, 81.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0210989 s, 49.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260033 s, 40.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0273729 s, 38.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0190082 s, 55.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0194769 s, 53.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0279554 s, 37.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0346679 s, 30.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0240493 s, 43.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239431 s, 43.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144528 s, 72.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0244662 s, 42.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0338676 s, 31.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0288951 s, 36.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0308411 s, 34.0 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b5882800.lock_unused_count=2
ldlm.namespaces.lustre-OST0001-osc-ffff8800b5882800.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00712416 s, 147 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00852227 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00324064 s, 324 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117082 s, 89.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109415 s, 95.8 MB/s
1048576 bytes (1.0 MB) copied, 0.00964083 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00585448 s, 179 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00967649 s, 108 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b5882800.lock_unused_count=2
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00419694 s, 250 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00143092 s, 733 MB/s
1048576 bytes (1.0 MB) copied, 0.00424843 s, 247 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00125865 s, 833 MB/s
1048576 bytes (1.0 MB) copied, 0.0012796 s, 819 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129772 s, 80.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00140025 s, 749 MB/s
1048576 bytes (1.0 MB) copied, 0.00140253 s, 748 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg154-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91325 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7439 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg154-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg154-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.133925 s, 125 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105742 s, 99.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00953878 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189316 s, 55.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0273227 s, 38.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.029242 s, 35.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226417 s, 46.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0285692 s, 36.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189314 s, 55.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0289329 s, 36.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0466533 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413638 s, 25.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0360988 s, 29.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0474338 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.034358 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0386229 s, 27.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.054145 s, 19.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0112725 s, 93.0 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168283 s, 62.3 MB/s
, 0.0110101 s, 95.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116917 s, 89.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170591 s, 61.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174731 s, 60.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140511 s, 74.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143884 s, 72.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158614 s, 66.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0160086 s, 65.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165034 s, 63.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167857 s, 62.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167916 s, 62.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.016314 s, 64.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0172071 s, 60.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187159 s, 56.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg154-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg154-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76338 s, 10.7 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74166 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg154-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg154-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75989 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74792 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg154-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg154-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo