== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 03:50:43 (1727769043)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.213141 s, 78.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0266361 s, 39.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253228 s, 41.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0683381 s, 15.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0681469 s, 15.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.049635 s, 21.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0825125 s, 12.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0836649 s, 12.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0954662 s, 11.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.104856 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.123105 s, 8.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.127365 s, 8.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.117034 s, 9.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.119111 s, 8.8 MB/s
1048576 bytes (1.0 MB) copied, 0.108866 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138981 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.130561 s, 8.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0591902 s, 17.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0605311 s, 17.3 MB/s
, 0.00670625 s, 156 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00832521 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.065588 s, 16.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00266156 s, 394 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152713 s, 68.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0700229 s, 15.0 MB/s
1048576 bytes (1.0 MB) copied, 0.00678772 s, 154 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0407438 s, 25.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0579622 s, 18.1 MB/s
, 0.0433502 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0611706 s, 17.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0493277 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0509187 s, 20.6 MB/s
1+0 records in
1+0 records out
ldlm.namespaces.lustre-OST0000-osc-ffff8800ad0cc800.lock_unused_count=8
ldlm.namespaces.lustre-OST0001-osc-ffff8800ad0cc800.lock_unused_count=2
1048576 bytes (1.0 MB) copied, 0.0926343 s, 11.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.99226 s, 21.0 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.83459 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.38@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.38@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.177027 s, 94.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0238867 s, 43.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0325562 s, 32.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0384161 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0274917 s, 38.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0233634 s, 44.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133265 s, 78.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0346441 s, 30.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0491814 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0321736 s, 32.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.063489 s, 16.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0499809 s, 21.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.101489 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0910305 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0951233 s, 11.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.08406 s, 12.5 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800ad0cc800.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800ad0cc800.lock_unused_count=5
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.104588 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124264 s, 84.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0222182 s, 47.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328983 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.021661 s, 48.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0537275 s, 19.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0164036 s, 63.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145594 s, 72.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00679601 s, 154 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0018304 s, 573 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0426762 s, 24.6 MB/s
, 0.0359012 s, 29.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0258827 s, 40.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0224079 s, 46.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175953 s, 59.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0341852 s, 30.7 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800ad0cc800.lock_unused_count=2
1048576 bytes (1.0 MB) copied, 0.0517196 s, 20.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.9579 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.83301 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.38@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.210518 s, 79.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0305585 s, 34.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0311716 s, 33.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0227078 s, 46.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253064 s, 41.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336388 s, 31.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0322936 s, 32.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0565566 s, 18.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0914551 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.104144 s, 10.1 MB/s
, 0.102746 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106956 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0978141 s, 10.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.122577 s, 8.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.153718 s, 6.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.164172 s, 6.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.147904 s, 7.1 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800ad0cc800.lock_unused_count=3
ldlm.namespaces.lustre-OST0001-osc-ffff8800ad0cc800.lock_unused_count=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0367917 s, 28.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00176282 s, 595 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800ad0cc800.lock_unused_count=2
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00816101 s, 128 MB/s
1048576 bytes (1.0 MB) copied, 0.00681569 s, 154 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0982841 s, 10.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0998819 s, 10.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00273531 s, 383 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.08675 s, 12.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0869293 s, 12.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.073469 s, 14.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0740285 s, 14.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0737008 s, 14.2 MB/s
, 0.0624887 s, 16.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0596099 s, 17.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0368766 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0419153 s, 25.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.00318 s, 21.0 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93951 s, 21.2 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.166535 s, 101 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186553 s, 56.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119135 s, 88.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0201967 s, 51.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0586404 s, 17.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0579103 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0654906 s, 16.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0610896 s, 17.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0640569 s, 16.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0670882 s, 15.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.086655 s, 12.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0761876 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0779638 s, 13.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0801159 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0930042 s, 11.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0848282 s, 12.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.101318 s, 10.3 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800ad0cc800.lock_unused_count=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0455783 s, 23.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0381774 s, 27.5 MB/s
1048576 bytes (1.0 MB) copied, 0.04444 s, 23.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0419411 s, 25.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0398862 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0397437 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0406999 s, 25.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0458086 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0405596 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0424053 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0409627 s, 25.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0399485 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0541317 s, 19.4 MB/s
, 0.0404097 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0482454 s, 21.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0513728 s, 20.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80158 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81467 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96079 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82402 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.137307 s, 122 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0098975 s, 106 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198206 s, 52.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196339 s, 53.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0135821 s, 77.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192894 s, 54.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0219437 s, 47.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130679 s, 80.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182998 s, 57.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254173 s, 41.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0343548 s, 30.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402673 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0465651 s, 22.5 MB/s
, 0.0532421 s, 19.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0310702 s, 33.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0634049 s, 16.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0518262 s, 20.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0207639 s, 50.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0315132 s, 33.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.023392 s, 44.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.024442 s, 42.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0247227 s, 42.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253367 s, 41.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0269665 s, 38.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0267778 s, 39.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357338 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0312063 s, 33.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0313551 s, 33.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327548 s, 32.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0366495 s, 28.6 MB/s
, 0.0365641 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.038005 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0415978 s, 25.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4944 s, 5.4 MB/s
Write runtime is 20 s, speed is 5.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4793 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.195328 s, 85.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142738 s, 73.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.021672 s, 48.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0259239 s, 40.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0313611 s, 33.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0510326 s, 20.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0358938 s, 29.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0537827 s, 19.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226087 s, 46.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0515836 s, 20.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0386485 s, 27.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413444 s, 25.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0474133 s, 22.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0371987 s, 28.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0268105 s, 39.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328983 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0512691 s, 20.5 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800ad0cc800.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00334743 s, 313 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175438 s, 59.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015949 s, 65.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0159509 s, 65.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0238284 s, 44.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.021818 s, 48.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142896 s, 73.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0211568 s, 49.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152203 s, 68.9 MB/s
, 0.0269482 s, 38.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187281 s, 56.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0287774 s, 36.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0213232 s, 49.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0224843 s, 46.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0212215 s, 49.4 MB/s
1048576 bytes (1.0 MB) copied, 0.022577 s, 46.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.00607 s, 20.9 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.79318 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.189182 s, 88.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142789 s, 73.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176292 s, 59.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402985 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0352313 s, 29.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0378328 s, 27.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0403414 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0455417 s, 23.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0457492 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.061913 s, 16.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.056258 s, 18.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.073165 s, 14.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0755638 s, 13.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0753747 s, 13.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0689643 s, 15.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0758336 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0753739 s, 13.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.018525 s, 56.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200654 s, 52.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0257067 s, 40.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228392 s, 45.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00970258 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0201085 s, 52.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0087159 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145566 s, 72.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00201696 s, 520 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0225733 s, 46.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0436437 s, 24.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0371325 s, 28.2 MB/s
, 0.03687 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00326191 s, 321 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00413341 s, 254 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00184632 s, 568 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80151 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77162 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.78274 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76021 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo