== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 01:22:26 (1735107746)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.118047 s, 142 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013994 s, 74.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130054 s, 80.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0459352 s, 22.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0399299 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0418617 s, 25.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0562752 s, 18.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0530877 s, 19.8 MB/s
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0531044 s, 19.7 MB/s
, 0.0554674 s, 18.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0458713 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0544332 s, 19.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.053515 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0508451 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0522298 s, 20.1 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800abdac000.lock_unused_count=6
ldlm.namespaces.lustre-OST0001-osc-ffff8800abdac000.lock_unused_count=6
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0504163 s, 20.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0584665 s, 17.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128242 s, 81.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0076531 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011142 s, 94.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0158523 s, 66.1 MB/s
, 0.0139137 s, 75.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146484 s, 71.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00978952 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122056 s, 85.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133599 s, 78.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014211 s, 73.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0161803 s, 64.8 MB/s
, 0.0161537 s, 64.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0164642 s, 63.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0166744 s, 62.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181266 s, 57.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0218204 s, 48.1 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800abdac000.lock_unused_count=4
ldlm.namespaces.lustre-OST0001-osc-ffff8800abdac000.lock_unused_count=1
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88926 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73053 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.38@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.38@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.115374 s, 145 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146422 s, 71.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133495 s, 78.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0343633 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0390805 s, 26.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402982 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0393048 s, 26.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0438537 s, 23.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0533199 s, 19.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0502182 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0513799 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0604107 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0628856 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0535208 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0596603 s, 17.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0623656 s, 16.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0598312 s, 17.5 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800abdac000.lock_unused_count=3
ldlm.namespaces.lustre-OST0001-osc-ffff8800abdac000.lock_unused_count=4
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0780993 s, 13.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0766332 s, 13.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0763069 s, 13.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0805578 s, 13.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0777375 s, 13.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0789432 s, 13.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0794419 s, 13.2 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0800069 s, 13.1 MB/s
, 0.0799969 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0804634 s, 13.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0809551 s, 13.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0822105 s, 12.8 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800abdac000.lock_unused_count=2
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0818077 s, 12.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0830632 s, 12.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0844779 s, 12.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0837856 s, 12.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88615 s, 21.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72851 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.38@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.11915 s, 141 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186865 s, 56.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0231341 s, 45.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345786 s, 30.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0288098 s, 36.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0270745 s, 38.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0392437 s, 26.7 MB/s
, 0.0378508 s, 27.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412725 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412666 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0246208 s, 42.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0364153 s, 28.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0351721 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0419655 s, 25.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0504696 s, 20.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0284515 s, 36.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198245 s, 52.9 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800abdac000.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800abdac000.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107547 s, 97.5 MB/s
, 0.00763032 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0087973 s, 119 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00993624 s, 106 MB/s
, 0.0092386 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135496 s, 77.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0102378 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113199 s, 92.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0114742 s, 91.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114099 s, 91.9 MB/s
, 0.0102968 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115822 s, 90.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121179 s, 86.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0114464 s, 91.6 MB/s
, 0.0114497 s, 91.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140504 s, 74.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89026 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.87804 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.109338 s, 153 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124287 s, 84.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133669 s, 78.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264199 s, 39.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385213 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357652 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412641 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0396056 s, 26.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0329379 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0355246 s, 29.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0414875 s, 25.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0391204 s, 26.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0317892 s, 33.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0331066 s, 31.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0406933 s, 25.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.041902 s, 25.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0382643 s, 27.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00852481 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0087335 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111858 s, 93.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110274 s, 95.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101409 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010142 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00938355 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115175 s, 91.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111865 s, 93.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012889 s, 81.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00172621 s, 607 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118697 s, 88.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0132727 s, 79.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105639 s, 99.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105598 s, 99.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152874 s, 68.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74066 s, 10.8 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73069 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88201 s, 21.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72696 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.10336 s, 162 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101825 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00985691 s, 106 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0297521 s, 35.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0322944 s, 32.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0332321 s, 31.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.031526 s, 33.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328475 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0272794 s, 38.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0354698 s, 29.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0427758 s, 24.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0420614 s, 24.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0457889 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.044943 s, 23.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413812 s, 25.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380695 s, 27.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0436247 s, 24.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0084012 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00611905 s, 171 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102048 s, 103 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00336587 s, 312 MB/s
, 0.00847351 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00807079 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00809852 s, 129 MB/s
1048576 bytes (1.0 MB) copied, 0.00806581 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0092721 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00972224 s, 108 MB/s
, 0.00971975 s, 108 MB/s
1+0 records in
1+0 records out
, 0.0103373 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00867954 s, 121 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0105791 s, 99.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0107909 s, 97.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00963956 s, 109 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.6323 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4287 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.117434 s, 143 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00925896 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014378 s, 72.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0252104 s, 41.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0207829 s, 50.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0262863 s, 39.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0194111 s, 54.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0405484 s, 25.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0302291 s, 34.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0370104 s, 28.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0446627 s, 23.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0508233 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0436412 s, 24.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0453463 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0407225 s, 25.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0387479 s, 27.1 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800abdac000.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800abdac000.lock_unused_count=7
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0726247 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00646401 s, 162 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00158935 s, 660 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00150015 s, 699 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00153028 s, 685 MB/s
, 0.0015457 s, 678 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0137031 s, 76.5 MB/s
, 0.0166763 s, 62.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0086122 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123221 s, 85.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132169 s, 79.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174288 s, 60.2 MB/s
, 0.0171473 s, 61.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00274135 s, 383 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012964 s, 80.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132162 s, 79.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00227005 s, 462 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800abdac000.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800abdac000.lock_unused_count=1
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88952 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72881 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.103206 s, 163 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00904955 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147535 s, 71.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0224409 s, 46.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163875 s, 64.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0244831 s, 42.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0268923 s, 39.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0276341 s, 37.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0355011 s, 29.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0331523 s, 31.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0363363 s, 28.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327839 s, 32.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.031551 s, 33.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0370306 s, 28.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.037651 s, 27.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0434639 s, 24.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0555344 s, 18.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00761657 s, 138 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00849892 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00666929 s, 157 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00975043 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00866594 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00944807 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00885199 s, 118 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00959422 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00880918 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112533 s, 93.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00868639 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00919186 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117574 s, 89.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0094853 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100097 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117858 s, 89.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg238-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg238-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74402 s, 10.8 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72867 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73884 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73083 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg238-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg238-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
pdsh@oleg238-client: oleg238-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo