== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 06:13:10 (1734433990)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.136491 s, 123 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114043 s, 91.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114622 s, 91.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0360244 s, 29.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385791 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0332101 s, 31.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0464795 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0452941 s, 23.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.046244 s, 22.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0465737 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.046706 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0467504 s, 22.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0492194 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0463372 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0441979 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0469622 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0516866 s, 20.3 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6329000.lock_unused_count=7
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6329000.lock_unused_count=3
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138465 s, 75.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00917796 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00274414 s, 382 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0195634 s, 53.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200974 s, 52.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0025987 s, 403 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
, 0.0325599 s, 32.2 MB/s
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0139752 s, 75.0 MB/s
, 0.044345 s, 23.6 MB/s
1048576 bytes (1.0 MB) copied, 0.032639 s, 32.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155237 s, 67.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0451966 s, 23.2 MB/s
, 0.0171373 s, 61.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0230476 s, 45.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00334708 s, 313 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.047133 s, 22.2 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6329000.lock_unused_count=3
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg414-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92242 s, 21.3 MB/s
Write runtime is 5 s, speed is 20.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76602 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg414-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg414-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.14@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.14@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.126671 s, 132 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153778 s, 68.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183102 s, 57.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0363947 s, 28.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0494044 s, 21.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0512644 s, 20.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0528261 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0500663 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0510021 s, 20.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0538662 s, 19.5 MB/s
, 0.0546217 s, 19.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0550887 s, 19.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0599318 s, 17.5 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0580982 s, 18.0 MB/s
, 0.0572783 s, 18.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0609558 s, 17.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0635732 s, 16.5 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6329000.lock_unused_count=3
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134571 s, 77.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122744 s, 85.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138683 s, 75.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014035 s, 74.7 MB/s
, 0.016139 s, 65.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0223731 s, 46.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157907 s, 66.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184011 s, 57.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170796 s, 61.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174974 s, 59.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150216 s, 69.8 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6329000.lock_unused_count=2
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171293 s, 61.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151215 s, 69.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0217669 s, 48.2 MB/s
, 0.0159737 s, 65.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165394 s, 63.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg414-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93045 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75851 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg414-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg414-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.14@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.12878 s, 130 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128436 s, 81.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0283952 s, 36.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0278287 s, 37.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0277005 s, 37.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0448485 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0317578 s, 33.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0405473 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0249223 s, 42.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0371647 s, 28.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0481442 s, 21.8 MB/s
, 0.0352871 s, 29.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402028 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286376 s, 36.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0313369 s, 33.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0474493 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0368683 s, 28.4 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6329000.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00960162 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014055 s, 74.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132893 s, 78.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00956096 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150924 s, 69.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0114599 s, 91.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102969 s, 102 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01246 s, 84.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0125134 s, 83.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124973 s, 83.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132535 s, 79.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139268 s, 75.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0139677 s, 75.1 MB/s
, 0.013887 s, 75.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158005 s, 66.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016666 s, 62.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg414-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89324 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88334 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg414-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg414-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.124 s, 135 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120556 s, 87.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122221 s, 85.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148415 s, 70.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0166261 s, 63.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0487418 s, 21.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.043096 s, 24.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0588405 s, 17.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0590551 s, 17.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0472974 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0465665 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.062107 s, 16.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0642481 s, 16.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0630967 s, 16.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0545272 s, 19.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0521081 s, 20.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0545205 s, 19.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104225 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136375 s, 76.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123648 s, 84.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103542 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013445 s, 78.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140218 s, 74.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111648 s, 93.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131389 s, 79.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0130469 s, 80.4 MB/s
1048576 bytes (1.0 MB) copied, 0.012898 s, 81.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156428 s, 67.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0140028 s, 74.9 MB/s
, 0.014532 s, 72.2 MB/s
, 0.0140416 s, 74.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0153421 s, 68.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177753 s, 59.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg414-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75431 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74139 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg414-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
oleg414-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89279 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74217 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg414-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
oleg414-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.111464 s, 151 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012015 s, 87.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130726 s, 80.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0317645 s, 33.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0282963 s, 37.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328131 s, 32.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0381187 s, 27.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0397932 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.036301 s, 28.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0400196 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0466676 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0471477 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0431874 s, 24.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0392243 s, 26.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0433681 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0522819 s, 20.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0465665 s, 22.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010184 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00924065 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00889756 s, 118 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00957868 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010124 s, 104 MB/s
1048576 bytes (1.0 MB) copied, 0.0124984 s, 83.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0095138 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00996273 s, 105 MB/s
, 0.0103047 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100441 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00985597 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105177 s, 99.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00934127 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0105099 s, 99.8 MB/s
1048576 bytes (1.0 MB) copied, 0.00979237 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121966 s, 86.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg414-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.6394 s, 5.3 MB/s
Write runtime is 20 s, speed is 5.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4458 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg414-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
oleg414-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.19176 s, 87.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112031 s, 93.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0319894 s, 32.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0294654 s, 35.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0362936 s, 28.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0384066 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0417011 s, 25.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0310102 s, 33.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0247846 s, 42.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0320935 s, 32.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0388695 s, 27.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0360676 s, 29.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191499 s, 54.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0409135 s, 25.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.04072 s, 25.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0529721 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0484807 s, 21.6 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6329000.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6329000.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128887 s, 81.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134474 s, 78.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013923 s, 75.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165648 s, 63.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146906 s, 71.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0199061 s, 52.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156551 s, 67.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163137 s, 64.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0168495 s, 62.2 MB/s
, 0.0168936 s, 62.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0178654 s, 58.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192894 s, 54.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183883 s, 57.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0186074 s, 56.4 MB/s
, 0.0198245 s, 52.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0197133 s, 53.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg414-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91506 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75802 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg414-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg414-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.129901 s, 129 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122259 s, 85.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119792 s, 87.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01359 s, 77.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0358258 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0375794 s, 27.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0377784 s, 27.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0438874 s, 23.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0473677 s, 22.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0500264 s, 21.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413245 s, 25.4 MB/s
1048576 bytes (1.0 MB) copied, 0.046752 s, 22.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0460535 s, 22.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0546508 s, 19.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.058169 s, 18.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0511096 s, 20.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0525434 s, 20.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0125848 s, 83.3 MB/s
, 0.0161535 s, 64.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132791 s, 79.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133524 s, 78.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141461 s, 74.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144276 s, 72.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148352 s, 70.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165473 s, 63.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133913 s, 78.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135546 s, 77.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144801 s, 72.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175574 s, 59.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159372 s, 65.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016228 s, 64.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0164081 s, 63.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186943 s, 56.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg414-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg414-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76372 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74378 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg414-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg414-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7427 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73613 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg414-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg414-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo