== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 12:15:16 (1740676516)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.118441 s, 142 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116651 s, 89.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153294 s, 68.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016085 s, 65.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0466146 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0475274 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0487996 s, 21.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0560551 s, 18.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0431965 s, 24.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.058978 s, 17.8 MB/s
, 0.0604872 s, 17.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0470685 s, 22.3 MB/s
, 0.0431147 s, 24.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0544576 s, 19.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0511604 s, 20.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0491023 s, 21.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0552976 s, 19.0 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012bf75000.lock_unused_count=5
ldlm.namespaces.lustre-OST0001-osc-ffff88012bf75000.lock_unused_count=6
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107458 s, 97.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00935145 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00961462 s, 109 MB/s
1048576 bytes (1.0 MB) copied, 0.0150336 s, 69.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124792 s, 84.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139629 s, 75.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163909 s, 64.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0132148 s, 79.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125744 s, 83.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128284 s, 81.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142629 s, 73.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143564 s, 73.0 MB/s
, 0.0138813 s, 75.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015041 s, 69.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142275 s, 73.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172378 s, 60.8 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff88012bf75000.lock_unused_count=1
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89178 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73727 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg405-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg405-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.5@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.5@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.108557 s, 155 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00825131 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172615 s, 60.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169685 s, 61.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0206668 s, 50.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175833 s, 59.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0353046 s, 29.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.023248 s, 45.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0291015 s, 36.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329322 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281419 s, 37.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.038465 s, 27.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0644119 s, 16.3 MB/s
, 0.088219 s, 11.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0840859 s, 12.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0702711 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0799506 s, 13.1 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012bf75000.lock_unused_count=5
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0283386 s, 37.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0265165 s, 39.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0274238 s, 38.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0298605 s, 35.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0285875 s, 36.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0289472 s, 36.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.028019 s, 37.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0302903 s, 34.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0297854 s, 35.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0295039 s, 35.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0289879 s, 36.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0305699 s, 34.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0302249 s, 34.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0298596 s, 35.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0299717 s, 35.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327723 s, 32.0 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff88012bf75000.lock_unused_count=1
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89588 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73234 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg405-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg405-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.5@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.117256 s, 143 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0096332 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130726 s, 80.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121007 s, 86.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182855 s, 57.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184938 s, 56.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0298702 s, 35.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0323406 s, 32.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0241139 s, 43.5 MB/s
, 0.0189741 s, 55.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.024245 s, 43.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0373015 s, 28.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0238869 s, 43.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0406869 s, 25.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0110406 s, 95.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0290711 s, 36.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118092 s, 88.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00587948 s, 178 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00858321 s, 122 MB/s
, 0.0083897 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00775599 s, 135 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00927609 s, 113 MB/s
, 0.00919315 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0121002 s, 86.7 MB/s
, 0.0109745 s, 95.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00998994 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00156113 s, 672 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106255 s, 98.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107939 s, 97.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0094695 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0096533 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108456 s, 96.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128349 s, 81.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89284 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.87782 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg405-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg405-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.110402 s, 152 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00938226 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137486 s, 76.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0311351 s, 33.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0381995 s, 27.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0324534 s, 32.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0422418 s, 24.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0344671 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0424167 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0315679 s, 33.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0362013 s, 29.0 MB/s
, 0.0330108 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0371851 s, 28.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0426582 s, 24.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.036858 s, 28.4 MB/s
, 0.0400304 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413422 s, 25.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104325 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00238529 s, 440 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00238287 s, 440 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00286505 s, 366 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117633 s, 89.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119619 s, 87.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00171841 s, 610 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00644932 s, 163 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152568 s, 68.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013625 s, 77.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142064 s, 73.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142945 s, 73.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122567 s, 85.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122147 s, 85.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155607 s, 67.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161263 s, 65.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74684 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73067 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg405-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg405-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88451 s, 21.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7304 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg405-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg405-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.126328 s, 133 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110699 s, 94.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101151 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242805 s, 43.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0331739 s, 31.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
, 0.0182633 s, 57.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0274285 s, 38.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0210808 s, 49.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0369118 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0361174 s, 29.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275532 s, 38.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286216 s, 36.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0363727 s, 28.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0415438 s, 25.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114983 s, 91.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0399127 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0602795 s, 17.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00898924 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00992054 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00857876 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118424 s, 88.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00992446 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106569 s, 98.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107599 s, 97.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108057 s, 97.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105925 s, 99.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0113251 s, 92.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01179 s, 88.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111728 s, 93.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011344 s, 92.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126454 s, 82.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012987 s, 80.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143276 s, 73.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.6393 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4338 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg405-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg405-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.123616 s, 136 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00930577 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129909 s, 80.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327166 s, 32.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0381694 s, 27.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0373886 s, 28.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0388911 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0444978 s, 23.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0439231 s, 23.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0399532 s, 26.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0450784 s, 23.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0428645 s, 24.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0473142 s, 22.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0523079 s, 20.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0463779 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0552276 s, 19.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0439365 s, 23.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00824913 s, 127 MB/s
, 0.00774091 s, 135 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00820413 s, 128 MB/s
1048576 bytes (1.0 MB) copied, 0.0114428 s, 91.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00979493 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111602 s, 94.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110794 s, 94.6 MB/s
1048576 bytes (1.0 MB) copied, 0.00329355 s, 318 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105575 s, 99.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00937789 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0072946 s, 144 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00921741 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00974573 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00961627 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100831 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00294378 s, 356 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89319 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73303 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg405-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg405-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.108055 s, 155 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0101746 s, 103 MB/s
1048576 bytes (1.0 MB) copied, 0.0102001 s, 103 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0304025 s, 34.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0252235 s, 41.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0322307 s, 32.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0378188 s, 27.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0381762 s, 27.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0263932 s, 39.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0266929 s, 39.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.041743 s, 25.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402509 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345876 s, 30.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0364735 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0403387 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0354178 s, 29.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0415019 s, 25.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00819989 s, 128 MB/s
, 0.0095681 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00871346 s, 120 MB/s
1048576 bytes (1.0 MB) copied, 0.010948 s, 95.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00996403 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103819 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104691 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00530868 s, 198 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00708259 s, 148 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00710864 s, 148 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115574 s, 90.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106906 s, 98.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105192 s, 99.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0116576 s, 89.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0105291 s, 99.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125209 s, 83.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg405-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg405-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74491 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73183 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg405-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg405-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74044 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73156 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg405-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg405-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo