== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 00:41:50 (1733809310)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.12649 s, 133 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122225 s, 85.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0207475 s, 50.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0283982 s, 36.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017972 s, 58.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0224141 s, 46.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0433759 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.040527 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0359254 s, 29.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0374479 s, 28.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0463369 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.045121 s, 23.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149005 s, 70.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0360377 s, 29.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0463088 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.03877 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0504204 s, 20.8 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff88012fec4800.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00850891 s, 123 MB/s
, 0.010625 s, 98.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00570787 s, 184 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137861 s, 76.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00986256 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136815 s, 76.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00963986 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00879219 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00883569 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00921261 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00965663 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109263 s, 96.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111725 s, 93.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113896 s, 92.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118814 s, 88.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128948 s, 81.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89895 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75291 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg218-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg218-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.18@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.18@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.126576 s, 133 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198196 s, 52.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0221864 s, 47.3 MB/s
1048576 bytes (1.0 MB) copied, 0.013113 s, 80.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0395138 s, 26.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0383587 s, 27.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0283208 s, 37.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.046496 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0498934 s, 21.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0641327 s, 16.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0628699 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0630492 s, 16.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0587223 s, 17.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0709881 s, 14.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0585346 s, 17.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0728525 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.077189 s, 13.6 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012fec4800.lock_unused_count=6
ldlm.namespaces.lustre-OST0001-osc-ffff88012fec4800.lock_unused_count=3
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0693 s, 15.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.076254 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.067767 s, 15.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0758577 s, 13.8 MB/s
, 0.0738895 s, 14.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0781601 s, 13.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.069628 s, 15.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0784877 s, 13.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0708067 s, 14.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0711001 s, 14.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.072116 s, 14.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0729877 s, 14.4 MB/s
, 0.0704986 s, 14.9 MB/s
1048576 bytes (1.0 MB) copied, 0.072578 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0738054 s, 14.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0741905 s, 14.1 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff88012fec4800.lock_unused_count=5
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90697 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74131 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg218-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg218-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.18@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.125262 s, 134 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104465 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125887 s, 83.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365572 s, 28.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0517428 s, 20.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0536949 s, 19.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.049395 s, 21.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0544466 s, 19.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0491382 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.047311 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0433459 s, 24.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0605195 s, 17.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0528877 s, 19.8 MB/s
, 0.0450062 s, 23.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0517518 s, 20.3 MB/s
, 0.0464802 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0484439 s, 21.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00922987 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00805347 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00711978 s, 147 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00718679 s, 146 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00817568 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0101085 s, 104 MB/s
1048576 bytes (1.0 MB) copied, 0.0109277 s, 96.0 MB/s
, 0.0090761 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143058 s, 73.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113367 s, 92.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109031 s, 96.2 MB/s
, 0.0110231 s, 95.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107426 s, 97.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0117632 s, 89.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111931 s, 93.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134722 s, 77.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89276 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88776 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg218-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg218-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.126781 s, 132 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102906 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00843726 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153045 s, 68.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0215624 s, 48.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131517 s, 79.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00847776 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0282873 s, 37.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0227645 s, 46.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0358572 s, 29.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0419201 s, 25.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0391946 s, 26.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0427912 s, 24.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0284473 s, 36.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0465827 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02902 s, 36.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0263674 s, 39.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00942412 s, 111 MB/s
, 0.00826035 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00697627 s, 150 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00720455 s, 146 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00809578 s, 130 MB/s
1048576 bytes (1.0 MB) copied, 0.0132956 s, 78.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0083294 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125094 s, 83.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101754 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010479 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107114 s, 97.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0122856 s, 85.3 MB/s
, 0.0123946 s, 84.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0141559 s, 74.1 MB/s
, 0.0145957 s, 71.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0142897 s, 73.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75454 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.737 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg218-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
oleg218-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89483 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73121 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg218-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg218-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.117328 s, 143 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125784 s, 83.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159268 s, 65.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151697 s, 69.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0279448 s, 37.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0297475 s, 35.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0331827 s, 31.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.028886 s, 36.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0339884 s, 30.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0315546 s, 33.2 MB/s
, 0.0372676 s, 28.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0411854 s, 25.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0395308 s, 26.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0433037 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0424277 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0369459 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0435296 s, 24.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110936 s, 94.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00713784 s, 147 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100552 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117931 s, 88.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128405 s, 81.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149559 s, 70.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119698 s, 87.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145489 s, 72.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010706 s, 97.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154262 s, 68.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0098996 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109469 s, 95.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114228 s, 91.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118507 s, 88.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129957 s, 80.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147805 s, 70.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.6388 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4337 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg218-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
oleg218-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.109139 s, 154 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159882 s, 65.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106579 s, 98.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203187 s, 51.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200774 s, 52.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0210214 s, 49.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0302521 s, 34.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281436 s, 37.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0509967 s, 20.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0577743 s, 18.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0351019 s, 29.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0499391 s, 21.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0544718 s, 19.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0438539 s, 23.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0421921 s, 24.9 MB/s
1048576 bytes (1.0 MB) copied, 0.045252 s, 23.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0517912 s, 20.2 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012fec4800.lock_unused_count=5
ldlm.namespaces.lustre-OST0001-osc-ffff88012fec4800.lock_unused_count=6
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00185163 s, 566 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff88012fec4800.lock_unused_count=1
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.018974 s, 55.3 MB/s
, 0.0235821 s, 44.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00177041 s, 592 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0208223 s, 50.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187601 s, 55.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0177415 s, 59.1 MB/s
1048576 bytes (1.0 MB) copied, 0.00188365 s, 557 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281907 s, 37.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0236094 s, 44.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0168729 s, 62.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114886 s, 91.3 MB/s
, 0.0127965 s, 81.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113729 s, 92.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00879974 s, 119 MB/s
, 0.0064116 s, 164 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89531 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73425 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg218-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg218-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.116779 s, 144 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118445 s, 88.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105847 s, 99.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0257847 s, 40.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0313095 s, 33.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0301167 s, 34.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0341008 s, 30.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0335255 s, 31.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0446157 s, 23.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0423038 s, 24.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0482524 s, 21.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385716 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0493386 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0499737 s, 21.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0376584 s, 27.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.047097 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0483907 s, 21.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00993191 s, 106 MB/s
1048576 bytes (1.0 MB) copied, 0.00917964 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124001 s, 84.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105512 s, 99.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114428 s, 91.6 MB/s
, 0.00922605 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117009 s, 89.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118452 s, 88.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158411 s, 66.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0143385 s, 73.1 MB/s
, 0.0143026 s, 73.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012891 s, 81.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133527 s, 78.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142981 s, 73.3 MB/s
, 0.0143641 s, 73.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0164956 s, 63.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg218-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg218-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75197 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73329 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg218-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg218-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73507 s, 10.8 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7264 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg218-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg218-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo