== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 07:40:48 (1734439248)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.133042 s, 126 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101718 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131382 s, 79.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0464432 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0448791 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0494345 s, 21.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0525402 s, 20.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.04726 s, 22.2 MB/s
, 0.0427479 s, 24.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0484834 s, 21.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0491121 s, 21.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0579826 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0574671 s, 18.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0562304 s, 18.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0610422 s, 17.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0514021 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0566662 s, 18.5 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b605d000.lock_unused_count=6
ldlm.namespaces.lustre-OST0001-osc-ffff8800b605d000.lock_unused_count=4
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00827637 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108284 s, 96.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00950529 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147086 s, 71.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107294 s, 97.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138122 s, 75.9 MB/s
, 0.0115388 s, 90.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119478 s, 87.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156156 s, 67.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142166 s, 73.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150741 s, 69.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157683 s, 66.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173379 s, 60.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174139 s, 60.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177194 s, 59.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0195652 s, 53.6 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b605d000.lock_unused_count=4
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89276 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74297 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg144-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
oleg144-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.44@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.44@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.108365 s, 155 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106517 s, 98.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119318 s, 87.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0188597 s, 55.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161715 s, 64.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0271732 s, 38.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0369804 s, 28.4 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357375 s, 29.3 MB/s
, 0.0343736 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.039467 s, 26.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0423161 s, 24.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0379863 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.036141 s, 29.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0448704 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0392465 s, 26.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0437846 s, 23.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0411056 s, 25.5 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b605d000.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff8800b605d000.lock_unused_count=2
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105477 s, 99.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00206462 s, 508 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00342655 s, 306 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00963297 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00977061 s, 107 MB/s
1048576 bytes (1.0 MB) copied, 0.0111975 s, 93.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106091 s, 98.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011468 s, 91.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104578 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128165 s, 81.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0106165 s, 98.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140919 s, 74.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120085 s, 87.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120774 s, 86.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141434 s, 74.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0166137 s, 63.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89506 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73116 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg144-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg144-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.44@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.0981615 s, 171 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117993 s, 88.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119066 s, 88.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00752186 s, 139 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101627 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0282063 s, 37.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0244259 s, 42.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0303106 s, 34.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0406383 s, 25.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0205751 s, 51.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0396784 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0358366 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0391074 s, 26.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0364111 s, 28.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0406041 s, 25.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0334963 s, 31.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0382585 s, 27.4 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b605d000.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00839434 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104395 s, 100 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00458218 s, 229 MB/s
1048576 bytes (1.0 MB) copied, 0.00999798 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136708 s, 76.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126116 s, 83.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00382462 s, 274 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125895 s, 83.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111574 s, 94.0 MB/s
, 0.00665365 s, 158 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105576 s, 99.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0136523 s, 76.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129822 s, 80.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139505 s, 75.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128481 s, 81.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145365 s, 72.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89101 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88461 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg144-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg144-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.102308 s, 164 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00875436 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107324 s, 97.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138142 s, 75.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0374718 s, 28.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402639 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0437379 s, 24.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0314662 s, 33.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0387698 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0478402 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0481503 s, 21.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0457697 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0358166 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0396246 s, 26.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0514739 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0487892 s, 21.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0445165 s, 23.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138049 s, 76.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111501 s, 94.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110909 s, 94.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011889 s, 88.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117879 s, 89.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136951 s, 76.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012774 s, 82.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133569 s, 78.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139904 s, 74.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135974 s, 77.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016703 s, 62.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151168 s, 69.4 MB/s
, 0.0148455 s, 70.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155191 s, 67.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156119 s, 67.2 MB/s
1048576 bytes (1.0 MB) copied, 0.016009 s, 65.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74535 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.735 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg144-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
oleg144-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88406 s, 21.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72926 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg144-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg144-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.105334 s, 159 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100045 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130364 s, 80.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114958 s, 91.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0245426 s, 42.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0261797 s, 40.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015395 s, 68.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02678 s, 39.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.019136 s, 54.8 MB/s
, 0.0216109 s, 48.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0216326 s, 48.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0320624 s, 32.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0317413 s, 33.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0221278 s, 47.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00725484 s, 145 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0481978 s, 21.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0470398 s, 22.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146269 s, 71.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134132 s, 78.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125205 s, 83.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131911 s, 79.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0126062 s, 83.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013598 s, 77.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149317 s, 70.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0164375 s, 63.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0132448 s, 79.2 MB/s
, 0.00273299 s, 384 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00949972 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144411 s, 72.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014697 s, 71.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00262483 s, 399 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149778 s, 70.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176196 s, 59.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.6445 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4356 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg144-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg144-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.111465 s, 151 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0112565 s, 93.2 MB/s
, 0.0126735 s, 82.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275564 s, 38.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260938 s, 40.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0219634 s, 47.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0340482 s, 30.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0386705 s, 27.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0418454 s, 25.1 MB/s
, 0.0384869 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0344249 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0420943 s, 24.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0381552 s, 27.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0425147 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0388554 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0455283 s, 23.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0448905 s, 23.4 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b605d000.lock_unused_count=2
ldlm.namespaces.lustre-OST0001-osc-ffff8800b605d000.lock_unused_count=2
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00571185 s, 184 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00998983 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00660517 s, 159 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112446 s, 93.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00466258 s, 225 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0083946 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00868044 s, 121 MB/s
1048576 bytes (1.0 MB) copied, 0.00870199 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012701 s, 82.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107717 s, 97.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108868 s, 96.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105468 s, 99.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01078 s, 97.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107 s, 98.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107249 s, 97.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00151385 s, 693 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89153 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73338 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg144-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg144-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.115848 s, 145 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00963257 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112593 s, 93.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0217414 s, 48.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0258106 s, 40.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0299514 s, 35.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0266691 s, 39.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0268516 s, 39.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357295 s, 29.3 MB/s
, 0.0293461 s, 35.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0352681 s, 29.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0353981 s, 29.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0291866 s, 35.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0416615 s, 25.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402206 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0323042 s, 32.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365657 s, 28.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00740645 s, 142 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00611836 s, 171 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00623457 s, 168 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100003 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00739128 s, 142 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00900868 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103818 s, 101 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00132565 s, 791 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00971638 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107518 s, 97.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00911668 s, 115 MB/s
1048576 bytes (1.0 MB) copied, 0.0111674 s, 93.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0095442 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104355 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112747 s, 93.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119924 s, 87.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg144-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg144-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74153 s, 10.8 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7304 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg144-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg144-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74391 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7297 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg144-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg144-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
pdsh@oleg144-client: oleg144-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo