== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 07:54:24 (1740488064)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.149539 s, 112 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0220574 s, 47.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.020709 s, 50.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0503697 s, 20.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.049272 s, 21.3 MB/s
1048576 bytes (1.0 MB) copied, 0.033166 s, 31.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0534594 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.051356 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0609846 s, 17.2 MB/s
, 0.046416 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0587674 s, 17.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0543147 s, 19.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0525697 s, 19.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0549355 s, 19.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0614766 s, 17.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0681321 s, 15.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0553966 s, 18.9 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b62fc000.lock_unused_count=5
ldlm.namespaces.lustre-OST0001-osc-ffff8800b62fc000.lock_unused_count=2
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00935503 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00805336 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00285572 s, 367 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0125948 s, 83.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00233988 s, 448 MB/s
, 0.00942257 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00380162 s, 276 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00928929 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00945703 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104115 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00968656 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101024 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100655 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011559 s, 90.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122292 s, 85.7 MB/s
, 0.0114229 s, 91.8 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b62fc000.lock_unused_count=1
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg155-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89499 s, 21.4 MB/s
Write runtime is 5 s, speed is 20.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74261 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg155-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg155-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.55@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.55@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.156173 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0097225 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165454 s, 63.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0337038 s, 31.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0358425 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0269464 s, 38.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0230434 s, 45.5 MB/s
, 0.0420924 s, 24.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.037486 s, 28.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.03302 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0289054 s, 36.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.039473 s, 26.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0252518 s, 41.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0387273 s, 27.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0316428 s, 33.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0424297 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.049464 s, 21.2 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b62fc000.lock_unused_count=2
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00247698 s, 423 MB/s
, 0.015119 s, 69.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123839 s, 84.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0145507 s, 72.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0212586 s, 49.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00586802 s, 179 MB/s
1048576 bytes (1.0 MB) copied, 0.0167455 s, 62.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00233679 s, 449 MB/s
1048576 bytes (1.0 MB) copied, 0.00244441 s, 429 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00228224 s, 459 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00280822 s, 373 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.017752 s, 59.1 MB/s
1048576 bytes (1.0 MB) copied, 0.00927019 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00884153 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0215577 s, 48.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00238469 s, 440 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg155-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92156 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74571 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg155-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg155-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.55@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.166692 s, 101 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169498 s, 61.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0221549 s, 47.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0233164 s, 45.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184548 s, 56.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123588 s, 84.8 MB/s
, 0.0101899 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149327 s, 70.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0344086 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0263614 s, 39.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0530659 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.053084 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0399824 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0399534 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.044604 s, 23.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0314923 s, 33.3 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff8800b62fc000.lock_unused_count=7
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0291483 s, 36.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00931371 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00143951 s, 728 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00815827 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124544 s, 84.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157512 s, 66.6 MB/s
, 0.0114004 s, 92.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00890939 s, 118 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135337 s, 77.5 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00685387 s, 153 MB/s
, 0.00351513 s, 298 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00399972 s, 262 MB/s
1048576 bytes (1.0 MB) copied, 0.0105071 s, 99.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00116633 s, 899 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00131912 s, 795 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00150425 s, 697 MB/s
, 0.00373539 s, 281 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b62fc000.lock_unused_count=1
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg155-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91258 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89337 s, 21.4 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg155-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg155-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.172954 s, 97.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226082 s, 46.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0294847 s, 35.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0311505 s, 33.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.028678 s, 36.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0213939 s, 49.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0205301 s, 51.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0160984 s, 65.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0256436 s, 40.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0398162 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0358811 s, 29.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172567 s, 60.8 MB/s
1048576 bytes (1.0 MB) copied, 0.00897628 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0576732 s, 18.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0563528 s, 18.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0559874 s, 18.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0323147 s, 32.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112791 s, 93.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112212 s, 93.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00955925 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00314993 s, 333 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149172 s, 70.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173739 s, 60.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00292786 s, 358 MB/s
1048576 bytes (1.0 MB) copied, 0.00438266 s, 239 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151262 s, 69.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00274914 s, 381 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015536 s, 67.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148379 s, 70.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167386 s, 62.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149359 s, 70.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163664 s, 64.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0202223 s, 51.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg155-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76802 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74027 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg155-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg155-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89185 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74757 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg155-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg155-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.14154 s, 119 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013053 s, 80.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118481 s, 88.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0382797 s, 27.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0233332 s, 44.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0341774 s, 30.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0312064 s, 33.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0421964 s, 24.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328314 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0473314 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0489787 s, 21.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0492218 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.046395 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0477524 s, 22.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0479667 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0555572 s, 18.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0559351 s, 18.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119717 s, 87.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109406 s, 95.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146846 s, 71.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105096 s, 99.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00955581 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00951168 s, 110 MB/s
1048576 bytes (1.0 MB) copied, 0.00979826 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105515 s, 99.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113955 s, 92.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114802 s, 91.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116203 s, 90.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129255 s, 81.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130736 s, 80.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132693 s, 79.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134461 s, 78.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151903 s, 69.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg155-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4556 s, 5.4 MB/s
Write runtime is 20 s, speed is 5.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4368 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg155-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg155-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.153442 s, 109 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136725 s, 76.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0194797 s, 53.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228936 s, 45.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385607 s, 27.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0489972 s, 21.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0471606 s, 22.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0423937 s, 24.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.051658 s, 20.3 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0531866 s, 19.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0432829 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.044495 s, 23.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0499609 s, 21.0 MB/s
1+0 records in
1+0 records out
, 0.0606737 s, 17.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0453921 s, 23.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0436917 s, 24.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0507688 s, 20.7 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff8800b62fc000.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139872 s, 75.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142072 s, 73.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146096 s, 71.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128333 s, 81.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0158902 s, 66.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192188 s, 54.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0159521 s, 65.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155344 s, 67.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169918 s, 61.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162304 s, 64.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146779 s, 71.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125962 s, 83.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014406 s, 72.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150103 s, 69.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152329 s, 68.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163528 s, 64.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg155-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91527 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74171 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg155-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg155-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.112218 s, 150 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00730032 s, 144 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00830986 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130394 s, 80.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157418 s, 66.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0276268 s, 38.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0263568 s, 39.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.030437 s, 34.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0346033 s, 30.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0325585 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0263057 s, 39.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0375584 s, 27.9 MB/s
, 0.031579 s, 33.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0296606 s, 35.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412221 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0375455 s, 27.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0422863 s, 24.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00644431 s, 163 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00762174 s, 138 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102166 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00773331 s, 136 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00826038 s, 127 MB/s
1048576 bytes (1.0 MB) copied, 0.00970264 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00815653 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00921692 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106029 s, 98.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00957095 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00973289 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00930057 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105893 s, 99.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106803 s, 98.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0108272 s, 96.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0097811 s, 107 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg155-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg155-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73462 s, 10.8 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72408 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg155-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg155-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76271 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg155-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg155-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
pdsh@oleg155-client: oleg155-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo