== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 03:06:17 (1733645177)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.0959608 s, 175 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00746037 s, 141 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00648463 s, 162 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169553 s, 61.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182254 s, 57.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0126128 s, 83.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0210068 s, 49.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169189 s, 62.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0244217 s, 42.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134227 s, 78.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0426537 s, 24.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0330269 s, 31.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0346312 s, 30.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380784 s, 27.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0474111 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0416355 s, 25.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0374387 s, 28.0 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a5f1000.lock_unused_count=7
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00774231 s, 135 MB/s
, 0.0076991 s, 136 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00779804 s, 134 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108649 s, 96.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105974 s, 98.9 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106521 s, 98.4 MB/s
, 0.00962284 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00897147 s, 117 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00940987 s, 111 MB/s
1048576 bytes (1.0 MB) copied, 0.0106441 s, 98.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103893 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109969 s, 95.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118612 s, 88.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128673 s, 81.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115102 s, 91.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106523 s, 98.4 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a5f1000.lock_unused_count=3
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg425-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88112 s, 21.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72877 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg425-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg425-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.25@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.25@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.187238 s, 89.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141937 s, 73.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200099 s, 52.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.032864 s, 31.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.045459 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286303 s, 36.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.03431 s, 30.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0444324 s, 23.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0493671 s, 21.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0415375 s, 25.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0396986 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0356719 s, 29.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0452374 s, 23.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327436 s, 32.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.039698 s, 26.4 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a5f1000.lock_unused_count=5
ldlm.namespaces.lustre-OST0001-osc-ffff88012a5f1000.lock_unused_count=7
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0445585 s, 23.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.050548 s, 20.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.108704 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.108083 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109943 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107566 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109402 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.108194 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109491 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110374 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109573 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109458 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109347 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110443 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110292 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110483 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110953 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110512 s, 9.5 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a5f1000.lock_unused_count=2
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg425-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88795 s, 21.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73785 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg425-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg425-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.25@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.10366 s, 162 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00758535 s, 138 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00931248 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203262 s, 51.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260606 s, 40.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.025666 s, 40.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242997 s, 43.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0368428 s, 28.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0383652 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0374358 s, 28.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336984 s, 31.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413791 s, 25.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0456742 s, 23.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.043423 s, 24.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0419164 s, 25.0 MB/s
1048576 bytes (1.0 MB) copied, 0.045794 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.045115 s, 23.2 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a5f1000.lock_unused_count=3
ldlm.namespaces.lustre-OST0001-osc-ffff88012a5f1000.lock_unused_count=3
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00636794 s, 165 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00762705 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00908491 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00803435 s, 131 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00904213 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0094043 s, 111 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00952334 s, 110 MB/s
, 0.00834554 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00896536 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00925459 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00994109 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00991105 s, 106 MB/s
1048576 bytes (1.0 MB) copied, 0.0114674 s, 91.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109885 s, 95.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0107848 s, 97.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106538 s, 98.4 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a5f1000.lock_unused_count=4
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg425-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88331 s, 21.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.87335 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg425-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg425-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.0915281 s, 183 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00956775 s, 110 MB/s
1048576 bytes (1.0 MB) copied, 0.00926586 s, 113 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0124453 s, 84.3 MB/s
, 0.0224576 s, 46.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0349819 s, 30.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0352903 s, 29.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0341922 s, 30.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0296709 s, 35.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0302461 s, 34.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357594 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0314363 s, 33.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0395206 s, 26.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380793 s, 27.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0386261 s, 27.1 MB/s
, 0.0382997 s, 27.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0348788 s, 30.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00606374 s, 173 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00616519 s, 170 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00520288 s, 202 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0070675 s, 148 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00868147 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102127 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0045128 s, 232 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00809929 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00761968 s, 138 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00618828 s, 169 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00902831 s, 116 MB/s
1048576 bytes (1.0 MB) copied, 0.00819896 s, 128 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00850712 s, 123 MB/s
, 0.00350855 s, 299 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00386776 s, 271 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00114061 s, 919 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg425-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73574 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72087 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg425-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg425-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90315 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73588 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg425-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg425-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.135847 s, 124 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103789 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00961641 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146346 s, 71.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155491 s, 67.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01999 s, 52.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0325291 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0375023 s, 28.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0353391 s, 29.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402258 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0428477 s, 24.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.028698 s, 36.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0391165 s, 26.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.036584 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0394952 s, 26.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413436 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0422948 s, 24.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00488628 s, 215 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00521703 s, 201 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0085065 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00759097 s, 138 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0064426 s, 163 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00664869 s, 158 MB/s
, 0.00685441 s, 153 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00732557 s, 143 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00817042 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0054154 s, 194 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.006508 s, 161 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00708543 s, 148 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00761464 s, 138 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0076478 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00767279 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00957152 s, 110 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg425-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.8165 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4242 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg425-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
oleg425-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.0970734 s, 173 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00833033 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012941 s, 81.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0223624 s, 46.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0266599 s, 39.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0355981 s, 29.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0341514 s, 30.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0359281 s, 29.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.031128 s, 33.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0255241 s, 41.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.037891 s, 27.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0338179 s, 31.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0383694 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385337 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336479 s, 31.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0355786 s, 29.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0218186 s, 48.1 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a5f1000.lock_unused_count=2
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00606793 s, 173 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00768129 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00714115 s, 147 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00669766 s, 157 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00748479 s, 140 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101319 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00779192 s, 135 MB/s
1048576 bytes (1.0 MB) copied, 0.00878689 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0094936 s, 110 MB/s
, 0.00886887 s, 118 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00864429 s, 121 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00815497 s, 129 MB/s
, 0.00936781 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0027973 s, 375 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00114486 s, 916 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff88012a5f1000.lock_unused_count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106837 s, 98.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg425-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88372 s, 21.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72087 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg425-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg425-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.0925635 s, 181 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00756949 s, 139 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175881 s, 59.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175996 s, 59.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02354 s, 44.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196135 s, 53.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0262541 s, 39.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186275 s, 56.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242982 s, 43.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0291906 s, 35.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239118 s, 43.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385636 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0437414 s, 24.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0384283 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0497906 s, 21.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0405514 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0408406 s, 25.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00814262 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00650022 s, 161 MB/s
1048576 bytes (1.0 MB) copied, 0.00548314 s, 191 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00827444 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00758508 s, 138 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00726063 s, 144 MB/s
1048576 bytes (1.0 MB) copied, 0.00717575 s, 146 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00821235 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00715527 s, 147 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00675018 s, 155 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00119013 s, 881 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00125977 s, 832 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00129019 s, 813 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00829527 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00930301 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103835 s, 101 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg425-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg425-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73347 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72201 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg425-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg425-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7445 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72282 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg425-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg425-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
pdsh@oleg425-client: oleg425-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo