== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 07:30:05 (1746617405)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.239348 s, 70.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121679 s, 86.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0251144 s, 41.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0274426 s, 38.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0267965 s, 39.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0277757 s, 37.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402365 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365444 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0342938 s, 30.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0325311 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0379111 s, 27.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0516849 s, 20.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0459349 s, 22.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0625351 s, 16.8 MB/s
, 0.0677903 s, 15.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0565479 s, 18.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0699786 s, 15.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00697815 s, 150 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00666322 s, 157 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107466 s, 97.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00726448 s, 144 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00965352 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0098295 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00870068 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00878878 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012066 s, 86.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00841094 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00860743 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00834577 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00866265 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0089304 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00979881 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106417 s, 98.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg444-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92324 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72649 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.44@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.44@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.253004 s, 66.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104151 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113646 s, 92.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254462 s, 41.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0226266 s, 46.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0279304 s, 37.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0190196 s, 55.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0321185 s, 32.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0426278 s, 24.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0459724 s, 22.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0481278 s, 21.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0675281 s, 15.5 MB/s
, 0.0738859 s, 14.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.278203 s, 3.8 MB/s
, 0.26636 s, 3.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.317838 s, 3.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.316051 s, 3.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00815082 s, 129 MB/s
, 0.00813759 s, 129 MB/s
, 0.00998362 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00812702 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107179 s, 97.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00909702 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00959439 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103538 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120651 s, 86.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109875 s, 95.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119809 s, 87.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115907 s, 90.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124244 s, 84.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012267 s, 85.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132771 s, 79.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139603 s, 75.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg444-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92554 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72768 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.44@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.230498 s, 72.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120694 s, 86.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00771021 s, 136 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0227979 s, 46.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0273565 s, 38.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0166372 s, 63.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0301933 s, 34.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167731 s, 62.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0201444 s, 52.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0420263 s, 25.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385911 s, 27.2 MB/s
, 0.0227287 s, 46.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0435904 s, 24.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0377754 s, 27.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0517348 s, 20.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0501318 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0454274 s, 23.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00687939 s, 152 MB/s
1048576 bytes (1.0 MB) copied, 0.00620284 s, 169 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00678807 s, 154 MB/s
, 0.00900058 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109386 s, 95.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00421534 s, 249 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00829364 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100491 s, 104 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00533487 s, 197 MB/s
, 0.0110904 s, 94.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00922207 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00777364 s, 135 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00930991 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116086 s, 90.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0117766 s, 89.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105799 s, 99.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg444-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92192 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.87579 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.427346 s, 39.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133288 s, 78.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0214685 s, 48.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.026778 s, 39.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0296694 s, 35.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281899 s, 37.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0246211 s, 42.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.024754 s, 42.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0276593 s, 37.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.077872 s, 13.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0957213 s, 11.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0864767 s, 12.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0982855 s, 10.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.339566 s, 3.1 MB/s
, 0.337671 s, 3.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.39078 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.384913 s, 2.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00838639 s, 125 MB/s
, 0.00927537 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00937094 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101583 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103559 s, 101 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0134125 s, 78.2 MB/s
, 0.0113374 s, 92.5 MB/s
, 0.00577868 s, 181 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106928 s, 98.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00153096 s, 685 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116214 s, 90.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119192 s, 88.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117919 s, 88.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127394 s, 82.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125405 s, 83.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142942 s, 73.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg444-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77599 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73165 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.9071 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72787 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.839685 s, 20.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00772243 s, 136 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140019 s, 74.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0311628 s, 33.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0314989 s, 33.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328194 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239923 s, 43.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0320579 s, 32.7 MB/s
, 0.0300352 s, 34.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.17211 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.174439 s, 6.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.187182 s, 5.6 MB/s
1048576 bytes (1.0 MB) copied, 0.186174 s, 5.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.192106 s, 5.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.189421 s, 5.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.203921 s, 5.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.207094 s, 5.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00734496 s, 143 MB/s
, 0.00747408 s, 140 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00858303 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00794026 s, 132 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00872456 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109842 s, 95.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00821194 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00770938 s, 136 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00820279 s, 128 MB/s
, 0.00790382 s, 133 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00814306 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00822994 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00893501 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00985521 s, 106 MB/s
, 0.00885529 s, 118 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010827 s, 96.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg444-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.8275 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4284 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.228893 s, 73.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112788 s, 93.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0280292 s, 37.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0220625 s, 47.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242359 s, 43.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254433 s, 41.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0323485 s, 32.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0291485 s, 36.0 MB/s
1048576 bytes (1.0 MB) copied, 0.029199 s, 35.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0314138 s, 33.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.052061 s, 20.1 MB/s
, 0.0514834 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.047665 s, 22.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0601315 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0611354 s, 17.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0501974 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0540021 s, 19.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00657586 s, 159 MB/s
, 0.00795581 s, 132 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00893813 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00745559 s, 141 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00766075 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0012268 s, 855 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00990215 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00923738 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00947489 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110433 s, 95.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.01083 s, 96.8 MB/s
, 0.00984721 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010055 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0111829 s, 93.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0105821 s, 99.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124861 s, 84.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg444-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91538 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72891 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.428134 s, 39.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0065507 s, 160 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0185481 s, 56.5 MB/s
, 0.0184553 s, 56.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109629 s, 95.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151776 s, 69.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015336 s, 68.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0208503 s, 50.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162565 s, 64.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0751111 s, 14.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0876321 s, 12.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0933407 s, 11.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0967688 s, 10.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.333386 s, 3.1 MB/s
, 0.33381 s, 3.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.377292 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.375411 s, 2.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0101133 s, 104 MB/s
, 0.00822302 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00702776 s, 149 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100718 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00845726 s, 124 MB/s
1048576 bytes (1.0 MB) copied, 0.00859761 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00956348 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100585 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103053 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103769 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0087585 s, 120 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00916856 s, 114 MB/s
1048576 bytes (1.0 MB) copied, 0.0097021 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00941597 s, 111 MB/s
1048576 bytes (1.0 MB) copied, 0.0101934 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141034 s, 74.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg444-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg444-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77682 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73252 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77149 s, 10.7 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7404 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo