== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 12:19:05 (1736788745)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.276695 s, 60.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167353 s, 62.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0235751 s, 44.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0339224 s, 30.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0346148 s, 30.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264149 s, 39.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336827 s, 31.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281002 s, 37.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0351107 s, 29.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172892 s, 60.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0429527 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327074 s, 32.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0267556 s, 39.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0330326 s, 31.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281535 s, 37.2 MB/s
, 0.0352291 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147119 s, 71.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121556 s, 86.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159937 s, 65.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136119 s, 77.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159237 s, 65.8 MB/s
, 0.015565 s, 67.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0170949 s, 61.3 MB/s
, 0.012272 s, 85.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133174 s, 78.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176858 s, 59.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014313 s, 73.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140296 s, 74.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137107 s, 76.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142065 s, 73.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0164845 s, 63.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152979 s, 68.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170977 s, 61.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.00706 s, 20.9 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74504 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.26@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.26@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.233964 s, 71.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00990602 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171038 s, 61.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0201671 s, 52.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0212033 s, 49.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0312303 s, 33.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0295596 s, 35.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0269844 s, 38.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0405009 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0316297 s, 33.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0437151 s, 24.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0407359 s, 25.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0360524 s, 29.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0448954 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0367576 s, 28.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.271339 s, 3.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.270398 s, 3.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00998018 s, 105 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0147959 s, 70.9 MB/s
, 0.0128059 s, 81.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00328968 s, 319 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108734 s, 96.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128079 s, 81.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144702 s, 72.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161801 s, 64.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181739 s, 57.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0176971 s, 59.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0185748 s, 56.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176733 s, 59.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192656 s, 54.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189285 s, 55.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0190058 s, 55.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0199077 s, 52.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.98118 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74536 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.26@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.25585 s, 65.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118663 s, 88.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117156 s, 89.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192929 s, 54.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174392 s, 60.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0269084 s, 39.0 MB/s
, 0.0317847 s, 33.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0520743 s, 20.1 MB/s
, 0.0531232 s, 19.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0390667 s, 26.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0578082 s, 18.1 MB/s
, 0.0533777 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0333704 s, 31.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0428849 s, 24.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.044239 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0706509 s, 14.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0573018 s, 18.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00877203 s, 120 MB/s
, 0.00877248 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00733399 s, 143 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00811409 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00447546 s, 234 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00945397 s, 111 MB/s
1048576 bytes (1.0 MB) copied, 0.00954519 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00964517 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142165 s, 73.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114378 s, 91.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111957 s, 93.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107209 s, 97.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011942 s, 87.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117155 s, 89.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109064 s, 96.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011207 s, 93.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.94633 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88456 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.452945 s, 37.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189763 s, 55.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357861 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0434266 s, 24.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385315 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0439357 s, 23.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0555358 s, 18.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0530078 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0492474 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0455518 s, 23.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0772431 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0658031 s, 15.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0632991 s, 16.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.277797 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.279401 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.308093 s, 3.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.312903 s, 3.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00931048 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00807492 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00862775 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00933849 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104965 s, 99.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102098 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00970865 s, 108 MB/s
1048576 bytes (1.0 MB) copied, 0.0131967 s, 79.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109347 s, 95.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106273 s, 98.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121966 s, 86.0 MB/s
, 0.010799 s, 97.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013831 s, 75.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112209 s, 93.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121776 s, 86.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131912 s, 79.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80179 s, 10.7 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74354 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91958 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74857 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.846247 s, 19.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00778076 s, 135 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02038 s, 51.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0149094 s, 70.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116534 s, 90.0 MB/s
1048576 bytes (1.0 MB) copied, 0.016561 s, 63.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0258226 s, 40.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159285 s, 65.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196524 s, 53.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.18101 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.194467 s, 5.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.192415 s, 5.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.202101 s, 5.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.224322 s, 4.7 MB/s
, 0.227787 s, 4.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.229195 s, 4.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.23147 s, 4.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0140585 s, 74.6 MB/s
, 0.0154761 s, 67.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142533 s, 73.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0190048 s, 55.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163582 s, 64.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168786 s, 62.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179581 s, 58.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144605 s, 72.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01542 s, 68.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153211 s, 68.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0160643 s, 65.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182906 s, 57.3 MB/s
1+0 records in
1+0 records out
, 0.0177181 s, 59.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0158118 s, 66.3 MB/s
, 0.0159785 s, 65.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0208344 s, 50.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7484 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4381 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.274725 s, 61.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143598 s, 73.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.023543 s, 44.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0267236 s, 39.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264604 s, 39.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169805 s, 61.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0294145 s, 35.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0119007 s, 88.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173533 s, 60.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0428229 s, 24.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0470532 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0486968 s, 21.5 MB/s
, 0.015531 s, 67.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.046909 s, 22.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0462358 s, 22.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0523697 s, 20.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0533847 s, 19.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00844142 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00843856 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00722117 s, 145 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00766621 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00876901 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.009025 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00852411 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00892269 s, 118 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142153 s, 73.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112254 s, 93.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134669 s, 77.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114785 s, 91.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00973896 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123996 s, 84.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106602 s, 98.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114773 s, 91.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.99562 s, 21.0 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74809 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.444745 s, 37.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00872803 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00691423 s, 152 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00992864 s, 106 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0320687 s, 32.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017082 s, 61.4 MB/s
, 0.0178642 s, 58.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0262579 s, 39.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.032437 s, 32.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0728811 s, 14.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0909542 s, 11.5 MB/s
, 0.0909416 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0914174 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.231025 s, 4.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.222199 s, 4.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.265448 s, 4.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.270231 s, 3.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0109828 s, 95.5 MB/s
, 0.0126316 s, 83.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163445 s, 64.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00260732 s, 402 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148661 s, 70.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171496 s, 61.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161064 s, 65.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00762913 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157139 s, 66.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143545 s, 73.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014374 s, 72.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147588 s, 71.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161902 s, 64.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159554 s, 65.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016049 s, 65.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174071 s, 60.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg426-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg426-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81942 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74647 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8032 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7459 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo