== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 17:49:23 (1746136163)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.464445 s, 36.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0397194 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0279983 s, 37.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0348658 s, 30.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.022134 s, 47.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.101652 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.134076 s, 7.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138338 s, 7.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.141137 s, 7.4 MB/s
, 0.146108 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.137994 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.161329 s, 6.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.14454 s, 7.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.178551 s, 5.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.155144 s, 6.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.18588 s, 5.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.205481 s, 5.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0500234 s, 21.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0291247 s, 36.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0301724 s, 34.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0308158 s, 34.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402427 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0338184 s, 31.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0400127 s, 26.2 MB/s
, 0.03838 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0400021 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0417102 s, 25.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413461 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0422697 s, 24.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0466571 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0467469 s, 22.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0514268 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0607152 s, 17.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg125-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 7.1094 s, 14.7 MB/s
Write runtime is 8 s, speed is 12.500000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.84055 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.25@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.25@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.401146 s, 41.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0295086 s, 35.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0428743 s, 24.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149273 s, 70.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0415564 s, 25.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0663475 s, 15.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0783766 s, 13.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0881499 s, 11.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0836645 s, 12.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.140456 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.103065 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.167564 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.154765 s, 6.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.155643 s, 6.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.144254 s, 7.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.137462 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.159075 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0283526 s, 37.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0314711 s, 33.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0320574 s, 32.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0279153 s, 37.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0233558 s, 44.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0315834 s, 33.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0265266 s, 39.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275879 s, 38.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.029599 s, 35.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.029808 s, 35.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0326763 s, 32.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0312153 s, 33.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0309801 s, 33.8 MB/s
, 0.0338652 s, 31.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.03209 s, 32.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0449832 s, 23.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg125-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 7.65455 s, 13.7 MB/s
Write runtime is 9 s, speed is 11.111111 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81233 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.25@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.304482 s, 55.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.026459 s, 39.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0494301 s, 21.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0847318 s, 12.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0916373 s, 11.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.103507 s, 10.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0921159 s, 11.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.128076 s, 8.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107351 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.115487 s, 9.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.127436 s, 8.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138238 s, 7.6 MB/s
1048576 bytes (1.0 MB) copied, 0.160038 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.159287 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.162286 s, 6.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.171165 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.181743 s, 5.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0342663 s, 30.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0424275 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168818 s, 62.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0444513 s, 23.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.047529 s, 22.1 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0580811 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0415129 s, 25.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0420959 s, 24.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0119858 s, 87.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0465886 s, 22.5 MB/s
1048576 bytes (1.0 MB) copied, 0.019842 s, 52.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0467136 s, 22.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0528039 s, 19.9 MB/s
, 0.0530076 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0512926 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0552069 s, 19.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg125-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 7.26217 s, 14.4 MB/s
Write runtime is 8 s, speed is 12.500000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.99075 s, 21.0 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.389605 s, 43.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117686 s, 89.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0532995 s, 19.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0209185 s, 50.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102052 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.108724 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0762029 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0894763 s, 11.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.111825 s, 9.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.113461 s, 9.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.135971 s, 7.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.111113 s, 9.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0935601 s, 11.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.213425 s, 4.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.246619 s, 4.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.248079 s, 4.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.281121 s, 3.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0559572 s, 18.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0590059 s, 17.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0581341 s, 18.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0424208 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0680676 s, 15.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0495722 s, 21.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0506457 s, 20.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0510185 s, 20.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0403189 s, 26.0 MB/s
1+0 records in
1+0 records out
, 0.0430046 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0812808 s, 12.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0409203 s, 25.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0628038 s, 16.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.048068 s, 21.8 MB/s
, 0.0481445 s, 21.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0626166 s, 16.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg125-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.4108 s, 10.1 MB/s
Write runtime is 12 s, speed is 8.333333 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.85109 s, 10.6 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 7.50095 s, 14.0 MB/s
Write runtime is 8 s, speed is 12.500000 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8281 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.321133 s, 52.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0220626 s, 47.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328861 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0211503 s, 49.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.092395 s, 11.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0755321 s, 13.9 MB/s
1048576 bytes (1.0 MB) copied, 0.100735 s, 10.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102707 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102808 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0985764 s, 10.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0816238 s, 12.8 MB/s
1048576 bytes (1.0 MB) copied, 0.101901 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.108808 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.116195 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.13048 s, 8.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.133348 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138333 s, 7.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275184 s, 38.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0287128 s, 36.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0295729 s, 35.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00448372 s, 234 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239396 s, 43.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200881 s, 52.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.043028 s, 24.4 MB/s
1048576 bytes (1.0 MB) copied, 0.00177025 s, 592 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0564207 s, 18.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106183 s, 98.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0464045 s, 22.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0398741 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0491903 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0498902 s, 21.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0511905 s, 20.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.043677 s, 24.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg125-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.6791 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.5325 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.288577 s, 58.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0316097 s, 33.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0433288 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.112038 s, 9.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.154374 s, 6.8 MB/s
1048576 bytes (1.0 MB) copied, 0.165866 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138997 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.181808 s, 5.8 MB/s
, 0.173154 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.20413 s, 5.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.210208 s, 5.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.203818 s, 5.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.17653 s, 5.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.181614 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.227727 s, 4.6 MB/s
1048576 bytes (1.0 MB) copied, 0.217327 s, 4.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.198516 s, 5.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0383336 s, 27.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0397313 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0425178 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191339 s, 54.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0450279 s, 23.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196177 s, 53.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0231932 s, 45.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.026001 s, 40.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0285453 s, 36.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0307151 s, 34.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0325574 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0311176 s, 33.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0322853 s, 32.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0354184 s, 29.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412515 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0445178 s, 23.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg125-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 7.15858 s, 14.6 MB/s
Write runtime is 8 s, speed is 12.500000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.83736 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.275481 s, 60.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132398 s, 79.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145931 s, 71.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0265627 s, 39.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0571817 s, 18.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0761307 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0843855 s, 12.4 MB/s
, 0.0658744 s, 15.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109903 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.114498 s, 9.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.125821 s, 8.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.146847 s, 7.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.144629 s, 7.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.230442 s, 4.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.250734 s, 4.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.325234 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.30964 s, 3.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0307447 s, 34.1 MB/s
, 0.0317271 s, 33.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167872 s, 62.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014328 s, 73.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0109183 s, 96.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0231996 s, 45.2 MB/s
, 0.0237785 s, 44.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0331459 s, 31.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126998 s, 82.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0341379 s, 30.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413877 s, 25.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00207938 s, 504 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00350536 s, 299 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0015225 s, 689 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00207178 s, 506 MB/s
1048576 bytes (1.0 MB) copied, 0.00198202 s, 529 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg125-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg125-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.554 s, 9.9 MB/s
Write runtime is 12 s, speed is 8.333333 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82231 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.0496 s, 10.4 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80531 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo