== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 18:11:33 (1746137493)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.274022 s, 61.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0391839 s, 26.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.035187 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0710853 s, 14.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0786019 s, 13.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.079036 s, 13.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0921783 s, 11.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.125808 s, 8.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.115374 s, 9.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.11515 s, 9.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.10894 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.129876 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.133283 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.132397 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.131716 s, 8.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.148189 s, 7.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.152524 s, 6.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.020022 s, 52.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0197898 s, 53.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203665 s, 51.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0382896 s, 27.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0272195 s, 38.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0293862 s, 35.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0308099 s, 34.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.032583 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0265403 s, 39.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0267419 s, 39.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0286502 s, 36.6 MB/s
, 0.0291323 s, 36.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385419 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412272 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0334945 s, 31.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0395491 s, 26.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.66717 s, 18.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77965 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.40@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.40@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.254869 s, 65.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155952 s, 67.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0206402 s, 50.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.044936 s, 23.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0490664 s, 21.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0480481 s, 21.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0415262 s, 25.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0460662 s, 22.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0403492 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0466315 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0533597 s, 19.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0452118 s, 23.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0600427 s, 17.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.265184 s, 4.0 MB/s
, 0.258241 s, 4.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.31637 s, 3.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.312054 s, 3.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0359802 s, 29.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0367581 s, 28.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0343158 s, 30.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02793 s, 37.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365872 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00462247 s, 227 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00158634 s, 661 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0272244 s, 38.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413205 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00153061 s, 685 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.022152 s, 47.3 MB/s
, 0.0018614 s, 563 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0448455 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286572 s, 36.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0482488 s, 21.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0244696 s, 42.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.13296 s, 20.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80571 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.40@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.357037 s, 47.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.020501 s, 51.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.039232 s, 26.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0379944 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.127438 s, 8.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0922785 s, 11.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0999776 s, 10.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.121104 s, 8.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.145492 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.139213 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.142753 s, 7.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.146229 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.126023 s, 8.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.15645 s, 6.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.176455 s, 5.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.220981 s, 4.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.193592 s, 5.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0468353 s, 22.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196345 s, 53.4 MB/s
, 0.0194829 s, 53.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0136584 s, 76.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00638139 s, 164 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00969012 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00877988 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00904457 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00190327 s, 551 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0589207 s, 17.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0598741 s, 17.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0476238 s, 22.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0639203 s, 16.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0339146 s, 30.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0256597 s, 40.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00669884 s, 157 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.56291 s, 18.8 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.94295 s, 21.2 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.461686 s, 36.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0214038 s, 49.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0532588 s, 19.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0598153 s, 17.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0630693 s, 16.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.055431 s, 18.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0529875 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0459051 s, 22.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0707188 s, 14.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0609599 s, 17.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0649279 s, 16.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0719885 s, 14.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.13771 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.142005 s, 7.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.142069 s, 7.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.146776 s, 7.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.155653 s, 6.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0248795 s, 42.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0256233 s, 40.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183431 s, 57.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0267122 s, 39.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.028984 s, 36.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0202622 s, 51.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168566 s, 62.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0343796 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0341362 s, 30.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0225506 s, 46.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0234482 s, 44.7 MB/s
, 0.0239284 s, 43.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0280445 s, 37.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0241299 s, 43.5 MB/s
, 0.0254295 s, 41.2 MB/s
, 0.0264231 s, 39.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.0759 s, 10.4 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77793 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.28503 s, 19.8 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80094 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.922896 s, 18.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0197868 s, 53.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0160869 s, 65.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187493 s, 55.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0701816 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0727312 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0727851 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0603357 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.062775 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.130057 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.133317 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.156248 s, 6.7 MB/s
, 0.160979 s, 6.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.156654 s, 6.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.216305 s, 4.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.24133 s, 4.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.255631 s, 4.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0410797 s, 25.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00184365 s, 569 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00194596 s, 539 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0830457 s, 12.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0962144 s, 10.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0483604 s, 21.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0696807 s, 15.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0760014 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.034876 s, 30.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0321321 s, 32.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0587387 s, 17.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412325 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0259399 s, 40.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00699231 s, 150 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181264 s, 57.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123513 s, 84.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7328 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.5252 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.343675 s, 48.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00713749 s, 147 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176193 s, 59.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165392 s, 63.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0229782 s, 45.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.048025 s, 21.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0541907 s, 19.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0508406 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.061883 s, 16.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0639951 s, 16.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0642814 s, 16.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0656199 s, 16.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0699258 s, 15.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0751389 s, 14.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.070516 s, 14.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0770676 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0759254 s, 13.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141655 s, 74.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155177 s, 67.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0164849 s, 63.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0235754 s, 44.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0204826 s, 51.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176734 s, 59.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184863 s, 56.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198945 s, 52.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174342 s, 60.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171984 s, 61.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184564 s, 56.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0206367 s, 50.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0213154 s, 49.2 MB/s
, 0.0214316 s, 48.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0217135 s, 48.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0222898 s, 47.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.4453 s, 19.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82869 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.460624 s, 36.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127422 s, 82.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328596 s, 31.9 MB/s
, 0.0307925 s, 34.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.038613 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0486348 s, 21.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0466846 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0540243 s, 19.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0525412 s, 20.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0508842 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0967551 s, 10.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0972803 s, 10.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0954988 s, 11.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.317555 s, 3.3 MB/s
, 0.316577 s, 3.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.367139 s, 2.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.371629 s, 2.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0214114 s, 49.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182864 s, 57.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137323 s, 76.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101499 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172721 s, 60.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0269696 s, 38.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108449 s, 96.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114899 s, 91.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145567 s, 72.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149207 s, 70.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156918 s, 66.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156652 s, 66.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159449 s, 65.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169008 s, 62.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153795 s, 68.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191284 s, 54.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg440-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg440-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.95277 s, 10.5 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.78385 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.94656 s, 10.5 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8013 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo