== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 21:22:56 (1746148976)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.363972 s, 46.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0197467 s, 53.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0377391 s, 27.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0815453 s, 12.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102316 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.121772 s, 8.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.119036 s, 8.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.12775 s, 8.2 MB/s
1048576 bytes (1.0 MB) copied, 0.122151 s, 8.6 MB/s
1048576 bytes (1.0 MB) copied, 0.138764 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.13685 s, 7.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.12894 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.154493 s, 6.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.162874 s, 6.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.172913 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.189961 s, 5.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.193217 s, 5.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0256501 s, 40.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0204129 s, 51.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264411 s, 39.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0261459 s, 40.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.038029 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254205 s, 41.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253569 s, 41.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0261902 s, 40.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0259111 s, 40.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0290389 s, 36.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0330397 s, 31.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345846 s, 30.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.037973 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0374333 s, 28.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0389032 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0488282 s, 21.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg216-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.32983 s, 16.6 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8191 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.16@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.16@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.335796 s, 50.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260053 s, 40.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0335763 s, 31.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0662703 s, 15.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0716845 s, 14.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0859891 s, 12.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.073132 s, 14.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0807493 s, 13.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.074502 s, 14.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0949949 s, 11.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0805444 s, 13.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0835878 s, 12.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0835506 s, 12.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.201814 s, 5.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.209571 s, 5.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.256526 s, 4.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.264547 s, 4.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00828628 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242086 s, 43.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0243649 s, 43.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0210622 s, 49.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0369595 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0166871 s, 62.8 MB/s
, 0.0148406 s, 70.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192439 s, 54.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196185 s, 53.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.021222 s, 49.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0212737 s, 49.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0245513 s, 42.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275385 s, 38.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0298521 s, 35.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0319845 s, 32.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0320209 s, 32.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg216-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.95409 s, 17.6 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82744 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.16@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.329755 s, 50.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254746 s, 41.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0320487 s, 32.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0715979 s, 14.6 MB/s
, 0.0773758 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0773272 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0769474 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110409 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0894217 s, 11.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106324 s, 9.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.1009 s, 10.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.119965 s, 8.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0999569 s, 10.5 MB/s
, 0.102288 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.120532 s, 8.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.112554 s, 9.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.129656 s, 8.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00567376 s, 185 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0418753 s, 25.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0506812 s, 20.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0250287 s, 41.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.007002 s, 150 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.043388 s, 24.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0458959 s, 22.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00478 s, 219 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0443298 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0368592 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.040936 s, 25.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0463641 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0348747 s, 30.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0256501 s, 40.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0360061 s, 29.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0212437 s, 49.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg216-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.81108 s, 18.0 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95643 s, 21.2 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.281867 s, 59.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183208 s, 57.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0274431 s, 38.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0501552 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0954206 s, 11.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0755432 s, 13.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.118497 s, 8.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.101576 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109667 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.132578 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.113434 s, 9.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138873 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.160221 s, 6.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.17021 s, 6.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.13447 s, 7.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.150706 s, 7.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.154675 s, 6.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345185 s, 30.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0456989 s, 22.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0394078 s, 26.6 MB/s
, 0.0404577 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0323816 s, 32.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0481503 s, 21.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.039811 s, 26.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0450287 s, 23.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0450722 s, 23.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365382 s, 28.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.061232 s, 17.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0474636 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380993 s, 27.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0406196 s, 25.8 MB/s
, 0.0526329 s, 19.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0425705 s, 24.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg216-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.0421 s, 10.4 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.84587 s, 10.6 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.19213 s, 16.9 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8282 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.284875 s, 58.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0261415 s, 40.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0422127 s, 24.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0756439 s, 13.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0894234 s, 11.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0940913 s, 11.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0778405 s, 13.5 MB/s
1048576 bytes (1.0 MB) copied, 0.080956 s, 13.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0967435 s, 10.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0924875 s, 11.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0898906 s, 11.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0680187 s, 15.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0979919 s, 10.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107419 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.116411 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.115267 s, 9.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.122642 s, 8.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0267142 s, 39.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242544 s, 43.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0284305 s, 36.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.029326 s, 35.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0357359 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327933 s, 32.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.029005 s, 36.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0344502 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0398032 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0310327 s, 33.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0322823 s, 32.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0322223 s, 32.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0378656 s, 27.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0359214 s, 29.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0282499 s, 37.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0310895 s, 33.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg216-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7167 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.5183 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.195364 s, 85.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139349 s, 75.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174871 s, 60.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0367762 s, 28.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412534 s, 25.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0560543 s, 18.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0493122 s, 21.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0606839 s, 17.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.065157 s, 16.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0625562 s, 16.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0689568 s, 15.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0666407 s, 15.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0741944 s, 14.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0733152 s, 14.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0661143 s, 15.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0725689 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0721857 s, 14.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0171032 s, 61.3 MB/s
1048576 bytes (1.0 MB) copied, 0.021158 s, 49.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173427 s, 60.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184063 s, 57.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0165482 s, 63.4 MB/s
, 0.016278 s, 64.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146654 s, 71.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0216246 s, 48.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161133 s, 65.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167889 s, 62.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0180916 s, 58.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186027 s, 56.4 MB/s
, 0.018768 s, 55.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0199655 s, 52.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0220492 s, 47.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02238 s, 46.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg216-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.41252 s, 19.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.83283 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.271117 s, 61.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0359785 s, 29.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0221512 s, 47.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0395831 s, 26.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186168 s, 56.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192888 s, 54.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0484395 s, 21.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.137579 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109473 s, 9.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.133169 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.180927 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.174802 s, 6.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.225667 s, 4.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.204531 s, 5.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.222012 s, 4.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.234148 s, 4.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.209136 s, 5.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0266314 s, 39.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0244307 s, 42.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.024766 s, 42.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0297825 s, 35.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0283034 s, 37.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0303322 s, 34.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0300336 s, 34.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0342926 s, 30.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0342414 s, 30.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0356448 s, 29.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0392583 s, 26.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.039596 s, 26.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0427163 s, 24.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0411237 s, 25.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0463057 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0542343 s, 19.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg216-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg216-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.0169 s, 10.5 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.86448 s, 10.6 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.98385 s, 10.5 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8307 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo