== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 22:16:51 (1746152211)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.456534 s, 36.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0298967 s, 35.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0601145 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0804273 s, 13.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0899271 s, 11.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0923216 s, 11.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0983652 s, 10.7 MB/s
1048576 bytes (1.0 MB) copied, 0.113455 s, 9.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.119269 s, 8.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.132358 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.140617 s, 7.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.153043 s, 6.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.155449 s, 6.7 MB/s
1048576 bytes (1.0 MB) copied, 0.135661 s, 7.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.144877 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.223682 s, 4.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.22189 s, 4.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0578706 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00372598 s, 281 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00319249 s, 328 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0511272 s, 20.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0947066 s, 11.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00221236 s, 474 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0704835 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.080068 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0673291 s, 15.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0717839 s, 14.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0959334 s, 10.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0339923 s, 30.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0669898 s, 15.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0317271 s, 33.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0301558 s, 34.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203324 s, 51.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.20857 s, 16.9 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.83661 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.20@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.20@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.36412 s, 46.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328855 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0307186 s, 34.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226543 s, 46.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.104637 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.117686 s, 8.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0896883 s, 11.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105646 s, 9.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.142903 s, 7.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.112384 s, 9.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.133466 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.120688 s, 8.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.157045 s, 6.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.137467 s, 7.6 MB/s
1048576 bytes (1.0 MB) copied, 0.17026 s, 6.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.178377 s, 5.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.23068 s, 4.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0352267 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0310528 s, 33.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0315197 s, 33.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.032438 s, 32.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0708259 s, 14.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0364763 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0358177 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0423252 s, 24.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0420002 s, 25.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0416893 s, 25.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.03754 s, 27.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0400902 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0418913 s, 25.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380571 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0398666 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0317345 s, 33.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 7.06271 s, 14.8 MB/s
Write runtime is 8 s, speed is 12.500000 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.83647 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.20@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.389861 s, 43.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0382071 s, 27.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0449545 s, 23.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.131405 s, 8.0 MB/s
1048576 bytes (1.0 MB) copied, 0.145418 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.14555 s, 7.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.130591 s, 8.0 MB/s
1048576 bytes (1.0 MB) copied, 0.112001 s, 9.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.129531 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.145425 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.123809 s, 8.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.137371 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.124045 s, 8.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.133099 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.176169 s, 6.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.182529 s, 5.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.174307 s, 6.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0318156 s, 33.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0230385 s, 45.5 MB/s
, 0.0692061 s, 15.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0643849 s, 16.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00222754 s, 471 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0601767 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0606335 s, 17.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0562749 s, 18.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0698402 s, 15.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0534774 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0638901 s, 16.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0812947 s, 12.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0556045 s, 18.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.019772 s, 53.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181934 s, 57.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0224807 s, 46.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.92276 s, 15.1 MB/s
Write runtime is 8 s, speed is 12.500000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.99285 s, 21.0 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.54124 s, 31.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175499 s, 59.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0495587 s, 21.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0342026 s, 30.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0676233 s, 15.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107425 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0551915 s, 19.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.136816 s, 7.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138222 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.140984 s, 7.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106727 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.196394 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.144453 s, 7.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.207191 s, 5.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.21164 s, 5.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.157912 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.225371 s, 4.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127392 s, 82.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114821 s, 91.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00185496 s, 565 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00393831 s, 266 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0256748 s, 40.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00236734 s, 443 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0769255 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0828973 s, 12.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0644261 s, 16.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0331603 s, 31.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00726014 s, 144 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0355198 s, 29.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0879354 s, 11.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0350769 s, 29.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0318816 s, 32.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0263192 s, 39.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.1714 s, 10.3 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82672 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.38638 s, 16.4 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81973 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.922322 s, 18.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0461073 s, 22.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0899665 s, 11.7 MB/s
, 0.0949023 s, 11.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.129202 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0961747 s, 10.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.128393 s, 8.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.135498 s, 7.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.116613 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.120696 s, 8.7 MB/s
, 0.113612 s, 9.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.152698 s, 6.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.170835 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.27528 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.330007 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.302374 s, 3.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.314759 s, 3.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00202986 s, 517 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0479286 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00675962 s, 155 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0498335 s, 21.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0919852 s, 11.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129755 s, 80.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0981291 s, 10.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.101451 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102173 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.082617 s, 12.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.060573 s, 17.3 MB/s
, 0.0521938 s, 20.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0843209 s, 12.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110097 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0658884 s, 15.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0561494 s, 18.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7545 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.556 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.497462 s, 33.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017714 s, 59.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0397544 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0387651 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0367538 s, 28.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0510518 s, 20.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.09956 s, 10.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.152929 s, 6.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.139354 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.147288 s, 7.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.162341 s, 6.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.142042 s, 7.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.191787 s, 5.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.196659 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.201234 s, 5.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.216639 s, 4.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.207969 s, 5.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00222589 s, 471 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00207304 s, 506 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00288371 s, 364 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00545788 s, 192 MB/s
, 0.00456597 s, 230 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00250178 s, 419 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0589887 s, 17.8 MB/s
, 0.0636889 s, 16.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0642156 s, 16.3 MB/s
, 0.00651782 s, 161 MB/s
1048576 bytes (1.0 MB) copied, 0.0403537 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0502389 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0569963 s, 18.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0338811 s, 30.9 MB/s
, 0.0483196 s, 21.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.024955 s, 42.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 7.6973 s, 13.6 MB/s
Write runtime is 9 s, speed is 11.111111 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.84934 s, 10.6 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.476898 s, 35.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0321325 s, 32.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0344039 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0701971 s, 14.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0773821 s, 13.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0901601 s, 11.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0758641 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0800974 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0754594 s, 13.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0819216 s, 12.8 MB/s
, 0.0849278 s, 12.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0880016 s, 11.9 MB/s
, 0.0937998 s, 11.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.133306 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.130246 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.143559 s, 7.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.14344 s, 7.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0292862 s, 35.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00339405 s, 309 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00377768 s, 278 MB/s
1048576 bytes (1.0 MB) copied, 0.0323879 s, 32.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00168943 s, 621 MB/s
, 0.035975 s, 29.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00571372 s, 184 MB/s
, 0.027968 s, 37.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281384 s, 37.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0316082 s, 33.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0323952 s, 32.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00611268 s, 172 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0250385 s, 41.9 MB/s
, 0.0346963 s, 30.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328413 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011679 s, 89.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.0718 s, 10.4 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81388 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.91729 s, 10.6 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82192 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo