== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 20:44:47 (1746146687)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.149944 s, 112 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135588 s, 77.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102344 s, 102 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0346129 s, 30.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0320977 s, 32.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0332497 s, 31.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0322365 s, 32.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.038916 s, 26.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412422 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0420686 s, 24.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0477683 s, 22.0 MB/s
, 0.0442693 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0447136 s, 23.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0587584 s, 17.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0604229 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0713419 s, 14.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0683532 s, 15.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183686 s, 57.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0214845 s, 48.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0219175 s, 47.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0147468 s, 71.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151445 s, 69.2 MB/s
, 0.0162354 s, 64.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017606 s, 59.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181582 s, 57.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0210093 s, 49.9 MB/s
, 0.0209279 s, 50.1 MB/s
, 0.0210726 s, 49.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0277361 s, 37.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0253832 s, 41.3 MB/s
, 0.0224721 s, 46.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.024323 s, 43.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0267221 s, 39.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg359-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.06988 s, 20.7 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76666 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.59@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.59@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.109245 s, 154 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010852 s, 96.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122905 s, 85.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0267561 s, 39.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228029 s, 46.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0302482 s, 34.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0312931 s, 33.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0359499 s, 29.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0375866 s, 27.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0360515 s, 29.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0287695 s, 36.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0406598 s, 25.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0373995 s, 28.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.208265 s, 5.0 MB/s
1048576 bytes (1.0 MB) copied, 0.210971 s, 5.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.260785 s, 4.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.264255 s, 4.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0111322 s, 94.2 MB/s
, 0.0162265 s, 64.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00988228 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00838051 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00861466 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162096 s, 64.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011861 s, 88.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016489 s, 63.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131668 s, 79.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133558 s, 78.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.019591 s, 53.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012448 s, 84.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133389 s, 78.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132615 s, 79.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134386 s, 78.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150504 s, 69.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg359-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96783 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74524 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.59@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.205721 s, 81.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0290144 s, 36.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200101 s, 52.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0283369 s, 37.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0229849 s, 45.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0235233 s, 44.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413004 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0477264 s, 22.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0741986 s, 14.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0730823 s, 14.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.082495 s, 12.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0901689 s, 11.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0978687 s, 10.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0946784 s, 11.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.122574 s, 8.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.122584 s, 8.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.125721 s, 8.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0308665 s, 34.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0317618 s, 33.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327089 s, 32.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0265265 s, 39.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0271649 s, 38.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0280112 s, 37.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0295432 s, 35.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0358678 s, 29.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0377232 s, 27.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0340212 s, 30.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0320771 s, 32.7 MB/s
, 0.0267679 s, 39.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0348278 s, 30.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0281719 s, 37.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0287188 s, 36.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328449 s, 31.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg359-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.06718 s, 20.7 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91699 s, 21.3 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.12508 s, 134 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135719 s, 77.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181632 s, 57.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0493232 s, 21.3 MB/s
, 0.0492897 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0471788 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0476566 s, 22.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0399437 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0578883 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402347 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0609111 s, 17.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0484114 s, 21.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0565765 s, 18.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.10496 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.116105 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106052 s, 9.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.108349 s, 9.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137564 s, 76.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119158 s, 88.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107168 s, 97.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00509574 s, 206 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116389 s, 90.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00856108 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120934 s, 86.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00748073 s, 140 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00820955 s, 128 MB/s
, 0.00803396 s, 131 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143979 s, 72.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00906262 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100321 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105141 s, 99.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108962 s, 96.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109664 s, 95.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg359-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81853 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76675 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.40829 s, 19.4 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.79339 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.139222 s, 121 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101931 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00788701 s, 133 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00815656 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328956 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0452594 s, 23.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0472358 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0461186 s, 22.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0411576 s, 25.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0932923 s, 11.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.11625 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.119743 s, 8.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.121694 s, 8.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.128862 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.131845 s, 8.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.127964 s, 8.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138549 s, 7.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0252694 s, 41.5 MB/s
, 0.0160689 s, 65.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165307 s, 63.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174779 s, 60.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179026 s, 58.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.019081 s, 55.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0200885 s, 52.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0181348 s, 57.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0250304 s, 41.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0201742 s, 52.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0202454 s, 51.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.024367 s, 43.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0216877 s, 48.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0221566 s, 47.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0232737 s, 45.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0248596 s, 42.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg359-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7281 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.5074 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.151571 s, 111 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106037 s, 98.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139316 s, 75.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159788 s, 65.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0237217 s, 44.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0291745 s, 35.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0323122 s, 32.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0473451 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328962 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0638893 s, 16.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0545691 s, 19.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0503828 s, 20.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0574034 s, 18.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0726882 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.079982 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0793602 s, 13.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.073592 s, 14.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00601958 s, 174 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0036496 s, 287 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0235148 s, 44.6 MB/s
, 0.0233911 s, 44.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137829 s, 76.1 MB/s
, 0.0201316 s, 52.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.027798 s, 37.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226794 s, 46.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168508 s, 62.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0221331 s, 47.4 MB/s
, 0.00346545 s, 303 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0221585 s, 47.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0283323 s, 37.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0266809 s, 39.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00879633 s, 119 MB/s
1048576 bytes (1.0 MB) copied, 0.00835134 s, 126 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg359-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.99858 s, 21.0 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76662 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.130004 s, 129 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124995 s, 83.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149907 s, 69.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116621 s, 89.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154816 s, 67.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121685 s, 86.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0315041 s, 33.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402508 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0218039 s, 48.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.101653 s, 10.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.104529 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0998172 s, 10.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0987406 s, 10.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.124079 s, 8.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.138288 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.141085 s, 7.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.143361 s, 7.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0278599 s, 37.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00154164 s, 680 MB/s
, 0.00156651 s, 669 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132322 s, 79.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0321952 s, 32.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157547 s, 66.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.032842 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0305571 s, 34.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0261152 s, 40.2 MB/s
, 0.0350244 s, 29.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00373538 s, 281 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0236846 s, 44.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112435 s, 93.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0348779 s, 30.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0351312 s, 29.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.022233 s, 47.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg359-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg359-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.83198 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77495 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.86978 s, 10.6 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.79344 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo