== sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 17:52:56 (1748987576)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.212821 s, 78.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187516 s, 55.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0113415 s, 92.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260728 s, 40.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0286138 s, 36.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0214183 s, 49.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116484 s, 90.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0305394 s, 34.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150948 s, 69.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0580093 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0662058 s, 15.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0593894 s, 17.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0687233 s, 15.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0920322 s, 11.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0903772 s, 11.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0751684 s, 13.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0789444 s, 13.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131793 s, 79.6 MB/s
, 0.0152435 s, 68.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135314 s, 77.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143474 s, 73.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152729 s, 68.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168571 s, 62.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138606 s, 75.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0150866 s, 69.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0140665 s, 74.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00581547 s, 180 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0187618 s, 55.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0146935 s, 71.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017239 s, 60.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174632 s, 60.0 MB/s
1048576 bytes (1.0 MB) copied, 0.00722081 s, 145 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177248 s, 59.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg253-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96079 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74202 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.53@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.53@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.207661 s, 80.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01196 s, 87.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00749002 s, 140 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0205902 s, 50.9 MB/s
, 0.00796005 s, 132 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196939 s, 53.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179735 s, 58.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00921501 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0237382 s, 44.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0589037 s, 17.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0803193 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0858941 s, 12.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0730593 s, 14.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.146269 s, 7.2 MB/s
, 0.142623 s, 7.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.257324 s, 4.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.249227 s, 4.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110817 s, 94.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01205 s, 87.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130225 s, 80.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117497 s, 89.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0180551 s, 58.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014613 s, 71.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152833 s, 68.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170017 s, 61.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0152539 s, 68.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0154323 s, 67.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017365 s, 60.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0178194 s, 58.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0180184 s, 58.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0168417 s, 62.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183116 s, 57.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0205665 s, 51.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg253-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96444 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74391 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.53@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.213725 s, 78.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203879 s, 51.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147541 s, 71.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0265448 s, 39.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155449 s, 67.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200351 s, 52.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177422 s, 59.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176546 s, 59.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146374 s, 71.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0567203 s, 18.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0811093 s, 12.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0793506 s, 13.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0713577 s, 14.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0884401 s, 11.9 MB/s
, 0.0741071 s, 14.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0850344 s, 12.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0934485 s, 11.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151023 s, 69.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115178 s, 91.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145789 s, 71.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133756 s, 78.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130499 s, 80.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0137008 s, 76.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00290048 s, 362 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163954 s, 64.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0169395 s, 61.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139828 s, 75.0 MB/s
, 0.0168093 s, 62.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156451 s, 67.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015373 s, 68.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00274448 s, 382 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161617 s, 64.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0195929 s, 53.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg253-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95918 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89546 s, 21.4 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.361543 s, 46.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0201353 s, 52.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336958 s, 31.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385719 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0304078 s, 34.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.044858 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0477871 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0417077 s, 25.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.042536 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.152521 s, 6.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.17969 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.169752 s, 6.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.167714 s, 6.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.318589 s, 3.3 MB/s
, 0.300928 s, 3.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.353108 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.344528 s, 3.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126287 s, 83.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012702 s, 82.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141902 s, 73.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0315957 s, 33.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162434 s, 64.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0196748 s, 53.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0175566 s, 59.7 MB/s
1048576 bytes (1.0 MB) copied, 0.017725 s, 59.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0211218 s, 49.6 MB/s
1048576 bytes (1.0 MB) copied, 0.019214 s, 54.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0344416 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.020023 s, 52.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0361182 s, 29.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02006 s, 52.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0202482 s, 51.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0204927 s, 51.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg253-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80468 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74322 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95282 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74402 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.660501 s, 25.4 MB/s
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192311 s, 54.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0193578 s, 54.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131641 s, 79.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0188552 s, 55.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0229141 s, 45.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00982404 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0185531 s, 56.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0360732 s, 29.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.352813 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.362599 s, 2.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.369222 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.364735 s, 2.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.372065 s, 2.8 MB/s
, 0.387389 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.390137 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.394735 s, 2.7 MB/s
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101316 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118776 s, 88.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177571 s, 59.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146365 s, 71.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00972821 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122726 s, 85.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0156737 s, 66.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177046 s, 59.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0202551 s, 51.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143669 s, 73.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016936 s, 61.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159994 s, 65.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157948 s, 66.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0175975 s, 59.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168645 s, 62.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189486 s, 55.3 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg253-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7061 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4444 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.215259 s, 77.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146436 s, 71.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.023925 s, 43.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0283938 s, 36.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0381556 s, 27.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0222141 s, 47.2 MB/s
, 0.0127937 s, 82.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0221231 s, 47.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0190398 s, 55.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0554015 s, 18.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0528828 s, 19.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0693527 s, 15.1 MB/s
1+0 records in
1+0 records out
, 0.0654212 s, 16.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0728737 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0768109 s, 13.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.088524 s, 11.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0785723 s, 13.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.012413 s, 84.5 MB/s
, 0.0163382 s, 64.2 MB/s
1+0 records in
1+0 records out
, 0.0122939 s, 85.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0149013 s, 70.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014793 s, 70.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00787538 s, 133 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148676 s, 70.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152345 s, 68.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00948553 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176856 s, 59.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176156 s, 59.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172838 s, 60.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00548097 s, 191 MB/s
, 0.0169992 s, 61.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0163598 s, 64.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0188522 s, 55.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg253-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96105 s, 21.1 MB/s
Write runtime is 5 s, speed is 20.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74399 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.360556 s, 46.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253627 s, 41.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345345 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0259537 s, 40.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0377861 s, 27.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0251355 s, 41.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226598 s, 46.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.027976 s, 37.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.024688 s, 42.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.151827 s, 6.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.145828 s, 7.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.331683 s, 3.2 MB/s
1048576 bytes (1.0 MB) copied, 0.349793 s, 3.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.360229 s, 2.9 MB/s
, 0.351747 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.38773 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.401989 s, 2.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124395 s, 84.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0137312 s, 76.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0159608 s, 65.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144234 s, 72.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0171383 s, 61.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0188793 s, 55.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169961 s, 61.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0180624 s, 58.1 MB/s
, 0.0204561 s, 51.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165799 s, 63.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0196824 s, 53.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0194125 s, 54.0 MB/s
, 0.0193736 s, 54.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0433411 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0209674 s, 50.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.022331 s, 47.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg253-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg253-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80589 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74263 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80062 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74288 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo