== sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 19:11:17 (1736986277)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.197928 s, 84.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0172374 s, 60.8 MB/s
, 0.0140919 s, 74.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345846 s, 30.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0259945 s, 40.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0353691 s, 29.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0366743 s, 28.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0304428 s, 34.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0349358 s, 30.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0678878 s, 15.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0830433 s, 12.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0750674 s, 14.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0815089 s, 12.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0897303 s, 11.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0878478 s, 11.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0814307 s, 12.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0862348 s, 12.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00677154 s, 155 MB/s
, 0.00949294 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00774029 s, 135 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116239 s, 90.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00924319 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00850866 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00904037 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00914236 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103572 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105197 s, 99.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101537 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104035 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113978 s, 92.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0121826 s, 86.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0115855 s, 90.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106762 s, 98.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93719 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73338 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.21@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.21@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.207838 s, 80.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101582 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181173 s, 57.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0298409 s, 35.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.03228 s, 32.5 MB/s
, 0.0305328 s, 34.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.032179 s, 32.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0347137 s, 30.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0390481 s, 26.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0572722 s, 18.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0745725 s, 14.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0792722 s, 13.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0749974 s, 14.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.157931 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.156863 s, 6.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.257382 s, 4.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.2535 s, 4.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.011807 s, 88.8 MB/s
, 0.0150986 s, 69.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101842 s, 103 MB/s
1048576 bytes (1.0 MB) copied, 0.0137668 s, 76.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124938 s, 83.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109687 s, 95.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123468 s, 84.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125803 s, 83.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126228 s, 83.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137567 s, 76.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136536 s, 76.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146485 s, 71.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015648 s, 67.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154147 s, 68.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154973 s, 67.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173173 s, 60.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93608 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74025 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.21@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.192961 s, 86.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106356 s, 98.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254693 s, 41.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0197531 s, 53.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0266601 s, 39.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0278349 s, 37.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0308197 s, 34.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175546 s, 59.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253698 s, 41.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0620479 s, 16.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0705757 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0751391 s, 14.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0774216 s, 13.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0877289 s, 12.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.085983 s, 12.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0757125 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0878163 s, 11.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126406 s, 83.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147354 s, 71.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112366 s, 93.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136915 s, 76.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172183 s, 60.9 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139594 s, 75.1 MB/s
, 0.0130864 s, 80.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126009 s, 83.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137462 s, 76.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013468 s, 77.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148029 s, 70.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143173 s, 73.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148706 s, 70.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143063 s, 73.3 MB/s
1048576 bytes (1.0 MB) copied, 0.018652 s, 56.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169032 s, 62.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95402 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89646 s, 21.4 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.342816 s, 48.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107653 s, 97.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00861233 s, 122 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0307916 s, 34.1 MB/s
, 0.0233088 s, 45.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0339968 s, 30.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0319789 s, 32.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0325829 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0369304 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.171006 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.186174 s, 5.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.184601 s, 5.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.176374 s, 5.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.273454 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.274663 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.313282 s, 3.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.314729 s, 3.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00765316 s, 137 MB/s
, 0.00774353 s, 135 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118164 s, 88.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143769 s, 72.9 MB/s
, 0.0106033 s, 98.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125189 s, 83.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0096725 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111752 s, 93.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013301 s, 78.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115756 s, 90.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143571 s, 73.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0136191 s, 77.0 MB/s
, 0.013596 s, 77.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141411 s, 74.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147113 s, 71.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159893 s, 65.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77842 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7378 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91521 s, 21.3 MB/s
Write runtime is 5 s, speed is 20.000000 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73343 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.640426 s, 26.2 MB/s
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109491 s, 95.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00868285 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101089 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00958464 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0325669 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0318131 s, 33.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0339801 s, 30.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0324373 s, 32.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.365725 s, 2.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.369266 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.378721 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.376818 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.386716 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.384462 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.390253 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.391581 s, 2.7 MB/s
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00930589 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00702386 s, 149 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103851 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00816664 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00818514 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00976492 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00914746 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106615 s, 98.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00964343 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00915298 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00901197 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108504 s, 96.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00993124 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00945822 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011221 s, 93.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123122 s, 85.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.8344 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4343 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.192761 s, 87.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109267 s, 96.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191567 s, 54.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0152836 s, 68.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0247198 s, 42.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254924 s, 41.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.030002 s, 35.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0279893 s, 37.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.034143 s, 30.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0678807 s, 15.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0701659 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0813927 s, 12.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0726819 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0832982 s, 12.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0888151 s, 11.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0930331 s, 11.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0963803 s, 10.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00876885 s, 120 MB/s
1048576 bytes (1.0 MB) copied, 0.00999124 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00811701 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00707118 s, 148 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00897171 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133997 s, 78.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00992598 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00792712 s, 132 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00793259 s, 132 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00901364 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00899746 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00987106 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109442 s, 95.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0117298 s, 89.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0111039 s, 94.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011943 s, 87.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93997 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73229 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.339053 s, 49.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0076375 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115471 s, 90.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148529 s, 70.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184563 s, 56.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0236144 s, 44.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0249971 s, 41.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0283669 s, 37.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0193267 s, 54.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.171073 s, 6.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.174709 s, 6.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.182346 s, 5.8 MB/s
1048576 bytes (1.0 MB) copied, 0.185565 s, 5.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.319445 s, 3.3 MB/s
, 0.32546 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.372072 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.372389 s, 2.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0065831 s, 159 MB/s
, 0.00399765 s, 262 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00741281 s, 141 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00598397 s, 175 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00629465 s, 167 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102816 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00992707 s, 106 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00789117 s, 133 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0080575 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00925797 s, 113 MB/s
1048576 bytes (1.0 MB) copied, 0.00994268 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00991314 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106274 s, 98.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112263 s, 93.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0114137 s, 91.9 MB/s
, 0.0114006 s, 92.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77208 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7291 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76339 s, 10.7 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72964 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo