== sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 18:24:12 (1751322252)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.212764 s, 78.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016755 s, 62.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0322633 s, 32.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0300536 s, 34.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0227501 s, 46.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013436 s, 78.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02796 s, 37.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0256104 s, 40.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0246439 s, 42.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0610366 s, 17.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0603154 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.068996 s, 15.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0536229 s, 19.6 MB/s
, 0.0677271 s, 15.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0610707 s, 17.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0853214 s, 12.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0728212 s, 14.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102778 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169828 s, 61.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0143668 s, 73.0 MB/s
, 0.0109437 s, 95.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142084 s, 73.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125182 s, 83.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0190548 s, 55.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00245722 s, 427 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.015795 s, 66.4 MB/s
, 0.0164493 s, 63.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00251259 s, 417 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017171 s, 61.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163319 s, 64.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162157 s, 64.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163805 s, 64.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196923 s, 53.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg203-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95315 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73728 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.3@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.3@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.207757 s, 80.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179607 s, 58.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0310038 s, 33.8 MB/s
, 0.0187858 s, 55.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0229195 s, 45.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402818 s, 26.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0437775 s, 24.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0363434 s, 28.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0393354 s, 26.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0582046 s, 18.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0771749 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0821659 s, 12.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0829469 s, 12.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.14658 s, 7.2 MB/s
1048576 bytes (1.0 MB) copied, 0.154461 s, 6.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.246195 s, 4.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.24334 s, 4.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146949 s, 71.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125855 s, 83.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00351151 s, 299 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140646 s, 74.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156756 s, 66.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.019894 s, 52.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167115 s, 62.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176902 s, 59.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.018853 s, 55.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0180482 s, 58.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0182023 s, 57.6 MB/s
, 0.0165598 s, 63.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184171 s, 56.9 MB/s
1+0 records in
1+0 records out
, 0.0184583 s, 56.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0167606 s, 62.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0207774 s, 50.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg203-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.97281 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74128 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.3@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.208721 s, 80.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0227655 s, 46.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0243378 s, 43.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0208347 s, 50.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0321872 s, 32.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0405028 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00795306 s, 132 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0205809 s, 50.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0294289 s, 35.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.062373 s, 16.8 MB/s
, 0.0695572 s, 15.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0852683 s, 12.3 MB/s
, 0.0744521 s, 14.1 MB/s
, 0.069516 s, 15.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0627614 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0785598 s, 13.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0875816 s, 12.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127186 s, 82.4 MB/s
, 0.0147506 s, 71.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128166 s, 81.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00265367 s, 395 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154755 s, 67.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0194433 s, 53.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159164 s, 65.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00235439 s, 445 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156265 s, 67.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00283515 s, 370 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01644 s, 63.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170023 s, 61.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0157724 s, 66.5 MB/s
, 0.0184547 s, 56.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169598 s, 61.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0236375 s, 44.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg203-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.9674 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89087 s, 21.4 MB/s
Read runtime is 5 s, speed is 20.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.365198 s, 45.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131191 s, 79.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0225728 s, 46.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0327573 s, 32.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140778 s, 74.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0210817 s, 49.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0294215 s, 35.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0390593 s, 26.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0202485 s, 51.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.150427 s, 7.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.134052 s, 7.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.34301 s, 3.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.332403 s, 3.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.361707 s, 2.9 MB/s
, 0.352059 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.375614 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.391695 s, 2.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176287 s, 59.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181018 s, 57.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0190285 s, 55.1 MB/s
1048576 bytes (1.0 MB) copied, 0.024824 s, 42.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0356607 s, 29.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0202234 s, 51.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0206576 s, 50.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.02129 s, 49.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0377573 s, 27.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228638 s, 45.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0230953 s, 45.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0333762 s, 31.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0247533 s, 42.4 MB/s
, 0.024401 s, 43.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0240039 s, 43.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.025076 s, 41.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg203-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80098 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74193 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95753 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74197 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.662411 s, 25.3 MB/s
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.018569 s, 56.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226616 s, 46.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0387863 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0256798 s, 40.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0351006 s, 29.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0437731 s, 24.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0402484 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0332348 s, 31.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.351598 s, 3.0 MB/s
, 0.348374 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.364853 s, 2.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.378255 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.371998 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.377098 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.386299 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.397174 s, 2.6 MB/s
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113941 s, 92.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115359 s, 90.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142175 s, 73.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129815 s, 80.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141913 s, 73.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143376 s, 73.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00688744 s, 152 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196942 s, 53.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171368 s, 61.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015433 s, 67.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149961 s, 69.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148803 s, 70.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151455 s, 69.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156884 s, 66.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014076 s, 74.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0194025 s, 54.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg203-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7049 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4432 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.215937 s, 77.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134925 s, 77.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264025 s, 39.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0187385 s, 56.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158413 s, 66.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182031 s, 57.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00947783 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275008 s, 38.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0289749 s, 36.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.059094 s, 17.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0781009 s, 13.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0728794 s, 14.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0860466 s, 12.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0873521 s, 12.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106386 s, 9.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102866 s, 10.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0868168 s, 12.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01162 s, 90.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111689 s, 93.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124394 s, 84.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142032 s, 73.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184168 s, 56.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174082 s, 60.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158105 s, 66.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0164915 s, 63.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00523243 s, 200 MB/s
, 0.00258596 s, 405 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141937 s, 73.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143253 s, 73.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153999 s, 68.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162147 s, 64.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171275 s, 61.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0188626 s, 55.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg203-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95802 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74069 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.363364 s, 46.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0236283 s, 44.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012768 s, 82.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0262813 s, 39.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106256 s, 98.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156267 s, 67.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253846 s, 41.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0231587 s, 45.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200494 s, 52.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.151172 s, 6.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.166338 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.157854 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.167056 s, 6.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.343699 s, 3.1 MB/s
, 0.349853 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.35627 s, 2.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.344661 s, 3.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0161748 s, 64.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148652 s, 70.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013837 s, 75.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141031 s, 74.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154699 s, 67.8 MB/s
, 0.0151451 s, 69.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00283072 s, 370 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155858 s, 67.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143274 s, 73.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0189459 s, 55.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170902 s, 61.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172255 s, 60.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169193 s, 62.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168407 s, 62.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189071 s, 55.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0205782 s, 51.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg203-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg203-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82966 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74063 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80486 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74136 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo