== sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 16:36:35 (1748982995)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.190285 s, 88.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0164566 s, 63.7 MB/s
, 0.0257491 s, 40.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0301304 s, 34.8 MB/s
, 0.0353808 s, 29.6 MB/s
, 0.0305601 s, 34.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0285244 s, 36.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329418 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.041058 s, 25.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0751305 s, 14.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0840457 s, 12.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0758019 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0843999 s, 12.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0962329 s, 10.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.080513 s, 13.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0997274 s, 10.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0858565 s, 12.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00938387 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00899184 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00839085 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00859006 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0086429 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107186 s, 97.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108225 s, 96.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130275 s, 80.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00140736 s, 745 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012166 s, 86.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119932 s, 87.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122072 s, 85.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105806 s, 99.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141717 s, 74.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110389 s, 95.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138974 s, 75.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.9355 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73606 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.54@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.54@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.16468 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192326 s, 54.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128624 s, 81.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0401913 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0272102 s, 38.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0427924 s, 24.5 MB/s
, 0.0410314 s, 25.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0420881 s, 24.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0348942 s, 30.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0299995 s, 35.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0319407 s, 32.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.129018 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.14395 s, 7.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.231396 s, 4.5 MB/s
, 0.240506 s, 4.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.325952 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.332341 s, 3.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153357 s, 68.4 MB/s
, 0.0125933 s, 83.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187175 s, 56.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0149624 s, 70.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156492 s, 67.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0157714 s, 66.5 MB/s
, 0.0160612 s, 65.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169172 s, 62.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158797 s, 66.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162468 s, 64.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179255 s, 58.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170117 s, 61.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174585 s, 60.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179925 s, 58.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0185753 s, 56.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203336 s, 51.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96722 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74515 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.54@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.181012 s, 92.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0201877 s, 51.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329853 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0248662 s, 42.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0433508 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365733 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0388054 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0398074 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.04602 s, 22.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0621809 s, 16.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0547398 s, 19.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0641225 s, 16.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0681045 s, 15.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0657231 s, 16.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0722202 s, 14.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0772052 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0798472 s, 13.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107555 s, 97.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136116 s, 77.0 MB/s
, 0.0138675 s, 75.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126134 s, 83.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154508 s, 67.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176653 s, 59.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0199712 s, 52.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122572 s, 85.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142772 s, 73.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144952 s, 72.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136662 s, 76.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159405 s, 65.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00364626 s, 288 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171007 s, 61.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0160609 s, 65.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189117 s, 55.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96211 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89548 s, 21.4 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.277699 s, 60.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203787 s, 51.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0348069 s, 30.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327024 s, 32.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0404917 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.03004 s, 34.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.032699 s, 32.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254189 s, 41.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0324813 s, 32.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.136008 s, 7.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.120424 s, 8.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.321833 s, 3.3 MB/s
1048576 bytes (1.0 MB) copied, 0.313913 s, 3.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.340671 s, 3.1 MB/s
, 0.343766 s, 3.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.387395 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.374366 s, 2.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133966 s, 78.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129543 s, 80.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112217 s, 93.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103409 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110842 s, 94.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127107 s, 82.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128059 s, 81.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132812 s, 79.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140441 s, 74.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147832 s, 70.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157156 s, 66.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141065 s, 74.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132361 s, 79.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0172895 s, 60.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0177693 s, 59.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162833 s, 64.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.79969 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73713 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96643 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74824 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.477746 s, 35.1 MB/s
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161762 s, 64.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198966 s, 52.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0341737 s, 30.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0233547 s, 44.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0304236 s, 34.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0493027 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0410601 s, 25.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0467976 s, 22.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.335938 s, 3.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.355102 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.347675 s, 3.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.350722 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.363377 s, 2.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.347435 s, 3.0 MB/s
1048576 bytes (1.0 MB) copied, 0.355247 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.352687 s, 3.0 MB/s
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132399 s, 79.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102291 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104057 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109373 s, 95.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0115383 s, 90.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182182 s, 57.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132691 s, 79.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144662 s, 72.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0190479 s, 55.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145847 s, 71.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159958 s, 65.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.015343 s, 68.3 MB/s
, 0.0149792 s, 70.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0166556 s, 63.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171517 s, 61.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187224 s, 56.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7329 s, 5.3 MB/s
Write runtime is 20 s, speed is 5.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/1000/500/1000, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4447 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.210003 s, 79.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114545 s, 91.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013419 s, 78.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0305688 s, 34.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0296098 s, 35.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0265202 s, 39.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329922 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0261776 s, 40.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0274951 s, 38.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0557539 s, 18.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0706254 s, 14.8 MB/s
, 0.0715113 s, 14.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0638418 s, 16.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0710697 s, 14.8 MB/s
, 0.0739193 s, 14.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0854557 s, 12.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0786855 s, 13.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115149 s, 91.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0107386 s, 97.6 MB/s
, 0.0113059 s, 92.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011462 s, 91.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141127 s, 74.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175848 s, 59.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168947 s, 62.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00308873 s, 339 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171082 s, 61.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0231714 s, 45.3 MB/s
, 0.0168105 s, 62.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159747 s, 65.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159178 s, 65.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155155 s, 67.6 MB/s
, 0.0155057 s, 67.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174465 s, 60.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96196 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74766 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.263027 s, 63.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142761 s, 73.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0294967 s, 35.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0319064 s, 32.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0217206 s, 48.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0207591 s, 50.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0303527 s, 34.5 MB/s
, 0.0356305 s, 29.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0282544 s, 37.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.125853 s, 8.3 MB/s
1048576 bytes (1.0 MB) copied, 0.137437 s, 7.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.217339 s, 4.8 MB/s
, 0.239421 s, 4.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.290879 s, 3.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.288728 s, 3.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.329688 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.319093 s, 3.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127543 s, 82.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0166508 s, 63.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147229 s, 71.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130589 s, 80.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147345 s, 71.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150028 s, 69.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162455 s, 64.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169518 s, 61.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163233 s, 64.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141506 s, 74.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146111 s, 71.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0158435 s, 66.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154934 s, 67.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016069 s, 65.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174395 s, 60.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169602 s, 61.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81414 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74274 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8093 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74435 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo