== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 14:00:53 (1741629653)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.237583 s, 70.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00751062 s, 140 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137988 s, 76.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00821002 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112056 s, 93.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.026191 s, 40.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0295866 s, 35.4 MB/s
, 0.0151843 s, 69.1 MB/s
, 0.0312081 s, 33.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162744 s, 64.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0459623 s, 22.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0457955 s, 22.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0452841 s, 23.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.047749 s, 22.0 MB/s
, 0.0540428 s, 19.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0361253 s, 29.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.050914 s, 20.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00742413 s, 141 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00760063 s, 138 MB/s
1048576 bytes (1.0 MB) copied, 0.00965597 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105098 s, 99.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00918232 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00961622 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00493212 s, 213 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135859 s, 77.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00933671 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121039 s, 86.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00843911 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00912898 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0096055 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00985149 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126394 s, 83.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123979 s, 84.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg445-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92661 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73058 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.45@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.45@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.232303 s, 72.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00833606 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115871 s, 90.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242372 s, 43.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.028576 s, 36.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0309978 s, 33.8 MB/s
, 0.0228683 s, 45.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328193 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0310602 s, 33.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0394283 s, 26.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0437172 s, 24.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336449 s, 31.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0453771 s, 23.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.276039 s, 3.8 MB/s
, 0.271437 s, 3.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.315419 s, 3.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.318526 s, 3.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120583 s, 87.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0065666 s, 160 MB/s
, 0.00804998 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00874086 s, 120 MB/s
1048576 bytes (1.0 MB) copied, 0.00930859 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126623 s, 82.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100453 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109154 s, 96.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105157 s, 99.7 MB/s
, 0.0121522 s, 86.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104718 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115785 s, 90.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117092 s, 89.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125318 s, 83.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126724 s, 82.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117633 s, 89.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg445-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93307 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7308 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.45@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.231242 s, 72.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133952 s, 78.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0216833 s, 48.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0347917 s, 30.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0369948 s, 28.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0295971 s, 35.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0359289 s, 29.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0456125 s, 23.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0484471 s, 21.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385886 s, 27.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0466166 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0441249 s, 23.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0407765 s, 25.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0453629 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0427917 s, 24.5 MB/s
1048576 bytes (1.0 MB) copied, 0.04724 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0544542 s, 19.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00679322 s, 154 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00762328 s, 138 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116885 s, 89.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111032 s, 94.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00915194 s, 115 MB/s
, 0.00415648 s, 252 MB/s
1048576 bytes (1.0 MB) copied, 0.00434626 s, 241 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011137 s, 94.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109658 s, 95.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115668 s, 90.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011642 s, 90.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120371 s, 87.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131196 s, 79.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123221 s, 85.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123686 s, 84.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133522 s, 78.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg445-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92526 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88007 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.428892 s, 39.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191767 s, 54.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0267356 s, 39.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151472 s, 69.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0172726 s, 60.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0298666 s, 35.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0330275 s, 31.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196087 s, 53.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0246084 s, 42.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0703542 s, 14.9 MB/s
, 0.0743314 s, 14.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.224958 s, 4.7 MB/s
, 0.224212 s, 4.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.277817 s, 3.8 MB/s
, 0.280701 s, 3.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.415014 s, 2.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.432198 s, 2.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00626679 s, 167 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.006218 s, 169 MB/s
1048576 bytes (1.0 MB) copied, 0.0107121 s, 97.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00949415 s, 110 MB/s
1048576 bytes (1.0 MB) copied, 0.00763159 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0111691 s, 93.9 MB/s
1048576 bytes (1.0 MB) copied, 0.00894754 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101866 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102562 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00836746 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101915 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103451 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105479 s, 99.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.010331 s, 101 MB/s
1048576 bytes (1.0 MB) copied, 0.0104171 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117954 s, 88.9 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg445-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77928 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72976 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91248 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7338 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.834209 s, 20.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116426 s, 90.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011492 s, 91.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228251 s, 45.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0204644 s, 51.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0250289 s, 41.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0227935 s, 46.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0258922 s, 40.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226528 s, 46.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.173343 s, 6.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.180625 s, 5.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.178913 s, 5.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.189298 s, 5.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.201016 s, 5.2 MB/s
, 0.201757 s, 5.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.201847 s, 5.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.209694 s, 5.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
, 0.00714544 s, 147 MB/s
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0070997 s, 148 MB/s
, 0.00646692 s, 162 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105885 s, 99.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00755566 s, 139 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112453 s, 93.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00208338 s, 503 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00840756 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00960836 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00969518 s, 108 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0113841 s, 92.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0120006 s, 87.4 MB/s
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0134317 s, 78.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0123925 s, 84.6 MB/s
, 0.0117916 s, 88.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013019 s, 80.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg445-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.8306 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4296 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.23563 s, 71.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00876542 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114612 s, 91.5 MB/s
, 0.0102073 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0211011 s, 49.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0355892 s, 29.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0354489 s, 29.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0416523 s, 25.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0426366 s, 24.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.040482 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0305351 s, 34.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357471 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0386474 s, 27.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0340569 s, 30.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0371683 s, 28.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0394965 s, 26.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0425443 s, 24.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00995848 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00655175 s, 160 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00722734 s, 145 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0084592 s, 124 MB/s
1048576 bytes (1.0 MB) copied, 0.00193913 s, 541 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110024 s, 95.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00865554 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00958667 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107797 s, 97.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0121434 s, 86.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0122652 s, 85.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115879 s, 90.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108417 s, 96.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121331 s, 86.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136634 s, 76.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156163 s, 67.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg445-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.94341 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75034 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.431655 s, 38.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.012497 s, 83.9 MB/s
, 0.0120837 s, 86.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170306 s, 61.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121202 s, 86.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0117905 s, 88.9 MB/s
, 0.0117538 s, 89.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143067 s, 73.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00839921 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.076148 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0895833 s, 11.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0875004 s, 12.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0894701 s, 11.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.368643 s, 2.8 MB/s
, 0.385536 s, 2.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.423414 s, 2.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.437035 s, 2.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00819679 s, 128 MB/s
, 0.00950675 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00644237 s, 163 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00652672 s, 161 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00849826 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00848268 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0082918 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00855764 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00921488 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00767705 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00619344 s, 169 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00901408 s, 116 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00970717 s, 108 MB/s
, 0.0101302 s, 104 MB/s
1048576 bytes (1.0 MB) copied, 0.0110285 s, 95.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104239 s, 101 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg445-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg445-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7699 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73071 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76696 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73072 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo