== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 08:43:02 (1743511382)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.240756 s, 69.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109401 s, 95.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0203779 s, 51.5 MB/s
, 0.0174137 s, 60.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0293663 s, 35.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0255664 s, 41.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0339457 s, 30.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0369219 s, 28.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0333574 s, 31.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0336553 s, 31.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0512145 s, 20.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0521347 s, 20.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.047545 s, 22.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0578778 s, 18.1 MB/s
, 0.0582641 s, 18.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.066408 s, 15.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0676141 s, 15.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00760843 s, 138 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00680428 s, 154 MB/s
1048576 bytes (1.0 MB) copied, 0.00871512 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00713452 s, 147 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00399081 s, 263 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00151606 s, 692 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00904699 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00957099 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00319292 s, 328 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00880699 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00658847 s, 159 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00903597 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104303 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00938593 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00934661 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112162 s, 93.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92834 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72993 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.10@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.10@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.237456 s, 70.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00836865 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00779877 s, 134 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129561 s, 80.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179958 s, 58.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157437 s, 66.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126082 s, 83.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136161 s, 77.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0244742 s, 42.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00749227 s, 140 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0398122 s, 26.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385763 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0403331 s, 26.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.274669 s, 3.8 MB/s
, 0.274421 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.305122 s, 3.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.310175 s, 3.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0130546 s, 80.3 MB/s
, 0.00432115 s, 243 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00722508 s, 145 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00153357 s, 684 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00158293 s, 662 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170879 s, 61.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00189118 s, 554 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00231819 s, 452 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162458 s, 64.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174304 s, 60.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141767 s, 74.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0114696 s, 91.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0160261 s, 65.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00270566 s, 388 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00828208 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00740905 s, 142 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93311 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72952 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.10@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.266728 s, 62.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119664 s, 87.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140623 s, 74.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0308866 s, 33.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0318369 s, 32.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0381396 s, 27.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0397032 s, 26.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365037 s, 28.7 MB/s
1048576 bytes (1.0 MB) copied, 0.029793 s, 35.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0387586 s, 27.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0365981 s, 28.7 MB/s
, 0.0328941 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0361874 s, 29.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0348039 s, 30.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0325936 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328482 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0443283 s, 23.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174713 s, 60.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130935 s, 80.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0095219 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161356 s, 65.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144809 s, 72.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0148398 s, 70.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156489 s, 67.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0114144 s, 91.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129169 s, 81.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124864 s, 84.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149569 s, 70.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0145113 s, 72.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144138 s, 72.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152448 s, 68.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182192 s, 57.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00215987 s, 485 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.00529 s, 20.9 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88288 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.449627 s, 37.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228429 s, 45.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111439 s, 94.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264174 s, 39.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00985285 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0257619 s, 40.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0250556 s, 41.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0255901 s, 41.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.023405 s, 44.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0574177 s, 18.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0600029 s, 17.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0900409 s, 11.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0861815 s, 12.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.368012 s, 2.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.359566 s, 2.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.400448 s, 2.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.4247 s, 2.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119302 s, 87.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147757 s, 71.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016345 s, 64.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128221 s, 81.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122852 s, 85.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124916 s, 83.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0142735 s, 73.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00274234 s, 382 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138239 s, 75.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131005 s, 80.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154177 s, 68.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143978 s, 72.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146616 s, 71.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013992 s, 74.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145255 s, 72.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184796 s, 56.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80526 s, 10.7 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74562 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91945 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73759 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.833844 s, 20.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107981 s, 97.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168223 s, 62.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0180266 s, 58.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0164706 s, 63.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264286 s, 39.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260795 s, 40.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0226732 s, 46.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0257298 s, 40.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.176018 s, 6.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.183159 s, 5.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.189629 s, 5.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.189319 s, 5.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.200196 s, 5.2 MB/s
1048576 bytes (1.0 MB) copied, 0.204698 s, 5.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.195148 s, 5.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.210464 s, 5.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128709 s, 81.5 MB/s
, 0.0120003 s, 87.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168197 s, 62.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011107 s, 94.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0201295 s, 52.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013291 s, 78.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161228 s, 65.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0147464 s, 71.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146837 s, 71.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124037 s, 84.5 MB/s
, 0.0147215 s, 71.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169889 s, 61.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00266919 s, 393 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200728 s, 52.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192622 s, 54.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0194126 s, 54.0 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7128 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4393 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.244632 s, 68.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017209 s, 60.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017896 s, 58.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115817 s, 90.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187301 s, 56.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0213805 s, 49.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0220885 s, 47.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0299457 s, 35.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0328956 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0231672 s, 45.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183868 s, 57.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.027023 s, 38.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0319963 s, 32.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0287118 s, 36.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0449568 s, 23.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0324947 s, 32.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00900354 s, 116 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00798323 s, 131 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00806569 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00921514 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100079 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132067 s, 79.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0153656 s, 68.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108777 s, 96.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110195 s, 95.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135978 s, 77.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118517 s, 88.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.012583 s, 83.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0119171 s, 88.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129859 s, 80.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130567 s, 80.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127881 s, 82.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149689 s, 70.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92666 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72958 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.446702 s, 37.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0233312 s, 44.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0232804 s, 45.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0246626 s, 42.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0211213 s, 49.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0283715 s, 37.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131114 s, 80.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157214 s, 66.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141468 s, 74.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.062769 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0732251 s, 14.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0748532 s, 14.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0768311 s, 13.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.325139 s, 3.2 MB/s
, 0.317021 s, 3.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.350427 s, 3.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.375504 s, 2.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0109737 s, 95.6 MB/s
, 0.0110755 s, 94.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0121185 s, 86.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143186 s, 73.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120857 s, 86.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128422 s, 81.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0130467 s, 80.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119655 s, 87.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128859 s, 81.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161249 s, 65.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138137 s, 75.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139433 s, 75.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141353 s, 74.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154872 s, 67.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136496 s, 76.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0164814 s, 63.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.85112 s, 10.6 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73703 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.77662 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73274 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo