== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 16:19:10 (1746130750)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.382117 s, 43.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0352997 s, 29.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0759387 s, 13.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.118294 s, 8.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.137699 s, 7.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.136031 s, 7.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.162329 s, 6.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.170659 s, 6.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.179079 s, 5.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.213241 s, 4.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.198988 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.210291 s, 5.0 MB/s
1048576 bytes (1.0 MB) copied, 0.215496 s, 4.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.241365 s, 4.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.219818 s, 4.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.239373 s, 4.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.269822 s, 3.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0379385 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275291 s, 38.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0282138 s, 37.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0311343 s, 33.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.026768 s, 39.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0279788 s, 37.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0331913 s, 31.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0276396 s, 37.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161339 s, 65.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182222 s, 57.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0213898 s, 49.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0215337 s, 48.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0233758 s, 44.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.023619 s, 44.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.025511 s, 41.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0261468 s, 40.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.84796 s, 15.3 MB/s
Write runtime is 8 s, speed is 12.500000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82991 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.49@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.49@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.216124 s, 77.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253723 s, 41.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0308289 s, 34.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0212559 s, 49.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0706002 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0688292 s, 15.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0628643 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.064272 s, 16.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0720341 s, 14.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0474638 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0485362 s, 21.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0578903 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0891323 s, 11.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0680286 s, 15.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0983156 s, 10.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0942718 s, 11.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.104088 s, 10.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0252115 s, 41.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264458 s, 39.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00210214 s, 499 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0371438 s, 28.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.037347 s, 28.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0222754 s, 47.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
, 0.0397699 s, 26.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0461466 s, 22.7 MB/s
1048576 bytes (1.0 MB) copied, 0.00578471 s, 181 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0370353 s, 28.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0370004 s, 28.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0423205 s, 24.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0324517 s, 32.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0291427 s, 36.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0290919 s, 36.0 MB/s
1048576 bytes (1.0 MB) copied, 0.042407 s, 24.7 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.3456 s, 16.5 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82036 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.49@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.201691 s, 83.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159705 s, 65.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0215752 s, 48.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0193284 s, 54.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.08944 s, 11.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0969219 s, 10.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.104374 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.111721 s, 9.4 MB/s
1048576 bytes (1.0 MB) copied, 0.10823 s, 9.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.117289 s, 8.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105133 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106587 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.122253 s, 8.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.133064 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.127446 s, 8.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.149988 s, 7.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.142073 s, 7.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00216988 s, 483 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0738348 s, 14.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0846034 s, 12.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0578676 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0914531 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0671754 s, 15.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0691811 s, 15.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0630504 s, 16.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0769498 s, 13.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0783955 s, 13.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0602864 s, 17.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0944292 s, 11.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0238196 s, 44.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0397216 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0603988 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0495927 s, 21.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.58725 s, 15.9 MB/s
Write runtime is 8 s, speed is 12.500000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.97064 s, 21.1 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.319651 s, 52.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0199285 s, 52.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0213507 s, 49.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0527716 s, 19.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0639939 s, 16.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.071685 s, 14.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0717532 s, 14.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0773148 s, 13.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0832012 s, 12.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0808031 s, 13.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0820027 s, 12.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.088872 s, 11.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0919068 s, 11.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.153727 s, 6.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.151861 s, 6.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.147824 s, 7.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.153297 s, 6.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0306814 s, 34.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0310833 s, 33.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0500493 s, 21.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0520469 s, 20.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0526794 s, 19.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0549034 s, 19.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0445381 s, 23.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0599056 s, 17.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0455117 s, 23.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0594149 s, 17.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0623994 s, 16.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0501009 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0660335 s, 15.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0532704 s, 19.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.07019 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0727543 s, 14.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.1378 s, 10.3 MB/s
Write runtime is 12 s, speed is 8.333333 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.81876 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.51363 s, 16.1 MB/s
Write runtime is 8 s, speed is 12.500000 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.84104 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.250558 s, 67.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345114 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104972 s, 99.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00895296 s, 117 MB/s
1048576 bytes (1.0 MB) copied, 0.0334798 s, 31.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0254497 s, 41.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0987523 s, 10.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0803253 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0978149 s, 10.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.118317 s, 8.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.12601 s, 8.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.106691 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.132979 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.148164 s, 7.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.154495 s, 6.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.129348 s, 8.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.163589 s, 6.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0453602 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.044408 s, 23.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327933 s, 32.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0464839 s, 22.6 MB/s
, 0.0333731 s, 31.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329243 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0330037 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0380109 s, 27.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0368747 s, 28.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0429589 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0417397 s, 25.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0435249 s, 24.1 MB/s
, 0.0433862 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0465508 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0464784 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0513664 s, 20.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7189 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.5262 s, 5.4 MB/s
Read runtime is 20 s, speed is 5.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.297205 s, 56.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0211644 s, 49.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0251653 s, 41.7 MB/s
, 0.0192125 s, 54.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0943744 s, 11.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0790177 s, 13.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0871736 s, 12.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0982554 s, 10.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107486 s, 9.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.102796 s, 10.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.13527 s, 7.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.131207 s, 8.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.146881 s, 7.1 MB/s
1048576 bytes (1.0 MB) copied, 0.13572 s, 7.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.152284 s, 6.9 MB/s
, 0.144756 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.175858 s, 6.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00588301 s, 178 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0543354 s, 19.3 MB/s
, 0.0544821 s, 19.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.06114 s, 17.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0552171 s, 19.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0558393 s, 18.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0693776 s, 15.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0541379 s, 19.4 MB/s
, 0.0336713 s, 31.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0391867 s, 26.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0601411 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00728399 s, 144 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0468225 s, 22.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0298064 s, 35.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0289123 s, 36.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177393 s, 59.1 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.54729 s, 16.0 MB/s
Write runtime is 7 s, speed is 14.285714 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.83258 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.315561 s, 53.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_w]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131201 s, 79.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0324496 s, 32.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.045539 s, 23.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0534503 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0452113 s, 23.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0565805 s, 18.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0492003 s, 21.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0428712 s, 24.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0647215 s, 16.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0563905 s, 18.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0496933 s, 21.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0589559 s, 17.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.132413 s, 7.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.144825 s, 7.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.229183 s, 4.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.25101 s, 4.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [declare] [-a] [pids_r]
execvp fails running declare (13): Permission denied
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.051304 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0525002 s, 20.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0540012 s, 19.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00458854 s, 229 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0294579 s, 35.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0680909 s, 15.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0686405 s, 15.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.024257 s, 43.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00209535 s, 500 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0491745 s, 21.3 MB/s
, 0.0362937 s, 28.9 MB/s
, 0.0703165 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0731261 s, 14.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.045479 s, 23.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0138798 s, 75.5 MB/s
, 0.0139084 s, 75.4 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg349-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg349-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 10.1942 s, 10.3 MB/s
Write runtime is 12 s, speed is 8.333333 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80919 s, 10.7 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.97202 s, 10.5 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.78919 s, 10.7 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
sleep 5 for ZFS MDS
sleep 5 for ZFS OST
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo