== sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 15:52:35 (1733172755)
ost.OSS.ost_io.nrs_policies=tbf
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.117964 s, 142 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111472 s, 94.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154827 s, 67.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.030676 s, 34.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0367593 s, 28.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0356766 s, 29.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.040151 s, 26.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.040928 s, 25.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0400759 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0514028 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0441871 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0394734 s, 26.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0526557 s, 19.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0533482 s, 19.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0488579 s, 21.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0543446 s, 19.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0501812 s, 20.9 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff880137393000.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00939033 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107883 s, 97.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00621659 s, 169 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00668748 s, 157 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00860989 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010436 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100182 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00922206 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00896161 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00959302 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102233 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00995944 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115654 s, 90.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110041 s, 95.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112622 s, 93.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121172 s, 86.5 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg225-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89215 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73055 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg225-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg225-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.25@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.25@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.118889 s, 141 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145822 s, 71.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.023406 s, 44.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336933 s, 31.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0290603 s, 36.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0239286 s, 43.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0307651 s, 34.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0401051 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0457615 s, 22.9 MB/s
1048576 bytes (1.0 MB) copied, 0.048226 s, 21.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0411919 s, 25.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0475123 s, 22.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122482 s, 85.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0379445 s, 27.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0353859 s, 29.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174698 s, 60.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0405696 s, 25.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00972991 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00778394 s, 135 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00708207 s, 148 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00875102 s, 120 MB/s
, 0.00897086 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122644 s, 85.5 MB/s
1048576 bytes (1.0 MB) copied, 0.00826289 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00960294 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101738 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00969326 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116005 s, 90.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120767 s, 86.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126601 s, 82.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130653 s, 80.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132296 s, 79.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157087 s, 66.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg225-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89352 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72925 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg225-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg225-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.25@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.142012 s, 118 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.014447 s, 72.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0166186 s, 63.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0346699 s, 30.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0407833 s, 25.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.042339 s, 24.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0501611 s, 20.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0479556 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0521231 s, 20.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0432567 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0525019 s, 20.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0407694 s, 25.7 MB/s
, 0.0495845 s, 21.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0424605 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0473198 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0448714 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0553953 s, 18.9 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff880137393000.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00890596 s, 118 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011179 s, 93.8 MB/s
1048576 bytes (1.0 MB) copied, 0.010892 s, 96.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110289 s, 95.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122838 s, 85.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136119 s, 77.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104314 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00606638 s, 173 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125173 s, 83.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120205 s, 87.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0112431 s, 93.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0129679 s, 80.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113067 s, 92.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135009 s, 77.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127161 s, 82.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156993 s, 66.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg225-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.89097 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88176 s, 21.5 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg225-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg225-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.126527 s, 133 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171208 s, 61.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176608 s, 59.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0164952 s, 63.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141909 s, 73.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345011 s, 30.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191598 s, 54.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0296557 s, 35.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0303023 s, 34.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0399799 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336493 s, 31.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0427053 s, 24.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0450456 s, 23.3 MB/s
, 0.0328256 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0319728 s, 32.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0395379 s, 26.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125598 s, 83.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00792842 s, 132 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00739411 s, 142 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00657937 s, 159 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00709648 s, 148 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130099 s, 80.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00979785 s, 107 MB/s
1048576 bytes (1.0 MB) copied, 0.0129931 s, 80.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00995754 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112797 s, 93.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0117857 s, 89.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0114434 s, 91.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0123747 s, 84.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124969 s, 83.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127881 s, 82.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124578 s, 84.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122474 s, 85.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg225-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.742 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72738 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg225-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg225-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.88553 s, 21.5 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73225 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg225-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg225-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.13249 s, 127 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113624 s, 92.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148642 s, 70.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154982 s, 67.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0362872 s, 28.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0352518 s, 29.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0384026 s, 27.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0436633 s, 24.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0485427 s, 21.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.04258 s, 24.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.034217 s, 30.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0413924 s, 25.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0497982 s, 21.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0381448 s, 27.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0460886 s, 22.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0481273 s, 21.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0485531 s, 21.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00863123 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00954312 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109224 s, 96.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00975984 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105581 s, 99.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0106524 s, 98.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129981 s, 80.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0094868 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101958 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00994654 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102241 s, 103 MB/s
, 0.0100555 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113802 s, 92.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01171 s, 89.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118451 s, 88.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120294 s, 87.2 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg225-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.6393 s, 5.3 MB/s
Write runtime is 20 s, speed is 5.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4333 s, 5.4 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg225-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg225-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.128326 s, 131 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125507 s, 83.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106705 s, 98.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170621 s, 61.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0277485 s, 37.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0316007 s, 33.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0262953 s, 39.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0400442 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0397931 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0365212 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124285 s, 84.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0407514 s, 25.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0471309 s, 22.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0385218 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0386269 s, 27.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0185614 s, 56.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0301655 s, 34.8 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff880137393000.lock_unused_count=2
ldlm.namespaces.lustre-OST0001-osc-ffff880137393000.lock_unused_count=1
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0106425 s, 98.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112064 s, 93.6 MB/s
, 0.0106137 s, 98.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131494 s, 79.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00430873 s, 243 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155363 s, 67.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00577041 s, 182 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126642 s, 82.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00637676 s, 164 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00735445 s, 143 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108793 s, 96.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112169 s, 93.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112771 s, 93.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119936 s, 87.4 MB/s
, 0.0116951 s, 89.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146105 s, 71.8 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg225-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.8926 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73357 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg225-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg225-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.128521 s, 131 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012465 s, 84.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143106 s, 73.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0379057 s, 27.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0367274 s, 28.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0301071 s, 34.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0422527 s, 24.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0305097 s, 34.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0443159 s, 23.7 MB/s
, 0.0321894 s, 32.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0435592 s, 24.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0419459 s, 25.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0429276 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.03966 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0405461 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0444836 s, 23.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0426529 s, 24.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0091373 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101027 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00827696 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122554 s, 85.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108347 s, 96.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0100026 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104624 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116222 s, 90.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111426 s, 94.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105004 s, 99.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108524 s, 96.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105236 s, 99.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0114782 s, 91.4 MB/s
1048576 bytes (1.0 MB) copied, 0.010802 s, 97.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115729 s, 90.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108565 s, 96.6 MB/s
removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg225-client.virtnet'
removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg225-client.virtnet'
removed directory: '/mnt/lustre/d77k.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74707 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72998 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg225-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg225-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73692 s, 10.8 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73307 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77k.sanityn
Waiting for MDT destroys to complete
oleg225-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg225-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
pdsh@oleg225-client: oleg225-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo