== sanityn test 77kb: check different granular NID/UID/GID/JobID/OPCode combination ========================================================== 04:16:47 (1726301807)
ost.OSS.ost_io.nrs_policies=tbf jobid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.26994 s, 62.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146676 s, 71.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196631 s, 53.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0469038 s, 22.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0448804 s, 23.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0511353 s, 20.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0569456 s, 18.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0534732 s, 19.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0578491 s, 18.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0591634 s, 17.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0507978 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0467989 s, 22.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0501394 s, 20.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0469288 s, 22.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.057319 s, 18.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0495191 s, 21.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0608779 s, 17.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0127808 s, 82.0 MB/s
, 0.0146168 s, 71.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0144207 s, 72.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0126663 s, 82.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0127088 s, 82.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149767 s, 70.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200351 s, 52.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0160569 s, 65.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163237 s, 64.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0154851 s, 67.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162173 s, 64.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177103 s, 59.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0178818 s, 58.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0178042 s, 58.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0180242 s, 58.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0188043 s, 55.8 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg210-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95755 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74186 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf nid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.10@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.10@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.292636 s, 57.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181094 s, 57.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.026026 s, 40.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186309 s, 56.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0368835 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0356608 s, 29.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0383865 s, 27.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0494938 s, 21.2 MB/s
, 0.0550652 s, 19.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0830582 s, 12.6 MB/s
, 0.0865709 s, 12.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.139341 s, 7.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.153298 s, 6.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.244826 s, 4.3 MB/s
, 0.253262 s, 4.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.286803 s, 3.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.285357 s, 3.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0150211 s, 69.8 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174556 s, 60.1 MB/s
, 0.0132196 s, 79.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0148549 s, 70.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150779 s, 69.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162116 s, 64.7 MB/s
, 0.0161251 s, 65.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159606 s, 65.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184306 s, 56.9 MB/s
, 0.0182195 s, 57.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0173022 s, 60.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173158 s, 60.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0192279 s, 54.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0189855 s, 55.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196527 s, 53.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0212335 s, 49.4 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg210-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.99726 s, 21.0 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74312 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf nid+jobid
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.10@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.271591 s, 61.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176455 s, 59.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0211812 s, 49.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191078 s, 54.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0495735 s, 21.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0512839 s, 20.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0449726 s, 23.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0467251 s, 22.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0390377 s, 26.9 MB/s
, 0.0459142 s, 22.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0615712 s, 17.0 MB/s
1048576 bytes (1.0 MB) copied, 0.052031 s, 20.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0388111 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0490007 s, 21.4 MB/s
1048576 bytes (1.0 MB) copied, 0.040216 s, 26.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0577015 s, 18.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0630525 s, 16.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151667 s, 69.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147394 s, 71.1 MB/s
, 0.0102861 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0180062 s, 58.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0153829 s, 68.2 MB/s
, 0.0153898 s, 68.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189103 s, 55.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159111 s, 65.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176856 s, 59.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157439 s, 66.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0186596 s, 56.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171373 s, 61.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174481 s, 60.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176382 s, 59.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017956 s, 58.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0193018 s, 54.3 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg210-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95087 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.97238 s, 21.1 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf jobid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.469026 s, 35.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0180508 s, 58.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.040823 s, 25.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.027387 s, 38.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0316538 s, 33.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0564343 s, 18.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0604128 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0551321 s, 19.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0715026 s, 14.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.063837 s, 16.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0830543 s, 12.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0714086 s, 14.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0690125 s, 15.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.325276 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.313755 s, 3.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.356432 s, 2.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.366119 s, 2.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00979582 s, 107 MB/s
1048576 bytes (1.0 MB) copied, 0.0105084 s, 99.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128696 s, 81.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161401 s, 65.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0116037 s, 90.4 MB/s
, 0.0192706 s, 54.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162666 s, 64.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137686 s, 76.2 MB/s
, 0.0166277 s, 63.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104186 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151595 s, 69.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0160625 s, 65.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157108 s, 66.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168803 s, 62.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189969 s, 55.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0195007 s, 53.8 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg210-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.86447 s, 10.6 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74227 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.94914 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74045 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf uid+gid
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.875603 s, 19.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0236002 s, 44.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0364699 s, 28.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0426748 s, 24.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0465348 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0587344 s, 17.9 MB/s
, 0.0604152 s, 17.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0615519 s, 17.0 MB/s
, 0.0586078 s, 17.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.147246 s, 7.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.157822 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.172043 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.173268 s, 6.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.19097 s, 5.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.19371 s, 5.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.204555 s, 5.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.207868 s, 5.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.015267 s, 68.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133965 s, 78.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179395 s, 58.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0134138 s, 78.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150532 s, 69.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171338 s, 61.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149213 s, 70.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0148938 s, 70.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167696 s, 62.5 MB/s
, 0.0180888 s, 58.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0171573 s, 61.1 MB/s
, 0.0174268 s, 60.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0205698 s, 51.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0187907 s, 55.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191425 s, 54.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196466 s, 53.4 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg210-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7496 s, 5.3 MB/s
Write runtime is 20 s, speed is 5.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.9355 s, 5.3 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf uid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.272163 s, 61.6 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012259 s, 85.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175591 s, 59.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0234649 s, 44.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0251517 s, 41.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130434 s, 80.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162352 s, 64.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0197985 s, 53.0 MB/s
, 0.021356 s, 49.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0471984 s, 22.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0272203 s, 38.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0507262 s, 20.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0443239 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0264503 s, 39.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0440669 s, 23.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0392643 s, 26.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0401171 s, 26.1 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.016036 s, 65.4 MB/s
, 0.0110559 s, 94.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.012864 s, 81.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00951717 s, 110 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167481 s, 62.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139067 s, 75.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121929 s, 86.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0077456 s, 135 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0169991 s, 61.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150054 s, 69.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0152481 s, 68.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0169001 s, 62.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0170422 s, 61.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151874 s, 69.0 MB/s
, 0.0152919 s, 68.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0167308 s, 62.7 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg210-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.94742 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7407 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.139859 s, 120 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113148 s, 92.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110237 s, 95.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0077057 s, 136 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0234869 s, 44.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0327073 s, 32.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0333816 s, 31.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0255129 s, 41.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0236235 s, 44.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0304559 s, 34.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0087655 s, 120 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0644784 s, 16.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0669123 s, 15.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0570288 s, 18.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0708137 s, 14.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.125983 s, 8.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.141949 s, 7.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0125119 s, 83.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129729 s, 80.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130928 s, 80.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136879 s, 76.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143077 s, 73.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0214242 s, 48.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156529 s, 67.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183751 s, 57.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0185682 s, 56.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184228 s, 56.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0243234 s, 43.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0207974 s, 50.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192547 s, 54.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200075 s, 52.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0227919 s, 46.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0236693 s, 44.3 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg210-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg210-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8332 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74317 s, 10.8 MB/s
Read runtime is 10 s, speed is 10.000000 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.79797 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74301 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo