== sanityn test 77kb: check different granular NID/UID/GID/JobID/OPCode combination ========================================================== 05:35:00 (1733394900)
ost.OSS.ost_io.nrs_policies=tbf jobid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.240474 s, 69.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010659 s, 98.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0235384 s, 44.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.017082 s, 61.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0223435 s, 46.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0411912 s, 25.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0388619 s, 27.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0341782 s, 30.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0400927 s, 26.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0446729 s, 23.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0457715 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0394376 s, 26.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0535343 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0485658 s, 21.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0435001 s, 24.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0433601 s, 24.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0551833 s, 19.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113918 s, 92.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0130943 s, 80.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173282 s, 60.5 MB/s
, 0.017341 s, 60.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143868 s, 72.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00563292 s, 186 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148851 s, 70.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0135031 s, 77.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0124319 s, 84.3 MB/s
, 0.0153991 s, 68.1 MB/s
, 0.0154441 s, 67.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00322443 s, 325 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015434 s, 67.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156961 s, 66.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156399 s, 67.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0166698 s, 62.9 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg205-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95741 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72928 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf nid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.5@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.5@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.111457 s, 151 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00851171 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112416 s, 93.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0221845 s, 47.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0294184 s, 35.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0364913 s, 28.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0221165 s, 47.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0280939 s, 37.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0361305 s, 29.0 MB/s
1048576 bytes (1.0 MB) copied, 0.0394368 s, 26.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0407655 s, 25.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0412443 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0333388 s, 31.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.205314 s, 5.1 MB/s
, 0.213555 s, 4.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.258794 s, 4.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.25956 s, 4.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
, 0.0149262 s, 70.3 MB/s
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0170192 s, 61.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0120133 s, 87.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172687 s, 60.7 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168783 s, 62.1 MB/s
, 0.0164723 s, 63.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131216 s, 79.9 MB/s
, 0.0127237 s, 82.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0160859 s, 65.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0163235 s, 64.2 MB/s
, 0.0161435 s, 65.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0176392 s, 59.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0193675 s, 54.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0178084 s, 58.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0189424 s, 55.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203741 s, 51.5 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg205-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.98694 s, 21.0 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73546 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf nid+jobid
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.5@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.137355 s, 122 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00797713 s, 131 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101529 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156048 s, 67.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.025613 s, 40.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163463 s, 64.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129556 s, 80.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0377969 s, 27.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0246613 s, 42.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412725 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0293964 s, 35.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0263027 s, 39.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0330982 s, 31.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.027706 s, 37.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0309094 s, 33.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0280608 s, 37.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329037 s, 31.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147034 s, 71.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127417 s, 82.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136876 s, 76.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132538 s, 79.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0162334 s, 64.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139067 s, 75.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0173557 s, 60.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127096 s, 82.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00268065 s, 391 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00235652 s, 445 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163134 s, 64.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00310209 s, 338 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118188 s, 88.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128566 s, 81.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00833431 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00329655 s, 318 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg205-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95507 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96274 s, 21.1 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf jobid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.447897 s, 37.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00882942 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.023105 s, 45.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140871 s, 74.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101891 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0744082 s, 14.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0713959 s, 14.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0668924 s, 15.7 MB/s
, 0.0782219 s, 13.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0919513 s, 11.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0877431 s, 12.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0836678 s, 12.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0801525 s, 13.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.086986 s, 12.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0910596 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.223533 s, 4.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.23307 s, 4.5 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00843384 s, 124 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00618642 s, 169 MB/s
, 0.00190338 s, 551 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00733095 s, 143 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0082424 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00434505 s, 241 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00871237 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00200575 s, 523 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00646819 s, 162 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105295 s, 99.6 MB/s
, 0.00969323 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105497 s, 99.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00243155 s, 431 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106245 s, 98.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105334 s, 99.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00193539 s, 542 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg205-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.82103 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74632 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91664 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73732 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf uid+gid
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.864501 s, 19.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0222069 s, 47.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0243408 s, 43.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0258304 s, 40.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0325317 s, 32.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0389605 s, 26.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329288 s, 31.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0414633 s, 25.3 MB/s
, 0.0329302 s, 31.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.149922 s, 7.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.178307 s, 5.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.17445 s, 6.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.175066 s, 6.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.175467 s, 6.0 MB/s
1048576 bytes (1.0 MB) copied, 0.177214 s, 5.9 MB/s
, 0.19789 s, 5.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.183018 s, 5.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00666719 s, 157 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00927979 s, 113 MB/s
, 0.00779948 s, 134 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00808113 s, 130 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132973 s, 78.9 MB/s
1048576 bytes (1.0 MB) copied, 0.00927588 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00994117 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115417 s, 90.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121966 s, 86.0 MB/s
1+0 records in
1+0 records out
, 0.0106758 s, 98.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0128826 s, 81.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127158 s, 82.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138617 s, 75.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0133046 s, 78.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128016 s, 81.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155948 s, 67.2 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg205-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.7299 s, 5.3 MB/s
Write runtime is 21 s, speed is 4.761904 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.9383 s, 5.3 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf uid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.095168 s, 176 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112264 s, 93.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0092063 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155783 s, 67.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0140453 s, 74.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0176403 s, 59.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0247397 s, 42.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0216231 s, 48.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123708 s, 84.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0355982 s, 29.5 MB/s
, 0.0231034 s, 45.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0334291 s, 31.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0396966 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.040502 s, 25.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228772 s, 45.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0291263 s, 36.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0425058 s, 24.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0098271 s, 107 MB/s
, 0.00673167 s, 156 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.005859 s, 179 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00872203 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00783675 s, 134 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00796276 s, 132 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00321547 s, 326 MB/s
, 0.00980131 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00936106 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00929175 s, 113 MB/s
1048576 bytes (1.0 MB) copied, 0.00938888 s, 112 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0102467 s, 102 MB/s
, 0.0105777 s, 99.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107307 s, 97.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105127 s, 99.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113245 s, 92.6 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg205-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92151 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73638 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.136717 s, 123 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0155381 s, 67.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184434 s, 56.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0185555 s, 56.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.030816 s, 34.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0237868 s, 44.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0427021 s, 24.6 MB/s
1048576 bytes (1.0 MB) copied, 0.04766 s, 22.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0412624 s, 25.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.032537 s, 32.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0389193 s, 26.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357857 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0368243 s, 28.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0414916 s, 25.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0538059 s, 19.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0912557 s, 11.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0949127 s, 11.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138661 s, 75.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105002 s, 99.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0166062 s, 63.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111356 s, 94.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0129964 s, 80.7 MB/s
, 0.0129847 s, 80.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0136968 s, 76.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140698 s, 74.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0149237 s, 70.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0160768 s, 65.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147817 s, 70.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173077 s, 60.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200847 s, 52.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0172052 s, 60.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0207512 s, 50.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0196157 s, 53.5 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg205-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg205-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.8213 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74397 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.80398 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74098 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo