== sanityn test 77kb: check different granular NID/UID/GID/JobID/OPCode combination ========================================================== 04:14:35 (1726301675)
ost.OSS.ost_io.nrs_policies=tbf jobid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.138856 s, 121 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011402 s, 92.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0128728 s, 81.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0555156 s, 18.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0520334 s, 20.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0519471 s, 20.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0682817 s, 15.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.065549 s, 16.0 MB/s
1048576 bytes (1.0 MB) copied, 0.055665 s, 18.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0716352 s, 14.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0646283 s, 16.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.062333 s, 16.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0705226 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0659129 s, 15.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0674786 s, 15.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0751048 s, 14.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0696891 s, 15.0 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a55f800.lock_unused_count=4
ldlm.namespaces.lustre-OST0001-osc-ffff88012a55f800.lock_unused_count=6
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0202927 s, 51.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183825 s, 57.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183534 s, 57.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0201833 s, 52.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0179351 s, 58.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0205071 s, 51.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0188991 s, 55.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0190714 s, 55.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0195712 s, 53.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.019679 s, 53.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0206106 s, 50.9 MB/s
, 0.0187399 s, 56.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.021494 s, 48.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.022511 s, 46.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0222661 s, 47.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0241142 s, 43.5 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a55f800.lock_unused_count=1
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg453-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91201 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74097 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
Waiting for MDT destroys to complete
oleg453-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg453-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf nid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.53@tcp}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.53@tcp}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.15053 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115478 s, 90.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0151802 s, 69.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0271987 s, 38.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0434254 s, 24.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0397423 s, 26.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0505184 s, 20.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0489016 s, 21.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0389794 s, 26.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0478338 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0574656 s, 18.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0577999 s, 18.1 MB/s
, 0.0536025 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0535695 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0576579 s, 18.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0563758 s, 18.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0568781 s, 18.4 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a55f800.lock_unused_count=2
ldlm.namespaces.lustre-OST0001-osc-ffff88012a55f800.lock_unused_count=4
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.015571 s, 67.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0156912 s, 66.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0174014 s, 60.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0215093 s, 48.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0176415 s, 59.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0175071 s, 59.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0197121 s, 53.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0247859 s, 42.3 MB/s
, 0.0179064 s, 58.6 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a55f800.lock_unused_count=2
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0235553 s, 44.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203001 s, 51.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0218256 s, 48.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.020221 s, 51.9 MB/s
, 0.0200942 s, 52.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0209473 s, 50.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260362 s, 40.3 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg453-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90915 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73978 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
Waiting for MDT destroys to complete
oleg453-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg453-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf nid+jobid
ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.53@tcp}&jobid={dd.500} rate=20
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.162497 s, 103 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175011 s, 59.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0205877 s, 50.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0200714 s, 52.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0182405 s, 57.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00920371 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260241 s, 40.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0514766 s, 20.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0213875 s, 49.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0288355 s, 36.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0524025 s, 20.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.048902 s, 21.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0521948 s, 20.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0345063 s, 30.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0563996 s, 18.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.037604 s, 27.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.044789 s, 23.4 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a55f800.lock_unused_count=1
ldlm.namespaces.lustre-OST0001-osc-ffff88012a55f800.lock_unused_count=7
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00707548 s, 148 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107741 s, 97.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00811848 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00814195 s, 129 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00690734 s, 152 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0103524 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0150397 s, 69.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.009088 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00921714 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0108076 s, 97.0 MB/s
1048576 bytes (1.0 MB) copied, 0.00934658 s, 112 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00940438 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109498 s, 95.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110879 s, 94.6 MB/s
1048576 bytes (1.0 MB) copied, 0.0113303 s, 92.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011678 s, 89.8 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a55f800.lock_unused_count=2
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg453-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.90837 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.97405 s, 21.1 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
Waiting for MDT destroys to complete
oleg453-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg453-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf jobid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.148962 s, 113 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0121836 s, 86.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122534 s, 85.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00747767 s, 140 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0188939 s, 55.5 MB/s
, 0.0295175 s, 35.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0207116 s, 50.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0139976 s, 74.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0231746 s, 45.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0317945 s, 33.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168179 s, 62.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0360771 s, 29.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0295181 s, 35.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0235592 s, 44.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0237707 s, 44.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.036268 s, 28.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242707 s, 43.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0102796 s, 102 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0126048 s, 83.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186654 s, 56.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0110332 s, 95.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175182 s, 59.9 MB/s
, 0.0141956 s, 73.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145014 s, 72.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0145624 s, 72.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0162304 s, 64.6 MB/s
, 0.014207 s, 73.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163408 s, 64.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173985 s, 60.3 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0175702 s, 59.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0158284 s, 66.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0163104 s, 64.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0203714 s, 51.5 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg453-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7632 s, 10.7 MB/s
Write runtime is 10 s, speed is 10.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.7397 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
Waiting for MDT destroys to complete
oleg453-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
oleg453-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.9053 s, 21.4 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74156 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
Waiting for MDT destroys to complete
oleg453-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg453-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf uid+gid
ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.151532 s, 111 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0083956 s, 125 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101256 s, 104 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00960113 s, 109 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101454 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0289979 s, 36.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0214643 s, 48.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0186502 s, 56.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0199956 s, 52.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0330871 s, 31.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0336718 s, 31.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.033913 s, 30.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0456164 s, 23.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0370341 s, 28.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.034717 s, 30.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0376306 s, 27.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0452004 s, 23.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119762 s, 87.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0138236 s, 75.9 MB/s
, 0.0175033 s, 59.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127745 s, 82.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0177942 s, 58.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0067445 s, 155 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0146451 s, 71.6 MB/s
, 0.0159576 s, 65.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165559 s, 63.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0168344 s, 62.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0140054 s, 74.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0191851 s, 54.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.016793 s, 62.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016788 s, 62.5 MB/s
1048576 bytes (1.0 MB) copied, 0.016727 s, 62.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.016999 s, 61.7 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg453-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 5, read rate: 5
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.4667 s, 5.4 MB/s
Write runtime is 20 s, speed is 5.000000 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 19.9513 s, 5.3 MB/s
Read runtime is 21 s, speed is 4.761904 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
Waiting for MDT destroys to complete
oleg453-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg453-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf uid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.14165 s, 118 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00853126 s, 123 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.018939 s, 55.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0390465 s, 26.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0508918 s, 20.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0578306 s, 18.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0449724 s, 23.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0634295 s, 16.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0553102 s, 19.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0599206 s, 17.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0556034 s, 18.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0506863 s, 20.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0546521 s, 19.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0587422 s, 17.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.059572 s, 17.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0536112 s, 19.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0652327 s, 16.1 MB/s
ldlm.namespaces.lustre-OST0000-osc-ffff88012a55f800.lock_unused_count=2
ldlm.namespaces.lustre-OST0001-osc-ffff88012a55f800.lock_unused_count=5
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00772865 s, 136 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00245669 s, 427 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0052412 s, 200 MB/s
1048576 bytes (1.0 MB) copied, 0.0138129 s, 75.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0157663 s, 66.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187514 s, 55.9 MB/s
1048576 bytes (1.0 MB) copied, 0.00831122 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0184936 s, 56.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00186552 s, 562 MB/s
ldlm.namespaces.lustre-OST0001-osc-ffff88012a55f800.lock_unused_count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00257564 s, 407 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00175541 s, 597 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00234277 s, 448 MB/s
, 0.0023327 s, 450 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00877828 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123894 s, 84.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0275991 s, 38.0 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg453-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.91739 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74291 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
Waiting for MDT destroys to complete
oleg453-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg453-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw
ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur
ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet] [bs=1M] [count=16]
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.130983 s, 128 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_w]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00877765 s, 119 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00908433 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00828638 s, 127 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0292023 s, 35.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0187658 s, 55.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0321376 s, 32.6 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0314628 s, 33.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0416379 s, 25.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.029418 s, 35.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0312903 s, 33.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0410717 s, 25.5 MB/s
1048576 bytes (1.0 MB) copied, 0.0320631 s, 32.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0386892 s, 27.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0364831 s, 28.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0442583 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0470092 s, 22.3 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [declare] [-a] [pids_r]
execvp fails running declare (2): No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137287 s, 76.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0159364 s, 65.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122454 s, 85.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01341 s, 78.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00576226 s, 182 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0143057 s, 73.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0147067 s, 71.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0165363 s, 63.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0161799 s, 64.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0166829 s, 62.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0175183 s, 59.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170962 s, 61.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0173905 s, 60.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0171642 s, 61.1 MB/s
, 0.0199329 s, 52.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0217888 s, 48.1 MB/s
removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg453-client.virtnet'
removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg453-client.virtnet'
removed directory: '/mnt/lustre/d77kb.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 10, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.76442 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74113 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
Waiting for MDT destroys to complete
oleg453-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg453-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.75673 s, 10.7 MB/s
Write runtime is 11 s, speed is 9.090909 IOPS
Verify the read rate is under TBF control
running as uid/gid/euid/egid 500/500/500/500, groups:
 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct]
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.74271 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kb.sanityn
Waiting for MDT destroys to complete
oleg453-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported
oleg453-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
pdsh@oleg453-client: oleg453-server: ssh exited with exit code 1
ost.OSS.ost_io.nrs_tbf_rule=stop ext_a
ost.OSS.ost_io.nrs_tbf_rule=stop ext_b
ost.OSS.ost_io.nrs_policies=fifo