-----============= acceptance-small: sanityn ============----- Thu Dec  5 04:34:38 EST 2024
mgs: CentOS Linux release 7.9.2009 (Core)
MGS_OS_VERSION_ID=7
MGS_OS_ID=centos
MGS_OS_VERSION_CODE=117440512
MGS_OS_ID_LIKE=rhel fedora centos
mds1: CentOS Linux release 7.9.2009 (Core)
MDS1_OS_ID_LIKE=rhel fedora centos
MDS1_OS_ID=centos
MDS1_OS_VERSION_ID=7
MDS1_OS_VERSION_CODE=117440512
ost1: CentOS Linux release 7.9.2009 (Core)
OST1_OS_VERSION_CODE=117440512
OST1_OS_VERSION_ID=7
OST1_OS_ID_LIKE=rhel fedora centos
OST1_OS_ID=centos
client: CentOS Linux release 7.9.2009 (Core)
CLIENT_OS_ID=centos
CLIENT_OS_ID_LIKE=rhel fedora centos
CLIENT_OS_VERSION_ID=7
CLIENT_OS_VERSION_CODE=117440512
excepting tests: 28 102
skipping tests SLOW=no: 33a
=== sanityn: start setup 04:34:42 (1733391282) ===
Starting client oleg218-client.virtnet:  -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre2
Started clients oleg218-client.virtnet: 
192.168.202.118@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project)
oleg218-client.virtnet: executing check_config_client /mnt/lustre
oleg218-client.virtnet: Checking config lustre mounted on /mnt/lustre
Checking servers environments
Checking clients oleg218-client.virtnet environments
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff88012aaa2000.idle_timeout=debug
osc.lustre-OST0000-osc-ffff88012aaa7000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff88012aaa2000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff88012aaa7000.idle_timeout=debug
disable quota as required
oleg218-server: oleg218-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
=== sanityn: finish setup 04:34:49 (1733391289) ===
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106808 s, 98.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [true]
running as uid/gid/euid/egid 500/500/500/500, groups:
 [touch] [/mnt/lustre/d0_runas_test/f6968]

== sanityn test 77kf: Verify ID range with comparator works correctly ========================================================== 04:34:50 (1733391290)
using SAVE_PROJECT_SUPPORTED=0
ost.OSS.ost_io.nrs_policies=tbf projid
ost.OSS.ost_io.nrs_tbf_rule=start ext_rw projid>={100}&projid<{200} rate=20
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.266551 s, 62.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0192358 s, 54.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0243168 s, 43.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0479391 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0310982 s, 33.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.059116 s, 17.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0547643 s, 19.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0586552 s, 17.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0628874 s, 16.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.068224 s, 15.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0634348 s, 16.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0993532 s, 10.6 MB/s
, 0.107683 s, 9.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.27227 s, 3.9 MB/s
, 0.254643 s, 4.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.308938 s, 3.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.310493 s, 3.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.118996 s, 8.8 MB/s
1048576 bytes (1.0 MB) copied, 0.120567 s, 8.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.130353 s, 8.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.117561 s, 8.9 MB/s
1048576 bytes (1.0 MB) copied, 0.12551 s, 8.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.116601 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.116764 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.116792 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.112561 s, 9.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.116567 s, 9.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.110187 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.117905 s, 8.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.111037 s, 9.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.109857 s, 9.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.105173 s, 10.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.120417 s, 8.7 MB/s
removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg218-client.virtnet'
removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg218-client.virtnet'
removed directory: '/mnt/lustre/d77kf.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.9963 s, 21.0 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.95685 s, 21.2 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77kf.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.236023 s, 71.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0170691 s, 61.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0131974 s, 79.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.018105 s, 57.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.01777 s, 59.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0294338 s, 35.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0181073 s, 57.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113174 s, 92.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0101714 s, 103 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0188264 s, 55.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0400866 s, 26.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.052067 s, 20.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0338817 s, 30.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.273665 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.277322 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.325931 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.328063 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183852 s, 57.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0185902 s, 56.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0208983 s, 50.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0222438 s, 47.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198656 s, 52.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0198505 s, 52.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0210255 s, 49.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0208866 s, 50.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0210951 s, 49.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0219899 s, 47.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228481 s, 45.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0228118 s, 46.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0230514 s, 45.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0233689 s, 44.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0242878 s, 43.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0240925 s, 43.5 MB/s
removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg218-client.virtnet'
removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg218-client.virtnet'
removed directory: '/mnt/lustre/d77kf.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.94257 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.97157 s, 21.1 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77kf.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_rw
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf projid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_w projid>={100}&projid<{200}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r projid>={100}&projid<{200}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.248787 s, 67.4 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.013322 s, 78.7 MB/s
1048576 bytes (1.0 MB) copied, 0.0192864 s, 54.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0245207 s, 42.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0241533 s, 43.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0458727 s, 22.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0357422 s, 29.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385257 s, 27.2 MB/s
, 0.0340741 s, 30.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0726556 s, 14.4 MB/s
1048576 bytes (1.0 MB) copied, 0.0666663 s, 15.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.114214 s, 9.2 MB/s
, 0.107361 s, 9.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.1567 s, 6.7 MB/s
1048576 bytes (1.0 MB) copied, 0.159345 s, 6.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.26758 s, 3.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.250625 s, 4.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00957258 s, 110 MB/s
, 0.00844312 s, 124 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104854 s, 100 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0122021 s, 85.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0106003 s, 98.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0095426 s, 110 MB/s
1048576 bytes (1.0 MB) copied, 0.0109027 s, 96.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0099134 s, 106 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0015862 s, 661 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.011655 s, 90.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115738 s, 90.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111294 s, 94.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0123783 s, 84.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0127651 s, 82.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0116681 s, 89.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141183 s, 74.3 MB/s
removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg218-client.virtnet'
removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg218-client.virtnet'
removed directory: '/mnt/lustre/d77kf.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93307 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73207 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kf.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.235017 s, 71.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0132721 s, 79.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00870501 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0269245 s, 38.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0319067 s, 32.9 MB/s
1048576 bytes (1.0 MB) copied, 0.0274566 s, 38.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0329109 s, 31.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0355127 s, 29.5 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0385232 s, 27.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0516307 s, 20.3 MB/s
1048576 bytes (1.0 MB) copied, 0.0478256 s, 21.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.047611 s, 22.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0390037 s, 26.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.275755 s, 3.8 MB/s
, 0.262447 s, 4.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.323063 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.325444 s, 3.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00502421 s, 209 MB/s
, 0.0100258 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0068628 s, 153 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00547108 s, 192 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00684906 s, 153 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.006843 s, 153 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108505 s, 96.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00748495 s, 140 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0087759 s, 119 MB/s
1048576 bytes (1.0 MB) copied, 0.00875902 s, 120 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010874 s, 96.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109523 s, 95.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00978351 s, 107 MB/s
, 0.0113012 s, 92.8 MB/s
, 0.0116288 s, 90.2 MB/s
1048576 bytes (1.0 MB) copied, 0.0116398 s, 90.1 MB/s
removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg218-client.virtnet'
removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg218-client.virtnet'
removed directory: '/mnt/lustre/d77kf.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.93755 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73413 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kf.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_policies=fifo
PASS 77kf (136s)
cleanup: ======================================================
== sanityn test complete, duration 148 sec =============== 04:37:06 (1733391426)
=== sanityn: start cleanup 04:37:07 (1733391427) ===
Stopping clients: oleg218-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg218-client.virtnet /mnt/lustre2 opts:
=== sanityn: finish cleanup 04:37:07 (1733391427) ===