-----============= acceptance-small: sanityn ============----- Thu Dec  5 04:34:41 EST 2024
mgs: CentOS Linux release 7.9.2009 (Core)
MGS_OS_VERSION_ID=7
MGS_OS_ID=centos
MGS_OS_VERSION_CODE=117440512
MGS_OS_ID_LIKE=rhel fedora centos
mds1: CentOS Linux release 7.9.2009 (Core)
MDS1_OS_ID_LIKE=rhel fedora centos
MDS1_OS_ID=centos
MDS1_OS_VERSION_ID=7
MDS1_OS_VERSION_CODE=117440512
ost1: CentOS Linux release 7.9.2009 (Core)
OST1_OS_VERSION_CODE=117440512
OST1_OS_VERSION_ID=7
OST1_OS_ID_LIKE=rhel fedora centos
OST1_OS_ID=centos
client: CentOS Linux release 7.9.2009 (Core)
CLIENT_OS_ID=centos
CLIENT_OS_ID_LIKE=rhel fedora centos
CLIENT_OS_VERSION_ID=7
CLIENT_OS_VERSION_CODE=117440512
excepting tests: 28 102
skipping tests SLOW=no: 33a
=== sanityn: start setup 04:34:45 (1733391285) ===
Starting client oleg235-client.virtnet:  -o user_xattr,flock 192.168.202.135@tcp:/lustre /mnt/lustre2
Started clients oleg235-client.virtnet: 
192.168.202.135@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project)
oleg235-client.virtnet: executing check_config_client /mnt/lustre
oleg235-client.virtnet: Checking config lustre mounted on /mnt/lustre
Checking servers environments
Checking clients oleg235-client.virtnet environments
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800abb0d800.idle_timeout=debug
osc.lustre-OST0000-osc-ffff8800b63b7800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800abb0d800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b63b7800.idle_timeout=debug
disable quota as required
oleg235-server: oleg235-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
=== sanityn: finish setup 04:34:54 (1733391294) ===
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137953 s, 76.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups:
 [true]
running as uid/gid/euid/egid 500/500/500/500, groups:
 [touch] [/mnt/lustre/d0_runas_test/f7697]

== sanityn test 77kf: Verify ID range with comparator works correctly ========================================================== 04:34:55 (1733391295)
using SAVE_PROJECT_SUPPORTED=0
ost.OSS.ost_io.nrs_policies=tbf projid
ost.OSS.ost_io.nrs_tbf_rule=start ext_rw projid>={100}&projid<{200} rate=20
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.33051 s, 50.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0141218 s, 74.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0182309 s, 57.5 MB/s
, 0.0116457 s, 90.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0284785 s, 36.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0431372 s, 24.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0295778 s, 35.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0530523 s, 19.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0493963 s, 21.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0517208 s, 20.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0499148 s, 21.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0424546 s, 24.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0603827 s, 17.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.23036 s, 4.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.234563 s, 4.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.27811 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.28043 s, 3.7 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0340141 s, 30.8 MB/s
, 0.0232328 s, 45.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0290534 s, 36.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0307837 s, 34.1 MB/s
, 0.0261207 s, 40.1 MB/s
1048576 bytes (1.0 MB) copied, 0.0431061 s, 24.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0405441 s, 25.9 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0168601 s, 62.2 MB/s
, 0.0191067 s, 54.9 MB/s
, 0.0151893 s, 69.0 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0129397 s, 81.0 MB/s
, 0.00306824 s, 342 MB/s
1048576 bytes (1.0 MB) copied, 0.0225785 s, 46.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0109216 s, 96.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00816938 s, 128 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00731422 s, 143 MB/s
removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg235-client.virtnet'
removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg235-client.virtnet'
removed directory: '/mnt/lustre/d77kf.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.01076 s, 20.9 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.97132 s, 21.1 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77kf.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.232624 s, 72.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00924977 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0069961 s, 150 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.024468 s, 42.9 MB/s
, 0.0334887 s, 31.3 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0284641 s, 36.8 MB/s
1048576 bytes (1.0 MB) copied, 0.0315228 s, 33.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0267615 s, 39.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0382899 s, 27.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0260712 s, 40.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0464785 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0506456 s, 20.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0418134 s, 25.1 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.265526 s, 3.9 MB/s
, 0.265108 s, 4.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.327935 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.329434 s, 3.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0227226 s, 46.1 MB/s
, 0.0242034 s, 43.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0252521 s, 41.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0223838 s, 46.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0243709 s, 43.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0233669 s, 44.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0257657 s, 40.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0246214 s, 42.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0252277 s, 41.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0236256 s, 44.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0244825 s, 42.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0249079 s, 42.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0250192 s, 41.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0253297 s, 41.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0259408 s, 40.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0278669 s, 37.6 MB/s
removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg235-client.virtnet'
removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg235-client.virtnet'
removed directory: '/mnt/lustre/d77kf.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 20
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.9374 s, 21.2 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96803 s, 21.1 MB/s
Read runtime is 6 s, speed is 16.666666 IOPS
cleanup_tbf /mnt/lustre/d77kf.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_rw
ost.OSS.ost_io.nrs_policies=fifo
ost.OSS.ost_io.nrs_policies=tbf projid+opcode
ost.OSS.ost_io.nrs_tbf_rule=start ext_w projid>={100}&projid<{200}&opcode={ost_write} rate=20
ost.OSS.ost_io.nrs_tbf_rule=start ext_r projid>={100}&projid<{200}&opcode={ost_read} rate=10
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.252607 s, 66.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137254 s, 76.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0195016 s, 53.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0231736 s, 45.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0435749 s, 24.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0442047 s, 23.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0465016 s, 22.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0703781 s, 14.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0690725 s, 15.2 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.106332 s, 9.9 MB/s
, 0.112448 s, 9.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.161733 s, 6.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.153746 s, 6.8 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.255479 s, 4.1 MB/s
, 0.271606 s, 3.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.314346 s, 3.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.329102 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00864254 s, 121 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.009669 s, 108 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00584254 s, 179 MB/s
, 0.00673341 s, 156 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00676841 s, 155 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00861754 s, 122 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00944351 s, 111 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00765482 s, 137 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00901991 s, 116 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00909211 s, 115 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00924686 s, 113 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.010006 s, 105 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104294 s, 101 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107723 s, 97.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0113034 s, 92.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108078 s, 97.0 MB/s
removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg235-client.virtnet'
removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg235-client.virtnet'
removed directory: '/mnt/lustre/d77kf.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.92857 s, 21.3 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.72776 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kf.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
16+0 records in
16+0 records out
16777216 bytes (17 MB) copied, 0.233247 s, 71.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00849805 s, 123 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00991607 s, 106 MB/s
1048576 bytes (1.0 MB) copied, 0.0189112 s, 55.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148662 s, 70.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.024362 s, 43.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148744 s, 70.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0148657 s, 70.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00889744 s, 118 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00773133 s, 136 MB/s
1+0 records in
1+0 records out
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0419749 s, 25.0 MB/s
, 0.0419738 s, 25.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0368811 s, 28.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.266796 s, 3.9 MB/s
, 0.272881 s, 3.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.320231 s, 3.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.326524 s, 3.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.0104471 s, 100 MB/s
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
, 0.00960652 s, 109 MB/s
1048576 bytes (1.0 MB) copied, 0.00802618 s, 131 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00918791 s, 114 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00834124 s, 126 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00894921 s, 117 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00982808 s, 107 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0105634 s, 99.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107099 s, 97.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0107559 s, 97.5 MB/s
, 0.0106 s, 98.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0118628 s, 88.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0119592 s, 87.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0137804 s, 76.1 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115795 s, 90.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0138459 s, 75.7 MB/s
removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg235-client.virtnet'
removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg235-client.virtnet'
removed directory: '/mnt/lustre/d77kf.sanityn'
cpu_npartitions on ost1 is 1
Limited write rate: 20, read rate: 10
Verify the write rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 4.96656 s, 21.1 MB/s
Write runtime is 6 s, speed is 16.666666 IOPS
Verify the read rate is under TBF control
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 9.73302 s, 10.8 MB/s
Read runtime is 11 s, speed is 9.090909 IOPS
cleanup_tbf /mnt/lustre/d77kf.sanityn
sleep 5 for ZFS zfs
sleep 5 for ZFS zfs
Waiting for MDT destroys to complete
ost.OSS.ost_io.nrs_tbf_rule=stop ext_w
ost.OSS.ost_io.nrs_tbf_rule=stop ext_r
ost.OSS.ost_io.nrs_policies=fifo
PASS 77kf (135s)
cleanup: ======================================================
== sanityn test complete, duration 150 sec =============== 04:37:11 (1733391431)
=== sanityn: start cleanup 04:37:11 (1733391431) ===
Stopping clients: oleg235-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg235-client.virtnet /mnt/lustre2 opts:
=== sanityn: finish cleanup 04:37:12 (1733391432) ===