-----============= acceptance-small: sanity-hsm ============----- Tue Jun 17 12:45:03 EDT 2025
mgs: CentOS Linux release 7.9.2009 (Core)
MGS_OS_VERSION_ID=7
MGS_OS_ID=centos
MGS_OS_VERSION_CODE=117440512
MGS_OS_ID_LIKE=rhel fedora centos
mds1: CentOS Linux release 7.9.2009 (Core)
MDS1_OS_ID_LIKE=rhel fedora centos
MDS1_OS_ID=centos
MDS1_OS_VERSION_ID=7
MDS1_OS_VERSION_CODE=117440512
ost1: CentOS Linux release 7.9.2009 (Core)
OST1_OS_VERSION_CODE=117440512
OST1_OS_VERSION_ID=7
OST1_OS_ID_LIKE=rhel fedora centos
OST1_OS_ID=centos
client: CentOS Linux release 7.9.2009 (Core)
CLIENT_OS_ID=centos
CLIENT_OS_ID_LIKE=rhel fedora centos
CLIENT_OS_VERSION_ID=7
CLIENT_OS_VERSION_CODE=117440512
oleg409-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanity-hsm.*ex: No such file or directory
excepting tests: 
=== sanity-hsm: start setup 12:45:11 (1750178711) ===
Starting client oleg409-client.virtnet:  -o user_xattr,flock 192.168.204.109@tcp:/lustre /mnt/lustre2
Started clients oleg409-client.virtnet: 
192.168.204.109@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
oleg409-client.virtnet: executing check_config_client /mnt/lustre
oleg409-client.virtnet: Checking config lustre mounted on /mnt/lustre
Checking servers environments
Checking clients oleg409-client.virtnet environments
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800b5a00800.idle_timeout=debug
osc.lustre-OST0000-osc-ffff8800b6223800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b5a00800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b6223800.idle_timeout=debug
disable quota as required
oleg409-server: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
osd-ldiskfs.track_declares_assert=1
=== sanity-hsm: finish setup 12:45:25 (1750178725) ===
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [true]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [touch] [/mnt/lustre/d0_runas_test/f7935]
nobody:x:99:
Killing existing copytools on oleg409-client.virtnet
pkill: pidfile not valid
Try `pkill --help' for more information.
Set HSM on and start
Set sanity-hsm HSM policy

== sanity-hsm test 1A: lfs hsm flags root/non-root access ========================================================== 12:45:48 (1750178748)
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [touch] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_clear] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--exists] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm]
lfs hsm_set: change hsm flags for '/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm' failed: Operation not permitted
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm]
PASS 1A (3s)

== sanity-hsm test 1a: mmap & cat a HSM released file ==== 12:45:51 (1750178751)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0619641 s, 16.9 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Verifying released state: PASS 1a (4s)

== sanity-hsm test 1b: Archive, Release and Restore composite file ========================================================== 12:45:55 (1750178755)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0208782 s, 50.2 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
archive /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm
Waiting 200s for 'SUCCEED'
release /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm
verify released state: 
pass
restore /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm
verify restored state: 
pass
PASS 1b (4s)

== sanity-hsm test 1c: Check setting archive-id in lfs hsm_set ========================================================== 12:45:59 (1750178759)
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [touch] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm]
verifying archive number is 32
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm]
verifying archive number is still 32
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm]
verifying archive number is 33
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm]
verifying archive number is 16
PASS 1c (3s)

== sanity-hsm test 1d: Archive, Release and Restore DoM file ========================================================== 12:46:02 (1750178762)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0340912 s, 30.8 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
archive /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm
Waiting 200s for 'SUCCEED'
release /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm
verify released state: 
pass
restore /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm
verify restored state: 
pass
PASS 1d (5s)

== sanity-hsm test 1e: Archive, Release and Restore SEL file ========================================================== 12:46:07 (1750178767)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.028848 s, 36.3 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
archive /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm
release /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm
verify released state: 
pass
restore /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
verify restored state: 
pass
PASS 1e (5s)

== sanity-hsm test 1f: DoM file release after restore ==== 12:46:12 (1750178772)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.033598 s, 31.2 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
archive /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm
Waiting 200s for 'SUCCEED'
release /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm
verify released state: 
pass
restore /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
verify restored state: 
pass
release again /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm
/mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm: (0x0000000d) released exists archived, archive_id:1
verify released state: 
pass
PASS 1f (6s)

== sanity-hsm test 2: Check file dirtyness when doing setattr ========================================================== 12:46:18 (1750178778)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00203896 s, 0.0 kB/s
0x200000403:0x8:0x0
PASS 2 (2s)

== sanity-hsm test 3: Check file dirtyness when opening for write ========================================================== 12:46:20 (1750178780)
Succeed in opening file "/mnt/lustre/d3.sanity-hsm/f3.sanity-hsm"(flags=O_WRONLY)
3+0 records in
3+0 records out
3+0 records in
3+0 records out
PASS 3 (3s)

== sanity-hsm test 4: Useless cancel must not be registered ========================================================== 12:46:23 (1750178783)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0562661 s, 18.6 MB/s
PASS 4 (3s)

== sanity-hsm test 8: Test default archive number ======== 12:46:26 (1750178786)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_8/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
PASS 8 (4s)

== sanity-hsm test 9A: Use of explicit archive number, with dedicated copytool ========================================================== 12:46:30 (1750178790)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_9A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 3  "/mnt/lustre2"'
oleg409-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool
oleg409-client.virtnet: executing get_client_uuid /mnt/lustre2
found agent uuid=94a00271-e790-468b-8c8b-c344f94cbb5f archive_id=3 requests=[current:0 ok:0 errors:0] on mds1
found agent uuid=94a00271-e790-468b-8c8b-c344f94cbb5f archive_id=3 requests=[current:0 ok:0 errors:0] on mds2
Waiting 200s for 'SUCCEED'
PASS 9A (11s)

== sanity-hsm test 9a: Multiple remote agents ============ 12:46:41 (1750178801)

 SKIP: sanity-hsm test_9a Need 3 or more clients, have 1
SKIP 9a (1s)

== sanity-hsm test 10a: Archive a file =================== 12:46:42 (1750178802)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Verifying content
Verifying hsm state 
Verifying archive number is 2
PASS 10a (2s)

== sanity-hsm test 10b: Archive of non dirty file must work without doing request ========================================================== 12:46:44 (1750178804)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
PASS 10b (4s)

== sanity-hsm test 10c: Check forbidden archive ========== 12:46:48 (1750178808)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Cannot send HSM request (use of /mnt/lustre/d10c.sanity-hsm/f10c.sanity-hsm): Operation not permitted
PASS 10c (3s)

== sanity-hsm test 10d: Archive a file on the default archive id ========================================================== 12:46:51 (1750178811)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
PASS 10d (3s)

== sanity-hsm test 11a: Import a file ==================== 12:46:54 (1750178814)
lhsmtool_posix: 1750178814.877810 lhsmtool_posix[22891]: action=1 src=d11a.sanity-hsm/f11a.sanity-hsm dst=/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750178814.883483 lhsmtool_posix[22891]: importing '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm'
lhsmtool_posix: 1750178814.911728 lhsmtool_posix[22891]: imported '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//0424/0x200000402:0x26:0x0'=='/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm'
lhsmtool_posix: 1750178814.911806 lhsmtool_posix[22891]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
Verifying released state: Verifying imported size 159=159
Verifying released pattern: released
Verifying new fid 0x200000402:0x26:0x0 in archive
PASS 11a (2s)

== sanity-hsm test 11b: Import a deleted file using its FID ========================================================== 12:46:56 (1750178816)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_11b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
lhsmtool_posix: 1750178819.193205 lhsmtool_posix[23649]: action=1 src=0x200000403:0x11:0x0 dst=/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750178819.199625 lhsmtool_posix[23649]: Resolving [0x200000403:0x11:0x0] to /tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0
lhsmtool_posix: 1750178819.199699 lhsmtool_posix[23649]: importing '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0'
lhsmtool_posix: 1750178819.217180 lhsmtool_posix[23649]: imported '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//042a/0x200000402:0x28:0x0'=='/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0'
lhsmtool_posix: 1750178819.217251 lhsmtool_posix[23649]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm: OK
PASS 11b (5s)

== sanity-hsm test 11c: Import a file to a directory with a pool ========================================================== 12:47:01 (1750178821)
Creating new pool test_11c
oleg409-server: Pool lustre.test_11c created
Adding targets to pool
oleg409-server: OST lustre-OST0001_UUID added to pool lustre.test_11c
lhsmtool_posix: 1750178827.305646 lhsmtool_posix[24425]: action=1 src=d11c.sanity-hsm/f11c.sanity-hsm dst=/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750178827.314707 lhsmtool_posix[24425]: importing '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm'
lhsmtool_posix: 1750178827.342018 lhsmtool_posix[24425]: imported '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//0428/0x200000402:0x2a:0x0'=='/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm'
lhsmtool_posix: 1750178827.342094 lhsmtool_posix[24425]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
Destroy the created pools: test_11c
lustre.test_11c
oleg409-server: OST lustre-OST0001_UUID removed from pool lustre.test_11c
oleg409-server: Pool lustre.test_11c destroyed
PASS 11c (14s)

== sanity-hsm test 12a: Restore an imported file explicitly ========================================================== 12:47:15 (1750178835)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
lhsmtool_posix: 1750178835.814024 lhsmtool_posix[25223]: action=1 src=d12a.sanity-hsm/f12a.sanity-hsm dst=/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750178835.818523 lhsmtool_posix[25223]: importing '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm'
lhsmtool_posix: 1750178835.842116 lhsmtool_posix[25223]: imported '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//042f/0x200000402:0x2d:0x0'=='/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm'
lhsmtool_posix: 1750178835.842487 lhsmtool_posix[25223]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
Verifying released state: 
Verifying file state: 
PASS 12a (3s)

== sanity-hsm test 12b: Restore an imported file implicitly ========================================================== 12:47:18 (1750178838)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
lhsmtool_posix: 1750178838.925219 lhsmtool_posix[25987]: action=1 src=d12b.sanity-hsm/f12b.sanity-hsm dst=/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750178838.928442 lhsmtool_posix[25987]: importing '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm'
lhsmtool_posix: 1750178838.956997 lhsmtool_posix[25987]: imported '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//042d/0x200000402:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm'
lhsmtool_posix: 1750178838.957071 lhsmtool_posix[25987]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
Verifying released state: 
Verifying file state after restore: 
PASS 12b (3s)

== sanity-hsm test 12c: Restore a file with stripe of 2 == 12:47:21 (1750178841)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.0702203 s, 74.7 MB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
/mnt/lustre/d12c.sanity-hsm/f12c.sanity-hsm: OK
PASS 12c (5s)

== sanity-hsm test 12d: Restore of a non archived, non released file must work ========================================================== 12:47:26 (1750178846)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
PASS 12d (3s)

== sanity-hsm test 12e: Check forbidden restore ========== 12:47:30 (1750178850)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
/mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1
Cannot send HSM request (use of /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm): Operation not permitted
PASS 12e (4s)

== sanity-hsm test 12f: Restore a released file explicitly ========================================================== 12:47:34 (1750178854)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Verifying file state: PASS 12f (4s)

== sanity-hsm test 12g: Restore a released file implicitly ========================================================== 12:47:38 (1750178858)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
PASS 12g (4s)

== sanity-hsm test 12h: Restore a released file implicitly from a second node ========================================================== 12:47:42 (1750178862)

 SKIP: sanity-hsm test_12h Need 2 or more clients, have 1
SKIP 12h (1s)

== sanity-hsm test 12m: Archive/release/implicit restore ========================================================== 12:47:43 (1750178863)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12m/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
PASS 12m (4s)

== sanity-hsm test 12n: Import/implicit restore/release == 12:47:47 (1750178867)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12n/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
lhsmtool_posix: 1750178868.339113 lhsmtool_posix[31281]: action=1 src=d12n.sanity-hsm/f12n.sanity-hsm dst=/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750178868.343204 lhsmtool_posix[31281]: importing '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm'
lhsmtool_posix: 1750178868.364611 lhsmtool_posix[31281]: imported '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//043f/0x200000402:0x3d:0x0'=='/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm'
lhsmtool_posix: 1750178868.364694 lhsmtool_posix[31281]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
PASS 12n (3s)

== sanity-hsm test 12o: Layout-swap failure during Restore leaves file released ========================================================== 12:47:50 (1750178870)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12o/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
fail_loc=0x152
mdt.lustre-MDT0000.hsm.policy=+NRA
mdt.lustre-MDT0001.hsm.policy=+NRA
diff: /mnt/lustre/d12o.sanity-hsm/f12o.sanity-hsm: No data available
mdt.lustre-MDT0000.hsm.policy=-NRA
mdt.lustre-MDT0001.hsm.policy=-NRA
fail_loc=0
mdt.lustre-MDT0000.hsm_control=purge
mdt.lustre-MDT0001.hsm_control=purge
Waiting 200s for 'SUCCEED'
PASS 12o (21s)

== sanity-hsm test 12p: implicit restore of a file on copytool mount point ========================================================== 12:48:11 (1750178891)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12p/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
PASS 12p (6s)

== sanity-hsm test 12q: file attributes are refreshed after restore ========================================================== 12:48:17 (1750178897)
Starting client: oleg409-client.virtnet:  -o user_xattr,flock 192.168.204.109@tcp:/lustre /mnt/lustre3
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre3"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0375613 s, 27.9 MB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre3"'
Stopping client oleg409-client.virtnet /mnt/lustre3 (opts:)
PASS 12q (10s)

== sanity-hsm test 12r: lseek restores released file ===== 12:48:27 (1750178907)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12r/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
PASS 12r (4s)

== sanity-hsm test 12s: race between restore requests ==== 12:48:31 (1750178911)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12s/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
fail_loc=0x8000018b
PASS 12s (10s)

== sanity-hsm test 12t: Multiple parallel reads for a HSM imported file ========================================================== 12:48:41 (1750178921)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12t/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
lhsmtool_posix: 1750178924.414153 lhsmtool_posix[3436]: action=1 src=0x200000403:0x2a:0x0 dst=/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750178924.419949 lhsmtool_posix[3436]: Resolving [0x200000403:0x2a:0x0] to /tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0
lhsmtool_posix: 1750178924.420198 lhsmtool_posix[3436]: importing '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0'
lhsmtool_posix: 1750178924.439148 lhsmtool_posix[3436]: imported '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//0449/0x200000402:0x4b:0x0'=='/tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0'
lhsmtool_posix: 1750178924.439495 lhsmtool_posix[3436]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
PASS 12t (5s)

== sanity-hsm test 12u: Multiple reads on multiple HSM imported files in parallel ========================================================== 12:48:46 (1750178926)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12u/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
lhsmtool_posix: 1750178932.185605 lhsmtool_posix[4370]: action=1 src=0x200000403:0x2c:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0 mount_point=/mnt/lustre2
lhsmtool_posix: 1750178932.190997 lhsmtool_posix[4370]: Resolving [0x200000403:0x2c:0x0] to /tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0
lhsmtool_posix: 1750178932.191161 lhsmtool_posix[4370]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0'
lhsmtool_posix: 1750178932.209363 lhsmtool_posix[4370]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//044f/0x200000402:0x4d:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0'
lhsmtool_posix: 1750178932.209424 lhsmtool_posix[4370]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
lhsmtool_posix: 1750178932.296000 lhsmtool_posix[4399]: action=1 src=0x200000403:0x2d:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1 mount_point=/mnt/lustre2
lhsmtool_posix: 1750178932.298814 lhsmtool_posix[4399]: Resolving [0x200000403:0x2d:0x0] to /tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0
lhsmtool_posix: 1750178932.298943 lhsmtool_posix[4399]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0'
lhsmtool_posix: 1750178932.312916 lhsmtool_posix[4399]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//044c/0x200000402:0x4e:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0'
lhsmtool_posix: 1750178932.312984 lhsmtool_posix[4399]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
lhsmtool_posix: 1750178932.393968 lhsmtool_posix[4428]: action=1 src=0x200000403:0x2e:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2 mount_point=/mnt/lustre2
lhsmtool_posix: 1750178932.398864 lhsmtool_posix[4428]: Resolving [0x200000403:0x2e:0x0] to /tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0
lhsmtool_posix: 1750178932.398937 lhsmtool_posix[4428]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0'
lhsmtool_posix: 1750178932.419113 lhsmtool_posix[4428]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//044d/0x200000402:0x4f:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0'
lhsmtool_posix: 1750178932.419217 lhsmtool_posix[4428]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
lhsmtool_posix: 1750178932.505132 lhsmtool_posix[4457]: action=1 src=0x200000403:0x2f:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3 mount_point=/mnt/lustre2
lhsmtool_posix: 1750178932.509167 lhsmtool_posix[4457]: Resolving [0x200000403:0x2f:0x0] to /tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0
lhsmtool_posix: 1750178932.509226 lhsmtool_posix[4457]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0'
lhsmtool_posix: 1750178932.525711 lhsmtool_posix[4457]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//0452/0x200000402:0x50:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0'
lhsmtool_posix: 1750178932.525786 lhsmtool_posix[4457]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
lhsmtool_posix: 1750178932.603998 lhsmtool_posix[4486]: action=1 src=0x200000403:0x30:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4 mount_point=/mnt/lustre2
lhsmtool_posix: 1750178932.607858 lhsmtool_posix[4486]: Resolving [0x200000403:0x30:0x0] to /tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0
lhsmtool_posix: 1750178932.607927 lhsmtool_posix[4486]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0'
lhsmtool_posix: 1750178932.624715 lhsmtool_posix[4486]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0453/0x200000402:0x51:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0'
lhsmtool_posix: 1750178932.624786 lhsmtool_posix[4486]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
lhsmtool_posix: 1750178932.707172 lhsmtool_posix[4515]: action=1 src=0x200000403:0x31:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5 mount_point=/mnt/lustre2
lhsmtool_posix: 1750178932.710223 lhsmtool_posix[4515]: Resolving [0x200000403:0x31:0x0] to /tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0
lhsmtool_posix: 1750178932.710281 lhsmtool_posix[4515]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0'
lhsmtool_posix: 1750178932.724728 lhsmtool_posix[4515]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0450/0x200000402:0x52:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0'
lhsmtool_posix: 1750178932.724796 lhsmtool_posix[4515]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
lhsmtool_posix: 1750178932.805881 lhsmtool_posix[4544]: action=1 src=0x200000403:0x32:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6 mount_point=/mnt/lustre2
lhsmtool_posix: 1750178932.809819 lhsmtool_posix[4544]: Resolving [0x200000403:0x32:0x0] to /tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0
lhsmtool_posix: 1750178932.809881 lhsmtool_posix[4544]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0'
lhsmtool_posix: 1750178932.824651 lhsmtool_posix[4544]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0451/0x200000402:0x53:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0'
lhsmtool_posix: 1750178932.824715 lhsmtool_posix[4544]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
lhsmtool_posix: 1750178932.908956 lhsmtool_posix[4573]: action=1 src=0x200000403:0x33:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7 mount_point=/mnt/lustre2
lhsmtool_posix: 1750178932.911957 lhsmtool_posix[4573]: Resolving [0x200000403:0x33:0x0] to /tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0
lhsmtool_posix: 1750178932.912026 lhsmtool_posix[4573]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0'
lhsmtool_posix: 1750178932.927082 lhsmtool_posix[4573]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0456/0x200000402:0x54:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0'
lhsmtool_posix: 1750178932.927159 lhsmtool_posix[4573]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
lhsmtool_posix: 1750178933.009324 lhsmtool_posix[4602]: action=1 src=0x200000403:0x34:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8 mount_point=/mnt/lustre2
lhsmtool_posix: 1750178933.013211 lhsmtool_posix[4602]: Resolving [0x200000403:0x34:0x0] to /tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0
lhsmtool_posix: 1750178933.013304 lhsmtool_posix[4602]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0'
lhsmtool_posix: 1750178933.029129 lhsmtool_posix[4602]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0457/0x200000402:0x55:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0'
lhsmtool_posix: 1750178933.029217 lhsmtool_posix[4602]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
lhsmtool_posix: 1750178933.113021 lhsmtool_posix[4631]: action=1 src=0x200000403:0x35:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9 mount_point=/mnt/lustre2
lhsmtool_posix: 1750178933.118748 lhsmtool_posix[4631]: Resolving [0x200000403:0x35:0x0] to /tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0
lhsmtool_posix: 1750178933.118812 lhsmtool_posix[4631]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0'
lhsmtool_posix: 1750178933.132858 lhsmtool_posix[4631]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0454/0x200000402:0x56:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0'
lhsmtool_posix: 1750178933.133188 lhsmtool_posix[4631]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
PASS 12u (9s)

== sanity-hsm test 13: Recursively import and restore a directory ========================================================== 12:48:55 (1750178935)
lhsmtool_posix: 1750178937.935527 lhsmtool_posix[6455]: action=1 src=subdir dst=/mnt/lustre/d13.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750178937.988371 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10'
lhsmtool_posix: 1750178938.016303 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//045b/0x200000402:0x59:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10'
lhsmtool_posix: 1750178938.016472 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9'
lhsmtool_posix: 1750178938.039337 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//0458/0x200000402:0x5a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9'
lhsmtool_posix: 1750178938.039503 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8'
lhsmtool_posix: 1750178938.061105 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//0459/0x200000402:0x5b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8'
lhsmtool_posix: 1750178938.061263 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7'
lhsmtool_posix: 1750178938.079097 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//045e/0x200000402:0x5c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7'
lhsmtool_posix: 1750178938.079236 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6'
lhsmtool_posix: 1750178938.098200 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//045f/0x200000402:0x5d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6'
lhsmtool_posix: 1750178938.098483 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5'
lhsmtool_posix: 1750178938.121591 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//045c/0x200000402:0x5e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5'
lhsmtool_posix: 1750178938.121969 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4'
lhsmtool_posix: 1750178938.141914 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//045d/0x200000402:0x5f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4'
lhsmtool_posix: 1750178938.142078 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3'
lhsmtool_posix: 1750178938.170066 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//0462/0x200000402:0x60:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3'
lhsmtool_posix: 1750178938.170226 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2'
lhsmtool_posix: 1750178938.194942 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//0463/0x200000402:0x61:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2'
lhsmtool_posix: 1750178938.195112 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1'
lhsmtool_posix: 1750178938.217648 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//0460/0x200000402:0x62:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1'
lhsmtool_posix: 1750178938.223513 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10'
lhsmtool_posix: 1750178938.249982 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//0408/0x240000402:0xa:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10'
lhsmtool_posix: 1750178938.250284 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9'
lhsmtool_posix: 1750178938.263505 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//0409/0x240000402:0xb:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9'
lhsmtool_posix: 1750178938.263922 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8'
lhsmtool_posix: 1750178938.289222 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//040e/0x240000402:0xc:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8'
lhsmtool_posix: 1750178938.289407 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7'
lhsmtool_posix: 1750178938.308157 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//040f/0x240000402:0xd:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7'
lhsmtool_posix: 1750178938.308310 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6'
lhsmtool_posix: 1750178938.327078 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//040c/0x240000402:0xe:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6'
lhsmtool_posix: 1750178938.327233 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5'
lhsmtool_posix: 1750178938.342676 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//040d/0x240000402:0xf:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5'
lhsmtool_posix: 1750178938.342824 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4'
lhsmtool_posix: 1750178938.369301 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//0412/0x240000402:0x10:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4'
lhsmtool_posix: 1750178938.369720 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3'
lhsmtool_posix: 1750178938.387797 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//0413/0x240000402:0x11:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3'
lhsmtool_posix: 1750178938.387989 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2'
lhsmtool_posix: 1750178938.404914 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//0410/0x240000402:0x12:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2'
lhsmtool_posix: 1750178938.405072 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1'
lhsmtool_posix: 1750178938.420780 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//0411/0x240000402:0x13:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1'
lhsmtool_posix: 1750178938.429958 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10'
lhsmtool_posix: 1750178938.456130 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//0466/0x200000402:0x64:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10'
lhsmtool_posix: 1750178938.456293 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9'
lhsmtool_posix: 1750178938.473017 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//0467/0x200000402:0x65:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9'
lhsmtool_posix: 1750178938.473168 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8'
lhsmtool_posix: 1750178938.488433 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//0464/0x200000402:0x66:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8'
lhsmtool_posix: 1750178938.488685 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7'
lhsmtool_posix: 1750178938.504920 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//0465/0x200000402:0x67:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7'
lhsmtool_posix: 1750178938.505278 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6'
lhsmtool_posix: 1750178938.533932 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//046a/0x200000402:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6'
lhsmtool_posix: 1750178938.534177 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5'
lhsmtool_posix: 1750178938.556439 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//046b/0x200000402:0x69:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5'
lhsmtool_posix: 1750178938.556636 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4'
lhsmtool_posix: 1750178938.576621 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//0468/0x200000402:0x6a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4'
lhsmtool_posix: 1750178938.576787 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3'
lhsmtool_posix: 1750178938.595501 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//0469/0x200000402:0x6b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3'
lhsmtool_posix: 1750178938.595986 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2'
lhsmtool_posix: 1750178938.615236 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//046e/0x200000402:0x6c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2'
lhsmtool_posix: 1750178938.615488 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1'
lhsmtool_posix: 1750178938.632432 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//046f/0x200000402:0x6d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1'
lhsmtool_posix: 1750178938.635016 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10'
lhsmtool_posix: 1750178938.654323 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//0417/0x240000402:0x15:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10'
lhsmtool_posix: 1750178938.654532 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9'
lhsmtool_posix: 1750178938.672943 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//0414/0x240000402:0x16:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9'
lhsmtool_posix: 1750178938.673107 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8'
lhsmtool_posix: 1750178938.694746 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//0415/0x240000402:0x17:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8'
lhsmtool_posix: 1750178938.694977 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7'
lhsmtool_posix: 1750178938.713018 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//041a/0x240000402:0x18:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7'
lhsmtool_posix: 1750178938.713187 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6'
lhsmtool_posix: 1750178938.737970 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//041b/0x240000402:0x19:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6'
lhsmtool_posix: 1750178938.738202 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5'
lhsmtool_posix: 1750178938.758479 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//0418/0x240000402:0x1a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5'
lhsmtool_posix: 1750178938.758869 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4'
lhsmtool_posix: 1750178938.775896 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//0419/0x240000402:0x1b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4'
lhsmtool_posix: 1750178938.776064 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3'
lhsmtool_posix: 1750178938.799224 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//041e/0x240000402:0x1c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3'
lhsmtool_posix: 1750178938.799654 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2'
lhsmtool_posix: 1750178938.813928 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//041f/0x240000402:0x1d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2'
lhsmtool_posix: 1750178938.814094 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1'
lhsmtool_posix: 1750178938.833319 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//041c/0x240000402:0x1e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1'
lhsmtool_posix: 1750178938.840517 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10'
lhsmtool_posix: 1750178938.868286 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//046d/0x200000402:0x6f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10'
lhsmtool_posix: 1750178938.868580 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9'
lhsmtool_posix: 1750178938.886272 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//0472/0x200000402:0x70:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9'
lhsmtool_posix: 1750178938.887268 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8'
lhsmtool_posix: 1750178938.904720 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//0473/0x200000402:0x71:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8'
lhsmtool_posix: 1750178938.904872 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7'
lhsmtool_posix: 1750178938.926115 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//0470/0x200000402:0x72:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7'
lhsmtool_posix: 1750178938.926277 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6'
lhsmtool_posix: 1750178938.945257 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//0471/0x200000402:0x73:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6'
lhsmtool_posix: 1750178938.945430 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5'
lhsmtool_posix: 1750178938.970295 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//0476/0x200000402:0x74:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5'
lhsmtool_posix: 1750178938.970619 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4'
lhsmtool_posix: 1750178938.993095 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//0477/0x200000402:0x75:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4'
lhsmtool_posix: 1750178938.993309 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3'
lhsmtool_posix: 1750178939.013724 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//0474/0x200000402:0x76:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3'
lhsmtool_posix: 1750178939.013874 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2'
lhsmtool_posix: 1750178939.031050 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//0475/0x200000402:0x77:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2'
lhsmtool_posix: 1750178939.031213 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1'
lhsmtool_posix: 1750178939.048066 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//047a/0x200000402:0x78:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1'
lhsmtool_posix: 1750178939.051133 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10'
lhsmtool_posix: 1750178939.074158 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//0422/0x240000402:0x20:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10'
lhsmtool_posix: 1750178939.074307 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9'
lhsmtool_posix: 1750178939.090113 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//0423/0x240000402:0x21:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9'
lhsmtool_posix: 1750178939.090273 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8'
lhsmtool_posix: 1750178939.109111 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//0420/0x240000402:0x22:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8'
lhsmtool_posix: 1750178939.109281 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7'
lhsmtool_posix: 1750178939.124379 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//0421/0x240000402:0x23:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7'
lhsmtool_posix: 1750178939.124542 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6'
lhsmtool_posix: 1750178939.143155 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//0426/0x240000402:0x24:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6'
lhsmtool_posix: 1750178939.143278 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5'
lhsmtool_posix: 1750178939.164662 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//0427/0x240000402:0x25:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5'
lhsmtool_posix: 1750178939.164806 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4'
lhsmtool_posix: 1750178939.190717 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//0424/0x240000402:0x26:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4'
lhsmtool_posix: 1750178939.190874 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3'
lhsmtool_posix: 1750178939.213626 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//0425/0x240000402:0x27:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3'
lhsmtool_posix: 1750178939.213987 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2'
lhsmtool_posix: 1750178939.229777 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//042a/0x240000402:0x28:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2'
lhsmtool_posix: 1750178939.229963 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1'
lhsmtool_posix: 1750178939.242095 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//042b/0x240000402:0x29:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1'
lhsmtool_posix: 1750178939.251127 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10'
lhsmtool_posix: 1750178939.285939 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//0478/0x200000402:0x7a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10'
lhsmtool_posix: 1750178939.286383 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9'
lhsmtool_posix: 1750178939.306016 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//0479/0x200000402:0x7b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9'
lhsmtool_posix: 1750178939.306272 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8'
lhsmtool_posix: 1750178939.330740 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//047e/0x200000402:0x7c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8'
lhsmtool_posix: 1750178939.330940 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7'
lhsmtool_posix: 1750178939.352622 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//047f/0x200000402:0x7d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7'
lhsmtool_posix: 1750178939.352783 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6'
lhsmtool_posix: 1750178939.375254 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//047c/0x200000402:0x7e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6'
lhsmtool_posix: 1750178939.375434 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5'
lhsmtool_posix: 1750178939.396910 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//047d/0x200000402:0x7f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5'
lhsmtool_posix: 1750178939.397078 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4'
lhsmtool_posix: 1750178939.420113 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//0482/0x200000402:0x80:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4'
lhsmtool_posix: 1750178939.420315 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3'
lhsmtool_posix: 1750178939.434446 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//0483/0x200000402:0x81:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3'
lhsmtool_posix: 1750178939.434690 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2'
lhsmtool_posix: 1750178939.451945 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//0480/0x200000402:0x82:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2'
lhsmtool_posix: 1750178939.452115 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1'
lhsmtool_posix: 1750178939.469840 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//0481/0x200000402:0x83:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1'
lhsmtool_posix: 1750178939.477291 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10'
lhsmtool_posix: 1750178939.516085 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//0429/0x240000402:0x2b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10'
lhsmtool_posix: 1750178939.516907 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9'
lhsmtool_posix: 1750178939.532817 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//042e/0x240000402:0x2c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9'
lhsmtool_posix: 1750178939.532981 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8'
lhsmtool_posix: 1750178939.560729 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//042f/0x240000402:0x2d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8'
lhsmtool_posix: 1750178939.561048 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7'
lhsmtool_posix: 1750178939.583076 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//042c/0x240000402:0x2e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7'
lhsmtool_posix: 1750178939.583239 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6'
lhsmtool_posix: 1750178939.613039 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//042d/0x240000402:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6'
lhsmtool_posix: 1750178939.613198 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5'
lhsmtool_posix: 1750178939.637847 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//0432/0x240000402:0x30:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5'
lhsmtool_posix: 1750178939.638005 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4'
lhsmtool_posix: 1750178939.662990 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//0433/0x240000402:0x31:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4'
lhsmtool_posix: 1750178939.663403 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3'
lhsmtool_posix: 1750178939.682337 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//0430/0x240000402:0x32:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3'
lhsmtool_posix: 1750178939.682510 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2'
lhsmtool_posix: 1750178939.711844 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//0431/0x240000402:0x33:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2'
lhsmtool_posix: 1750178939.711977 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1'
lhsmtool_posix: 1750178939.731608 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//0436/0x240000402:0x34:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1'
lhsmtool_posix: 1750178939.741864 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10'
lhsmtool_posix: 1750178939.776433 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//0487/0x200000402:0x85:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10'
lhsmtool_posix: 1750178939.776747 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9'
lhsmtool_posix: 1750178939.800973 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//0484/0x200000402:0x86:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9'
lhsmtool_posix: 1750178939.801420 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8'
lhsmtool_posix: 1750178939.829650 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//0485/0x200000402:0x87:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8'
lhsmtool_posix: 1750178939.829822 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7'
lhsmtool_posix: 1750178939.850995 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//048a/0x200000402:0x88:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7'
lhsmtool_posix: 1750178939.851141 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6'
lhsmtool_posix: 1750178939.874055 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//048b/0x200000402:0x89:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6'
lhsmtool_posix: 1750178939.874810 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5'
lhsmtool_posix: 1750178939.894820 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//0488/0x200000402:0x8a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5'
lhsmtool_posix: 1750178939.895023 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4'
lhsmtool_posix: 1750178939.912951 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//0489/0x200000402:0x8b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4'
lhsmtool_posix: 1750178939.913467 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3'
lhsmtool_posix: 1750178939.934171 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//048e/0x200000402:0x8c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3'
lhsmtool_posix: 1750178939.934319 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2'
lhsmtool_posix: 1750178939.954952 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//048f/0x200000402:0x8d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2'
lhsmtool_posix: 1750178939.955102 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1'
lhsmtool_posix: 1750178939.980100 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//048c/0x200000402:0x8e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1'
lhsmtool_posix: 1750178939.983255 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10'
lhsmtool_posix: 1750178940.009993 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//0434/0x240000402:0x36:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10'
lhsmtool_posix: 1750178940.010144 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9'
lhsmtool_posix: 1750178940.030622 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//0435/0x240000402:0x37:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9'
lhsmtool_posix: 1750178940.030788 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8'
lhsmtool_posix: 1750178940.055375 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//043a/0x240000402:0x38:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8'
lhsmtool_posix: 1750178940.055524 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7'
lhsmtool_posix: 1750178940.080205 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//043b/0x240000402:0x39:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7'
lhsmtool_posix: 1750178940.080390 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6'
lhsmtool_posix: 1750178940.116492 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//0438/0x240000402:0x3a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6'
lhsmtool_posix: 1750178940.117033 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5'
lhsmtool_posix: 1750178940.142775 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//0439/0x240000402:0x3b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5'
lhsmtool_posix: 1750178940.142950 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4'
lhsmtool_posix: 1750178940.167485 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//043e/0x240000402:0x3c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4'
lhsmtool_posix: 1750178940.167687 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3'
lhsmtool_posix: 1750178940.193986 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//043f/0x240000402:0x3d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3'
lhsmtool_posix: 1750178940.194151 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2'
lhsmtool_posix: 1750178940.213587 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//043c/0x240000402:0x3e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2'
lhsmtool_posix: 1750178940.213825 lhsmtool_posix[6455]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1'
lhsmtool_posix: 1750178940.232402 lhsmtool_posix[6455]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//043d/0x240000402:0x3f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1'
lhsmtool_posix: 1750178940.232492 lhsmtool_posix[6455]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_13/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
PASS 13 (10s)

== sanity-hsm test 14: Rebind archived file to a new fid ========================================================== 12:49:05 (1750178945)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_14/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0478571 s, 21.9 MB/s
Waiting 200s for 'SUCCEED'
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00554225 s, 0.0 kB/s
rebind 0x200000403:0x73:0x0 to 0x200000403:0x74:0x0
lhsmtool_posix: 1750178947.900403 lhsmtool_posix[7488]: action=2 src=0x200000403:0x73:0x0 dst=0x200000403:0x74:0x0 mount_point=/mnt/lustre2
lhsmtool_posix: 1750178947.901165 lhsmtool_posix[7488]: rebind [0x200000403:0x73:0x0] to [0x200000403:0x74:0x0]
lhsmtool_posix: 1750178947.902283 lhsmtool_posix[7488]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
PASS 14 (5s)

== sanity-hsm test 15: Rebind a list of files ============ 12:49:10 (1750178950)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_15/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0456308 s, 23.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0429292 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0429555 s, 24.4 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0464734 s, 22.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0534261 s, 19.6 MB/s
Waiting 300s for ''
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00184703 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00259314 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00285065 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00214831 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00150073 s, 0.0 kB/s
rebind list of files
lhsmtool_posix: 1750178953.620898 lhsmtool_posix[8335]: action=2 src=/tmp/tmp.7935 dst=(null) mount_point=/mnt/lustre2
lhsmtool_posix: 1750178953.621717 lhsmtool_posix[8335]: rebind [0x200000403:0x76:0x0] to [0x200000403:0x7b:0x0]
lhsmtool_posix: 1750178953.621953 lhsmtool_posix[8335]: rebind [0x200000403:0x77:0x0] to [0x200000403:0x7c:0x0]
lhsmtool_posix: 1750178953.622012 lhsmtool_posix[8335]: rebind [0x200000403:0x78:0x0] to [0x200000403:0x7d:0x0]
lhsmtool_posix: 1750178953.622081 lhsmtool_posix[8335]: rebind [0x200000403:0x79:0x0] to [0x200000403:0x7e:0x0]
lhsmtool_posix: 1750178953.622150 lhsmtool_posix[8335]: rebind [0x200000403:0x7a:0x0] to [0x200000403:0x7f:0x0]
lhsmtool_posix: 1750178953.622212 lhsmtool_posix[8335]: 5 lines read from '/tmp/tmp.7935', 5 rebind successful
lhsmtool_posix: 1750178953.622262 lhsmtool_posix[8335]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
PASS 15 (10s)

== sanity-hsm test 16: Test CT bandwith control option === 12:49:20 (1750178960)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_16/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1  "/mnt/lustre2"'
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 0.0256971 s, 816 MB/s
Waiting 200s for 'SUCCEED'
Changed after 2s: from 'WAITING' to 'STARTED'
Waiting 190s for 'SUCCEED'
Waiting 180s for 'SUCCEED'
Updated after 21s: want 'SUCCEED' got 'SUCCEED'
PASS 16 (23s)

== sanity-hsm test 20: Release is not permitted ========== 12:49:43 (1750178983)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00200103 s, 0.0 kB/s
0x200000403:0x87:0x0
Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted
Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted
Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted
Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted
PASS 20 (2s)

== sanity-hsm test 21: Simple release tests ============== 12:49:45 (1750178985)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_21/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0437393 s, 24.0 MB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
PASS 21 (4s)

== sanity-hsm test 22: Could not swap a release file ===== 12:49:49 (1750178989)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_22/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0353976 s, 29.6 MB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0390381 s, 26.9 MB/s
0x200000403:0x8b:0x0
lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d22.sanity-hsm/test_swap' and '/mnt/lustre/d22.sanity-hsm/test_release': Operation not supported (95)
PASS 22 (4s)

== sanity-hsm test 23: Release does not change a/mtime (utime) ========================================================== 12:49:53 (1750178993)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_23/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0443701 s, 23.6 MB/s
PASS 23 (2s)

== sanity-hsm test 24a: Archive, release, and restore does not change a/mtime (i/o) ========================================================== 12:49:55 (1750178995)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0507896 s, 20.6 MB/s
Waiting 200s for 'SUCCEED'
Waiting 200s for 'SUCCEED'
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
192.168.204.109@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0
Stopping client oleg409-client.virtnet /mnt/lustre (opts:)
Starting client: oleg409-client.virtnet:  -o user_xattr,flock 192.168.204.109@tcp:/lustre /mnt/lustre
PASS 24a (7s)

== sanity-hsm test 24b: root can archive, release, and restore user files ========================================================== 12:50:02 (1750179002)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0377706 s, 27.8 MB/s
Waiting 200s for 'SUCCEED'
Waiting 200s for 'SUCCEED'
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm]
/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm: (0x00000009) exists archived, archive_id:1
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [md5sum] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm]
PASS 24b (6s)

== sanity-hsm test 24c: check that user,group,other request masks work ========================================================== 12:50:08 (1750179008)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0377282 s, 27.8 MB/s
0x200000403:0x94:0x0
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm]
Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm]
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0370973 s, 28.3 MB/s
0x200000403:0x95:0x0
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm]
Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm]
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0394127 s, 26.6 MB/s
0x200000403:0x96:0x0
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm]
Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm]
PASS 24c (7s)

== sanity-hsm test 24d: check that read-only mounts are respected ========================================================== 12:50:15 (1750179015)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0403297 s, 26.0 MB/s
0x200000403:0x97:0x0
/mnt/lustre/d24d.sanity-hsm/f24d.sanity-hsm
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 0
	obdidx		 objid		 objid		 group
	     0	            80	         0x50	   0x280000401

Starting client: oleg409-client.virtnet:  -o user_xattr,flock 192.168.204.109@tcp:/lustre /mnt/lustre3
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre3"'
Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system
Waiting 200s for 'SUCCEED'
Waiting 200s for 'SUCCEED'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.44167 s, 2.4 MB/s
Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system
Stopping client oleg409-client.virtnet /mnt/lustre3 (opts:)
PASS 24d (7s)

== sanity-hsm test 24e: tar succeeds on HSM released files ========================================================== 12:50:22 (1750179022)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0371678 s, 28.2 MB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
/mnt/lustre/d24e.sanity-hsm/f24e.sanity-hsm: (0x0000000d) released exists archived, archive_id:1
tar: Removing leading `/' from member names
PASS 24e (4s)

== sanity-hsm test 24f: root can archive, release, and restore tar files ========================================================== 12:50:26 (1750179026)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
3d2fd8331483d30d32d70431b70233ef /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm
f24f.sanity-hsm
f24f.sanity-hsm
Sum0 = 3d2fd8331483d30d32d70431b70233ef  /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm, sum1 = 3d2fd8331483d30d32d70431b70233ef  /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm
PASS 24f (3s)

== sanity-hsm test 24g: write by non-owner still sets dirty ========================================================== 12:50:29 (1750179029)
RUNAS = 'runas -u 500 -g 500'
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Please listen carefully as our options have changed.
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [tee] [/mnt/lustre/d24g.sanity-hsm/f24g.sanity-hsm]
To be electrocuted by your telephone, press #.
PASS 24g (3s)

== sanity-hsm test 25a: Restore lost file (HS_LOST flag) from import ========================================================== 12:50:32 (1750179032)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
lhsmtool_posix: 1750179033.588376 lhsmtool_posix[18202]: action=1 src=d25a.sanity-hsm/f25a.sanity-hsm dst=/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750179033.591198 lhsmtool_posix[18202]: importing '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm'
lhsmtool_posix: 1750179033.606088 lhsmtool_posix[18202]: imported '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//0406/0x240000404:0x2:0x0'=='/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm'
lhsmtool_posix: 1750179033.606159 lhsmtool_posix[18202]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
md5sum: /mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm: No data available
PASS 25a (3s)

== sanity-hsm test 25b: Restore lost file (HS_LOST flag) after release ========================================================== 12:50:35 (1750179035)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
md5sum: /mnt/lustre/d25b.sanity-hsm/f25b.sanity-hsm: No data available
PASS 25b (3s)

== sanity-hsm test 26A: Remove the archive of a valid file ========================================================== 12:50:38 (1750179038)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00199281 s, 0.0 kB/s
Waiting 200s for 'SUCCEED'
PASS 26A (4s)

== sanity-hsm test 26a: Remove Archive On Last Unlink (RAoLU) policy ========================================================== 12:50:42 (1750179042)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Waiting 200s for 'SUCCEED'
PASS 26a (10s)

== sanity-hsm test 26b: RAoLU policy when CDT off ======== 12:50:52 (1750179052)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
mdt.lustre-MDT0000.hsm_control=shutdown
mdt.lustre-MDT0001.hsm_control=shutdown
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
PASS 26b (10s)

== sanity-hsm test 26c: RAoLU effective when file closed ========================================================== 12:51:02 (1750179062)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
multiop /mnt/lustre/d26c.sanity-hsm/f26c.sanity-hsm vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.7935
Waiting 200s for 'SUCCEED'
Waiting 200s for 'SUCCEED'
Updated after 9s: want 'SUCCEED' got 'SUCCEED'
PASS 26c (19s)

== sanity-hsm test 26d: RAoLU when Client eviction ======= 12:51:21 (1750179081)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0416571 s, 25.2 MB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
multiop /mnt/lustre/d26d.sanity-hsm/f26d.sanity-hsm vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.7935
Waiting 200s for 'SUCCEED'
Updated after 3s: want 'SUCCEED' got 'SUCCEED'
PASS 26d (11s)

== sanity-hsm test 26e: RAoLU with a non-started coordinator ========================================================== 12:51:32 (1750179092)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.041 s, 25.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0594427 s, 17.6 MB/s
Waiting 200s for 'SUCCEED'
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
Failing mds1 on oleg409-server
Stopping /mnt/lustre-mds1 (opts:) on oleg409-server
12:51:57 (1750179117) shut down
facet: mds1 facet_host: oleg409-server facet_failover_host: oleg409-server
Failover mds1 to oleg409-server
mount facets: mds1
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg409-server: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Started lustre-MDT0000
12:52:11 (1750179131) targets are mounted
12:52:11 (1750179131) facet_failover done
oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
Check cookie from RAoLU request (last: 1750178891, remove: 1750178892)
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
PASS 26e (71s)

== sanity-hsm test 27a: Remove the archive of an imported file (Operation not permitted) ========================================================== 12:52:44 (1750179164)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
39+0 records in
39+0 records out
40894464 bytes (41 MB) copied, 0.410214 s, 99.7 MB/s
lhsmtool_posix: 1750179165.011703 lhsmtool_posix[25878]: action=1 src=d27a.sanity-hsm/f27a.sanity-hsm dst=/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750179165.014788 lhsmtool_posix[25878]: importing '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm'
lhsmtool_posix: 1750179165.036497 lhsmtool_posix[25878]: imported '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//0400/0x240000404:0x4:0x0'=='/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm'
lhsmtool_posix: 1750179165.036594 lhsmtool_posix[25878]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
Cannot send HSM request (use of /mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm): Operation not permitted
PASS 27a (2s)

== sanity-hsm test 27b: Remove the archive of a relased file (Operation not permitted) ========================================================== 12:52:46 (1750179166)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0017914 s, 0.0 kB/s
Waiting 200s for 'SUCCEED'
Cannot send HSM request (use of /mnt/lustre/d27b.sanity-hsm/f27b.sanity-hsm): Operation not permitted
PASS 27b (3s)

== sanity-hsm test 28: Concurrent archive/file remove ==== 12:52:49 (1750179169)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_28/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00158137 s, 0.0 kB/s
mdt.lustre-MDT0000.hsm_control=disabled
mdt.lustre-MDT0001.hsm_control=disabled
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
PASS 28 (5s)

== sanity-hsm test 29a: Tests --mntpath and --archive options ========================================================== 12:52:54 (1750179174)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 7  "/mnt/lustre"'
Cannot send HSM request (use of /mnt/lustre): Invalid argument
hsm: '/qwerty/uyt' is not a valid FID
PASS 29a (2s)

== sanity-hsm test 29b: Archive/delete/remove by FID from the archive. ========================================================== 12:52:56 (1750179176)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0463885 s, 22.6 MB/s
Waiting 200s for 'SUCCEED'
Waiting 200s for 'SUCCEED'
PASS 29b (5s)

== sanity-hsm test 29c: Archive/delete/remove by FID, using a file list. ========================================================== 12:53:01 (1750179181)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0375318 s, 27.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0443459 s, 23.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0362026 s, 29.0 MB/s
PASS 29c (4s)

== sanity-hsm test 29d: hsm_remove by FID with archive_id 0 for unlinked file cause ========================================================== 12:53:05 (1750179185)

 SKIP: sanity-hsm test_29d Need 3 or more clients, have 1
SKIP 29d (1s)

== sanity-hsm test 30a: Restore at exec (import case) ==== 12:53:06 (1750179186)

 SKIP: sanity-hsm test_30a Need 2 or more clients, have 1
SKIP 30a (1s)

== sanity-hsm test 30b: Restore at exec (release case) === 12:53:07 (1750179187)

 SKIP: sanity-hsm test_30b Need 2 or more clients, have 1
SKIP 30b (1s)

== sanity-hsm test 30c: Update during exec of released file must fail ========================================================== 12:53:08 (1750179188)

 SKIP: sanity-hsm test_30c Need 2 or more clients, have 1
SKIP 30c (0s)

== sanity-hsm test 31a: Import a large file and check size during restore ========================================================== 12:53:08 (1750179188)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
39+0 records in
39+0 records out
40894464 bytes (41 MB) copied, 0.399279 s, 102 MB/s
lhsmtool_posix: 1750179189.835476 lhsmtool_posix[31495]: action=1 src=d31a.sanity-hsm/f31a.sanity-hsm dst=/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750179189.838439 lhsmtool_posix[31495]: importing '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm'
lhsmtool_posix: 1750179189.851321 lhsmtool_posix[31495]: imported '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//040c/0x200000407:0xb:0x0'=='/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm'
lhsmtool_posix: 1750179189.851399 lhsmtool_posix[31495]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
size seen is right: 40894464 == 40894464
 done
PASS 31a (16s)

== sanity-hsm test 31b: Restore a large unaligned file and check size during restore ========================================================== 12:53:24 (1750179204)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
39+0 records in
39+0 records out
39000000 bytes (39 MB) copied, 0.593886 s, 65.7 MB/s
Waiting 200s for 'SUCCEED'
size seen is right: 39000000 == 39000000
 done
PASS 31b (19s)

== sanity-hsm test 31c: Restore a large aligned file and check size during restore ========================================================== 12:53:43 (1750179223)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
39+0 records in
39+0 records out
40894464 bytes (41 MB) copied, 1.12666 s, 36.3 MB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
size seen is right: 40894464 == 40894464
 done
PASS 31c (19s)

== sanity-hsm test 33: Kill a restore waiting process ==== 12:54:02 (1750179242)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0049093 s, 0.0 kB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_33/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'STARTED'
Copytool is continued on oleg409-client.virtnet
/home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2829:  1314 Terminated              md5sum $f > /dev/null
PASS 33 (9s)

== sanity-hsm test 34: Remove file during restore ======== 12:54:11 (1750179251)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_34/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1  "/mnt/lustre2"'
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00330604 s, 0.0 kB/s
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'STARTED'
Updated after 2s: want 'STARTED' got 'STARTED'
Copytool is continued on oleg409-client.virtnet
/home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2861: kill: (2061) - No such process
PASS 34 (6s)

== sanity-hsm test 35: Overwrite file during restore ===== 12:54:17 (1750179257)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_35/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1  "/mnt/lustre2"'
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00374423 s, 0.0 kB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'STARTED'
Updated after 2s: want 'STARTED' got 'STARTED'
Copytool is continued on oleg409-client.virtnet
/home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2898: kill: (2815) - No such process
PASS 35 (8s)

== sanity-hsm test 36: Move file during restore ========== 12:54:25 (1750179265)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_36/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1  "/mnt/lustre2"'
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00205281 s, 0.0 kB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'STARTED'
Copytool is continued on oleg409-client.virtnet
/home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2934: kill: (3567) - No such process
PASS 36 (8s)

== sanity-hsm test 37: re-archive a dirty file =========== 12:54:33 (1750179273)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_37/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0790099 s, 13.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0292284 s, 35.9 MB/s
Waiting 200s for 'SUCCEED'
Changed after 2s: from 'SUCCEED
WAITING' to 'SUCCEED
SUCCEED'
Updated after 5s: want 'SUCCEED' got 'SUCCEED'
PASS 37 (71s)

== sanity-hsm test 40: Parallel archive requests ========= 12:55:44 (1750179344)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_40/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdt.lustre-MDT0000.hsm_control=purge
mdt.lustre-MDT0001.hsm_control=purge
Wait for all requests being enqueued...100 archive requests submitted
100 archive requests submitted
100 archive requests submitted
100 archive requests submitted
OK
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
PASS 40 (104s)

== sanity-hsm test 50: Archive with large number of pending HSM actions ========================================================== 12:57:28 (1750179448)
Filesystem                  Inodes IUsed  IFree IUse% Mounted on
192.168.204.109@tcp:/lustre 524226  1219 523007    1% /mnt/lustre
 - create 2834 (time 1750179462.14 total 10.00 last 283.40)
 - create 6053 (time 1750179472.15 total 20.00 last 321.83)
 - create 9292 (time 1750179482.15 total 30.01 last 323.78)
total: 10000 create in 32.31 seconds: 309.49 ops/second
10000
 - unlinked 0 (time 1750179530 ; total 0 ; last 0)
total: 10000 unlinks in 33 seconds: 303.030304 unlinks/second
Start Phase files_num: 10000 time: 44
 - create 3366 (time 1750179575.28 total 10.00 last 336.53)
 - create 6570 (time 1750179585.28 total 20.00 last 320.37)
 - create 9967 (time 1750179595.28 total 30.00 last 339.69)
 - create 13285 (time 1750179605.29 total 40.01 last 331.54)
 - create 16308 (time 1750179615.29 total 50.01 last 302.26)
 - create 19947 (time 1750179625.29 total 60.01 last 363.87)
total: 20000 create in 60.16 seconds: 332.42 ops/second
 - unlinked 0 (time 1750179709 ; total 0 ; last 0)
 - unlinked 10000 (time 1750179740 ; total 31 ; last 31)
total: 20000 unlinks in 58 seconds: 344.827576 unlinks/second
Middle Phase files_num: 20000 time: 82
 - create 3767 (time 1750179778.57 total 10.00 last 376.69)
 - create 8417 (time 1750179788.57 total 20.00 last 464.95)
total: 10000 create in 23.75 seconds: 421.02 ops/second
40000
 - unlinked 0 (time 1750179822 ; total 0 ; last 0)
total: 10000 unlinks in 32 seconds: 312.500000 unlinks/second
End Phase files_num: 10000 time: 28
40000
mdt.lustre-MDT0000.hsm_control=purge
mdt.lustre-MDT0001.hsm_control=purge
PASS 50 (413s)

== sanity-hsm test 52: Opened for write file on an evicted client should be set dirty ========================================================== 13:04:21 (1750179861)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_52/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0689802 s, 15.2 MB/s
multiop /mnt/lustre/d52.sanity-hsm/f52.sanity-hsm vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.7935
pdsh@oleg409-client: oleg409-client: ssh exited with exit code 5
PASS 52 (8s)

== sanity-hsm test 53: Opened for read file on an evicted client should not be set dirty ========================================================== 13:04:29 (1750179869)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_53/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0574682 s, 18.2 MB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
multiop /mnt/lustre/d53.sanity-hsm/f53.sanity-hsm vo_c
TMPPIPE=/tmp/multiop_open_wait_pipe.7935
pdsh@oleg409-client: oleg409-client: ssh exited with exit code 5
PASS 53 (9s)

== sanity-hsm test 54: Write during an archive cancels it ========================================================== 13:04:38 (1750179878)
39+0 records in
39+0 records out
39000000 bytes (39 MB) copied, 0.414548 s, 94.1 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_54/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1  "/mnt/lustre2"'
mdt.lustre-MDT0000.hsm.policy=+NRA
mdt.lustre-MDT0001.hsm.policy=+NRA
Waiting 200s for 'FAILED'
Waiting 170s for 'FAILED'
Updated after 36s: want 'FAILED' got 'FAILED'
mdt.lustre-MDT0000.hsm.policy=-NRA
mdt.lustre-MDT0001.hsm.policy=-NRA
PASS 54 (44s)

== sanity-hsm test 55: Truncate during an archive cancels it ========================================================== 13:05:22 (1750179922)
39+0 records in
39+0 records out
39000000 bytes (39 MB) copied, 0.420387 s, 92.8 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_55/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1  "/mnt/lustre2"'
Waiting 200s for 'STARTED'
Updated after 2s: want 'STARTED' got 'STARTED'
mdt.lustre-MDT0000.hsm.policy=+NRA
mdt.lustre-MDT0001.hsm.policy=+NRA
mdt.lustre-MDT0000.hsm.policy=-NRA
mdt.lustre-MDT0001.hsm.policy=-NRA
PASS 55 (8s)

== sanity-hsm test 56: Setattr during an archive is ok === 13:05:30 (1750179930)
39+0 records in
39+0 records out
39000000 bytes (39 MB) copied, 0.5324 s, 73.3 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_56/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1  "/mnt/lustre2"'
Waiting 200s for 'STARTED'
Updated after 2s: want 'STARTED' got 'STARTED'
Waiting 200s for 'SUCCEED'
Waiting 190s for 'SUCCEED'
Waiting 170s for 'SUCCEED'
Updated after 36s: want 'SUCCEED' got 'SUCCEED'
PASS 56 (41s)

== sanity-hsm test 57: Archive a file with dirty cache on another node ========================================================== 13:06:11 (1750179971)

 SKIP: sanity-hsm test_57 Need 2 or more clients, have 1
SKIP 57 (1s)

== sanity-hsm test 58: Truncate a released file will trigger restore ========================================================== 13:06:13 (1750179973)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_58/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
truncate up from 1716 to 3432
Waiting 200s for 'SUCCEED'
/mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2
truncate down from 1716 to 858
Waiting 200s for 'SUCCEED'
/mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2
truncate to 0
Waiting 200s for 'SUCCEED'
/mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2
PASS 58 (12s)

== sanity-hsm test 59: Release stripeless file with non-zero size ========================================================== 13:06:25 (1750179985)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_59/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
PASS 59 (4s)

== sanity-hsm test 60: Changing progress update interval from default ========================================================== 13:06:29 (1750179989)
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.10996 s, 95.4 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_60/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 --update-interval 5 "/mnt/lustre2"'
Waiting 100s for 'start copy'
Expecting a progress update within 20 seconds... 5242880 bytes copied in 3 seconds.
Wait for on going archive hsm action to complete
Waiting 10s for 'copied'
Updated after 4s: want 'copied' got 'copied'
PASS 60 (14s)

== sanity-hsm test 61: Waiting archive of a removed file should fail ========================================================== 13:06:44 (1750180004)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_61/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdt.lustre-MDT0000.hsm_control=disabled
mdt.lustre-MDT0001.hsm_control=disabled
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
PASS 61 (7s)

== sanity-hsm test 70: Copytool logs JSON register/unregister events to FIFO ========================================================== 13:06:50 (1750180010)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_70/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --event-fifo /tmp/sanity-hsm.test_70.eBGQ/fifo "/mnt/lustre2"'
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
Register/Unregister events look OK.
PASS 70 (5s)

== sanity-hsm test 71: Copytool logs JSON archive events to FIFO ========================================================== 13:06:55 (1750180015)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_71/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_71.sW6X/fifo "/mnt/lustre2"'
mdt.lustre-MDT0000.hsm.policy=-NRA
mdt.lustre-MDT0001.hsm.policy=-NRA
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.05 s, 21.0 MB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Archive events look OK.
mdt.lustre-MDT0000.hsm.policy=-NRA
mdt.lustre-MDT0001.hsm.policy=-NRA
PASS 71 (8s)

== sanity-hsm test 72: Copytool logs JSON restore events to FIFO ========================================================== 13:07:04 (1750180024)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_72/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_72.cezP/fifo "/mnt/lustre2"'
16+0 records in
16+0 records out
16000000 bytes (16 MB) copied, 0.296734 s, 53.9 MB/s
lhsmtool_posix: 1750180026.503643 lhsmtool_posix[23698]: action=1 src=d72.sanity-hsm/f72.sanity-hsm dst=/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750180026.509952 lhsmtool_posix[23698]: importing '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm'
lhsmtool_posix: 1750180026.539327 lhsmtool_posix[23698]: imported '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//0bc5/0x200000bd2:0x17:0x0'=='/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm'
lhsmtool_posix: 1750180026.539402 lhsmtool_posix[23698]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
Verifying released state: 
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Restore events look OK.
PASS 72 (7s)

== sanity-hsm test 90: Archive/restore a file list ======= 13:07:11 (1750180031)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_90/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdt.lustre-MDT0000.hsm_control=purge
mdt.lustre-MDT0001.hsm_control=purge
Waiting 200s for ''
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Waiting 200s for ''
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
PASS 90 (74s)

== sanity-hsm test 100: Set coordinator /proc tunables === 13:08:25 (1750180105)
Testing mdt.lustre-MDT0000.hsm.loop_period
Set loop_period to 0 must failed
oleg409-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/loop_period=0: Invalid argument
oleg409-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/loop_period'='0': Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 22
oleg409-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/loop_period=0: Invalid argument
oleg409-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/loop_period'='0': Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 22
Testing mdt.lustre-MDT0000.hsm.grace_delay
Set grace_delay to 0 must failed
oleg409-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/grace_delay=0: Invalid argument
oleg409-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/grace_delay'='0': Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 22
oleg409-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/grace_delay=0: Invalid argument
oleg409-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/grace_delay'='0': Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 22
Testing mdt.lustre-MDT0000.hsm.active_request_timeout
Set active_request_timeout to 0 must failed
oleg409-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/active_request_timeout=0: Invalid argument
oleg409-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/active_request_timeout'='0': Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 22
oleg409-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/active_request_timeout=0: Invalid argument
oleg409-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/active_request_timeout'='0': Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 22
Testing mdt.lustre-MDT0000.hsm.max_requests
Set max_requests to 0 must failed
oleg409-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=0: Invalid argument
oleg409-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='0': Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 22
oleg409-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=0: Invalid argument
oleg409-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='0': Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 22
Testing mdt.lustre-MDT0000.hsm.default_archive_id
Set default_archive_id to 0 must failed
oleg409-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/default_archive_id=0: Invalid argument
oleg409-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/default_archive_id'='0': Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 22
oleg409-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/default_archive_id=0: Invalid argument
oleg409-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/default_archive_id'='0': Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 22
PASS 100 (18s)

== sanity-hsm test 102: Verify coordinator control ======= 13:08:43 (1750180123)
mdt.lustre-MDT0000.hsm_control=disabled
mdt.lustre-MDT0001.hsm_control=disabled
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
mdt.lustre-MDT0000.hsm_control=shutdown
mdt.lustre-MDT0001.hsm_control=shutdown
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
PASS 102 (9s)

== sanity-hsm test 103: Purge all requests =============== 13:08:52 (1750180132)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdt.lustre-MDT0000.hsm_control=purge
mdt.lustre-MDT0001.hsm_control=purge
Current requests
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
PASS 103 (8s)

== sanity-hsm test 103a: Purge pending restore requests == 13:09:00 (1750180140)
mdt.lustre-MDT0000.hsm.policy=-NBR
mdt.lustre-MDT0001.hsm.policy=-NBR
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
(0x200000403:0x148:0x0|0x200000403:0x149:0x0|0x200000403:0x14a:0x0|0x200000403:0x14b:0x0|0x200000403:0x14c:0x0|0x200000403:0x14d:0x0|0x200000403:0x14e:0x0|0x200000403:0x14f:0x0).*action=ARCHIVE.*status=SUCCEED
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
cat: /mnt/lustre/.lustre/fid/0x200000403:0x148:0x0: No data available
cat: /mnt/lustre/.lustre/fid/0x200000403:0x149:0x0: No data available
cat: /mnt/lustre/.lustre/fid/0x200000403:0x14b:0x0: No data available
cat: /mnt/lustre/.lustre/fid/0x200000403:0x14a:0x0: No data available
cat: /mnt/lustre/.lustre/fid/0x200000403:0x14c:0x0: No data available
cat: /mnt/lustre/.lustre/fid/0x200000403:0x14e:0x0: No data available
cat: /mnt/lustre/.lustre/fid/0x200000403:0x14d:0x0: No data available
cat: /mnt/lustre/.lustre/fid/0x200000403:0x14f:0x0: No data available
mdt.lustre-MDT0000.hsm_control=purge
mdt.lustre-MDT0001.hsm_control=purge
PASS 103a (12s)

== sanity-hsm test 104: Copy tool data field ============= 13:09:12 (1750180152)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00346068 s, 0.0 kB/s
mdt.lustre-MDT0000.hsm_control=purge
mdt.lustre-MDT0001.hsm_control=purge
PASS 104 (7s)

== sanity-hsm test 105: Restart of coordinator =========== 13:09:19 (1750180159)
mdt.lustre-MDT0000.hsm_control=disabled
mdt.lustre-MDT0001.hsm_control=disabled
mdt.lustre-MDT0000.hsm_control=shutdown
mdt.lustre-MDT0001.hsm_control=shutdown
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
mdt.lustre-MDT0000.hsm_control=disabled
mdt.lustre-MDT0001.hsm_control=disabled
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
mdt.lustre-MDT0000.hsm_control=purge
mdt.lustre-MDT0001.hsm_control=purge
PASS 105 (21s)

== sanity-hsm test 106: Copytool register/unregister ===== 13:09:41 (1750180181)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
oleg409-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool
oleg409-client.virtnet: executing get_client_uuid /mnt/lustre2
found agent uuid=94a00271-e790-468b-8c8b-c344f94cbb5f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1
found agent uuid=94a00271-e790-468b-8c8b-c344f94cbb5f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2
oleg409-client.virtnet: 30870
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
uuid not found in agent list on mds1
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
uuid not found in agent list on mds2
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
oleg409-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool
oleg409-client.virtnet: executing get_client_uuid /mnt/lustre2
found agent uuid=94a00271-e790-468b-8c8b-c344f94cbb5f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1
found agent uuid=94a00271-e790-468b-8c8b-c344f94cbb5f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2
PASS 106 (14s)

== sanity-hsm test 107: Copytool re-register after MDS restart ========================================================== 13:09:54 (1750180194)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_107/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Failing mds1 on oleg409-server
Stopping /mnt/lustre-mds1 (opts:) on oleg409-server
13:09:58 (1750180198) shut down
facet: mds1 facet_host: oleg409-server facet_failover_host: oleg409-server
Failover mds1 to oleg409-server
mount facets: mds1
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg409-server: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Started lustre-MDT0000
13:10:15 (1750180215) targets are mounted
13:10:15 (1750180215) facet_failover done
oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
PASS 107 (29s)

== sanity-hsm test 109: Policy display/change ============ 13:10:23 (1750180223)
mdt.lustre-MDT0000.hsm.policy=+NBR
mdt.lustre-MDT0000.hsm.policy=+NRA
mdt.lustre-MDT0000.hsm.policy=-NBR
mdt.lustre-MDT0000.hsm.policy=-NRA
mdt.lustre-MDT0000.hsm.policy=NRA NBR
Next set_param must failed
oleg409-server: error: set_param: setting /sys/kernel/debug/lustre/mdt/lustre-MDT0000/hsm/policy=wrong: Invalid argument
oleg409-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/policy'='wrong': Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 22
Back to default policy
mdt.lustre-MDT0000.hsm.policy=+NRA
mdt.lustre-MDT0001.hsm.policy=+NRA
mdt.lustre-MDT0000.hsm.policy=-NBR
mdt.lustre-MDT0001.hsm.policy=-NBR
PASS 109 (10s)

== sanity-hsm test 110a: Non blocking restore policy (import case) ========================================================== 13:10:33 (1750180233)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
lhsmtool_posix: 1750180234.473361 lhsmtool_posix[2602]: action=1 src=d110a.sanity-hsm/f110a.sanity-hsm dst=/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750180234.477266 lhsmtool_posix[2602]: importing '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm'
lhsmtool_posix: 1750180234.509858 lhsmtool_posix[2602]: imported '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//0bb6/0x200000bd2:0x64:0x0'=='/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm'
lhsmtool_posix: 1750180234.509930 lhsmtool_posix[2602]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
mdt.lustre-MDT0000.hsm.policy=+NBR
mdt.lustre-MDT0001.hsm.policy=+NBR
md5sum: /mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm: No data available
mdt.lustre-MDT0000.hsm.policy=-NBR
mdt.lustre-MDT0001.hsm.policy=-NBR
PASS 110a (5s)

== sanity-hsm test 110b: Non blocking restore policy (release case) ========================================================== 13:10:38 (1750180238)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdt.lustre-MDT0000.hsm.policy=+NBR
mdt.lustre-MDT0001.hsm.policy=+NBR
md5sum: /mnt/lustre/d110b.sanity-hsm/f110b.sanity-hsm: No data available
mdt.lustre-MDT0000.hsm.policy=-NBR
mdt.lustre-MDT0001.hsm.policy=-NBR
PASS 110b (5s)

== sanity-hsm test 111a: No retry policy (import case), restore will error ========================================================== 13:10:43 (1750180243)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
lhsmtool_posix: 1750180244.984074 lhsmtool_posix[4109]: action=1 src=d111a.sanity-hsm/f111a.sanity-hsm dst=/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750180244.996185 lhsmtool_posix[4109]: importing '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm'
lhsmtool_posix: 1750180245.022098 lhsmtool_posix[4109]: imported '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//0bbb/0x200000bd2:0x69:0x0'=='/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm'
lhsmtool_posix: 1750180245.022894 lhsmtool_posix[4109]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
mdt.lustre-MDT0000.hsm.policy=+NRA
mdt.lustre-MDT0001.hsm.policy=+NRA
Remove from backend: 0x200000bd2:0x69:0x0 = /tmp/arc1/sanity-hsm.test_111a/0bbb/0x200000bd2:0x69:0x0
Waiting 200s for 'FAILED'
Updated after 2s: want 'FAILED' got 'FAILED'
mdt.lustre-MDT0000.hsm.policy=-NRA
mdt.lustre-MDT0001.hsm.policy=-NRA
PASS 111a (7s)

== sanity-hsm test 111b: No retry policy (release case), restore will error ========================================================== 13:10:50 (1750180250)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdt.lustre-MDT0000.hsm.policy=+NRA
mdt.lustre-MDT0001.hsm.policy=+NRA
Remove from backend: 0x200000403:0x158:0x0 = /tmp/arc1/sanity-hsm.test_111b/055b/0x200000403:0x158:0x0
Waiting 200s for 'FAILED'
mdt.lustre-MDT0000.hsm.policy=-NRA
mdt.lustre-MDT0001.hsm.policy=-NRA
PASS 111b (8s)

== sanity-hsm test 112: State of recorded request ======== 13:10:58 (1750180258)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_112/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdt.lustre-MDT0000.hsm_control=disabled
mdt.lustre-MDT0001.hsm_control=disabled
/mnt/lustre/d112.sanity-hsm/f112.sanity-hsm: ARCHIVE waiting (from 0 to EOF)
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
PASS 112 (8s)

== sanity-hsm test 113: wrong stat after restore ========= 13:11:06 (1750180266)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0332547 s, 31.5 MB/s
Starting client: oleg409-client.virtnet:  -o user_xattr,flock 192.168.204.109@tcp:/lustre /mnt/lustre3
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_113/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre3"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Verifying released state: 
multiop /mnt/lustre/d113.sanity-hsm/f113.sanity-hsm voO_WRONLY:O_APPEND:_w4c
TMPPIPE=/tmp/multiop_open_wait_pipe.7935
  File: '/mnt/lustre2/d113.sanity-hsm/f113.sanity-hsm'
  Size: 1048576   	Blocks: 1          IO Block: 4194304 regular file
Device: 2c54f966h/743766374d	Inode: 144115205306057051  Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-06-17 13:11:06.000000000 -0400
Modify: 2025-06-17 13:11:06.000000000 -0400
Change: 2025-06-17 13:11:06.000000000 -0400
 Birth: -
Stopping client oleg409-client.virtnet /mnt/lustre3 (opts:)
PASS 113 (7s)

== sanity-hsm test 114: Incompatible request does not set other requests as STARTED ========================================================== 13:11:13 (1750180273)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00178065 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00325595 s, 0.0 kB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_114/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdt.lustre-MDT0000.hsm_control=disabled
mdt.lustre-MDT0001.hsm_control=disabled
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
PASS 114 (8s)

== sanity-hsm test 200: Register/Cancel archive ========== 13:11:21 (1750180281)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00306142 s, 0.0 kB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_200/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'STARTED'
Updated after 2s: want 'STARTED' got 'STARTED'
Waiting 200s for 'CANCELED'
Updated after 2s: want 'CANCELED' got 'CANCELED'
Copytool is continued on oleg409-client.virtnet
PASS 200 (7s)

== sanity-hsm test 201: Register/Cancel restore ========== 13:11:28 (1750180288)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_201/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
39+0 records in
39+0 records out
40894464 bytes (41 MB) copied, 0.587516 s, 69.6 MB/s
lhsmtool_posix: 1750180290.402510 lhsmtool_posix[8914]: action=1 src=d201.sanity-hsm/f201.sanity-hsm dst=/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750180290.412766 lhsmtool_posix[8914]: importing '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm'
lhsmtool_posix: 1750180290.456971 lhsmtool_posix[8914]: imported '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//0ba0/0x200000bd2:0x72:0x0'=='/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm'
lhsmtool_posix: 1750180290.457036 lhsmtool_posix[8914]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
mdt.lustre-MDT0000.hsm_control=disabled
mdt.lustre-MDT0001.hsm_control=disabled
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
PASS 201 (9s)

== sanity-hsm test 202: Register/Cancel remove =========== 13:11:37 (1750180297)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00483028 s, 0.0 kB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_202/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'STARTED'
Updated after 2s: want 'STARTED' got 'STARTED'
PASS 202 (8s)

== sanity-hsm test 220A: Changelog for archive =========== 13:11:45 (1750180305)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl1 cl1'
lustre-MDT0000.3 16HSM   17:11:48.911171371 2025.06.17 0x0 t=[0x200000403:0x163:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0001: clear the changelog for cl1 of all records
lustre-MDT0001: Deregistered changelog user #1
lustre-MDT0000: clear the changelog for cl1 of all records
lustre-MDT0000: Deregistered changelog user #1
PASS 220A (10s)

== sanity-hsm test 220a: Changelog for failed archive ==== 13:11:55 (1750180315)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl2 cl2'
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'STARTED'
Copytool is continued on oleg409-client.virtnet
lustre-MDT0000.6 16HSM   17:12:00.358868440 2025.06.17 0x2 t=[0x200000403:0x164:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x75:0x0]
lustre-MDT0001: clear the changelog for cl2 of all records
lustre-MDT0001: Deregistered changelog user #2
lustre-MDT0000: clear the changelog for cl2 of all records
lustre-MDT0000: Deregistered changelog user #2
PASS 220a (13s)

== sanity-hsm test 221: Changelog for archive canceled === 13:12:08 (1750180328)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00793841 s, 0.0 kB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_221/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1  "/mnt/lustre2"'
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl3 cl3'
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'STARTED'
Copytool is continued on oleg409-client.virtnet
lustre-MDT0000.8 16HSM   17:12:13.801251786 2025.06.17 0x7d t=[0x200000403:0x165:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x75:0x0]
lustre-MDT0001: clear the changelog for cl3 of all records
lustre-MDT0001: Deregistered changelog user #3
lustre-MDT0000: clear the changelog for cl3 of all records
lustre-MDT0000: Deregistered changelog user #3
PASS 221 (11s)

== sanity-hsm test 222a: Changelog for explicit restore == 13:12:19 (1750180339)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
lhsmtool_posix: 1750180340.814735 lhsmtool_posix[13096]: action=1 src=d222a.sanity-hsm/f222a.sanity-hsm dst=/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750180340.818143 lhsmtool_posix[13096]: importing '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm'
lhsmtool_posix: 1750180340.843093 lhsmtool_posix[13096]: imported '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//0bab/0x200000bd2:0x79:0x0'=='/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm'
lhsmtool_posix: 1750180340.843827 lhsmtool_posix[13096]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl4 cl4'
lustre-MDT0000.11 16HSM   17:12:22.944950594 2025.06.17 0x80 t=[0x200000bd2:0x79:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x75:0x0]
lustre-MDT0001: clear the changelog for cl4 of all records
lustre-MDT0001: Deregistered changelog user #4
lustre-MDT0000: clear the changelog for cl4 of all records
lustre-MDT0000: Deregistered changelog user #4
PASS 222a (9s)

== sanity-hsm test 222b: Changelog for implicit restore == 13:12:28 (1750180348)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl5 cl5'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
0e339f9bf886034c50444841bff8eb5d  /mnt/lustre/d222b.sanity-hsm/f222b.sanity-hsm
lustre-MDT0000.20 16HSM   17:12:34.116287400 2025.06.17 0x80 t=[0x200000403:0x167:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0001: clear the changelog for cl5 of all records
lustre-MDT0001: Deregistered changelog user #5
lustre-MDT0000: clear the changelog for cl5 of all records
lustre-MDT0000: Deregistered changelog user #5
PASS 222b (12s)

== sanity-hsm test 222c: Changelog for failed explicit restore ========================================================== 13:12:41 (1750180361)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
lhsmtool_posix: 1750180362.234543 lhsmtool_posix[14859]: action=1 src=d222c.sanity-hsm/f222c.sanity-hsm dst=/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750180362.239114 lhsmtool_posix[14859]: importing '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm'
lhsmtool_posix: 1750180362.264726 lhsmtool_posix[14859]: imported '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//0baf/0x200000bd2:0x7d:0x0'=='/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm'
lhsmtool_posix: 1750180362.264797 lhsmtool_posix[14859]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl6 cl6'
Copytool is suspended on oleg409-client.virtnet
Copytool is continued on oleg409-client.virtnet
lustre-MDT0000.22 16HSM   17:12:45.698175928 2025.06.17 0x82 t=[0x200000bd2:0x7d:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0001: clear the changelog for cl6 of all records
lustre-MDT0001: Deregistered changelog user #6
lustre-MDT0000: clear the changelog for cl6 of all records
lustre-MDT0000: Deregistered changelog user #6
PASS 222c (11s)

== sanity-hsm test 222d: Changelog for failed implicit restore ========================================================== 13:12:52 (1750180372)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl7 cl7'
Remove from backend: 0x200000403:0x169:0x0 = /tmp/arc1/sanity-hsm.test_222d/056a/0x200000403:0x169:0x0
md5sum: /mnt/lustre/d222d.sanity-hsm/f222d.sanity-hsm: No data available
lustre-MDT0000.29 16HSM   17:12:56.835992619 2025.06.17 0x82 t=[0x200000403:0x169:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0001: clear the changelog for cl7 of all records
lustre-MDT0001: Deregistered changelog user #7
lustre-MDT0000: clear the changelog for cl7 of all records
lustre-MDT0000: Deregistered changelog user #7
PASS 222d (11s)

== sanity-hsm test 223a: Changelog for restore canceled (import case) ========================================================== 13:13:03 (1750180383)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1  "/mnt/lustre2"'
39+0 records in
39+0 records out
40894464 bytes (41 MB) copied, 0.937373 s, 43.6 MB/s
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl8 cl8'
lhsmtool_posix: 1750180389.310434 lhsmtool_posix[16734]: action=1 src=d223a.sanity-hsm/f223a.sanity-hsm dst=/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm mount_point=/mnt/lustre2
lhsmtool_posix: 1750180389.318328 lhsmtool_posix[16734]: importing '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm'
lhsmtool_posix: 1750180389.367716 lhsmtool_posix[16734]: imported '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//0b50/0x200000bd2:0x82:0x0'=='/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm'
lhsmtool_posix: 1750180389.367791 lhsmtool_posix[16734]: process finished, errs: 0 major, 0 minor, rc=0 (Success)
Waiting 200s for 'CANCELED'
Waiting 200s for 'SUCCEED'
Waiting 190s for 'SUCCEED'
Waiting 180s for 'SUCCEED'
Updated after 27s: want 'SUCCEED' got 'SUCCEED'
lustre-MDT0000.36 16HSM   17:13:38.522159996 2025.06.17 0xfd t=[0x200000bd2:0x82:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x80:0x0]
lustre-MDT0001: clear the changelog for cl8 of all records
lustre-MDT0001: Deregistered changelog user #8
lustre-MDT0000: clear the changelog for cl8 of all records
lustre-MDT0000: Deregistered changelog user #8
PASS 223a (43s)

== sanity-hsm test 223b: Changelog for restore canceled (release case) ========================================================== 13:13:46 (1750180426)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0025495 s, 0.0 kB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1  "/mnt/lustre2"'
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl9 cl9'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'STARTED'
Updated after 2s: want 'STARTED' got 'STARTED'
Waiting 200s for 'CANCELED'
Updated after 2s: want 'CANCELED' got 'CANCELED'
Copytool is continued on oleg409-client.virtnet
lustre-MDT0000.43 16HSM   17:13:56.305749800 2025.06.17 0xfd t=[0x200000403:0x16d:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0001: clear the changelog for cl9 of all records
lustre-MDT0001: Deregistered changelog user #9
lustre-MDT0000: clear the changelog for cl9 of all records
lustre-MDT0000: Deregistered changelog user #9
PASS 223b (17s)

== sanity-hsm test 224A: Changelog for remove ============ 13:14:03 (1750180443)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl10 cl10'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
lustre-MDT0000.48 16HSM   17:14:10.408077379 2025.06.17 0x200 t=[0x200000403:0x16f:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0001: clear the changelog for cl10 of all records
lustre-MDT0001: Deregistered changelog user #10
lustre-MDT0000: clear the changelog for cl10 of all records
lustre-MDT0000: Deregistered changelog user #10
PASS 224A (16s)

== sanity-hsm test 224a: Changelog for failed remove ===== 13:14:20 (1750180460)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl11 cl11'
Remove from backend: 0x200000403:0x170:0x0 = /tmp/arc1/sanity-hsm.test_224a/0573/0x200000403:0x170:0x0
Copytool is suspended on oleg409-client.virtnet
Copytool is continued on oleg409-client.virtnet
lustre-MDT0000.53 16HSM   17:14:24.970855728 2025.06.17 0x202 t=[0x200000403:0x170:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x7c:0x0]
lustre-MDT0001: clear the changelog for cl11 of all records
lustre-MDT0001: Deregistered changelog user #11
lustre-MDT0000: clear the changelog for cl11 of all records
lustre-MDT0000: Deregistered changelog user #11
PASS 224a (12s)

== sanity-hsm test 225: Changelog for remove canceled ==== 13:14:32 (1750180472)
Test disabled
PASS 225 (4s)

== sanity-hsm test 226: changelog for last rm/mv with exiting archive ========================================================== 13:14:36 (1750180476)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_226/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
0x200000403:0x173:0x0
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl12 cl12'
Waiting 200s for 'SUCCEED'
Waiting 200s for 'SUCCEED'
lustre-MDT0000.54 16HSM   17:14:41.987993867 2025.06.17 0x280 t=[0x200000403:0x171:0x0] ef=0x13 u=0:0 nid=0@<0:0>
lustre-MDT0000.55 16HSM   17:14:42.058181125 2025.06.17 0x280 t=[0x200000403:0x171:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x75:0x0]
lustre-MDT0000.56 16HSM   17:14:42.058452355 2025.06.17 0x0 t=[0x200000403:0x171:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x75:0x0]
lustre-MDT0000.57 16HSM   17:14:44.060708107 2025.06.17 0x280 t=[0x200000403:0x172:0x0] ef=0x13 u=0:0 nid=0@<0:0>
lustre-MDT0000.58 16HSM   17:14:44.126755766 2025.06.17 0x280 t=[0x200000403:0x172:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x75:0x0]
lustre-MDT0000.59 16HSM   17:14:44.126821887 2025.06.17 0x0 t=[0x200000403:0x172:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x75:0x0]
lustre-MDT0000.60 06UNLNK 17:14:45.254749505 2025.06.17 0x3 t=[0x200000403:0x171:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x87:0x0] f226.sanity-hsm-1
lustre-MDT0000.60 06UNLNK 17:14:45.254749505 2025.06.17 0x3 t=[0x200000403:0x171:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x87:0x0] f226.sanity-hsm-1
lustre-MDT0000.61 08RENME 17:14:45.561524162 2025.06.17 0x3 t=[0x200000403:0x172:0x0] j=mv.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x200000bd2:0x87:0x0] f226.sanity-hsm-2 s=[0x200000403:0x173:0x0] sp=[0x200000bd2:0x87:0x0] f226.sanity-hsm-3
lustre-MDT0001: clear the changelog for cl12 of all records
lustre-MDT0001: Deregistered changelog user #12
lustre-MDT0000: clear the changelog for cl12 of all records
lustre-MDT0000: Deregistered changelog user #12
PASS 226 (15s)

== sanity-hsm test 227: changelog when explicit setting of HSM flags ========================================================== 13:14:51 (1750180491)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00510532 s, 0.0 kB/s
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl13 cl13'
lustre-MDT0001: clear the changelog for cl13 of all records
lustre-MDT0001: Deregistered changelog user #13
lustre-MDT0000: clear the changelog for cl13 of all records
lustre-MDT0000: Deregistered changelog user #13
PASS 227 (14s)

== sanity-hsm test 228: On released file, return extend to FIEMAP. For [cp,tar] --sparse ========================================================== 13:15:05 (1750180505)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_228/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0590178 s, 17.8 MB/s
Waiting 200s for 'SUCCEED'
/mnt/lustre/f228.sanity-hsm: 1 extent found
tar: Removing leading `/' from member names
mnt/lustre/f228.sanity-hsm
PASS 228 (9s)

== sanity-hsm test 250: Coordinator max request ========== 13:15:14 (1750180514)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_250/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00338635 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0099055 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00387313 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00462497 s, 0.0 kB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00368807 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00571586 s, 0.0 kB/s
Waiting 200s for 'SUCCEED'
Waiting 200s for 'SUCCEED'
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'STARTED'
Copytool is continued on oleg409-client.virtnet
PASS 250 (24s)

== sanity-hsm test 251: Coordinator request timeout ====== 13:15:38 (1750180538)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0052358 s, 0.0 kB/s
mdt.lustre-MDT0000.hsm_control=disabled
mdt.lustre-MDT0001.hsm_control=disabled
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_251/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'CANCELED'
PASS 251 (17s)

== sanity-hsm test 252: Timeout'ed running archive of a removed file should be canceled ========================================================== 13:15:55 (1750180555)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00349678 s, 0.0 kB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_252/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Copytool is suspended on oleg409-client.virtnet
Waiting 200s for 'STARTED'
Copytool is continued on oleg409-client.virtnet
PASS 252 (11s)

== sanity-hsm test 253: Check for wrong file size after release ========================================================== 13:16:06 (1750180566)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_253/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
10+0 records in
10+0 records out
10000000 bytes (10 MB) copied, 0.108648 s, 92.0 MB/s
fail_loc=0x807
Cannot send HSM request (use of /mnt/lustre/d253.sanity-hsm/f253.sanity-hsm): Invalid argument
could not release file
PASS 253 (5s)

== sanity-hsm test 254a: Request counters are initialized to zero ========================================================== 13:16:11 (1750180571)
PASS 254a (5s)

== sanity-hsm test 254b: Request counters are correctly incremented and decremented ========================================================== 13:16:17 (1750180577)
Will launch 35 requests of each type
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_254b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Checking archive requests
Copytool is suspended on oleg409-client.virtnet
Copytool is continued on oleg409-client.virtnet
Checking restore requests
Copytool is suspended on oleg409-client.virtnet
Copytool is continued on oleg409-client.virtnet
Checking remove requests
Copytool is suspended on oleg409-client.virtnet
Copytool is continued on oleg409-client.virtnet
PASS 254b (23s)

== sanity-hsm test 255: Copytool registration wakes the coordinator up ========================================================== 13:16:40 (1750180600)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00476738 s, 0.0 kB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00457642 s, 0.0 kB/s
0x200000403:0x1cb:0x0
mdt.lustre-MDT0000.hsm_control=shutdown
mdt.lustre-MDT0001.hsm_control=shutdown
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
PASS 255 (16s)

== sanity-hsm test 260a: Restore request have priority over other requests ========================================================== 13:16:57 (1750180617)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0641217 s, 16.4 MB/s
0x200000403:0x1cc:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0672143 s, 15.6 MB/s
0x200000403:0x1cd:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0584957 s, 17.9 MB/s
0x200000403:0x1ce:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0548546 s, 19.1 MB/s
0x200000403:0x1cf:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0529796 s, 19.8 MB/s
0x200000403:0x1d0:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0499331 s, 21.0 MB/s
0x200000403:0x1d1:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0613462 s, 17.1 MB/s
0x200000403:0x1d2:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0606218 s, 17.3 MB/s
0x200000403:0x1d3:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0538194 s, 19.5 MB/s
0x200000403:0x1d4:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0527501 s, 19.9 MB/s
0x200000403:0x1d5:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0601677 s, 17.4 MB/s
0x200000403:0x1d6:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0508002 s, 20.6 MB/s
0x200000403:0x1d7:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0490256 s, 21.4 MB/s
0x200000403:0x1d8:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0594598 s, 17.6 MB/s
0x200000403:0x1d9:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.058306 s, 18.0 MB/s
0x200000403:0x1da:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0675416 s, 15.5 MB/s
0x200000403:0x1db:0x0
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
ARCHIVE
ARCHIVE
RESTORE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
PASS 260a (22s)

== sanity-hsm test 260b: Restore request have priority over other requests ========================================================== 13:17:19 (1750180639)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0664183 s, 15.8 MB/s
0x200000403:0x1de:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0748473 s, 14.0 MB/s
0x200000403:0x1df:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0643235 s, 16.3 MB/s
0x200000403:0x1e0:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0558134 s, 18.8 MB/s
0x200000403:0x1e1:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0629531 s, 16.7 MB/s
0x200000403:0x1e2:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.081243 s, 12.9 MB/s
0x200000403:0x1e3:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0619638 s, 16.9 MB/s
0x200000403:0x1e4:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0627757 s, 16.7 MB/s
0x200000403:0x1e5:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.057206 s, 18.3 MB/s
0x200000403:0x1e6:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0565425 s, 18.5 MB/s
0x200000403:0x1e7:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0626233 s, 16.7 MB/s
0x200000403:0x1e8:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0703529 s, 14.9 MB/s
0x200000403:0x1e9:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.04938 s, 21.2 MB/s
0x200000403:0x1ea:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0595134 s, 17.6 MB/s
0x200000403:0x1eb:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0542582 s, 19.3 MB/s
0x200000403:0x1ec:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0808943 s, 13.0 MB/s
0x200000403:0x1ed:0x0
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
RESTORE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
PASS 260b (22s)

== sanity-hsm test 260c: Requests are not reordered on the 'hot' path of the coordinator ========================================================== 13:17:41 (1750180661)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0569816 s, 18.4 MB/s
0x200000403:0x1ef:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0539437 s, 19.4 MB/s
0x200000403:0x1f0:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0580971 s, 18.0 MB/s
0x200000403:0x1f1:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0611844 s, 17.1 MB/s
0x200000403:0x1f2:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0547498 s, 19.2 MB/s
0x200000403:0x1f3:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.055732 s, 18.8 MB/s
0x200000403:0x1f4:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0563876 s, 18.6 MB/s
0x200000403:0x1f5:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0519137 s, 20.2 MB/s
0x200000403:0x1f6:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0505364 s, 20.7 MB/s
0x200000403:0x1f7:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0622795 s, 16.8 MB/s
0x200000403:0x1f8:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0494623 s, 21.2 MB/s
0x200000403:0x1f9:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0583946 s, 18.0 MB/s
0x200000403:0x1fa:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0735376 s, 14.3 MB/s
0x200000403:0x1fb:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0544959 s, 19.2 MB/s
0x200000403:0x1fc:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0609908 s, 17.2 MB/s
0x200000403:0x1fd:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0678045 s, 15.5 MB/s
0x200000403:0x1fe:0x0
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre2"'
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
mdt.lustre-MDT0000.hsm_control=shutdown
mdt.lustre-MDT0001.hsm_control=shutdown
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Waiting 200s for 'SUCCEED'
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
ARCHIVE
RESTORE
PASS 260c (24s)

== sanity-hsm test 261: Report 0 bytes size after HSM release ========================================================== 13:18:06 (1750180686)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_261/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
2+0 records in
2+0 records out
8192 bytes (8.2 kB) copied, 0.00612661 s, 1.3 MB/s
/mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x00000009) exists archived, archive_id:2
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
/mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x0000000d) released exists archived, archive_id:2
PASS 261 (7s)

== sanity-hsm test 262: The client should return 1 block for HSM released files ========================================================== 13:18:13 (1750180693)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_262/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
2+0 records in
2+0 records out
8192 bytes (8.2 kB) copied, 0.0040407 s, 2.0 MB/s
Waiting 200s for 'SUCCEED'
PASS 262 (5s)

== sanity-hsm test 300: On disk coordinator state kept between MDT umount/mount ========================================================== 13:18:18 (1750180698)
Stop coordinator and remove coordinator state at mount
mdt.lustre-MDT0000.hsm_control=shutdown
mdt.lustre-MDT0001.hsm_control=shutdown
Failing mds1 on oleg409-server
Stopping /mnt/lustre-mds1 (opts:) on oleg409-server
13:18:24 (1750180704) shut down
facet: mds1 facet_host: oleg409-server facet_failover_host: oleg409-server
Failover mds1 to oleg409-server
mount facets: mds1
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg409-server: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Started lustre-MDT0000
13:18:39 (1750180719) targets are mounted
13:18:39 (1750180719) facet_failover done
oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
Set coordinator start at mount, and start coordinator
Failing mds1 on oleg409-server
Stopping /mnt/lustre-mds1 (opts:) on oleg409-server
13:19:09 (1750180749) shut down
facet: mds1 facet_host: oleg409-server facet_failover_host: oleg409-server
Failover mds1 to oleg409-server
mount facets: mds1
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg409-server: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Started lustre-MDT0000
13:19:24 (1750180764) targets are mounted
13:19:24 (1750180764) facet_failover done
oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
PASS 300 (75s)

== sanity-hsm test 301: HSM tunnable are persistent ====== 13:19:33 (1750180773)
Failing mds1 on oleg409-server
Stopping /mnt/lustre-mds1 (opts:) on oleg409-server
13:19:37 (1750180777) shut down
facet: mds1 facet_host: oleg409-server facet_failover_host: oleg409-server
Failover mds1 to oleg409-server
mount facets: mds1
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg409-server: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Started lustre-MDT0000
13:19:53 (1750180793) targets are mounted
13:19:53 (1750180793) facet_failover done
oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
PASS 301 (29s)

== sanity-hsm test 302: HSM tunnable are persistent when CDT is off ========================================================== 13:20:02 (1750180802)
mdt.lustre-MDT0000.hsm_control=shutdown
mdt.lustre-MDT0001.hsm_control=shutdown
Failing mds1 on oleg409-server
Stopping /mnt/lustre-mds1 (opts:) on oleg409-server
13:20:07 (1750180807) shut down
facet: mds1 facet_host: oleg409-server facet_failover_host: oleg409-server
Failover mds1 to oleg409-server
mount facets: mds1
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg409-server: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Started lustre-MDT0000
13:20:22 (1750180822) targets are mounted
13:20:22 (1750180822) facet_failover done
oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
Failing mds2 on oleg409-server
Stopping /mnt/lustre-mds2 (opts:) on oleg409-server
13:20:30 (1750180830) shut down
facet: mds2 facet_host: oleg409-server facet_failover_host: oleg409-server
Failover mds2 to oleg409-server
mount facets: mds2
Start mds2: mount -t lustre -o localrecov  /dev/mapper/mds2_flakey /mnt/lustre-mds2
oleg409-server: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Started lustre-MDT0001
13:20:45 (1750180845) targets are mounted
13:20:45 (1750180845) facet_failover done
oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid
mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec
PASS 302 (54s)

== sanity-hsm test 400: Single request is sent to the right MDT ========================================================== 13:20:56 (1750180856)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_400/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0668828 s, 15.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0702404 s, 14.9 MB/s
archive successful on mdt0
Waiting 200s for 'SUCCEED'
archive successful on mdt1
PASS 400 (6s)

== sanity-hsm test 401: Compound requests split and sent to their respective MDTs ========================================================== 13:21:03 (1750180863)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_401/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0665161 s, 15.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.093896 s, 11.2 MB/s
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
archive successful on mdt0
archive successful on mdt1
PASS 401 (7s)

== sanity-hsm test 402a: Copytool start fails if all MDTs are inactive ========================================================== 13:21:09 (1750180869)
deactivate lustre-MDT0000-mdc-ffff8800a83b2800 on oleg409-client.virtnet
deactivate lustre-MDT0001-mdc-ffff8800a83b2800 on oleg409-client.virtnet
deactivate lustre-MDT0000-mdc-ffff8800b6223800 on oleg409-client.virtnet
deactivate lustre-MDT0001-mdc-ffff8800b6223800 on oleg409-client.virtnet
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Failed to start copytool agt1 on 'oleg409-client.virtnet'
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
uuid not found in agent list on mds1
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
uuid not found in agent list on mds2
activate lustre-MDT0000-mdc-ffff8800a83b2800 on oleg409-client.virtnet
activate lustre-MDT0001-mdc-ffff8800a83b2800 on oleg409-client.virtnet
activate lustre-MDT0000-mdc-ffff8800b6223800 on oleg409-client.virtnet
activate lustre-MDT0001-mdc-ffff8800b6223800 on oleg409-client.virtnet
PASS 402a (6s)

== sanity-hsm test 402b: CDT must retry request upon slow start of CT ========================================================== 13:21:15 (1750180875)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
fail_loc=0x14d
fail_loc=0
Waiting 200s for 'SUCCEED'
Updated after 5s: want 'SUCCEED' got 'SUCCEED'
PASS 402b (20s)

== sanity-hsm test 403: Copytool starts with inactive MDT and register on reconnect ========================================================== 13:21:35 (1750180895)
deactivate lustre-MDT0001-mdc-ffff8800a83b2800 on oleg409-client.virtnet
deactivate lustre-MDT0001-mdc-ffff8800b6223800 on oleg409-client.virtnet
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_403/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
oleg409-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool
oleg409-client.virtnet: executing get_client_uuid /mnt/lustre2
found agent uuid=94a00271-e790-468b-8c8b-c344f94cbb5f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
uuid not found in agent list on mds2
oleg409-client.virtnet: 12658
activate lustre-MDT0001-mdc-ffff8800a83b2800 on oleg409-client.virtnet
activate lustre-MDT0001-mdc-ffff8800b6223800 on oleg409-client.virtnet
found agent uuid=94a00271-e790-468b-8c8b-c344f94cbb5f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1
found agent uuid=94a00271-e790-468b-8c8b-c344f94cbb5f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2
PASS 403 (8s)

== sanity-hsm test 404: Inactive MDT does not block requests for active MDTs ========================================================== 13:21:43 (1750180903)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_404/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0558812 s, 18.8 MB/s
deactivate lustre-MDT0001-mdc-ffff8800a83b2800 on oleg409-client.virtnet
deactivate lustre-MDT0001-mdc-ffff8800b6223800 on oleg409-client.virtnet
Waiting 200s for 'SUCCEED'
archive successful on mdt0
activate lustre-MDT0001-mdc-ffff8800a83b2800 on oleg409-client.virtnet
activate lustre-MDT0001-mdc-ffff8800b6223800 on oleg409-client.virtnet
PASS 404 (5s)

== sanity-hsm test 405: archive and release under striped directory ========================================================== 13:21:48 (1750180908)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_405/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0347142 s, 30.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0343159 s, 30.6 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0347768 s, 30.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0366701 s, 28.6 MB/s
archive successful on 0x240000bd2:0x1:0x0
archive successful on 0x240000bd2:0x2:0x0
archive successful on 0x240000bd2:0x3:0x0
archive successful on 0x2000032e3:0x6:0x0
PASS 405 (10s)

== sanity-hsm test 406: attempting to migrate HSM archived files is safe ========================================================== 13:21:58 (1750180918)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.05844 s, 17.9 MB/s
old fid 0x2000032e3:0x9:0x0
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_406/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
lfs migrate: /mnt/lustre/d406.sanity-hsm/f406.sanity-hsm migrate failed: Operation not supported (95)
lfs migrate: cb_migrate_mdt_fini: error completing migration of /mnt/lustre/d406.sanity-hsm: Directory not empty (39)
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
new fid 0x240000bd3:0x7:0x0
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
debug=+trace
Waiting 200s for 'SUCCEED'
PASS 406 (11s)

== sanity-hsm test 407: Check for double RESTORE records in llog ========================================================== 13:22:10 (1750180930)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00731274 s, 0.0 kB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_407/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
fail_val=5
fail_loc=0x164
Copytool is suspended on oleg409-client.virtnet
mdt.lustre-MDT0000.hsm.actions=
lrh=[type=10680000 len=136 idx=6/3] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x6851a3d7 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x6851a3d8 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x6851a3d9 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6851a3da action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6851a3db action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6851a3dc action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6851a3dd action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6851a3de action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
Failing mds1 on oleg409-server
Stopping /mnt/lustre-mds1 (opts:) on oleg409-server
13:22:19 (1750180939) shut down
facet: mds1 facet_host: oleg409-server facet_failover_host: oleg409-server
Failover mds1 to oleg409-server
mount facets: mds1
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg409-server: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Started lustre-MDT0000
13:22:35 (1750180955) targets are mounted
13:22:35 (1750180955) facet_failover done
oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
fail_loc=0
mdt.lustre-MDT0000.hsm.actions=
lrh=[type=10680000 len=136 idx=6/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x6851a3d8 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x6851a3d9 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6851a3da action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6851a3db action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6851a3dc action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6851a3dd action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6851a3de action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/11] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6851a3df action=RESTORE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[]
Copytool is continued on oleg409-client.virtnet
d41d8cd98f00b204e9800998ecf8427e  /mnt/lustre/d407.sanity-hsm/f407.sanity-hsm
d41d8cd98f00b204e9800998ecf8427e  /mnt/lustre2/d407.sanity-hsm/f407.sanity-hsm
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
mdt.lustre-MDT0000.hsm.actions=
lrh=[type=10680000 len=136 idx=6/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x6851a3d8 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x6851a3d9 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6851a3da action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6851a3db action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6851a3dc action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6851a3dd action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6851a3de action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
lrh=[type=10680000 len=136 idx=6/11] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xd:0x0] compound/cookie=0x0/0x6851a3df action=RESTORE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[]
PASS 407 (37s)

== sanity-hsm test 408: Verify fiemap on release file ==== 13:22:46 (1750180966)
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_408/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
1+0 records in
1+0 records out
65536 bytes (66 kB) copied, 0.0387978 s, 1.7 MB/s
1+0 records in
1+0 records out
65536 bytes (66 kB) copied, 0.0392526 s, 1.7 MB/s
  File: '/mnt/lustre/f408.sanity-hsm'
  Size: 262144    	Blocks: 256        IO Block: 4194304 regular file
Device: 2c54f966h/743766374d	Inode: 144115406615871500  Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-06-17 13:22:47.000000000 -0400
Modify: 2025-06-17 13:22:47.000000000 -0400
Change: 2025-06-17 13:22:47.000000000 -0400
 Birth: -
disk usage: 131072	/mnt/lustre/f408.sanity-hsm
file size: 262144	/mnt/lustre/f408.sanity-hsm
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
extent 0 in offset 0, length 262144
flags (0x3): LAST UNKNOWN
No unwritten extents, extents number 1, sum of lengths 262144, expected sum 262144
PASS 408 (6s)

== sanity-hsm test 409a: Coordinator should not stop when in use ========================================================== 13:22:52 (1750180972)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00456062 s, 0.0 kB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
fail_val=5
fail_loc=0x164
cat: /mnt/lustre/d409a.sanity-hsm/f409a.sanity-hsm: No data available
mdt.lustre-MDT0000.hsm_control=shutdown
mdt.lustre-MDT0001.hsm_control=shutdown
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
Killing existing copytools on oleg409-client.virtnet
Copytool is continued on oleg409-client.virtnet
copytools stopped in 0s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
mdt.lustre-MDT0000.hsm_control=enabled
mdt.lustre-MDT0001.hsm_control=enabled
PASS 409a (19s)

== sanity-hsm test 409b: getattr released file with CDT stopped after remount ========================================================== 13:23:12 (1750180992)
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00529744 s, 0.0 kB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
Failing mds1 on oleg409-server
Stopping /mnt/lustre-mds1 (opts:) on oleg409-server
13:23:39 (1750181019) shut down
facet: mds1 facet_host: oleg409-server facet_failover_host: oleg409-server
Failover mds1 to oleg409-server
mount facets: mds1
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg409-server: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Started lustre-MDT0000
13:23:56 (1750181036) targets are mounted
13:23:56 (1750181036) facet_failover done
oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
  File: '/mnt/lustre/d409b.sanity-hsm/f409b.sanity-hsm'
  Size: 0         	Blocks: 0          IO Block: 4194304 regular empty file
Device: 2c54f966h/743766374d	Inode: 144115406632648721  Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-06-17 13:23:13.000000000 -0400
Modify: 2025-06-17 13:23:13.000000000 -0400
Change: 2025-06-17 13:23:13.000000000 -0400
 Birth: -
PASS 409b (76s)

== sanity-hsm test 410: lfs data_version -s allows release of force-archived file ========================================================== 13:24:27 (1750181067)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0969922 s, 10.8 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_410/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Cannot send HSM request (use of /mnt/lustre/d410.sanity-hsm/f410.sanity-hsm): Operation not permitted
PASS 410 (5s)

== sanity-hsm test 411: hsm_ops rbac role ================ 13:24:32 (1750181072)
On MGS 192.168.204.109, active = nodemap.active=1
waiting 10 secs for sync
On MGS 192.168.204.109, test_411.trusted_nodemap = nodemap.test_411.trusted_nodemap=1
waiting 10 secs for sync
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
/mnt/lustre/d411.sanity-hsm/fileA: (0x00000000)
Waiting 200s for 'SUCCEED'
Waiting 200s for 'SUCCEED'
Killing existing copytools on oleg409-client.virtnet
copytools stopped in 0s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
On MGS 192.168.204.109, test_411.rbac = nodemap.test_411.rbac=file_perms,dne_ops,quota_ops,byfid_ops,chlg_ops,fscrypt_admin,server_upcall,ignore_root_prjquota,local_admin
waiting 10 secs for sync
/mnt/lustre/d411.sanity-hsm/fileB: (0x00000000)
Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied
Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Operation not permitted
Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied
Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied
lfs hsm_set: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied
lfs hsm_clear: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied
On MGS 192.168.204.109, active = nodemap.active=0
waiting 10 secs for sync
oleg409-server: error: nodemap 'test_411' does not exist
oleg409-server: nodemap_del: Invalid argument
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
On MGS 192.168.204.109, active = nodemap.active=0
waiting 10 secs for sync
PASS 411 (73s)

== sanity-hsm test 500: various LLAPI HSM tests ========== 13:25:45 (1750181145)
striped dir -i0 -c2 -H all_char /mnt/lustre/d500.sanity-hsm
 test1 @1750181185: Register/unregister copytool 2000x to check for leaks ...... pass
 test2 @1750181185: Re/un-register copytool multiple times without error ....... pass
 test3 @1750181185: Pass bad parameters to llapi_hsm_copytool_register() ....... pass
 test4 @1750181185: Bad parameters to llapi_hsm_copytool_unregister() .......... pass
 test5 @1750181185: Test llapi_hsm_copytool_recv() in non-blocking mode ........ pass
 test6 @1750181185: Test llapi_hsm_copytool_recv() with bogus parameters ....... pass
 test7 @1750181185: Test event polling (without actual traffic) ................ pass
 test50 @1750181185: Test llapi_hsm_state_get()/get_fd() ....................... pass
 test51 @1750181185: Test llapi_hsm_state_set_fd() ............................. pass
 test52 @1750181185: Test llapi_hsm_current_action() ........................... pass
 test100 @1750181185: Simple archive creation with no progress reported. ....... pass
 test101 @1750181189: Simple archive creation with progress every byte. ........ pass
 test102 @1750181192: Archive creation with progress every byte, backwards ..... pass
 test103 @1750181193: Archive creation with a single progress report ........... pass
 test104 @1750181194: Archive creation with two progress reports ............... pass
 test105 @1750181195: Archive creation with one bogus progress report .......... pass
 test106 @1750181196: Archive creation with one empty progress report .......... pass
 test107 @1750181197: Archive creation with one bogus progress report .......... pass
 test108 @1750181201: Archive creation with same progress report each time ..... pass
 test109 @1750181202: Archive creation with one report, with large number ...... pass
 test110 @1750181203: Archive with 10 progress reports, checking progress ...... pass
 test111 @1750181204: Archive with 10 reports in reverse, checking progress .... pass
 test112 @1750181205: Archive with 10 reports, duplicated, check progress ...... pass
 test113 @1750181206: Archive with 9 reports, with 20% overlapping coverage .... pass
PASS 500 (64s)

== sanity-hsm test 600: Changelog fields 'u=' and 'nid=' ========================================================== 13:26:49 (1750181209)
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl1 cl1'
mdd.lustre-MDT0000.changelog_mask=ALL
mdd.lustre-MDT0001.changelog_mask=ALL
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [touch] [/mnt/lustre/d600.sanity-hsm/f600.sanity-hsm]
Got NID '192.168.204.9@tcp'
lustre-MDT0001: clear the changelog for cl1 of all records
lustre-MDT0001: Deregistered changelog user #1
lustre-MDT0000: clear the changelog for cl1 of all records
lustre-MDT0000: Deregistered changelog user #1
PASS 600 (12s)

== sanity-hsm test 601: OPEN Changelog entry ============= 13:27:01 (1750181221)
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl2 cl2'
mdd.lustre-MDT0000.changelog_mask=ALL
mdd.lustre-MDT0001.changelog_mask=ALL
lustre-MDT0000: clear the changelog for cl2 of all records
lustre-MDT0001: clear the changelog for cl2 of all records
lustre-MDT0001.4 10OPEN  17:27:06.796346764 2025.06.17 0x1 t=[0x240000bd3:0xa:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.204.9@tcp m=r-- p=[0x240000bd3:0x9:0x0]
lustre-MDT0001: clear the changelog for cl2 of all records
lustre-MDT0001: Deregistered changelog user #2
lustre-MDT0000: clear the changelog for cl2 of all records
lustre-MDT0000: Deregistered changelog user #2
PASS 601 (13s)

== sanity-hsm test 602: Changelog record CLOSE only if open+write or OPEN recorded ========================================================== 13:27:14 (1750181234)
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl3 cl3'
mdd.lustre-MDT0000.changelog_mask=ALL
mdd.lustre-MDT0001.changelog_mask=ALL
lustre-MDT0000: clear the changelog for cl3 of all records
lustre-MDT0001: clear the changelog for cl3 of all records
lustre-MDT0000.9 11CLOSE 17:27:18.913972838 2025.06.17 0x1 t=[0x2000032e2:0x2e:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0000: clear the changelog for cl3 of all records
lustre-MDT0001: clear the changelog for cl3 of all records
lustre-MDT0000.10 10OPEN  17:27:19.178474994 2025.06.17 0x242 t=[0x2000032e2:0x2e:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.204.9@tcp m=-w- p=[0x240000bd3:0x9:0x0]
lustre-MDT0000.11 23GXATR 17:27:19.182457190 2025.06.17 0x0 t=[0x2000032e2:0x2e:0x0] j=bash.0 ef=0x1b u=0:0 nid=192.168.204.9@tcp x=user.job
lustre-MDT0000.12 13TRUNC 17:27:19.185677936 2025.06.17 0xe t=[0x2000032e2:0x2e:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x2000032e2:0x2d:0x0]
lustre-MDT0000.13 11CLOSE 17:27:19.208129343 2025.06.17 0x242 t=[0x2000032e2:0x2e:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0000.13 11CLOSE 17:27:19.208129343 2025.06.17 0x242 t=[0x2000032e2:0x2e:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
mdd.lustre-MDT0000.changelog_mask=-OPEN
mdd.lustre-MDT0001.changelog_mask=-OPEN
lustre-MDT0000: clear the changelog for cl3 of all records
lustre-MDT0001: clear the changelog for cl3 of all records
f
lustre-MDT0000: clear the changelog for cl3 of all records
lustre-MDT0001: clear the changelog for cl3 of all records
lustre-MDT0000.14 13TRUNC 17:27:20.628257631 2025.06.17 0xe t=[0x2000032e2:0x2e:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp p=[0x2000032e2:0x2d:0x0]
lustre-MDT0000.15 11CLOSE 17:27:20.642717474 2025.06.17 0x242 t=[0x2000032e2:0x2e:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0000.15 11CLOSE 17:27:20.642717474 2025.06.17 0x242 t=[0x2000032e2:0x2e:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0001: clear the changelog for cl3 of all records
lustre-MDT0001: Deregistered changelog user #3
lustre-MDT0000: clear the changelog for cl3 of all records
lustre-MDT0000: Deregistered changelog user #3
PASS 602 (14s)

== sanity-hsm test 603: GETXATTR Changelog entry ========= 13:27:28 (1750181248)
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl4 cl4'
mdd.lustre-MDT0000.changelog_mask=ALL
mdd.lustre-MDT0001.changelog_mask=ALL
lustre-MDT0000: clear the changelog for cl4 of all records
lustre-MDT0001: clear the changelog for cl4 of all records
getfattr: Removing leading '/' from absolute path names
# file: mnt/lustre/d603.sanity-hsm/f603.sanity-hsm
user.xattr1="value1"

lustre-MDT0001.11 23GXATR 17:27:32.918055691 2025.06.17 0x0 t=[0x240000bd3:0xc:0x0] j=getfattr.0 ef=0x1b u=0:0 nid=192.168.204.9@tcp x=user.xattr1
lustre-MDT0001: clear the changelog for cl4 of all records
lustre-MDT0001: Deregistered changelog user #4
lustre-MDT0000: clear the changelog for cl4 of all records
lustre-MDT0000: Deregistered changelog user #4
PASS 603 (12s)

== sanity-hsm test 604: NOPEN Changelog entry ============ 13:27:40 (1750181260)
mdd.lustre-MDT0000.changelog_deniednext=20
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl5 cl5'
mdd.lustre-MDT0000.changelog_mask=ALL
mdd.lustre-MDT0001.changelog_mask=ALL
lustre-MDT0000: clear the changelog for cl5 of all records
lustre-MDT0001: clear the changelog for cl5 of all records
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm]
cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied
lustre-MDT0000.20 24NOPEN 17:27:45.764161179 2025.06.17 0x1 t=[0x2000032e2:0x30:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.204.9@tcp m=r-- p=[0x200000bd2:0x87:0x0]
Got NID '192.168.204.9@tcp'
lustre-MDT0000: clear the changelog for cl5 of all records
lustre-MDT0001: clear the changelog for cl5 of all records
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm]
cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied
lustre-MDT0000: clear the changelog for cl5 of all records
lustre-MDT0001: clear the changelog for cl5 of all records
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm]
cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied
lustre-MDT0000.21 24NOPEN 17:28:06.512157305 2025.06.17 0x1 t=[0x2000032e2:0x30:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.204.9@tcp m=r-- p=[0x2000032e2:0x2d:0x0]
Got NID '192.168.204.9@tcp'
lustre-MDT0001: clear the changelog for cl5 of all records
lustre-MDT0001: Deregistered changelog user #5
lustre-MDT0000: clear the changelog for cl5 of all records
lustre-MDT0000: Deregistered changelog user #5
mdd.lustre-MDT0000.changelog_deniednext=60
PASS 604 (32s)

== sanity-hsm test 605: Test OPEN and CLOSE rate limit in Changelogs ========================================================== 13:28:13 (1750181293)
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl6 cl6'
mdd.lustre-MDT0000.changelog_mask=ALL
mdd.lustre-MDT0001.changelog_mask=ALL
lustre-MDT0000: clear the changelog for cl6 of all records
lustre-MDT0001: clear the changelog for cl6 of all records
lustre-MDT0000.25 10OPEN  17:28:16.718237776 2025.06.17 0x43 t=[0x2000032e2:0x32:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.204.9@tcp m=rw- p=[0x2000032e2:0x31:0x0]
lustre-MDT0000.25 10OPEN  17:28:16.718237776 2025.06.17 0x43 t=[0x2000032e2:0x32:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.204.9@tcp m=rw- p=[0x2000032e2:0x31:0x0]
lustre-MDT0000: clear the changelog for cl6 of all records
lustre-MDT0001: clear the changelog for cl6 of all records
lustre-MDT0000: clear the changelog for cl6 of all records
lustre-MDT0001: clear the changelog for cl6 of all records
lustre-MDT0000.26 10OPEN  17:28:17.627720808 2025.06.17 0x1 t=[0x2000032e2:0x32:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.204.9@tcp m=r-- p=[0x2000032e2:0x31:0x0]
lustre-MDT0000.27 11CLOSE 17:28:17.659851160 2025.06.17 0x1 t=[0x2000032e2:0x32:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0000.26 10OPEN  17:28:17.627720808 2025.06.17 0x1 t=[0x2000032e2:0x32:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.204.9@tcp m=r-- p=[0x2000032e2:0x31:0x0]
lustre-MDT0000.27 11CLOSE 17:28:17.659851160 2025.06.17 0x1 t=[0x2000032e2:0x32:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0000: clear the changelog for cl6 of all records
lustre-MDT0001: clear the changelog for cl6 of all records
lustre-MDT0000: clear the changelog for cl6 of all records
lustre-MDT0001: clear the changelog for cl6 of all records
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [cat] [/mnt/lustre/d605.sanity-hsm/f605.sanity-hsm]
lustre-MDT0000.28 10OPEN  17:28:18.632570883 2025.06.17 0x1 t=[0x2000032e2:0x32:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.204.9@tcp m=r-- p=[0x2000032e2:0x2d:0x0]
lustre-MDT0000.29 11CLOSE 17:28:18.645199191 2025.06.17 0x1 t=[0x2000032e2:0x32:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.204.9@tcp
lustre-MDT0000.28 10OPEN  17:28:18.632570883 2025.06.17 0x1 t=[0x2000032e2:0x32:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.204.9@tcp m=r-- p=[0x2000032e2:0x2d:0x0]
lustre-MDT0000.29 11CLOSE 17:28:18.645199191 2025.06.17 0x1 t=[0x2000032e2:0x32:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.204.9@tcp
lustre-MDT0000: clear the changelog for cl6 of all records
lustre-MDT0001: clear the changelog for cl6 of all records
lustre-MDT0000.30 11CLOSE 17:28:19.105121565 2025.06.17 0x43 t=[0x2000032e2:0x32:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0000.30 11CLOSE 17:28:19.105121565 2025.06.17 0x43 t=[0x2000032e2:0x32:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.9@tcp
lustre-MDT0001: clear the changelog for cl6 of all records
lustre-MDT0001: Deregistered changelog user #6
lustre-MDT0000: clear the changelog for cl6 of all records
lustre-MDT0000: Deregistered changelog user #6
PASS 605 (13s)

== sanity-hsm test 606: llog_reader groks changelog fields ========================================================== 13:28:26 (1750181306)
mdd.lustre-MDT0000.changelog_mask=+hsm
mdd.lustre-MDT0001.changelog_mask=+hsm
Registered 2 changelog users: 'cl7 cl7'
mdd.lustre-MDT0000.changelog_mask=ALL
mdd.lustre-MDT0001.changelog_mask=ALL
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [touch] [/mnt/lustre/d606.sanity-hsm/f606.sanity-hsm]
Stopping /mnt/lustre-mds1 (opts:) on oleg409-server
rec #2 type=1064553b len=64 offset 8256
Header size : 8192	 llh_size : 64
Time : Tue Jun 17 13:10:11 2025
Number of records: 1	cat_idx: 1	last_idx: 2
Flags: 0x2012
Target uuid : 
-----------------------
#02 (064)fid=[0x1:0x3b:0x0] path=O/1/d27/59
Got UID/GID 500:500
Got NID 192.168.204.9@tcp
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg409-server: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg409-client: oleg409-server: ssh exited with exit code 1
Started lustre-MDT0000
lustre-MDT0001: clear the changelog for cl7 of all records
lustre-MDT0001: Deregistered changelog user #7
lustre-MDT0000: clear the changelog for cl7 of all records
lustre-MDT0000: Deregistered changelog user #7
PASS 606 (26s)

== sanity-hsm test 607a: release a file that was migrated after being archived ========================================================== 13:28:52 (1750181332)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.107521 s, 9.8 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Waiting 200s for 'SUCCEED'
PASS 607a (9s)

== sanity-hsm test 607b: Migrate should not change the HSM attribute of dirty files ========================================================== 13:29:01 (1750181341)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.139285 s, 7.5 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
llite.lustre-ffff8800a83b2800.xattr_cache=0
llite.lustre-ffff8800b6223800.xattr_cache=0
/mnt/lustre/d607b.sanity-hsm/f607b.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1
llite.lustre-ffff8800a83b2800.xattr_cache=1
llite.lustre-ffff8800b6223800.xattr_cache=1
PASS 607b (7s)

== sanity-hsm test 607c: 'lfs swap_layouts' should set dirty flag on HSM file ========================================================== 13:29:08 (1750181348)
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.167433 s, 6.3 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0968373 s, 10.8 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0937068 s, 11.2 MB/s
Starting copytool 'agt1' on 'oleg409-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre2"'
Waiting 200s for 'SUCCEED'
lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-1' and '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-3': Operation not permitted (1)
PASS 607c (8s)
== sanity-hsm test complete, duration 2652 sec =========== 13:29:16 (1750181356)
=== sanity-hsm: start cleanup 13:29:17 (1750181357) ===
Stopping clients: oleg409-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg409-client.virtnet /mnt/lustre2 opts:
=== sanity-hsm: finish cleanup 13:29:36 (1750181376) ===