-----============= acceptance-small: sanity-hsm ============----- Sat Aug 15 20:02:53 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg316-server: /home/green/git/lustre-release/lustre/tests/except/sanity-hsm.ex - see MDS1_VERSION <= CLIENT_VERSION (34683137 <= 34683137) for LU-18924, go 100 105 excepting tests: === sanity-hsm: start setup 20:03:01 (1786838581) === Starting client oleg316-client.virtnet: -o user_xattr,flock 192.168.203.116@tcp:/lustre /mnt/lustre2 Mount client oleg316-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.203.116@tcp:/lustre /mnt/lustre2 Started clients oleg316-client.virtnet: 192.168.203.116@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) oleg316-client.virtnet: executing check_config_client /mnt/lustre oleg316-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg316-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9c7d43757000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9c7d4a8c7800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9c7d43757000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9c7d4a8c7800.idle_timeout=debug disable quota as required oleg316-server: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === sanity-hsm: finish setup 20:03:17 (1786838597) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8239] nobody:x:65534: Killing existing copytools on oleg316-client.virtnet with signal TERM pkill: pidfile not valid Try `pkill --help' for more information. Set HSM on and start Set sanity-hsm HSM policy == sanity-hsm test 1A: lfs hsm flags root/non-root access ========================================================== 20:03:41 (1786838621) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_clear] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--exists] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] lfs hsm_set: change hsm flags for '/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm' failed: Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] PASS 1A (3s) == sanity-hsm test 1a: mmap & cat a HSM released file ==== 20:03:44 (1786838624) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0724799 s, 14.5 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying released state: PASS 1a (6s) == sanity-hsm test 1b: Archive, Release and Restore composite file ========================================================== 20:03:50 (1786838630) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283888 s, 36.9 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' release /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm verify released state: pass restore /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm Waiting 200s for 'SUCCEED' verify restored state: pass PASS 1b (7s) == sanity-hsm test 1c: Check setting archive-id in lfs hsm_set ========================================================== 20:03:57 (1786838637) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is still 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 33 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 16 PASS 1c (3s) == sanity-hsm test 1d: Archive, Release and Restore DoM file ========================================================== 20:04:00 (1786838640) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287132 s, 36.5 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm Waiting 200s for 'SUCCEED' release /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm verify released state: pass restore /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm Waiting 200s for 'SUCCEED' verify restored state: pass PASS 1d (7s) == sanity-hsm test 1e: Archive, Release and Restore SEL file ========================================================== 20:04:07 (1786838647) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313147 s, 33.5 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' release /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm verify released state: pass restore /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm verify restored state: pass PASS 1e (6s) == sanity-hsm test 1f: DoM file release after restore ==== 20:04:13 (1786838653) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0387083 s, 27.1 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' release /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm verify released state: pass restore /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm verify restored state: pass release again /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 verify released state: pass PASS 1f (5s) == sanity-hsm test 2: Check file dirtyness when doing setattr ========================================================== 20:04:18 (1786838658) 0+0 records in 0+0 records out 0 bytes copied, 0.00188365 s, 0.0 kB/s 0x200000403:0x8:0x0 PASS 2 (3s) == sanity-hsm test 3: Check file dirtyness when opening for write ========================================================== 20:04:21 (1786838661) Succeed in opening file "/mnt/lustre/d3.sanity-hsm/f3.sanity-hsm"(flags=O_WRONLY) 3+0 records in 3+0 records out 3+0 records in 3+0 records out PASS 3 (3s) == sanity-hsm test 4: Useless cancel must not be registered ========================================================== 20:04:24 (1786838664) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0531779 s, 19.7 MB/s PASS 4 (3s) == sanity-hsm test 8: Test default archive number ======== 20:04:27 (1786838667) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_8/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' PASS 8 (5s) == sanity-hsm test 9A: Use of explicit archive number, with dedicated copytool ========================================================== 20:04:32 (1786838672) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_9A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 3 "/mnt/lustre2"' oleg316-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg316-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=3 requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=3 requests=[current:0 ok:0 errors:0] on mds2 Waiting 200s for 'SUCCEED' PASS 9A (14s) == sanity-hsm test 9a: Multiple remote agents ============ 20:04:46 (1786838686) SKIP: sanity-hsm test_9a Need 3 or more clients, have 1 SKIP 9a (1s) == sanity-hsm test 10a: Archive a file =================== 20:04:47 (1786838687) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 996: warning: command substitution: ignored null byte in input Verifying content Verifying hsm state Verifying archive number is 2 PASS 10a (5s) == sanity-hsm test 10b: Archive of non dirty file must work without doing request ========================================================== 20:04:52 (1786838692) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 10b (6s) == sanity-hsm test 10c: Check forbidden archive ========== 20:04:58 (1786838698) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d10c.sanity-hsm/f10c.sanity-hsm): Operation not permitted PASS 10c (3s) == sanity-hsm test 10d: Archive a file on the default archive id ========================================================== 20:05:01 (1786838701) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' PASS 10d (6s) == sanity-hsm test 11a: Import a file ==================== 20:05:07 (1786838707) lhsmtool_posix: 1786838708.257264 [22186]: action=1 src=d11a.sanity-hsm/f11a.sanity-hsm dst=/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786838708.262332 [22186]: importing '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1786838708.288368 [22186]: imported '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//0424/0x200000402:0x26:0x0'=='/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1786838708.288420 [22186]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying imported size 158=158 Verifying released pattern: raid0,released Verifying new fid 0x200000402:0x26:0x0 in archive PASS 11a (4s) == sanity-hsm test 11b: Import a deleted file using its FID ========================================================== 20:05:11 (1786838711) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_11b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' lhsmtool_posix: 1786838714.292079 [22885]: action=1 src=0x200000403:0x11:0x0 dst=/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786838714.300131 [22885]: Resolving [0x200000403:0x11:0x0] to /tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0 lhsmtool_posix: 1786838714.300210 [22885]: importing '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0' lhsmtool_posix: 1786838714.321217 [22885]: imported '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//042a/0x200000402:0x28:0x0'=='/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0' lhsmtool_posix: 1786838714.321259 [22885]: process finished, errs: 0 major, 0 minor, rc=0 (Success) /mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm: OK PASS 11b (5s) == sanity-hsm test 11c: Import a file to a directory with a pool ========================================================== 20:05:17 (1786838717) Creating new pool test_11c oleg316-server: Pool lustre.test_11c created Adding targets to pool oleg316-server: OST lustre-OST0001_UUID added to pool lustre.test_11c lhsmtool_posix: 1786838723.066556 [23599]: action=1 src=d11c.sanity-hsm/f11c.sanity-hsm dst=/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786838723.072421 [23599]: importing '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1786838723.105968 [23599]: imported '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//0428/0x200000402:0x2a:0x0'=='/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1786838723.106026 [23599]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Destroy the created pools: test_11c lustre.test_11c oleg316-server: OST lustre-OST0001_UUID removed from pool lustre.test_11c oleg316-server: Pool lustre.test_11c destroyed PASS 11c (14s) == sanity-hsm test 12a: Restore an imported file explicitly ========================================================== 20:05:31 (1786838731) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786838732.509904 [24336]: action=1 src=d12a.sanity-hsm/f12a.sanity-hsm dst=/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786838732.515540 [24336]: importing '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1786838732.540536 [24336]: imported '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//042f/0x200000402:0x2d:0x0'=='/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1786838732.540601 [24336]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying file state: PASS 12a (5s) == sanity-hsm test 12b: Restore an imported file implicitly ========================================================== 20:05:36 (1786838736) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786838737.916810 [25049]: action=1 src=d12b.sanity-hsm/f12b.sanity-hsm dst=/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786838737.921714 [25049]: importing '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1786838737.948561 [25049]: imported '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//042d/0x200000402:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1786838737.948674 [25049]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying file state after restore: PASS 12b (5s) == sanity-hsm test 12c: Restore a file with stripe of 2 == 20:05:41 (1786838741) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0654943 s, 80.1 MB/s Waiting 200s for 'SUCCEED' /mnt/lustre/d12c.sanity-hsm/f12c.sanity-hsm: OK PASS 12c (6s) == sanity-hsm test 12d: Restore of a non archived, non released file must work ========================================================== 20:05:47 (1786838747) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 12d (6s) == sanity-hsm test 12e: Check forbidden restore ========== 20:05:53 (1786838753) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 Cannot send HSM request (use of /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm): Operation not permitted PASS 12e (3s) == sanity-hsm test 12f: Restore a released file explicitly ========================================================== 20:05:56 (1786838756) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying file state: PASS 12f (8s) == sanity-hsm test 12g: Restore a released file implicitly ========================================================== 20:06:04 (1786838764) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12g (5s) == sanity-hsm test 12h: Restore a released file implicitly from a second node ========================================================== 20:06:09 (1786838769) SKIP: sanity-hsm test_12h Need 2 or more clients, have 1 SKIP 12h (2s) == sanity-hsm test 12m: Archive/release/implicit restore ========================================================== 20:06:11 (1786838771) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12m/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 12m (6s) == sanity-hsm test 12n: Import/implicit restore/release == 20:06:17 (1786838777) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12n/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786838778.900527 [29913]: action=1 src=d12n.sanity-hsm/f12n.sanity-hsm dst=/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786838778.905588 [29913]: importing '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1786838778.933878 [29913]: imported '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//043f/0x200000402:0x3d:0x0'=='/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1786838778.933944 [29913]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12n (5s) == sanity-hsm test 12o: Layout-swap failure during Restore leaves file released ========================================================== 20:06:22 (1786838782) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12o/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' fail_loc=0x152 mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA diff: /mnt/lustre/d12o.sanity-hsm/f12o.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA fail_loc=0 mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 12o (22s) == sanity-hsm test 12p: implicit restore of a file on copytool mount point ========================================================== 20:06:44 (1786838804) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12p/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12p (5s) == sanity-hsm test 12q: file attributes are refreshed after restore ========================================================== 20:06:49 (1786838809) Starting client: oleg316-client.virtnet: -o user_xattr,flock 192.168.203.116@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404438 s, 25.9 MB/s Waiting 200s for 'SUCCEED' Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Stopping client oleg316-client.virtnet /mnt/lustre3 (opts:) PASS 12q (12s) == sanity-hsm test 12r: lseek restores released file ===== 20:07:01 (1786838821) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12r/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' PASS 12r (5s) == sanity-hsm test 12s: race between restore requests ==== 20:07:06 (1786838826) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12s/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' fail_loc=0x8000018b PASS 12s (12s) == sanity-hsm test 12t: Multiple parallel reads for a HSM imported file ========================================================== 20:07:18 (1786838838) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12t/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' lhsmtool_posix: 1786838841.091309 [34082]: action=1 src=0x200000403:0x2a:0x0 dst=/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786838841.094783 [34082]: Resolving [0x200000403:0x2a:0x0] to /tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0 lhsmtool_posix: 1786838841.094837 [34082]: importing '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0' lhsmtool_posix: 1786838841.108469 [34082]: imported '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//0449/0x200000402:0x4b:0x0'=='/tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0' lhsmtool_posix: 1786838841.108518 [34082]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12t (5s) == sanity-hsm test 12u: Multiple reads on multiple HSM imported files in parallel ========================================================== 20:07:23 (1786838843) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12u/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786838848.819520 [34946]: action=1 src=0x200000403:0x2c:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0 mount_point=/mnt/lustre2 lhsmtool_posix: 1786838848.823198 [34946]: Resolving [0x200000403:0x2c:0x0] to /tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0 lhsmtool_posix: 1786838848.823255 [34946]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0' lhsmtool_posix: 1786838848.849380 [34946]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//044f/0x200000402:0x4d:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0' lhsmtool_posix: 1786838848.849445 [34946]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786838848.978943 [34975]: action=1 src=0x200000403:0x2d:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1 mount_point=/mnt/lustre2 lhsmtool_posix: 1786838848.984588 [34975]: Resolving [0x200000403:0x2d:0x0] to /tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0 lhsmtool_posix: 1786838848.984706 [34975]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0' lhsmtool_posix: 1786838849.003966 [34975]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//044c/0x200000402:0x4e:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0' lhsmtool_posix: 1786838849.004033 [34975]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786838849.131574 [35004]: action=1 src=0x200000403:0x2e:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2 mount_point=/mnt/lustre2 lhsmtool_posix: 1786838849.135182 [35004]: Resolving [0x200000403:0x2e:0x0] to /tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0 lhsmtool_posix: 1786838849.135236 [35004]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0' lhsmtool_posix: 1786838849.154870 [35004]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//044d/0x200000402:0x4f:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0' lhsmtool_posix: 1786838849.154915 [35004]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786838849.286924 [35033]: action=1 src=0x200000403:0x2f:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3 mount_point=/mnt/lustre2 lhsmtool_posix: 1786838849.290055 [35033]: Resolving [0x200000403:0x2f:0x0] to /tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0 lhsmtool_posix: 1786838849.290118 [35033]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0' lhsmtool_posix: 1786838849.317311 [35033]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//0452/0x200000402:0x50:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0' lhsmtool_posix: 1786838849.317387 [35033]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786838849.450944 [35062]: action=1 src=0x200000403:0x30:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4 mount_point=/mnt/lustre2 lhsmtool_posix: 1786838849.456023 [35062]: Resolving [0x200000403:0x30:0x0] to /tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0 lhsmtool_posix: 1786838849.456097 [35062]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0' lhsmtool_posix: 1786838849.482348 [35062]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0453/0x200000402:0x51:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0' lhsmtool_posix: 1786838849.482403 [35062]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786838849.610564 [35091]: action=1 src=0x200000403:0x31:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5 mount_point=/mnt/lustre2 lhsmtool_posix: 1786838849.614358 [35091]: Resolving [0x200000403:0x31:0x0] to /tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0 lhsmtool_posix: 1786838849.614416 [35091]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0' lhsmtool_posix: 1786838849.638052 [35091]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0450/0x200000402:0x52:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0' lhsmtool_posix: 1786838849.638115 [35091]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786838849.784241 [35120]: action=1 src=0x200000403:0x32:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6 mount_point=/mnt/lustre2 lhsmtool_posix: 1786838849.787747 [35120]: Resolving [0x200000403:0x32:0x0] to /tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0 lhsmtool_posix: 1786838849.788046 [35120]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0' lhsmtool_posix: 1786838849.807990 [35120]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0451/0x200000402:0x53:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0' lhsmtool_posix: 1786838849.808034 [35120]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786838849.926890 [35149]: action=1 src=0x200000403:0x33:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7 mount_point=/mnt/lustre2 lhsmtool_posix: 1786838849.930188 [35149]: Resolving [0x200000403:0x33:0x0] to /tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0 lhsmtool_posix: 1786838849.930240 [35149]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0' lhsmtool_posix: 1786838849.951312 [35149]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0456/0x200000402:0x54:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0' lhsmtool_posix: 1786838849.951383 [35149]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786838850.093151 [35178]: action=1 src=0x200000403:0x34:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8 mount_point=/mnt/lustre2 lhsmtool_posix: 1786838850.096753 [35178]: Resolving [0x200000403:0x34:0x0] to /tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0 lhsmtool_posix: 1786838850.096812 [35178]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0' lhsmtool_posix: 1786838850.118021 [35178]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0457/0x200000402:0x55:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0' lhsmtool_posix: 1786838850.118072 [35178]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786838850.274584 [35207]: action=1 src=0x200000403:0x35:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9 mount_point=/mnt/lustre2 lhsmtool_posix: 1786838850.278888 [35207]: Resolving [0x200000403:0x35:0x0] to /tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0 lhsmtool_posix: 1786838850.278964 [35207]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0' lhsmtool_posix: 1786838850.303820 [35207]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0454/0x200000402:0x56:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0' lhsmtool_posix: 1786838850.303904 [35207]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12u (10s) == sanity-hsm test 13: Recursively import and restore a directory ========================================================== 20:07:33 (1786838853) lhsmtool_posix: 1786838857.193088 [36965]: action=1 src=subdir dst=/mnt/lustre/d13.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786838857.243755 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1786838857.272069 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//045b/0x200000402:0x59:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1786838857.272325 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1786838857.298188 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//0458/0x200000402:0x5a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1786838857.299221 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1786838857.324868 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//0459/0x200000402:0x5b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1786838857.325461 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1786838857.343678 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//045e/0x200000402:0x5c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1786838857.343988 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1786838857.364700 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//045f/0x200000402:0x5d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1786838857.364900 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1786838857.388843 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//045c/0x200000402:0x5e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1786838857.389824 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1786838857.409856 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//045d/0x200000402:0x5f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1786838857.410020 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1786838857.429317 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//0462/0x200000402:0x60:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1786838857.429511 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1786838857.452179 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//0463/0x200000402:0x61:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1786838857.453094 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1786838857.470825 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//0460/0x200000402:0x62:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1786838857.474373 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1786838857.495598 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//0408/0x240000402:0xa:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1786838857.495811 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1786838857.513663 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//0409/0x240000402:0xb:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1786838857.513901 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1786838857.536643 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//040e/0x240000402:0xc:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1786838857.536870 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1786838857.558011 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//040f/0x240000402:0xd:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1786838857.558206 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1786838857.573985 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//040c/0x240000402:0xe:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1786838857.574150 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1786838857.591653 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//040d/0x240000402:0xf:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1786838857.592008 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1786838857.611668 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//0412/0x240000402:0x10:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1786838857.611886 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1786838857.643903 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//0413/0x240000402:0x11:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1786838857.644086 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1786838857.676361 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//0410/0x240000402:0x12:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1786838857.676602 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1786838857.698382 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//0411/0x240000402:0x13:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1786838857.707361 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1786838857.734516 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//0466/0x200000402:0x64:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1786838857.734709 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1786838857.749990 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//0467/0x200000402:0x65:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1786838857.750263 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1786838857.768798 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//0464/0x200000402:0x66:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1786838857.769019 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1786838857.786713 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//0465/0x200000402:0x67:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1786838857.786895 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1786838857.805555 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//046a/0x200000402:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1786838857.806197 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1786838857.826987 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//046b/0x200000402:0x69:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1786838857.827213 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1786838857.846946 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//0468/0x200000402:0x6a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1786838857.847544 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1786838857.865941 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//0469/0x200000402:0x6b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1786838857.866123 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1786838857.887305 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//046e/0x200000402:0x6c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1786838857.887511 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1786838857.909116 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//046f/0x200000402:0x6d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1786838857.913502 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1786838857.935807 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//0417/0x240000402:0x15:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1786838857.936016 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1786838857.957549 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//0414/0x240000402:0x16:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1786838857.957792 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1786838857.980153 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//0415/0x240000402:0x17:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1786838857.980310 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1786838857.996176 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//041a/0x240000402:0x18:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1786838857.996361 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1786838858.014604 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//041b/0x240000402:0x19:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1786838858.014821 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1786838858.031581 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//0418/0x240000402:0x1a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1786838858.031832 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1786838858.045289 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//0419/0x240000402:0x1b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1786838858.045458 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1786838858.057946 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//041e/0x240000402:0x1c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1786838858.058076 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1786838858.074076 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//041f/0x240000402:0x1d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1786838858.074238 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1786838858.093025 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//041c/0x240000402:0x1e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1786838858.100195 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1786838858.119804 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//046d/0x200000402:0x6f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1786838858.120021 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1786838858.133162 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//0472/0x200000402:0x70:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1786838858.133384 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1786838858.153482 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//0473/0x200000402:0x71:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1786838858.153734 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1786838858.172549 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//0470/0x200000402:0x72:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1786838858.173143 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1786838858.190124 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//0471/0x200000402:0x73:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1786838858.190579 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1786838858.210205 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//0476/0x200000402:0x74:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1786838858.210384 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1786838858.227161 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//0477/0x200000402:0x75:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1786838858.227353 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1786838858.242743 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//0474/0x200000402:0x76:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1786838858.242947 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1786838858.266309 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//0475/0x200000402:0x77:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1786838858.266480 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1786838858.288452 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//047a/0x200000402:0x78:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1786838858.293894 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1786838858.322280 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//0422/0x240000402:0x20:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1786838858.322536 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1786838858.341239 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//0423/0x240000402:0x21:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1786838858.341452 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1786838858.357922 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//0420/0x240000402:0x22:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1786838858.358120 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1786838858.379925 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//0421/0x240000402:0x23:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1786838858.380085 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1786838858.400465 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//0426/0x240000402:0x24:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1786838858.401276 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1786838858.421816 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//0427/0x240000402:0x25:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1786838858.422017 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1786838858.445284 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//0424/0x240000402:0x26:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1786838858.447493 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1786838858.468194 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//0425/0x240000402:0x27:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1786838858.468561 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1786838858.488017 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//042a/0x240000402:0x28:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1786838858.488184 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1786838858.508114 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//042b/0x240000402:0x29:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1786838858.519836 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1786838858.551670 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//0478/0x200000402:0x7a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1786838858.551870 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1786838858.568901 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//0479/0x200000402:0x7b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1786838858.569106 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1786838858.585901 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//047e/0x200000402:0x7c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1786838858.586069 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1786838858.606348 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//047f/0x200000402:0x7d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1786838858.606516 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1786838858.623463 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//047c/0x200000402:0x7e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1786838858.623648 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1786838858.639059 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//047d/0x200000402:0x7f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1786838858.639319 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1786838858.657226 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//0482/0x200000402:0x80:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1786838858.657461 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1786838858.677513 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//0483/0x200000402:0x81:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1786838858.677712 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1786838858.694782 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//0480/0x200000402:0x82:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1786838858.694970 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1786838858.712299 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//0481/0x200000402:0x83:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1786838858.715664 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1786838858.738500 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//0429/0x240000402:0x2b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1786838858.738703 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1786838858.756665 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//042e/0x240000402:0x2c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1786838858.756878 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1786838858.774604 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//042f/0x240000402:0x2d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1786838858.774869 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1786838858.793857 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//042c/0x240000402:0x2e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1786838858.794828 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1786838858.814588 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//042d/0x240000402:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1786838858.814930 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1786838858.835036 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//0432/0x240000402:0x30:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1786838858.835265 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1786838858.854459 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//0433/0x240000402:0x31:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1786838858.854694 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1786838858.874839 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//0430/0x240000402:0x32:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1786838858.875031 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1786838858.891024 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//0431/0x240000402:0x33:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1786838858.891378 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1786838858.906105 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//0436/0x240000402:0x34:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1786838858.914430 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1786838858.944278 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//0487/0x200000402:0x85:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1786838858.944456 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1786838858.959580 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//0484/0x200000402:0x86:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1786838858.959831 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1786838858.973036 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//0485/0x200000402:0x87:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1786838858.973260 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1786838858.995555 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//048a/0x200000402:0x88:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1786838858.995830 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1786838859.020542 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//048b/0x200000402:0x89:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1786838859.020785 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1786838859.043333 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//0488/0x200000402:0x8a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1786838859.043574 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1786838859.068431 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//0489/0x200000402:0x8b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1786838859.068676 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1786838859.091024 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//048e/0x200000402:0x8c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1786838859.091241 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1786838859.108094 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//048f/0x200000402:0x8d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1786838859.109233 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1786838859.128449 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//048c/0x200000402:0x8e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1786838859.135233 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1786838859.160024 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//0434/0x240000402:0x36:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1786838859.160242 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1786838859.181058 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//0435/0x240000402:0x37:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1786838859.181204 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1786838859.201416 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//043a/0x240000402:0x38:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1786838859.202067 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1786838859.220317 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//043b/0x240000402:0x39:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1786838859.220518 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1786838859.236803 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//0438/0x240000402:0x3a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1786838859.237026 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1786838859.256136 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//0439/0x240000402:0x3b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1786838859.256326 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1786838859.278063 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//043e/0x240000402:0x3c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1786838859.278545 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1786838859.295802 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//043f/0x240000402:0x3d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1786838859.296037 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1786838859.313567 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//043c/0x240000402:0x3e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1786838859.313943 [36965]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1786838859.333152 [36965]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//043d/0x240000402:0x3f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1786838859.333245 [36965]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_13/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 13 (12s) == sanity-hsm test 14: Rebind archived file to a new fid ========================================================== 20:07:45 (1786838865) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_14/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0677577 s, 15.5 MB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00170432 s, 0.0 kB/s rebind 0x200000403:0x73:0x0 to 0x200000403:0x74:0x0 lhsmtool_posix: 1786838867.443432 [37918]: action=2 src=0x200000403:0x73:0x0 dst=0x200000403:0x74:0x0 mount_point=/mnt/lustre2 lhsmtool_posix: 1786838867.444847 [37918]: rebind [0x200000403:0x73:0x0] to [0x200000403:0x74:0x0] lhsmtool_posix: 1786838867.445123 [37918]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 14 (5s) == sanity-hsm test 15: Rebind a list of files ============ 20:07:50 (1786838870) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_15/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571665 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0588279 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0503081 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.057139 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526285 s, 19.9 MB/s Waiting 300s for '' pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 0+0 records in 0+0 records out 0 bytes copied, 0.0018783 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00224296 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00241545 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00287974 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00282878 s, 0.0 kB/s rebind list of files lhsmtool_posix: 1786838874.292393 [38708]: action=2 src=/tmp/tmp.8239 dst=(null) mount_point=/mnt/lustre2 lhsmtool_posix: 1786838874.293236 [38708]: rebind [0x200000403:0x76:0x0] to [0x200000403:0x7b:0x0] lhsmtool_posix: 1786838874.293354 [38708]: rebind [0x200000403:0x77:0x0] to [0x200000403:0x7c:0x0] lhsmtool_posix: 1786838874.293391 [38708]: rebind [0x200000403:0x78:0x0] to [0x200000403:0x7d:0x0] lhsmtool_posix: 1786838874.293434 [38708]: rebind [0x200000403:0x79:0x0] to [0x200000403:0x7e:0x0] lhsmtool_posix: 1786838874.293484 [38708]: rebind [0x200000403:0x7a:0x0] to [0x200000403:0x7f:0x0] lhsmtool_posix: 1786838874.293541 [38708]: 5 lines read from '/tmp/tmp.8239', 5 rebind successful lhsmtool_posix: 1786838874.293556 [38708]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 15 (11s) == sanity-hsm test 16: Test CT bandwith control option === 20:08:01 (1786838881) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_16/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 0.0302197 s, 694 MB/s Waiting 200s for 'SUCCEED' Waiting 190s for 'SUCCEED' Waiting 180s for 'SUCCEED' Updated after 21s: want 'SUCCEED' got 'SUCCEED' PASS 16 (24s) == sanity-hsm test 20: Release is not permitted ========== 20:08:25 (1786838905) 0+0 records in 0+0 records out 0 bytes copied, 0.00172338 s, 0.0 kB/s 0x200000403:0x87:0x0 Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted PASS 20 (3s) == sanity-hsm test 21: Simple release tests ============== 20:08:28 (1786838908) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_21/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047673 s, 22.0 MB/s Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' PASS 21 (6s) == sanity-hsm test 22: Could not swap a release file ===== 20:08:34 (1786838914) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_22/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0525226 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0481571 s, 21.8 MB/s 0x200000403:0x8b:0x0 lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d22.sanity-hsm/test_swap' and '/mnt/lustre/d22.sanity-hsm/test_release': Operation not supported (95) PASS 22 (4s) == sanity-hsm test 23: Release does not change a/mtime (utime) ========================================================== 20:08:38 (1786838918) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_23/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0427583 s, 24.5 MB/s PASS 23 (3s) == sanity-hsm test 24a: Archive, release, and restore does not change a/mtime (i/o) ========================================================== 20:08:41 (1786838921) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0476268 s, 22.0 MB/s Waiting 200s for 'SUCCEED' Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s 192.168.203.116@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg316-client.virtnet /mnt/lustre (opts:) Starting client: oleg316-client.virtnet: -o user_xattr,flock 192.168.203.116@tcp:/lustre /mnt/lustre PASS 24a (7s) == sanity-hsm test 24b: root can archive, release, and restore user files ========================================================== 20:08:48 (1786838928) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539842 s, 19.4 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] /mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm: (0x00000009) exists archived, archive_id:1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [md5sum] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] PASS 24b (5s) == sanity-hsm test 24c: check that user,group,other request masks work ========================================================== 20:08:54 (1786838934) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0508746 s, 20.6 MB/s 0x200000403:0x94:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0532969 s, 19.7 MB/s 0x200000403:0x95:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0434851 s, 24.1 MB/s 0x200000403:0x96:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] PASS 24c (9s) == sanity-hsm test 24d: check that read-only mounts are respected ========================================================== 20:09:02 (1786838942) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0621974 s, 16.9 MB/s 0x200000403:0x97:0x0 /mnt/lustre/d24d.sanity-hsm/f24d.sanity-hsm lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 81 0x51 0x2c0000401 Starting client: oleg316-client.virtnet: -o user_xattr,flock 192.168.203.116@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.440831 s, 2.4 MB/s Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Stopping client oleg316-client.virtnet /mnt/lustre3 (opts:) PASS 24d (9s) == sanity-hsm test 24e: tar succeeds on HSM released files ========================================================== 20:09:11 (1786838951) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.070749 s, 14.8 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d24e.sanity-hsm/f24e.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 tar: Removing leading `/' from member names PASS 24e (6s) == sanity-hsm test 24f: root can archive, release, and restore tar files ========================================================== 20:09:17 (1786838957) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm f24f.sanity-hsm f24f.sanity-hsm Sum0 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm, sum1 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm PASS 24f (4s) == sanity-hsm test 24g: write by non-owner still sets dirty ========================================================== 20:09:21 (1786838961) RUNAS = 'runas -u 500 -g 500' Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Please listen carefully as our options have changed. Waiting 200s for 'SUCCEED' running as uid/gid/euid/egid 500/500/500/500, groups: 500 [tee] [/mnt/lustre/d24g.sanity-hsm/f24g.sanity-hsm] To be electrocuted by your telephone, press #. PASS 24g (4s) == sanity-hsm test 25a: Restore lost file (HS_LOST flag) from import ========================================================== 20:09:25 (1786838965) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786838966.699016 [47766]: action=1 src=d25a.sanity-hsm/f25a.sanity-hsm dst=/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786838966.702771 [47766]: importing '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1786838966.723225 [47766]: imported '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//0406/0x240000404:0x2:0x0'=='/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1786838966.723264 [47766]: process finished, errs: 0 major, 0 minor, rc=0 (Success) md5sum: /mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm: No data available PASS 25a (3s) == sanity-hsm test 25b: Restore lost file (HS_LOST flag) after release ========================================================== 20:09:28 (1786838968) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' md5sum: /mnt/lustre/d25b.sanity-hsm/f25b.sanity-hsm: No data available PASS 25b (4s) == sanity-hsm test 26A: Remove the archive of a valid file ========================================================== 20:09:32 (1786838972) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00396595 s, 0.0 kB/s Waiting 200s for 'SUCCEED' PASS 26A (5s) == sanity-hsm test 26a: Remove Archive On Last Unlink (RAoLU) policy ========================================================== 20:09:37 (1786838977) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 7s: want 'SUCCEED' got 'SUCCEED' PASS 26a (19s) == sanity-hsm test 26b: RAoLU policy when CDT off ======== 20:09:56 (1786838996) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' PASS 26b (13s) == sanity-hsm test 26c: RAoLU effective when file closed ========================================================== 20:10:09 (1786839009) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' multiop /mnt/lustre/d26c.sanity-hsm/f26c.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8239 Waiting 200s for 'SUCCEED' Updated after 8s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 9s: want 'SUCCEED' got 'SUCCEED' PASS 26c (28s) == sanity-hsm test 26d: RAoLU when Client eviction ======= 20:10:37 (1786839037) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.039119 s, 26.8 MB/s Waiting 200s for 'SUCCEED' multiop /mnt/lustre/d26d.sanity-hsm/f26d.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8239 Waiting 200s for 'SUCCEED' pdsh@oleg316-client: oleg316-client: ssh exited with exit code 5 PASS 26d (14s) == sanity-hsm test 26e: RAoLU with a non-started coordinator ========================================================== 20:10:51 (1786839051) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458166 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0438738 s, 23.9 MB/s Waiting 200s for 'SUCCEED' Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s Failing mds1 on oleg316-server Stopping /mnt/lustre-mds1 (opts:) on oleg316-server 20:11:17 (1786839077) shut down facet: mds1 facet_host: oleg316-server facet_failover_host: oleg316-server Failover mds1 to oleg316-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg316-server: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-MDT0000 20:11:32 (1786839092) targets are mounted 20:11:32 (1786839092) facet_failover done oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Check cookie from RAoLU request (last: 1786838764, remove: 1786838765) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' PASS 26e (77s) == sanity-hsm test 27a: Remove the archive of an imported file (Operation not permitted) ========================================================== 20:12:08 (1786839128) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.463001 s, 88.3 MB/s lhsmtool_posix: 1786839130.041194 [55007]: action=1 src=d27a.sanity-hsm/f27a.sanity-hsm dst=/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786839130.046058 [55007]: importing '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1786839130.081601 [55007]: imported '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//0400/0x240000404:0x4:0x0'=='/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1786839130.081691 [55007]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Cannot send HSM request (use of /mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm): Operation not permitted PASS 27a (4s) == sanity-hsm test 27b: Remove the archive of a relased file (Operation not permitted) ========================================================== 20:12:12 (1786839132) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00237468 s, 0.0 kB/s Cannot send HSM request (use of /mnt/lustre/d27b.sanity-hsm/f27b.sanity-hsm): Operation not permitted PASS 27b (3s) == sanity-hsm test 28: Concurrent archive/file remove ==== 20:12:15 (1786839135) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_28/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00206121 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 28 (6s) == sanity-hsm test 29a: Tests --mntpath and --archive options ========================================================== 20:12:21 (1786839141) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 7 "/mnt/lustre"' Cannot send HSM request (use of /mnt/lustre): Invalid argument hsm: '/qwerty/uyt' is not a valid FID PASS 29a (2s) == sanity-hsm test 29b: Archive/delete/remove by FID from the archive. ========================================================== 20:12:23 (1786839143) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0406364 s, 25.8 MB/s Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' PASS 29b (6s) == sanity-hsm test 29c: Archive/delete/remove by FID, using a file list. ========================================================== 20:12:29 (1786839149) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047312 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0467835 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.043156 s, 24.3 MB/s Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' PASS 29c (8s) == sanity-hsm test 29d: hsm_remove by FID with archive_id 0 for unlinked file cause ========================================================== 20:12:37 (1786839157) SKIP: sanity-hsm test_29d Need 3 or more clients, have 1 SKIP 29d (1s) == sanity-hsm test 30a: Restore at exec (import case) ==== 20:12:38 (1786839158) SKIP: sanity-hsm test_30a Need 2 or more clients, have 1 SKIP 30a (1s) == sanity-hsm test 30b: Restore at exec (release case) === 20:12:39 (1786839159) SKIP: sanity-hsm test_30b Need 2 or more clients, have 1 SKIP 30b (1s) == sanity-hsm test 30c: Update during exec of released file must fail ========================================================== 20:12:40 (1786839160) SKIP: sanity-hsm test_30c Need 2 or more clients, have 1 SKIP 30c (1s) == sanity-hsm test 31a: Import a large file and check size during restore ========================================================== 20:12:41 (1786839161) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.401569 s, 102 MB/s lhsmtool_posix: 1786839162.914858 [60171]: action=1 src=d31a.sanity-hsm/f31a.sanity-hsm dst=/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786839162.917096 [60171]: importing '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1786839162.926930 [60171]: imported '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//040c/0x200000407:0xb:0x0'=='/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1786839162.926978 [60171]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' size seen is right: 40894464 == 40894464 done PASS 31a (14s) == sanity-hsm test 31b: Restore a large unaligned file and check size during restore ========================================================== 20:12:55 (1786839175) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.421006 s, 92.6 MB/s Waiting 200s for 'SUCCEED' size seen is right: 39000000 == 39000000 done PASS 31b (15s) == sanity-hsm test 31c: Restore a large aligned file and check size during restore ========================================================== 20:13:10 (1786839190) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.435218 s, 94.0 MB/s Waiting 200s for 'SUCCEED' size seen is right: 40894464 == 40894464 done PASS 31c (15s) == sanity-hsm test 33: Kill a restore waiting process ==== 20:13:25 (1786839205) 0+0 records in 0+0 records out 0 bytes copied, 0.00222777 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_33/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg316-client.virtnet Waiting 200s for 'STARTED' Copytool is continued on oleg316-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2893: 62199 Terminated md5sum $f > /dev/null PASS 33 (5s) == sanity-hsm test 34: Remove file during restore ======== 20:13:30 (1786839210) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_34/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00222855 s, 0.0 kB/s Copytool is suspended on oleg316-client.virtnet Waiting 200s for 'STARTED' Copytool is continued on oleg316-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2925: kill: (62867) - No such process PASS 34 (5s) == sanity-hsm test 35: Overwrite file during restore ===== 20:13:35 (1786839215) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_35/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00248952 s, 0.0 kB/s Copytool is suspended on oleg316-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Copytool is continued on oleg316-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2962: kill: (63551) - No such process PASS 35 (5s) == sanity-hsm test 36: Move file during restore ========== 20:13:40 (1786839220) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_36/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00245736 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Copytool is suspended on oleg316-client.virtnet Copytool is continued on oleg316-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2998: kill: (64241) - No such process PASS 36 (6s) == sanity-hsm test 37: re-archive a dirty file =========== 20:13:46 (1786839226) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_37/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051668 s, 20.3 MB/s Waiting 200s for 'SUCCEED' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217584 s, 48.2 MB/s Waiting 200s for 'SUCCEED' Changed after 2s: from 'SUCCEED WAITING' to 'SUCCEED SUCCEED' Updated after 3s: want 'SUCCEED' got 'SUCCEED' PASS 37 (68s) == sanity-hsm test 40: Parallel archive requests ========= 20:14:54 (1786839294) oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg316-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg316-client: oleg316-server: ssh exited with exit code 12 oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg316-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg316-client: oleg316-server: ssh exited with exit code 12 set max_requests=1152921504606846976 failed with 12 [ 899.237266] Lustre: lustre-MDT0000: No more memory to set HSM max_requests=1152921504606846976 (max request memory: 461MB, current total 6/975940): rc = -12 Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_40/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Wait for all requests being enqueued...100 archive requests submitted 100 archive requests submitted 100 archive requests submitted 100 archive requests submitted OK pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 PASS 40 (88s) == sanity-hsm test 50: Archive with large number of pending HSM actions ========================================================== 20:16:22 (1786839382) oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1000000: Cannot allocate memory oleg316-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg316-client: oleg316-server: ssh exited with exit code 12 oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=1000000: Cannot allocate memory oleg316-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg316-client: oleg316-server: ssh exited with exit code 12 Filesystem Inodes IUsed IFree IUse% Mounted on 192.168.203.116@tcp:/lustre 524222 1207 523015 1% /mnt/lustre - create 7601 (time 1786839395.02 total 10.00 last 760.05) total: 10000 create in 13.09 seconds: 763.81 ops/second 10000 - unlinked 0 (time 1786839425 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second Start Phase files_num: 10000 time: 26 - create 5713 (time 1786839450.33 total 10.00 last 571.23) - create 10000 (time 1786839457.19 total 16.86 last 625.35) - create 15936 (time 1786839467.19 total 26.86 last 593.47) total: 20000 create in 33.71 seconds: 593.26 ops/second - unlinked 0 (time 1786839520 ; total 0 ; last 0) - unlinked 10000 (time 1786839537 ; total 17 ; last 17) total: 20000 unlinks in 31 seconds: 645.161316 unlinks/second Middle Phase files_num: 20000 time: 45 - create 8351 (time 1786839562.89 total 10.00 last 835.08) total: 10000 create in 11.67 seconds: 857.02 ops/second 40000 - unlinked 0 (time 1786839587 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second End Phase files_num: 10000 time: 21 40000 mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 50 (224s) == sanity-hsm test 52: Opened for write file on an evicted client should be set dirty ========================================================== 20:20:06 (1786839606) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_52/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791867 s, 13.2 MB/s multiop /mnt/lustre/d52.sanity-hsm/f52.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8239 PASS 52 (5s) == sanity-hsm test 53: Opened for read file on an evicted client should not be set dirty ========================================================== 20:20:11 (1786839611) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_53/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357845 s, 29.3 MB/s multiop /mnt/lustre/d53.sanity-hsm/f53.sanity-hsm vo_c TMPPIPE=/tmp/multiop_open_wait_pipe.8239 PASS 53 (5s) == sanity-hsm test 54: Write during an archive cancels it ========================================================== 20:20:16 (1786839616) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.363896 s, 107 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_54/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Waiting 200s for 'FAILED' Waiting 170s for 'FAILED' Updated after 37s: want 'FAILED' got 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 54 (43s) == sanity-hsm test 55: Truncate during an archive cancels it ========================================================== 20:20:59 (1786839659) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.337798 s, 115 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_55/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' Waiting 200s for 'STARTED' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Waiting 200s for 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 55 (8s) == sanity-hsm test 56: Setattr during an archive is ok === 20:21:07 (1786839667) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.337422 s, 116 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_56/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Waiting 200s for 'SUCCEED' Waiting 190s for 'SUCCEED' Waiting 180s for 'SUCCEED' Updated after 37s: want 'SUCCEED' got 'SUCCEED' PASS 56 (42s) == sanity-hsm test 57: Archive a file with dirty cache on another node ========================================================== 20:21:49 (1786839709) SKIP: sanity-hsm test_57 Need 2 or more clients, have 1 SKIP 57 (1s) == sanity-hsm test 58: Truncate a released file will trigger restore ========================================================== 20:21:50 (1786839710) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_58/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' truncate up from 1516 to 3032 Waiting 200s for 'SUCCEED' /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate down from 1516 to 758 Waiting 200s for 'SUCCEED' /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate to 0 Waiting 200s for 'SUCCEED' /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 PASS 58 (9s) == sanity-hsm test 59: Release stripeless file with non-zero size ========================================================== 20:21:59 (1786839719) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_59/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 59 (4s) == sanity-hsm test 60: Changing progress update interval from default ========================================================== 20:22:03 (1786839723) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.086349 s, 121 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_60/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 --update-interval 5 "/mnt/lustre2"' Waiting 100s for 'start copy' Expecting a progress update within 20 seconds... 5242880 bytes copied in 4 seconds. Wait for on going archive hsm action to complete Waiting 10s for 'copied' Updated after 4s: want 'copied' got 'copied' PASS 60 (13s) == sanity-hsm test 61: Waiting archive of a removed file should fail ========================================================== 20:22:16 (1786839736) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_61/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 61 (4s) == sanity-hsm test 70: Copytool logs JSON register/unregister events to FIFO ========================================================== 20:22:20 (1786839740) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_70/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --event-fifo /tmp/sanity-hsm.test_70.NFq4/fifo "/mnt/lustre2"' Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s Register/Unregister events look OK. PASS 70 (4s) == sanity-hsm test 71: Copytool logs JSON archive events to FIFO ========================================================== 20:22:24 (1786839744) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_71/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_71.6ZMd/fifo "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341862 s, 30.7 MB/s Waiting 200s for 'SUCCEED' Archive events look OK. mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 71 (8s) == sanity-hsm test 72: Copytool logs JSON restore events to FIFO ========================================================== 20:22:32 (1786839752) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_72/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_72.b9nh/fifo "/mnt/lustre2"' 16+0 records in 16+0 records out 16000000 bytes (16 MB, 15 MiB) copied, 0.155386 s, 103 MB/s lhsmtool_posix: 1786839754.644930 [82182]: action=1 src=d72.sanity-hsm/f72.sanity-hsm dst=/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786839754.647217 [82182]: importing '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1786839754.657220 [82182]: imported '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//0bc5/0x200000bd2:0x17:0x0'=='/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1786839754.657267 [82182]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Waiting 200s for 'SUCCEED' Restore events look OK. PASS 72 (7s) == sanity-hsm test 90: Archive/restore a file list ======= 20:22:39 (1786839759) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_90/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Waiting 200s for '' pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 PASS 90 (68s) == sanity-hsm test 100: Set coordinator /proc tunables === 20:23:47 (1786839827) Testing mdt.lustre-MDT0000.hsm.loop_period Set loop_period to 0 must failed oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/loop_period=0: Invalid argument oleg316-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/loop_period'='0': Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 22 oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/loop_period=0: Invalid argument oleg316-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/loop_period'='0': Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.grace_delay Set grace_delay to 0 must failed oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/grace_delay=0: Invalid argument oleg316-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/grace_delay'='0': Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 22 oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/grace_delay=0: Invalid argument oleg316-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/grace_delay'='0': Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.active_request_timeout Set active_request_timeout to 0 must failed oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/active_request_timeout=0: Invalid argument oleg316-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 22 oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/active_request_timeout=0: Invalid argument oleg316-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.max_requests Set max_requests to 0 must failed oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=0: Invalid argument oleg316-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='0': Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 22 oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=0: Invalid argument oleg316-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='0': Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.default_archive_id Set default_archive_id to 0 must failed oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/default_archive_id=0: Invalid argument oleg316-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/default_archive_id'='0': Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 22 oleg316-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/default_archive_id=0: Invalid argument oleg316-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/default_archive_id'='0': Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 22 PASS 100 (16s) == sanity-hsm test 102: Verify coordinator control ======= 20:24:03 (1786839843) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 102 (7s) == sanity-hsm test 103: Purge all requests =============== 20:24:10 (1786839850) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Current requests pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 PASS 103 (5s) == sanity-hsm test 103a: Purge pending restore requests == 20:24:15 (1786839855) mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' (0x200000403:0x148:0x0|0x200000403:0x149:0x0|0x200000403:0x14a:0x0|0x200000403:0x14b:0x0|0x200000403:0x14c:0x0|0x200000403:0x14d:0x0|0x200000403:0x14e:0x0|0x200000403:0x14f:0x0).*action=ARCHIVE.*status=SUCCEED pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s cat: '/mnt/lustre/.lustre/fid/0x200000403:0x14a:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x148:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x149:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x14b:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x14c:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x14d:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x14e:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x14f:0x0': No data available mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 103a (10s) == sanity-hsm test 104: Copy tool data field ============= 20:24:25 (1786839865) 0+0 records in 0+0 records out 0 bytes copied, 0.00161469 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 104 (4s) == sanity-hsm test 105: Restart of coordinator =========== 20:24:29 (1786839869) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 105 (13s) == sanity-hsm test 106a: Copytool register/unregister ==== 20:24:42 (1786839882) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg316-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg316-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg316-client.virtnet: 88841 Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 uuid not found in agent list on mds2 Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg316-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg316-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 106a (11s) == sanity-hsm test 106b: Unregister agent after a copytool crash ========================================================== 20:24:53 (1786839893) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg316-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg316-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg316-client.virtnet: 90574 Killing existing copytools on oleg316-client.virtnet with signal KILL copytools stopped in 0s pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 uuid not found in agent list on mds2 Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 106b (8s) == sanity-hsm test 106c: Unregister copytool on client eviction ========================================================== 20:25:01 (1786839901) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg316-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg316-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg316-client.virtnet: 91805 oleg316-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg316-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg316-client.virtnet: executing get_client_uuid /mnt/lustre2 mdc.lustre-MDT0000-mdc-ffff9c7d4a8c7800.active=0 mdc.lustre-MDT0001-mdc-ffff9c7d4a8c7800.active=0 mdt.lustre-MDT0000.evict_client=d0afbd13-56e4-4b53-8abd-a00edad637cc mdt.lustre-MDT0001.evict_client=d0afbd13-56e4-4b53-8abd-a00edad637cc pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 uuid not found in agent list on mds2 mdc.lustre-MDT0000-mdc-ffff9c7d4a8c7800.active=1 mdc.lustre-MDT0001-mdc-ffff9c7d4a8c7800.active=1 oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 mdc.lustre-MDT0000-mdc-ffff9c7d4a8c7800.active=1 mdc.lustre-MDT0001-mdc-ffff9c7d4a8c7800.active=1 PASS 106c (15s) == sanity-hsm test 107: Copytool re-register after MDS restart ========================================================== 20:25:16 (1786839916) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_107/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Failing mds1 on oleg316-server Stopping /mnt/lustre-mds1 (opts:) on oleg316-server 20:25:20 (1786839920) shut down facet: mds1 facet_host: oleg316-server facet_failover_host: oleg316-server Failover mds1 to oleg316-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg316-server: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-MDT0000 20:25:34 (1786839934) targets are mounted 20:25:34 (1786839934) facet_failover done oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Waiting 200s for 'SUCCEED' PASS 107 (28s) == sanity-hsm test 109: Policy display/change ============ 20:25:44 (1786839944) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0000.hsm.policy=NRA NBR Next set_param must failed oleg316-server: error: set_param: setting /sys/kernel/debug/lustre/mdt/lustre-MDT0000/hsm/policy=wrong: Invalid argument oleg316-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/policy'='wrong': Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 22 Back to default policy mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 109 (6s) == sanity-hsm test 110a: Non blocking restore policy (import case) ========================================================== 20:25:50 (1786839950) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786839951.623588 [96408]: action=1 src=d110a.sanity-hsm/f110a.sanity-hsm dst=/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786839951.626023 [96408]: importing '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1786839951.634882 [96408]: imported '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//0bb6/0x200000bd2:0x64:0x0'=='/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1786839951.634931 [96408]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0001.hsm.policy=+NBR md5sum: /mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm: No data available Waiting 200s for 'SUCCEED' mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 110a (5s) == sanity-hsm test 110b: Non blocking restore policy (release case) ========================================================== 20:25:55 (1786839955) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0001.hsm.policy=+NBR md5sum: /mnt/lustre/d110b.sanity-hsm/f110b.sanity-hsm: No data available Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 110b (7s) == sanity-hsm test 111a: No retry policy (import case), restore will error ========================================================== 20:26:02 (1786839962) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786839963.147254 [97819]: action=1 src=d111a.sanity-hsm/f111a.sanity-hsm dst=/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786839963.149686 [97819]: importing '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1786839963.159469 [97819]: imported '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//0bba/0x200000bd2:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1786839963.159512 [97819]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Remove from backend: 0x200000bd2:0x68:0x0 = /tmp/arc1/sanity-hsm.test_111a/0bba/0x200000bd2:0x68:0x0 Waiting 200s for 'FAILED' Updated after 2s: want 'FAILED' got 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 111a (5s) == sanity-hsm test 111b: No retry policy (release case), restore will error ========================================================== 20:26:07 (1786839967) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Waiting 200s for 'SUCCEED' Remove from backend: 0x200000bd3:0x7:0x0 = /tmp/arc1/sanity-hsm.test_111b/0bd4/0x200000bd3:0x7:0x0 Waiting 200s for 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 111b (6s) == sanity-hsm test 112: State of recorded request ======== 20:26:13 (1786839973) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_112/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled /mnt/lustre/d112.sanity-hsm/f112.sanity-hsm: ARCHIVE waiting (from 0 to EOF) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 112 (5s) == sanity-hsm test 113: wrong stat after restore ========= 20:26:18 (1786839978) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327929 s, 32.0 MB/s Starting client: oleg316-client.virtnet: -o user_xattr,flock 192.168.203.116@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_113/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying released state: multiop /mnt/lustre/d113.sanity-hsm/f113.sanity-hsm voO_WRONLY:O_APPEND:_w4c TMPPIPE=/tmp/multiop_open_wait_pipe.8239 File: /mnt/lustre2/d113.sanity-hsm/f113.sanity-hsm Size: 1048576 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115238860488714 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-15 20:26:19.000000000 -0400 Modify: 2026-08-15 20:26:19.000000000 -0400 Change: 2026-08-15 20:26:19.000000000 -0400 Birth: 2026-08-15 20:26:18.000000000 -0400 Stopping client oleg316-client.virtnet /mnt/lustre3 (opts:) PASS 113 (5s) == sanity-hsm test 114: Incompatible request does not set other requests as STARTED ========================================================== 20:26:23 (1786839983) 0+0 records in 0+0 records out 0 bytes copied, 0.000947235 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.000966492 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_114/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 114 (5s) == sanity-hsm test 200: Register/Cancel archive ========== 20:26:28 (1786839988) 0+0 records in 0+0 records out 0 bytes copied, 0.00137319 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_200/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg316-client.virtnet Waiting 200s for 'STARTED' Copytool is continued on oleg316-client.virtnet PASS 200 (5s) == sanity-hsm test 201: Register/Cancel restore ========== 20:26:33 (1786839993) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_201/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.402244 s, 102 MB/s lhsmtool_posix: 1786839994.330097 [102226]: action=1 src=d201.sanity-hsm/f201.sanity-hsm dst=/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786839994.332421 [102226]: importing '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1786839994.341431 [102226]: imported '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//0ba0/0x200000bd2:0x72:0x0'=='/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1786839994.341460 [102226]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 201 (6s) == sanity-hsm test 202: Register/Cancel remove =========== 20:26:39 (1786839999) 0+0 records in 0+0 records out 0 bytes copied, 0.0011192 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_202/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg316-client.virtnet Waiting 200s for 'STARTED' PASS 202 (4s) == sanity-hsm test 220A: Changelog for archive =========== 20:26:43 (1786840003) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.3 16HSM 00:26:46.196613164 2026.08.16 0x0 t=[0x200000bd3:0x12:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 220A (7s) == sanity-hsm test 220a: Changelog for failed archive ==== 20:26:51 (1786840011) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' Copytool is suspended on oleg316-client.virtnet Copytool is continued on oleg316-client.virtnet lustre-MDT0000.6 16HSM 00:26:53.525964287 2026.08.16 0x2 t=[0x200000bd3:0x13:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 220a (6s) == sanity-hsm test 221: Changelog for archive canceled === 20:26:57 (1786840017) 0+0 records in 0+0 records out 0 bytes copied, 0.00123 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_221/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' Copytool is suspended on oleg316-client.virtnet Waiting 200s for 'CANCELED' Copytool is continued on oleg316-client.virtnet lustre-MDT0000.8 16HSM 00:27:01.392417776 2026.08.16 0x7d t=[0x200000bd3:0x14:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 221 (8s) == sanity-hsm test 222a: Changelog for explicit restore == 20:27:05 (1786840025) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786840025.968358 [106080]: action=1 src=d222a.sanity-hsm/f222a.sanity-hsm dst=/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786840025.970340 [106080]: importing '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1786840025.979295 [106080]: imported '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//0baa/0x200000bd2:0x78:0x0'=='/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1786840025.979328 [106080]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' lustre-MDT0000.11 16HSM 00:27:07.590513273 2026.08.16 0x80 t=[0x200000bd2:0x78:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 222a (6s) == sanity-hsm test 222b: Changelog for implicit restore == 20:27:11 (1786840031) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' Waiting 200s for 'SUCCEED' 4ef076ad20e5d4a2f4729d6ed41a3bf9 /mnt/lustre/d222b.sanity-hsm/f222b.sanity-hsm lustre-MDT0000.20 16HSM 00:27:15.849869568 2026.08.16 0x80 t=[0x200000bd3:0x16:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp p=[0x200000bd2:0x75:0x0] lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 PASS 222b (8s) == sanity-hsm test 222c: Changelog for failed explicit restore ========================================================== 20:27:19 (1786840039) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786840040.340919 [107713]: action=1 src=d222c.sanity-hsm/f222c.sanity-hsm dst=/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786840040.343187 [107713]: importing '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1786840040.353171 [107713]: imported '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//0baf/0x200000bd2:0x7d:0x0'=='/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1786840040.353219 [107713]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' Copytool is suspended on oleg316-client.virtnet Copytool is continued on oleg316-client.virtnet lustre-MDT0000.22 16HSM 00:27:22.129854308 2026.08.16 0x82 t=[0x200000bd2:0x7d:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 222c (6s) == sanity-hsm test 222d: Changelog for failed implicit restore ========================================================== 20:27:25 (1786840045) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' Waiting 200s for 'SUCCEED' Remove from backend: 0x200000bd3:0x18:0x0 = /tmp/arc1/sanity-hsm.test_222d/0bcb/0x200000bd3:0x18:0x0 md5sum: /mnt/lustre/d222d.sanity-hsm/f222d.sanity-hsm: No data available lustre-MDT0000.29 16HSM 00:27:30.353045228 2026.08.16 0x82 t=[0x200000bd3:0x18:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 222d (9s) == sanity-hsm test 223a: Changelog for restore canceled (import case) ========================================================== 20:27:34 (1786840054) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.388843 s, 105 MB/s mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl8 cl8' lhsmtool_posix: 1786840057.077390 [109459]: action=1 src=d223a.sanity-hsm/f223a.sanity-hsm dst=/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786840057.079573 [109459]: importing '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1786840057.089227 [109459]: imported '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//0b53/0x200000bd2:0x81:0x0'=='/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1786840057.089273 [109459]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Waiting 200s for 'STARTED' Waiting 200s for 'SUCCEED' Waiting 180s for 'SUCCEED' Updated after 29s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.36 16HSM 00:28:06.819247983 2026.08.16 0xfd t=[0x200000bd2:0x81:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001: clear the changelog for cl8 of all records lustre-MDT0001: Deregistered changelog user #8 lustre-MDT0000: clear the changelog for cl8 of all records lustre-MDT0000: Deregistered changelog user #8 PASS 223a (36s) == sanity-hsm test 223b: Changelog for restore canceled (release case) ========================================================== 20:28:10 (1786840090) 0+0 records in 0+0 records out 0 bytes copied, 0.00210044 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl9 cl9' Copytool is suspended on oleg316-client.virtnet Waiting 200s for 'STARTED' Copytool is continued on oleg316-client.virtnet lustre-MDT0000.43 16HSM 00:28:15.106914485 2026.08.16 0xfd t=[0x200000bd3:0x1c:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp p=[0x200000bd2:0x80:0x0] lustre-MDT0001: clear the changelog for cl9 of all records lustre-MDT0001: Deregistered changelog user #9 lustre-MDT0000: clear the changelog for cl9 of all records lustre-MDT0000: Deregistered changelog user #9 PASS 223b (8s) == sanity-hsm test 224A: Changelog for remove ============ 20:28:18 (1786840098) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl10 cl10' Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' lustre-MDT0000.48 16HSM 00:28:23.332930345 2026.08.16 0x200 t=[0x200000bd3:0x1e:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001: clear the changelog for cl10 of all records lustre-MDT0001: Deregistered changelog user #10 lustre-MDT0000: clear the changelog for cl10 of all records lustre-MDT0000: Deregistered changelog user #10 PASS 224A (9s) == sanity-hsm test 224a: Changelog for failed remove ===== 20:28:27 (1786840107) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl11 cl11' Waiting 200s for 'SUCCEED' Remove from backend: 0x200000bd3:0x1f:0x0 = /tmp/arc1/sanity-hsm.test_224a/0bcc/0x200000bd3:0x1f:0x0 Copytool is suspended on oleg316-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Copytool is continued on oleg316-client.virtnet lustre-MDT0000.53 16HSM 00:28:32.172411500 2026.08.16 0x202 t=[0x200000bd3:0x1f:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001: clear the changelog for cl11 of all records lustre-MDT0001: Deregistered changelog user #11 lustre-MDT0000: clear the changelog for cl11 of all records lustre-MDT0000: Deregistered changelog user #11 PASS 224a (8s) == sanity-hsm test 225: Changelog for remove canceled ==== 20:28:35 (1786840115) Test disabled PASS 225 (2s) == sanity-hsm test 226: changelog for last rm/mv with exiting archive ========================================================== 20:28:37 (1786840117) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_226/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0x200000bd3:0x22:0x0 mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl12 cl12' Waiting 200s for 'SUCCEED' lustre-MDT0000.54 16HSM 00:28:40.352817272 2026.08.16 0x280 t=[0x200000bd3:0x20:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.55 16HSM 00:28:40.371813161 2026.08.16 0x280 t=[0x200000bd3:0x20:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0000.56 16HSM 00:28:40.371892657 2026.08.16 0x0 t=[0x200000bd3:0x20:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0000.57 16HSM 00:28:41.376748133 2026.08.16 0x280 t=[0x200000bd3:0x21:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.58 16HSM 00:28:41.395823293 2026.08.16 0x280 t=[0x200000bd3:0x21:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0000.59 16HSM 00:28:41.395889211 2026.08.16 0x0 t=[0x200000bd3:0x21:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0000.60 06UNLNK 00:28:41.650712922 2026.08.16 0x3 t=[0x200000bd3:0x20:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp p=[0x200000bd2:0x86:0x0] f226.sanity-hsm-1 lustre-MDT0000.60 06UNLNK 00:28:41.650712922 2026.08.16 0x3 t=[0x200000bd3:0x20:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp p=[0x200000bd2:0x86:0x0] f226.sanity-hsm-1 lustre-MDT0000.61 08RENME 00:28:41.765884928 2026.08.16 0x3 t=[0x200000bd3:0x21:0x0] j=mv.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp p=[0x200000bd2:0x86:0x0] f226.sanity-hsm-2 s=[0x200000bd3:0x22:0x0] sp=[0x200000bd2:0x86:0x0] f226.sanity-hsm-3 lustre-MDT0001: clear the changelog for cl12 of all records lustre-MDT0001: Deregistered changelog user #12 lustre-MDT0000: clear the changelog for cl12 of all records lustre-MDT0000: Deregistered changelog user #12 PASS 226 (8s) == sanity-hsm test 227: changelog when explicit setting of HSM flags ========================================================== 20:28:45 (1786840125) 0+0 records in 0+0 records out 0 bytes copied, 0.00112509 s, 0.0 kB/s mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl13 cl13' lustre-MDT0001: clear the changelog for cl13 of all records lustre-MDT0001: Deregistered changelog user #13 lustre-MDT0000: clear the changelog for cl13 of all records lustre-MDT0000: Deregistered changelog user #13 PASS 227 (6s) == sanity-hsm test 228: On released file, return extend to FIEMAP. For [cp,tar] --sparse ========================================================== 20:28:51 (1786840131) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_228/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242532 s, 43.2 MB/s Waiting 200s for 'SUCCEED' /mnt/lustre/f228.sanity-hsm: 1 extent found tar: Removing leading `/' from member names mnt/lustre/f228.sanity-hsm PASS 228 (6s) == sanity-hsm test 250: Coordinator max request ========== 20:28:57 (1786840137) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_250/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00103691 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00106759 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00117846 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00114595 s, 0.0 kB/s Waiting 200s for 'SUCCEED' 0+0 records in 0+0 records out 0 bytes copied, 0.00102604 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00115517 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Copytool is suspended on oleg316-client.virtnet Waiting 200s for 'STARTED' Waiting 200s for 'STARTED' Waiting 200s for 'STARTED' Copytool is continued on oleg316-client.virtnet PASS 250 (15s) == sanity-hsm test 251: Coordinator request timeout ====== 20:29:12 (1786840152) 0+0 records in 0+0 records out 0 bytes copied, 0.00106128 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_251/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg316-client.virtnet Waiting 200s for 'STARTED' Waiting 200s for 'CANCELED' PASS 251 (10s) == sanity-hsm test 252: Timeout'ed running archive of a removed file should be canceled ========================================================== 20:29:22 (1786840162) 0+0 records in 0+0 records out 0 bytes copied, 0.00104766 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_252/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg316-client.virtnet Waiting 200s for 'STARTED' Waiting 200s for 'CANCELED' Copytool is continued on oleg316-client.virtnet PASS 252 (8s) == sanity-hsm test 253: Check for wrong file size after release ========================================================== 20:29:30 (1786840170) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_253/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 10+0 records in 10+0 records out 10000000 bytes (10 MB, 9.5 MiB) copied, 0.0791537 s, 126 MB/s Waiting 200s for 'SUCCEED' fail_loc=0x807 Cannot send HSM request (use of /mnt/lustre/d253.sanity-hsm/f253.sanity-hsm): Invalid argument could not release file PASS 253 (4s) == sanity-hsm test 254a: Request counters are initialized to zero ========================================================== 20:29:34 (1786840174) PASS 254a (2s) == sanity-hsm test 254b: Request counters are correctly incremented and decremented ========================================================== 20:29:36 (1786840176) Will launch 55 requests of each type Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_254b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Checking archive requests Copytool is suspended on oleg316-client.virtnet Waiting 90s for '55' Updated after 2s: want '55' got '55' Copytool is continued on oleg316-client.virtnet Checking restore requests Copytool is suspended on oleg316-client.virtnet Waiting 90s for '55' Updated after 2s: want '55' got '55' Copytool is continued on oleg316-client.virtnet Waiting 90s for '0' Checking remove requests Copytool is suspended on oleg316-client.virtnet Waiting 90s for '55' Copytool is continued on oleg316-client.virtnet PASS 254b (14s) == sanity-hsm test 255: Copytool registration wakes the coordinator up ========================================================== 20:29:50 (1786840190) 0+0 records in 0+0 records out 0 bytes copied, 0.00126503 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s 0+0 records in 0+0 records out 0 bytes copied, 0.00103841 s, 0.0 kB/s 0x200000bd3:0xa2:0x0 mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' PASS 255 (11s) == sanity-hsm test 256: HSM copytool register on DNE with stopped MDT must not panic ========================================================== 20:30:01 (1786840201) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=enabled PASS 256 (24s) == sanity-hsm test 260a: Restore request have priority over other requests ========================================================== 20:30:25 (1786840225) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313801 s, 33.4 MB/s 0x200000bd3:0xa3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0396707 s, 26.4 MB/s 0x200000bd3:0xa4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320894 s, 32.7 MB/s 0x200000bd3:0xa5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356595 s, 29.4 MB/s 0x200000bd3:0xa6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393269 s, 26.7 MB/s 0x200000bd3:0xa7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0349981 s, 30.0 MB/s 0x200000bd3:0xa8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293599 s, 35.7 MB/s 0x200000bd3:0xa9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332768 s, 31.5 MB/s 0x200000bd3:0xaa:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334745 s, 31.3 MB/s 0x200000bd3:0xab:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0367301 s, 28.5 MB/s 0x200000bd3:0xac:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331935 s, 31.6 MB/s 0x200000bd3:0xad:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336633 s, 31.1 MB/s 0x200000bd3:0xae:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310111 s, 33.8 MB/s 0x200000bd3:0xaf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301031 s, 34.8 MB/s 0x200000bd3:0xb0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316196 s, 33.2 MB/s 0x200000bd3:0xb1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321839 s, 32.6 MB/s 0x200000bd3:0xb2:0x0 Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' ARCHIVE ARCHIVE RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260a (14s) == sanity-hsm test 260b: Restore request have priority over other requests ========================================================== 20:30:39 (1786840239) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295086 s, 35.5 MB/s 0x200000bd3:0xb5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027694 s, 37.9 MB/s 0x200000bd3:0xb6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026963 s, 38.9 MB/s 0x200000bd3:0xb7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277117 s, 37.8 MB/s 0x200000bd3:0xb8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283116 s, 37.0 MB/s 0x200000bd3:0xb9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02741 s, 38.3 MB/s 0x200000bd3:0xba:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290959 s, 36.0 MB/s 0x200000bd3:0xbb:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306696 s, 34.2 MB/s 0x200000bd3:0xbc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275467 s, 38.1 MB/s 0x200000bd3:0xbd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290543 s, 36.1 MB/s 0x200000bd3:0xbe:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293381 s, 35.7 MB/s 0x200000bd3:0xbf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298693 s, 35.1 MB/s 0x200000bd3:0xc0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302446 s, 34.7 MB/s 0x200000bd3:0xc1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284261 s, 36.9 MB/s 0x200000bd3:0xc2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029323 s, 35.8 MB/s 0x200000bd3:0xc3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312433 s, 33.6 MB/s 0x200000bd3:0xc4:0x0 Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260b (14s) == sanity-hsm test 260c: Requests are not reordered on the 'hot' path of the coordinator ========================================================== 20:30:54 (1786840254) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317032 s, 33.1 MB/s 0x200000bd3:0xc6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290966 s, 36.0 MB/s 0x200000bd3:0xc7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270995 s, 38.7 MB/s 0x200000bd3:0xc8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270403 s, 38.8 MB/s 0x200000bd3:0xc9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296967 s, 35.3 MB/s 0x200000bd3:0xca:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271399 s, 38.6 MB/s 0x200000bd3:0xcb:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295877 s, 35.4 MB/s 0x200000bd3:0xcc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271991 s, 38.6 MB/s 0x200000bd3:0xcd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289745 s, 36.2 MB/s 0x200000bd3:0xce:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305961 s, 34.3 MB/s 0x200000bd3:0xcf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258164 s, 40.6 MB/s 0x200000bd3:0xd0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265947 s, 39.4 MB/s 0x200000bd3:0xd1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028166 s, 37.2 MB/s 0x200000bd3:0xd2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244241 s, 42.9 MB/s 0x200000bd3:0xd3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265945 s, 39.4 MB/s 0x200000bd3:0xd4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028404 s, 36.9 MB/s 0x200000bd3:0xd5:0x0 Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE RESTORE PASS 260c (16s) == sanity-hsm test 261: Report 0 bytes size after HSM release ========================================================== 20:31:09 (1786840269) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_261/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.00238347 s, 3.4 MB/s /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x00000009) exists archived, archive_id:2 Waiting 200s for 'SUCCEED' /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x0000000d) released exists archived, archive_id:2 PASS 261 (5s) == sanity-hsm test 262: The client should return 1 block for HSM released files ========================================================== 20:31:14 (1786840274) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_262/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.00266126 s, 3.1 MB/s Waiting 200s for 'SUCCEED' PASS 262 (4s) == sanity-hsm test 300: On disk coordinator state kept between MDT umount/mount ========================================================== 20:31:18 (1786840278) Stop coordinator and remove coordinator state at mount mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown Failing mds1 on oleg316-server Stopping /mnt/lustre-mds1 (opts:) on oleg316-server 20:31:27 (1786840287) shut down facet: mds1 facet_host: oleg316-server facet_failover_host: oleg316-server Failover mds1 to oleg316-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg316-server: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-MDT0000 20:31:41 (1786840301) targets are mounted 20:31:41 (1786840301) facet_failover done oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Set coordinator start at mount, and start coordinator Failing mds1 on oleg316-server Stopping /mnt/lustre-mds1 (opts:) on oleg316-server 20:32:11 (1786840331) shut down facet: mds1 facet_host: oleg316-server facet_failover_host: oleg316-server Failover mds1 to oleg316-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg316-server: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-MDT0000 20:32:25 (1786840345) targets are mounted 20:32:25 (1786840345) facet_failover done oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 300 (76s) == sanity-hsm test 301: HSM tunnable are persistent ====== 20:32:34 (1786840354) Failing mds1 on oleg316-server Stopping /mnt/lustre-mds1 (opts:) on oleg316-server 20:32:37 (1786840357) shut down facet: mds1 facet_host: oleg316-server facet_failover_host: oleg316-server Failover mds1 to oleg316-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg316-server: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-MDT0000 20:32:51 (1786840371) targets are mounted 20:32:51 (1786840371) facet_failover done oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 301 (26s) == sanity-hsm test 302: HSM tunnable are persistent when CDT is off ========================================================== 20:33:00 (1786840380) mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown Failing mds1 on oleg316-server Stopping /mnt/lustre-mds1 (opts:) on oleg316-server 20:33:03 (1786840383) shut down facet: mds1 facet_host: oleg316-server facet_failover_host: oleg316-server Failover mds1 to oleg316-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg316-server: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-MDT0000 20:33:17 (1786840397) targets are mounted 20:33:17 (1786840397) facet_failover done oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds2 on oleg316-server Stopping /mnt/lustre-mds2 (opts:) on oleg316-server 20:33:25 (1786840405) shut down facet: mds2 facet_host: oleg316-server facet_failover_host: oleg316-server Failover mds2 to oleg316-server mount facets: mds2 Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg316-server: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-MDT0001 20:33:39 (1786840419) targets are mounted 20:33:39 (1786840419) facet_failover done oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 302 (49s) == sanity-hsm test 400: Single request is sent to the right MDT ========================================================== 20:33:49 (1786840429) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_400/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328658 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03 s, 35.0 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' archive successful on mdt0 Waiting 200s for 'SUCCEED' archive successful on mdt1 PASS 400 (5s) == sanity-hsm test 401: Compound requests split and sent to their respective MDTs ========================================================== 20:33:54 (1786840434) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_401/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319698 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287671 s, 36.5 MB/s Waiting 200s for 'SUCCEED' archive successful on mdt0 archive successful on mdt1 PASS 401 (4s) == sanity-hsm test 402a: Copytool start fails if all MDTs are inactive ========================================================== 20:33:58 (1786840438) deactivate lustre-MDT0000-mdc-ffff9c7d4a414000 on oleg316-client.virtnet deactivate lustre-MDT0001-mdc-ffff9c7d4a414000 on oleg316-client.virtnet deactivate lustre-MDT0000-mdc-ffff9c7d4a8c7800 on oleg316-client.virtnet deactivate lustre-MDT0001-mdc-ffff9c7d4a8c7800 on oleg316-client.virtnet Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Failed to start copytool agt1 on 'oleg316-client.virtnet' pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 uuid not found in agent list on mds2 activate lustre-MDT0000-mdc-ffff9c7d4a414000 on oleg316-client.virtnet activate lustre-MDT0001-mdc-ffff9c7d4a414000 on oleg316-client.virtnet activate lustre-MDT0000-mdc-ffff9c7d4a8c7800 on oleg316-client.virtnet activate lustre-MDT0001-mdc-ffff9c7d4a8c7800 on oleg316-client.virtnet PASS 402a (3s) == sanity-hsm test 402b: CDT must retry request upon slow start of CT ========================================================== 20:34:01 (1786840441) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x14d fail_loc=0 Waiting 200s for 'SUCCEED' Updated after 6s: want 'SUCCEED' got 'SUCCEED' PASS 402b (19s) == sanity-hsm test 403: Copytool starts with inactive MDT and register on reconnect ========================================================== 20:34:20 (1786840460) deactivate lustre-MDT0001-mdc-ffff9c7d4a414000 on oleg316-client.virtnet deactivate lustre-MDT0001-mdc-ffff9c7d4a8c7800 on oleg316-client.virtnet Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_403/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg316-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg316-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 uuid not found in agent list on mds2 oleg316-client.virtnet: 136561 activate lustre-MDT0001-mdc-ffff9c7d4a414000 on oleg316-client.virtnet activate lustre-MDT0001-mdc-ffff9c7d4a8c7800 on oleg316-client.virtnet found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=d0afbd13-56e4-4b53-8abd-a00edad637cc archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 403 (6s) == sanity-hsm test 404: Inactive MDT does not block requests for active MDTs ========================================================== 20:34:26 (1786840466) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_404/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266224 s, 39.4 MB/s deactivate lustre-MDT0001-mdc-ffff9c7d4a414000 on oleg316-client.virtnet deactivate lustre-MDT0001-mdc-ffff9c7d4a8c7800 on oleg316-client.virtnet archive successful on mdt0 activate lustre-MDT0001-mdc-ffff9c7d4a414000 on oleg316-client.virtnet activate lustre-MDT0001-mdc-ffff9c7d4a8c7800 on oleg316-client.virtnet PASS 404 (3s) == sanity-hsm test 405: archive and release under striped directory ========================================================== 20:34:29 (1786840469) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_405/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231948 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188457 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203926 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196528 s, 53.4 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' archive successful on 0x240000bd2:0x1:0x0 archive successful on 0x240000bd2:0x2:0x0 archive successful on 0x240000bd2:0x3:0x0 archive successful on 0x2000032e3:0x6:0x0 PASS 405 (8s) == sanity-hsm test 406: attempting to migrate HSM archived files is safe ========================================================== 20:34:37 (1786840477) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286642 s, 36.6 MB/s old fid 0x2000032e3:0x9:0x0 Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_406/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lfs migrate: /mnt/lustre/d406.sanity-hsm/f406.sanity-hsm migrate failed: Operation not supported (95) lfs migrate: cb_migrate_mdt_fini: error completing migration of /mnt/lustre/d406.sanity-hsm: Directory not empty (39) Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' new fid 0x240000bd3:0x7:0x0 Waiting 200s for 'SUCCEED' debug=+trace Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 406 (9s) == sanity-hsm test 407: Check for double RESTORE records in llog ========================================================== 20:34:46 (1786840486) 0+0 records in 0+0 records out 0 bytes copied, 0.00209382 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_407/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' fail_val=5 fail_loc=0x164 Copytool is suspended on oleg316-client.virtnet mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/1] fid=[0x200000bd3:0xdc:0x0] dfid=[0x200000bd3:0xdc:0x0] compound/cookie=0x0/0x6a81054d action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/2] fid=[0x200000bd3:0xe0:0x0] dfid=[0x200000bd3:0xe0:0x0] compound/cookie=0x0/0x6a81054e action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/3] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x6a81054f action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x6a810550 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x6a810551 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6a810552 action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6a810553 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6a810554 action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6a810555 action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6a810556 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] Failing mds1 on oleg316-server Stopping /mnt/lustre-mds1 (opts:) on oleg316-server 20:34:54 (1786840494) shut down facet: mds1 facet_host: oleg316-server facet_failover_host: oleg316-server Failover mds1 to oleg316-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg316-server: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-MDT0000 20:35:08 (1786840508) targets are mounted 20:35:08 (1786840508) facet_failover done oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec fail_loc=0 mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/3] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x6a81054f action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x6a810550 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x6a810551 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6a810552 action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6a810553 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6a810554 action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6a810555 action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6a810556 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/11] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6a810557 action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] Copytool is continued on oleg316-client.virtnet d41d8cd98f00b204e9800998ecf8427e /mnt/lustre/d407.sanity-hsm/f407.sanity-hsm d41d8cd98f00b204e9800998ecf8427e /mnt/lustre2/d407.sanity-hsm/f407.sanity-hsm pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/3] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x6a81054f action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x6a810550 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x6a810551 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6a810552 action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6a810553 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6a810554 action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6a810555 action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6a810556 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/11] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xd:0x0] compound/cookie=0x0/0x6a810557 action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] PASS 407 (35s) == sanity-hsm test 408: Verify fiemap on release file ==== 20:35:21 (1786840521) Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_408/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0103283 s, 6.3 MB/s 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00813984 s, 8.1 MB/s File: /mnt/lustre/f408.sanity-hsm Size: 262144 Blocks: 256 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115406615871500 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-15 20:35:21.000000000 -0400 Modify: 2026-08-15 20:35:21.000000000 -0400 Change: 2026-08-15 20:35:21.000000000 -0400 Birth: 2026-08-15 20:35:21.000000000 -0400 disk usage: 131072 /mnt/lustre/f408.sanity-hsm file size: 262144 /mnt/lustre/f408.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' extent 0 in offset 0, length 262144 flags (0x3): LAST UNKNOWN No unwritten extents, extents number 1, sum of lengths 262144, expected sum 262144 PASS 408 (5s) == sanity-hsm test 409a: Coordinator should not stop when in use ========================================================== 20:35:26 (1786840526) 0+0 records in 0+0 records out 0 bytes copied, 0.00106214 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' fail_val=5 fail_loc=0x164 cat: /mnt/lustre/d409a.sanity-hsm/f409a.sanity-hsm: No data available mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 409a (14s) == sanity-hsm test 409b: getattr released file with CDT stopped after remount ========================================================== 20:35:40 (1786840540) 0+0 records in 0+0 records out 0 bytes copied, 0.00129457 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s Failing mds1 on oleg316-server Stopping /mnt/lustre-mds1 (opts:) on oleg316-server 20:36:05 (1786840565) shut down facet: mds1 facet_host: oleg316-server facet_failover_host: oleg316-server Failover mds1 to oleg316-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg316-server: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-MDT0000 20:36:19 (1786840579) targets are mounted 20:36:19 (1786840579) facet_failover done oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec File: /mnt/lustre/d409b.sanity-hsm/f409b.sanity-hsm Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115406632648721 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-15 20:35:41.000000000 -0400 Modify: 2026-08-15 20:35:41.000000000 -0400 Change: 2026-08-15 20:35:41.000000000 -0400 Birth: 2026-08-15 20:35:40.000000000 -0400 PASS 409b (69s) == sanity-hsm test 410: lfs data_version -s allows release of force-archived file ========================================================== 20:36:49 (1786840609) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736654 s, 14.2 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_410/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d410.sanity-hsm/f410.sanity-hsm): Operation not permitted PASS 410 (3s) == sanity-hsm test 411: hsm_ops rbac role ================ 20:36:52 (1786840612) On MGS 192.168.203.116, active = nodemap.active=1 On MGS 192.168.203.116, test_411.trusted_nodemap = nodemap.test_411.trusted_nodemap=1 Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d411.sanity-hsm/fileA: (0x00000000) Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg316-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' On MGS 192.168.203.116, test_411.rbac = nodemap.test_411.rbac=file_perms,dne_ops,quota_ops,byfid_ops,chlg_ops,fscrypt_admin,server_upcall,ignore_root_prjquota,immutable_flags,local_admin,pool_quota_ops,lqa_quota_ops,projid_set,foreign_ops /mnt/lustre/d411.sanity-hsm/fileB: (0x00000000) Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied lfs hsm_set: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied lfs hsm_clear: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied On MGS 192.168.203.116, active = nodemap.active=0 oleg316-server: error: nodemap 'test_411' does not exist oleg316-server: nodemap_del: Invalid argument pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 On MGS 192.168.203.116, active = nodemap.active=0 PASS 411 (16s) == sanity-hsm test 500: various LLAPI HSM tests ========== 20:37:08 (1786840628) striped dir -i0 -c2 -H crush /mnt/lustre/d500.sanity-hsm test1 @1786840643: Register/unregister copytool 2000x to check for leaks ...... pass test2 @1786840643: Re/un-register copytool multiple times without error ....... pass llapi_hsm_test: NULL archive numbers llapi_hsm_test: NULL archive numbers llapi_hsm_test: -1 requested when archive id >= 0 is supported test3 @1786840643: Pass bad parameters to llapi_hsm_copytool_register() ....... pass test4 @1786840643: Bad parameters to llapi_hsm_copytool_unregister() .......... pass test5 @1786840644: Test llapi_hsm_copytool_recv() in non-blocking mode ........ pass test6 @1786840644: Test llapi_hsm_copytool_recv() with bogus parameters ....... pass test7 @1786840644: Test event polling (without actual traffic) ................ pass test50 @1786840644: Test llapi_hsm_state_get()/get_fd() ....................... pass test51 @1786840644: Test llapi_hsm_state_set_fd() ............................. pass test52 @1786840644: Test llapi_hsm_current_action() ........................... pass test100 @1786840644: Simple archive creation with no progress reported. ....... pass test101 @1786840646: Simple archive creation with progress every byte. ........ pass test102 @1786840648: Archive creation with progress every byte, backwards ..... pass test103 @1786840649: Archive creation with a single progress report ........... pass test104 @1786840650: Archive creation with two progress reports ............... pass test105 @1786840651: Archive creation with one bogus progress report .......... pass test106 @1786840652: Archive creation with one empty progress report .......... pass test107 @1786840653: Archive creation with one bogus progress report .......... pass test108 @1786840655: Archive creation with same progress report each time ..... pass test109 @1786840656: Archive creation with one report, with large number ...... pass test110 @1786840657: Archive with 10 progress reports, checking progress ...... pass test111 @1786840658: Archive with 10 reports in reverse, checking progress .... pass test112 @1786840659: Archive with 10 reports, duplicated, check progress ...... pass test113 @1786840660: Archive with 9 reports, with 20% overlapping coverage .... pass PASS 500 (34s) == sanity-hsm test 600: Changelog fields 'u=' and 'nid=' ========================================================== 20:37:42 (1786840662) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d600.sanity-hsm/f600.sanity-hsm] Got NID '192.168.203.16@tcp' lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 600 (6s) == sanity-hsm test 601: OPEN Changelog entry ============= 20:37:48 (1786840668) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0000.4 10OPEN 00:37:50.676479270 2026.08.16 0x1 t=[0x2000032e2:0x1b:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.16@tcp m=r-- lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 601 (5s) == sanity-hsm test 602: Changelog record CLOSE only if open+write or OPEN recorded ========================================================== 20:37:54 (1786840674) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001.9 11CLOSE 00:37:56.269728893 2026.08.16 0x1 t=[0x240000bd3:0x1d:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001.10 10OPEN 00:37:56.374275112 2026.08.16 0x242 t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.16@tcp m=-w- lustre-MDT0001.11 23GXATR 00:37:56.376021405 2026.08.16 0x0 t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x1b u=0:0 nid=192.168.203.16@tcp x=user.job lustre-MDT0001.12 13TRUNC 00:37:56.377250276 2026.08.16 0xe t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp p=[0x240000bd3:0x1c:0x0] lustre-MDT0001.13 11CLOSE 00:37:56.382834551 2026.08.16 0x242 t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001.13 11CLOSE 00:37:56.382834551 2026.08.16 0x242 t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp mdd.lustre-MDT0000.changelog_mask=-OPEN mdd.lustre-MDT0001.changelog_mask=-OPEN lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records f lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001.14 13TRUNC 00:37:57.021366104 2026.08.16 0xe t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp p=[0x240000bd3:0x1c:0x0] lustre-MDT0001.15 11CLOSE 00:37:57.024604273 2026.08.16 0x242 t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001.15 11CLOSE 00:37:57.024604273 2026.08.16 0x242 t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 602 (6s) == sanity-hsm test 603: GETXATTR Changelog entry ========= 20:38:00 (1786840680) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0001: clear the changelog for cl4 of all records getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d603.sanity-hsm/f603.sanity-hsm user.xattr1="value1" lustre-MDT0000.10 23GXATR 00:38:02.487120639 2026.08.16 0x0 t=[0x2000032e2:0x1d:0x0] j=getfattr.0 ef=0x1b u=0:0 nid=192.168.203.16@tcp x=user.xattr1 lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 603 (5s) == sanity-hsm test 604: NOPEN Changelog entry ============ 20:38:05 (1786840685) mdd.lustre-MDT0000.changelog_deniednext=20 mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.15 24NOPEN 00:38:08.632223087 2026.08.16 0x1 t=[0x2000032e2:0x1f:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.16@tcp m=r-- p=[0x2000032e2:0x1e:0x0] Got NID '192.168.203.16@tcp' lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.16 24NOPEN 00:38:28.917824019 2026.08.16 0x1 t=[0x2000032e2:0x1f:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.16@tcp m=r-- Got NID '192.168.203.16@tcp' lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 mdd.lustre-MDT0000.changelog_deniednext=60 PASS 604 (27s) == sanity-hsm test 605: Test OPEN and CLOSE rate limit in Changelogs ========================================================== 20:38:32 (1786840712) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001.19 10OPEN 00:38:35.240321415 2026.08.16 0x43 t=[0x240000bd3:0x1f:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.16@tcp m=rw- lustre-MDT0001.19 10OPEN 00:38:35.240321415 2026.08.16 0x43 t=[0x240000bd3:0x1f:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.16@tcp m=rw- lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001.20 10OPEN 00:38:35.597862339 2026.08.16 0x1 t=[0x240000bd3:0x1f:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.16@tcp m=r-- p=[0x240000bd3:0x1c:0x0] lustre-MDT0001.21 11CLOSE 00:38:35.603371301 2026.08.16 0x1 t=[0x240000bd3:0x1f:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001.20 10OPEN 00:38:35.597862339 2026.08.16 0x1 t=[0x240000bd3:0x1f:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.16@tcp m=r-- p=[0x240000bd3:0x1c:0x0] lustre-MDT0001.21 11CLOSE 00:38:35.603371301 2026.08.16 0x1 t=[0x240000bd3:0x1f:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre/d605.sanity-hsm/f605.sanity-hsm] lustre-MDT0001.22 10OPEN 00:38:35.986410816 2026.08.16 0x1 t=[0x240000bd3:0x1f:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.16@tcp m=r-- p=[0x200000bd2:0x80:0x0] lustre-MDT0001.23 11CLOSE 00:38:35.989599957 2026.08.16 0x1 t=[0x240000bd3:0x1f:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.203.16@tcp lustre-MDT0001.22 10OPEN 00:38:35.986410816 2026.08.16 0x1 t=[0x240000bd3:0x1f:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.16@tcp m=r-- p=[0x200000bd2:0x80:0x0] lustre-MDT0001.23 11CLOSE 00:38:35.989599957 2026.08.16 0x1 t=[0x240000bd3:0x1f:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.203.16@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001.24 11CLOSE 00:38:36.177929587 2026.08.16 0x43 t=[0x240000bd3:0x1f:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001.24 11CLOSE 00:38:36.177929587 2026.08.16 0x43 t=[0x240000bd3:0x1f:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.16@tcp lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 605 (7s) == sanity-hsm test 606: llog_reader groks changelog fields ========================================================== 20:38:39 (1786840719) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d606.sanity-hsm/f606.sanity-hsm] Stopping /mnt/lustre-mds1 (opts:) on oleg316-server rec #2 type=1064553b len=64 offset 8256 Header size : 8192 llh_size : 64 Time : Sat Aug 15 20:25:32 2026 Number of records: 1 cat_idx: 0 last_idx: 2 Flags: 0x2012 Target uuid : ----------------------- #02 (064)fid=[0x1:0x3b:0x0] path=O/1/d27/59 Got UID/GID 500:500 Got NID 192.168.203.16@tcp Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg316-server: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-MDT0000 lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 606 (17s) == sanity-hsm test 607a: release file migrated after archive (non-blocking migrate) ========================================================== 20:38:56 (1786840736) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0652667 s, 16.1 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' PASS 607a (5s) == sanity-hsm test 607b: Migrate should not change the HSM attribute of dirty files ========================================================== 20:39:01 (1786840741) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0702326 s, 14.9 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' llite.lustre-ffff9c7d4a414000.xattr_cache=0 llite.lustre-ffff9c7d4a8c7800.xattr_cache=0 /mnt/lustre/d607b.sanity-hsm/f607b.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 llite.lustre-ffff9c7d4a414000.xattr_cache=1 llite.lustre-ffff9c7d4a8c7800.xattr_cache=1 PASS 607b (4s) == sanity-hsm test 607c: 'lfs swap_layouts' should set dirty flag on HSM file ========================================================== 20:39:05 (1786840745) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0702403 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0693633 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0635695 s, 16.5 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-1' and '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-3': Operation not permitted (1) PASS 607c (4s) == sanity-hsm test 607d: Migrate should not set HSM dirty flag for an empty file ========================================================== 20:39:09 (1786840749) 0+0 records in 0+0 records out 0 bytes copied, 0.00149377 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' PASS 607d (6s) == sanity-hsm test 607e: release file migrated after archive (blocking migrate) ========================================================== 20:39:15 (1786840755) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0630137 s, 16.6 MB/s Starting copytool 'agt1' on 'oleg316-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' PASS 607e (4s) == sanity-hsm test complete, duration 2185 sec =========== 20:39:19 (1786840759) === sanity-hsm: start cleanup 20:39:20 (1786840760) === Stopping clients: oleg316-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg316-client.virtnet /mnt/lustre2 opts: === sanity-hsm: finish cleanup 20:39:27 (1786840767) ===